Selenium and Web-Drivers in lambda function

0

Hello everyone, Our team is working on a web scraping project, that we'd like to upload to aws lambda. But, unfortunately, we've encountered some problems with web drivers. Based on our research, we are using: Python 3.8 Selenium v4.15.1 Chrome driver v121.0.6167.139 Headless-Chromium v1.0.0-57 (web drivers are packed together, and added to a layer through S3) as described in: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Synthetics_Library_python_selenium.html We've modeled our code according to those few tutorials we were able to find (such as https://www.youtube.com/watch?v=b49Y3NGJX68). But while trying to run even the simplest code, we encounter one of two errors: "Service /opt/chromedriver unexpectedly exited. Status code was: 127", or "Unable to locate or obtain driver for chrome".

We are using the following code structure: def lambda_handler(event, context):

logging.info('Web driver started')

# Define path
service = Service(executable_path=r'/opt/chromedriver')

# Define options
chrome_options = Options()

chrome_options.binary_location = '/opt/headless-chromium'

chrome_options.add_argument("--headless")

# Initialize Chrome
driver = webdriver.Chrome(service=service, options=chrome_options)
logging.info('Web driver defined')

# Open Google and then quit the driver
driver.get('https://www.google.ru')
logging.info('Web driver opened')

driver.quit()
logging.info('Web driver closed')

return "finished"

We'd be truly thankful if anyone could help us understand the reasons behind the mentioned errors.

  • Are the chromedriver and the headless-chromium in your Lambda layer built for the correct CPU architecture of your Lambda function? (i.e arm64 vs x86_64)

profile picture
Giorgi
gefragt vor einem Monat110 Aufrufe
Keine Antworten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen