Questions tagged with AWS Lambda

I am trying to invoke googleapis through AWS lambda. I am using a google service account to generate server-server authentication. we have stored the service account details in a JSON file name 'config.json'. Inorder to invoke API for google, we need to create auth object using the 'GoogleAuth' class and pass the config file path as a value to the 'keyfile' property. Though we have provided the correct path location of the file, lambda could not recognize the path and throw error. For path, I have tried absolute, and relative paths, using Path package with '__dirname", process.env.cwd(), used environment variable etc. I even tried with assets too. I am using AWS cdk to form cloud formation using nodejs. My intention is to invoke google apis using service account credentials with AWS lambda. ``` import {GoogleAuth} from 'google-auth-library' const auth = new GoogleAuth({ keyFile: 'path/to/file', scope: SCOPES }) ``` ``` undefined ERROR Uncaught Exception {"errorType":"Error","errorMessage":"ENOENT: no such file or directory, open '/keys/config.json'","code":"ENOENT","errno":-2,"syscall":"open","path":"/keys/config.json","stack":["Error: ENOENT: no such file or directory, open '/keys/config.json'"," at Object.openSync (node:fs:601:3)"," at Object.readFileSync (node:fs:469:35)"," at Object.<anonymous> (/var/task/index.js:533512:28)"," at Module._compile (node:internal/modules/cjs/loader:1254:14)"," at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)"," at Module.load (node:internal/modules/cjs/loader:1117:32)"," at Module._load (node:internal/modules/cjs/loader:958:12)"," at Module.require (node:internal/modules/cjs/loader:1141:19)"," at require (node:internal/modules/cjs/helpers:110:18)"," at _tryRequireFile (file:///var/runtime/index.mjs:912:37)"]} ```lg...

Hi, I'm trying to implement a system which essentially provides in-order item-level information about DDB updates to multiple consumers. I'm using DDB streams as Kinesis streams from DDB don't maintain order and deduping ([source](https://stackoverflow.com/a/74487638/19570509)). However, DDB streams can only have a max. of 2 concurrent consumers before being throttled. So what I'm trying to do is have the DDB stream trigger a lambda, which serializes the `com.amazonaws.services.lambda.runtime.events.DyanmodbEvent.DynamodbStreamRecord`, and passes it into a kinesis stream. A few questions: 1. What's the best method of serializing the DynamodbStreamRecord? I've seen some examples using KCL and the `RecordAdapter`, but that operates on the `com.amazonaws.services.dynamodbv2.model.Record` object, and not the lambda event objects `DynamodbStreamRecord` and `StreamRecord`. 1. When I'm writing code to send data into a kinesis stream, the `putRecord` API in the kinesis client requires a partition key parameter. If I'm aiming to maintain the same order on an item level in the kinesis stream, should the partition key that I supply for the `putRecord` call just be the partition key of the relevant item? Thanks!lg...

I am trying to list the databases in my DocumentDb from a Lambda function, and the task times out. It is a simple command it should not be timing out. What do I need to fix so that the command runs properly? ``` client = MongoClient(connectionString) logging.info("All the databases") logging.info(client.list_database_names()) ``` When I run the code on my VS Code IDE I get this message; > Exception has occurred: ServerSelectionTimeoutError docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed, Timeout: 30s, Topology Description: <TopologyDescription id: 641e0297b555a6d973f82353, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed')>]> File "C:\mma\Lambda_tweets\lambda_handler.py", line 47, in getDBConnection logging.info(client.list_database_names()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\mma\Lambda_tweets\lambda_handler.py", line 127, in lambda_handler getDBConnection() File "C:\mma\Lambda_tweets\lambda_handler.py", line 130, in <module> lambda_handler(None, None) pymongo.errors.ServerSelectionTimeoutError: docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed, Timeout: 30s, Topology Description: <TopologyDescription id: 641e0297b555a6d973f82353, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno 11001] getaddrinfo failed')>]> EDIT: When I extended the timeout as suggested in the comments, the error messages I was getting were as follows; > Response { "errorMessage": "docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known, Timeout: 30s, Topology Description: <TopologyDescription id: 641f88d7e7919b310a30cf51, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known')>]>", "errorType": "ServerSelectionTimeoutError", "requestId": "2f9fb04f-9519-41e9-b989-0dd8b6f3382b", "stackTrace": [ " File \"/var/task/lambda_function.py\", line 62, in lambda_handler\n getDBConnection()\n", " File \"/var/task/lambda_function.py\", line 51, in getDBConnection\n logging.info(client.list_database_names())\n", " File \"/opt/python/pymongo/mongo_client.py\", line 1867, in list_database_names\n return [doc[\"name\"] for doc in self.list_databases(session, nameOnly=True, comment=comment)]\n", " File \"/opt/python/pymongo/mongo_client.py\", line 1840, in list_databases\n res = admin._retryable_read_command(cmd, session=session)\n", " File \"/opt/python/pymongo/database.py\", line 849, in _retryable_read_command\n return self.__client._retryable_read(_cmd, read_preference, session)\n", " File \"/opt/python/pymongo/_csot.py\", line 105, in csot_wrapper\n return func(self, *args, **kwargs)\n", " File \"/opt/python/pymongo/mongo_client.py\", line 1441, in _retryable_read\n server = self._select_server(read_pref, session, address=address)\n", " File \"/opt/python/pymongo/mongo_client.py\", line 1257, in _select_server\n server = topology.select_server(server_selector)\n", " File \"/opt/python/pymongo/topology.py\", line 272, in select_server\n server = self._select_server(selector, server_selection_timeout, address)\n", " File \"/opt/python/pymongo/topology.py\", line 261, in _select_server\n servers = self.select_servers(selector, server_selection_timeout, address)\n", " File \"/opt/python/pymongo/topology.py\", line 223, in select_servers\n server_descriptions = self._select_servers_loop(selector, server_timeout, address)\n", " File \"/opt/python/pymongo/topology.py\", line 238, in _select_servers_loop\n raise ServerSelectionTimeoutError(\n" ] } > Function Logs START RequestId: 2f9fb04f-9519-41e9-b989-0dd8b6f3382b Version: $LATEST [INFO] 2023-03-25T23:50:47.126Z 2f9fb04f-9519-41e9-b989-0dd8b6f3382b get client /opt/python/pymongo/common.py:813: UserWarning: Unknown option ssl_ca_certs warnings.warn(str(exc)) [INFO] 2023-03-25T23:50:47.380Z 2f9fb04f-9519-41e9-b989-0dd8b6f3382b Get database testdb [INFO] 2023-03-25T23:50:47.380Z 2f9fb04f-9519-41e9-b989-0dd8b6f3382b All the databases [ERROR] ServerSelectionTimeoutError: docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known, Timeout: 30s, Topology Description: <TopologyDescription id: 641f88d7e7919b310a30cf51, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017: [Errno -2] Name or service not known')>]> Traceback (most recent call last):   File "/var/task/lambda_function.py", line 62, in lambda_handler     getDBConnection()   File "/var/task/lambda_function.py", line 51, in getDBConnection     logging.info(client.list_database_names())   File "/opt/python/pymongo/mongo_client.py", line 1867, in list_database_names     return [doc["name"] for doc in self.list_databases(session, nameOnly=True, comment=comment)]   File "/opt/python/pymongo/mongo_client.py", line 1840, in list_databases     res = admin._retryable_read_command(cmd, session=session)   File "/opt/python/pymongo/database.py", line 849, in _retryable_read_command     return self.__client._retryable_read(_cmd, read_preference, session)   File "/opt/python/pymongo/_csot.py", line 105, in csot_wrapper     return func(self, *args, **kwargs)   File "/opt/python/pymongo/mongo_client.py", line 1441, in _retryable_read     server = self._select_server(read_pref, session, address=address)   File "/opt/python/pymongo/mongo_client.py", line 1257, in _select_server     server = topology.select_server(server_selector)   File "/opt/python/pymongo/topology.py", line 272, in select_server     server = self._select_server(selector, server_selection_timeout, address)   File "/opt/python/pymongo/topology.py", line 261, in _select_server     servers = self.select_servers(selector, server_selection_timeout, address)   File "/opt/python/pymongo/topology.py", line 223, in select_servers     server_descriptions = self._select_servers_loop(selector, server_timeout, address)   File "/opt/python/pymongo/topology.py", line 238, in _select_servers_loop     raise ServerSelectionTimeoutError(END RequestId: 2f9fb04f-9519-41e9-b989-0dd8b6f3382b REPORT RequestId: 2f9fb04f-9519-41e9-b989-0dd8b6f3382b Duration: 30494.77 ms Billed Duration: 30495 ms Memory Size: 128 MB Max Memory Used: 66 MB Init Duration: 555.69 mslg...

I develop a NET 6 Web REST API service (->RDS PostgreSQL) deployed as Lambda. I develop it in VS 2022 on Local Windows machine and use local PGSQL. When deploy it to Lambda it works with RDS. It is ok. Now I need to add Authentication and Authorization methods to use RBAC. My question is: In AWS I should use Cognito Identity Provider. What to use during development as Cognito in local environment?lg...

I am trying to get a list of database names for my DocumentDB. I am using the pymongo library. I run this code; ``` logging.info("get client") #connectionString = os.environ["documentdb_connection"] connectionString = "mongodb://geoff:qzmpqzmp@docdb-2023-03-22-20-49-15.cluster-cwl5gnwixa5k.us-east-1.docdb.amazonaws.com:27017/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false" ##Specify the database to be used logging.info("Get database testdb") #db = client.testdb logging.info("All the databases") #db = client.testdb logging.info(client.list_database_names()) ``` I have been unable to connect to the specific database I wanted so I thought I would comment those lines out and instead list all of the databases to check if it exists. On the last line of that code I get a NameError; "name 'client' is not defined" How do I fix this?lg...

I want to access DocumentDb in my Lambda function. I try to configure my VPC in the Edit VPC page for my lambda function but I get this error message that stops me; > The provided execution role does not have permissions to call CreateNetworkInterface on EC2 How do I fix this?lg...

Hi, I am deploying a lambda function that utilizes the NLTK packages for preprocessing text. For the application to work I need to download the stop words, punkt and wordnet libraries. I have deployed using a docker image and SAM cli. When the function runs on AWS, I get a series of errors when trying to access the NLTK libraries. The first error I got was that '/home/sbx_user1051/' cannot be edited. After reading solutions on stack over flow, I was pointed in the direction of needing to store the NLTK libraries in the /tmp/ directory because that is the only directory that can be modified. Now, after redeploying the image with the changes to the code, I have the files stored in temp, but the lambda function does not search for that file when trying to access the stop words. It still tries to search for the file in these directories: - '/home/sbx_user1051/nltk_data' - '/var/lang/nltk_data' - '/var/lang/share/nltk_data' - '/var/lang/lib/nltk_data' - '/usr/share/nltk_data' - '/usr/local/share/nltk_data' - '/usr/lib/nltk_data' - '/usr/local/lib/nltk_data' What should I do about importing the NLTK libraries needed when running this function on aws lambda?lg...

I am currently working on a lambda function in which I have to send a message to an SQS queue. The Lambda function sits inside of a VPC to allow connection with a peered network that the Function makes requests to. Whenever I try to send the message to SQS however code execution seems to timeout consistently. I had the same issue when I was trying to send commands to DynamoDB. ``` import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs"; const sqsClient = new SQSClient({region: 'us-east-1'}); export const handler = async (event, context, callback) => { const response = await sqsClient.send(new SendMessageCommand(messageParams)); console.log(response); // <----- Doesn't reach here return callback(null, 'OK'); }; ``` IAM Permissions are all correct and the Security Group allows all traffic (When set to a VPC) So far, to specifically target the timeout problem, I've tried putting the function in a private subnet, public subnet, placing it in no VPC, replacing SDK v3 with aws-sdk v2 via a layer. None of these seem to have any impact on the issue. I haven't used VPC endpoints yet but I guess that shouldn't be necessary when the function is not connected to a VPC or in a public subnet?lg...

Hello, I store data in S3 as part of Amazon Data Exchange Products. I want to create an API product so that users can pull my data in csv or json format directly from the bucket. The bucket contains multiple csv's per day Do I need a lambda fronting an api in api gateway which will read the contents of the file first? Or, can I simply create an API which will parse the files and return data to the subscribers? I would then package that API up into a "data product" Ideally the data would be pulled as json by default, but they would also be able to pull it as csv. I will need to be able to give users the ability to structure a payload via SDK or CLI or some other IDE in which they specify various fields, date ranges etc... Thank you.lg...

So I have a lambda function that's the following: def lambda_handler(event, context): http = urllib3.PoolManager() return { "statusCode": 200, "headers": { "Content-Type": "application/json" }, "body": "{\"message\": \"Hello from Lambda!\"}", "test": event } If I test run the event with some event JSON, it returns what I want. But... when I test to run the lambda through my rest API I created I either get the error 502 when "Use Lambda Proxy integration" is enabled. Or when it's disabled event is always empty. I have tried to setup a link the following way: https://something.execute-api.eu-north-1.amazonaws.com/test/test/{id} Where you enter an {id} for example and I can catch it in the event. But how can I fix so event is an actual thing and actually getting any information? If you feel like im missing some information that might be important just ask, I don't know what could be useful.lg...

Hi there, I am trying to do some experiments to migrate from the data pipeline to the step function. I call the step function from the lambda function by passing a couple of parameters (name, env). My question is, how do I pass these parameters and clusterID (from the emr) to all next steps and use it? Also, is it possible to change the name of the cluster to the parameter I passed from the lambda function? Like "Parameters": {         "Name": "$.name",....? Please see the attached picture. lg...

Hello All, I've been trying to setup a Multi-region access point for S3. The basic problem is that I have users in Asia that I'm trying to get better performance for. The simple part. I've created two buckets, I've put an html file in each of them with a simple name of us-east-1 and ap-south-1. Initially those were private access only but for the purpose of getting anything working they are now public. They are setup in an mrap which for now is not replicating. I setup a cloudfront distribution and pointed it at the mrap but only ever get errors. https://corridor-ap-south-1.s3.ap-south-1.amazonaws.com/test/region-test/region.html - ap-south-1 html https://corridor-cdn.s3.amazonaws.com/test/region-test/region.html - us-east-1 html mrap alias: mbzcc59bo9dy4.mrap mrap access point? https://mbzcc59bo9dy4.mrap.accesspoint.s3-global.amazonaws.com/test/region-test/region.html The errors are: The authorization mechanism you have provided is not supported. Please use Signature Version 4. I hope I'm wrong, but is there a ridiculously stupid world where I have to put a signature on my object request? This seems like it would be a complete waste of money to setup a lambda to do this for all my object requests.lg...