REST API 504 Gateway Timeout Error triggering Lambda running on Flask

My GET requests works flawlessly with my REST API endpoints, but my POST request returns 504 -> { "message": "Endpoint request timed out" } in less than a second. I have timeout set to 3 minutes on both my Lambda and API Gateway Integration Request. My lambda code is essentially a webscraper with calls to 3 different API's like OpenAI etc. My Lambda works fine locally if I send a structured payload (Note: locally function executes in less than 30 seconds). I have my Lambda memory set to more than required. I have spent considerate amount of time trying to fix it but failed too. Would appreciate any suggestions/actions. I have attached CloudWatch logs below. This is for Lambda Function: 2024-08-26T13:47:41.212Z START RequestId: Version: $LATEST 2024-08-26T13:47:45.243Z https://www.aboutamazon.com/ 2024-08-26T13:48:03.672Z END RequestId: 2024-08-26T13:48:03.672Z REPORT RequestId: Duration: 22460.56 ms Billed Duration: 26498 ms Memory Size: 2048 MB

This is API Gateway Logs:

2024-08-26T13:47:36.444Z Method request body before transformations: {"companyName": "Amazon"} 2024-08-26T13:47:36.444Z Request validation succeeded for content type application/json 2024-08-26T13:47:36.444Z Endpoint request URI: https://lambda.us-east-1.amazonaws.com/funcname/invocations 2024-08-26T13:47:36.444Z Endpoint request headers: {X-Amz-Date=redacted, x-amzn-apigateway-api-id=redacted, Accept=application/json, User-Agent=AmazonAPIGateway_hd6y3jlnii, Host=lambda.us-east-1.amazonaws.com, X-Amz-Content-Sha256=redacted, X-Amzn-Trace-Id=Root=1-redacted-redacted, x-amzn-lambda-integration-tag=redacted, Authorization=********************************************************************************************************************************************************************************************************************************************************************************************************************************************5de667, X-Amz-Source-Arn=arn:aws:execute-api:us-east-1:redacted/prod/POST/getCompanyData, X-Amz-Security-Token=redacted [TRUNCATED] 2024-08-26T13:47:36.444ZEndpoint request body after transformations: {"resource":"/getCompanyData","path":"/getCompanyData","httpMethod":"POST","headers":{"Accept":"/","accept-encoding":"gzip, deflate, br","Content-Type":"application/json",""User-Agent":"Thunder Client (https://www.thunderclient.com)","X-Forwarded-Port":"443","X-Forwarded-Proto":"https"},"multiValueHeaders":{"Accept":["/*"],"accept-encoding":["gzip, deflate, br"],"Content-Type":["application/json"],"Host":ser-Agent":["Thunder Client (https://www.thunderclient.com)"],"X-Amzn-Trace-Id""X-Forwarded-For":["70.163.208.241"],"X-Forwarded-Port":["443"],"X-Forwarded-Proto":["https"]},"queryStringParameters":null,"multiValueQueryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"resourceId":" [TRUNCATED] Sending request to https://lambda.us-east-1.amazonaws.com/-03-31/functions/arn:aws:lambda:us-east-:function:companyscraper/invocations 2024-08-26T13:47:39.444Z Execution failed due to a timeout error 2024-08-26T13:47:39.444Z Method completed with status: 504

Please let me know if you need more information! Thank you!