By using AWS re:Post, you agree to the Terms of Use
/Microservices/

Microservices

Microservices are an architectural and organizational approach to software development where software is composed of small independent services that communicate over well-defined APIs. Microservices architectures make applications easier to scale and faster to develop, enabling innovation and accelerating time-to-market for new features.

Recent questions

see all
1/18

aws-sdk V3 timeout in lambda

Hello, I'm using NodeJS 14.x lambda to control an ecs service. As I do not need the ecs task to run permanently, I created a service inside the cluster so I can play around the desired count to start or stop it at will. I also created two lambdas, one for querying the current desired count and the current Public IP, another one for updating said desired count (to 0 or 1 should I want to start or stop it) I have packed aws-sdk v3 on a lambda layer to not have to package it on each lambda. Seems to work fine as I was getting runtime error > "Runtime.ImportModuleError: Error: Cannot find module '@aws-sdk/client-ecs'" But I do not anymore. The code is also working fine from my workstation as I'm able to execute it locally and I get the desired result (query to ecs api works fine) But All I get when testing from lambdas are Timeouts... It usually execute in less than 3 secondes on my local workstation but even with a lambda timeout set up at 3 minutes, this is what I get ``` START RequestId: XXXX-XX-XXXX Version: $LATEST 2022-01-11T23:57:59.528Z XXXX-XX-XXXX INFO before ecs client send END RequestId: XXXX-XX-XXXX REPORT RequestId: XXXX-XX-XXXX Duration: 195100.70 ms Billed Duration: 195000 ms Memory Size: 128 MB Max Memory Used: 126 MB Init Duration: 1051.68 ms 2022-01-12T00:01:14.533Z XXXX-XX-XXXX Task timed out after 195.10 seconds ``` The message `before ecs client send` is a console.log I made just before the ecs.send request for debug purposes I think I've set up the policy correctly, as well as the Lambda VPC with the default outbound rule to allow all protocol on all port to 0.0.0.0/0 so I I have no idea on where to look now. I have not found any way to debug aws-sdk V3 calls like you would do on V2 by adding a logger to the config. Maybe it could help understanding the issue....
1
answers
0
votes
5
views
Tomazed
asked 5 days ago

Why is HTTPD failing to start? Why is TLS failing to start? Missing certificate key is not missing!

For context, I followed this tutorial to configure SSL/TLS on an EC2 instance: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-amazon-linux-2.html Everything was working fine, I've installed a web application (Drupal 9) from composer-based repo, maintained my code, fine. I updated some packages with yum, update php, etc. I attempt to start Apache: ``` [ec2-user@ip-172-31-32-159 ~]$ sudo systemctl restart httpd Job for httpd.service failed. See "systemctl status httpd.service" and "journalctl -xe" for details. ``` I check `journalctl -xe` The important part appears to be: ``` -- Unit httpd-init.service has begun starting up. Jan 10 00:10:41 ip-172-31-32-159.us-east-2.compute.internal httpd-ssl-gencerts[9368]: Missing certificate key! Jan 10 00:10:41 ip-172-31-32-159.us-east-2.compute.internal systemd[1]: httpd-init.service: main process exited, code=exited, status=1/FAILURE Jan 10 00:10:41 ip-172-31-32-159.us-east-2.compute.internal systemd[1]: Failed to start One-time temporary TLS key generation for httpd.service. -- Subject: Unit httpd-init.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit httpd-init.service has failed. -- -- The result is failed. ``` Here is something interesting. I check `vim /etc/httpd/conf.d/ssl.conf` At line 100 is `SSLCertificateFile /etc/pki/tls/certs/localhost.crt` Okay, very good. The interesting thing is if I rename the file `sudo mv /etc/pki/tls/certs/localhost.crt /etc/pki/tls/certs/localhost.crt.bak`, and then try to start httpd `sudo systemctl start httpd`, returned is `Job for httpd.service failed because the control process exited with error code.` Checking `journalctl -xe` again, we recieve a different error: ``` -- Unit httpd.service has begun starting up. Jan 10 00:42:56 ip-172-31-32-159.us-east-2.compute.internal httpd[9841]: AH00526: Syntax error on line 100 of /etc/httpd/conf.d/ssl.conf: Jan 10 00:42:56 ip-172-31-32-159.us-east-2.compute.internal httpd[9841]: SSLCertificateFile: file '/etc/pki/tls/certs/localhost.crt' does not exist or Jan 10 00:42:56 ip-172-31-32-159.us-east-2.compute.internal systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE Jan 10 00:42:56 ip-172-31-32-159.us-east-2.compute.internal systemd[1]: Failed to start The Apache HTTP Server. ``` Renaming localhost.crt to localhost.crt.bak changes the error, breaks the link, and SSLCertificateFile appropriately does not exist. Changing localhost.crt.bak to localhost.crt restores the SSLCertificateFile link, and changes the error back to claiming there is a missing certificate key, when we can see it there: ``` Jan 10 00:47:07 ip-172-31-32-159.us-east-2.compute.internal httpd-ssl-gencerts[9884]: Missing certificate key! ``` What is going on here?
0
answers
0
votes
3
views
AWS-User-3495166
asked 7 days ago

CDK with typescript - error on cloud9

Hello Everyone, I tried https://github.com/fortejas/example-serverless-python-api on a cloud9 environment but I got the following error Commands that I used to setup: ``` mkdir sample-api cd sample-api/ cdk init app --language typescript . cd ~ git clone https://github.com/kasukur/example-serverless-python-api.git ls -lrt example-serverless-python-api/ cp -rf example-serverless-python-api/lambda-api/ ~/environment/sample-api/. cd ~/environment/sample-api/ Delete node_modules folder Delete package-lock.json npm i @aws-cdk/aws-lambda-python-alpha --force -g 
ec2-user:~/environment/sample-api $ cdk deploy ``` the error is ``` ec2-user:~/environment/sample-api $ cdk synth npm WARN exec The following package was not found and will be installed: ts-node /home/ec2-user/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:750 return new TSError(diagnosticText, diagnosticCodes); ^ TSError: ⨯ Unable to compile TypeScript: bin/sample-api.ts:4:10 - error TS2305: Module '"../lib/sample-api-stack"' has no exported member 'SampleApiStack'. 4 import { SampleApiStack } from '../lib/sample-api-stack'; ~~~~~~~~~~~~~~ at createTSError (/home/ec2-user/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:750:12) at reportTSError (/home/ec2-user/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:754:19) at getOutput (/home/ec2-user/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:941:36) at Object.compile (/home/ec2-user/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:1243:30) at Module.m._compile (/home/ec2-user/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:1370:30) at Module._extensions..js (node:internal/modules/cjs/loader:1153:10) at Object.require.extensions.<computed> [as .ts] (/home/ec2-user/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:1374:12) at Module.load (node:internal/modules/cjs/loader:981:32) at Function.Module._load (node:internal/modules/cjs/loader:822:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) { diagnosticText: `\x1B[96mbin/sample-api.ts\x1B[0m:\x1B[93m4\x1B[0m:\x1B[93m10\x1B[0m - \x1B[91merror\x1B[0m\x1B[90m TS2305: \x1B[0mModule '"../lib/sample-api-stack"' has no exported member 'SampleApiStack'.\n` + '\n' + "\x1B[7m4\x1B[0m import { SampleApiStack } from '../lib/sample-api-stack';\n" + '\x1B[7m \x1B[0m \x1B[91m ~~~~~~~~~~~~~~\x1B[0m\n', diagnosticCodes: [ 2305 ] } Subprocess exited with error 1 ``` Could someone please help with this Thank you
1
answers
0
votes
2
views
Sri
asked 9 days ago

jsii.errors.JSIIError: Cannot read properties of undefined (reading 'bindToGraph')

HI All This is my first implementation of StateMachineFragment. Goal: Attempting to create a class for re-usable lambda state. This class can take a parameter and pass this as payload to Lambda and the lambda will execute the right query based on the payload. Below is my POC code to 'classs-ify' the lambda and the call to statemachine. ``` from aws_cdk import ( Duration, Stack, # aws_sqs as sqs, aws_stepfunctions as _stepfunctions, aws_stepfunctions as sfn, aws_stepfunctions_tasks as _stepfunctions_tasks, aws_lambda as _lambda, ) from constructs import Construct class SubMachine(_stepfunctions.StateMachineFragment): def __init__(self, parent, id, *, jobTypeParam): super().__init__(parent, id) existingFunc = _lambda.Function.from_function_arn(self, "ExistingLambdaFunc", function_arn="arn:aws:lambda:us-east-1:958$#$#$#$:function:dummyFunction") lambda_invoked = _stepfunctions_tasks.LambdaInvoke(self, "someID", lambda_function=existingFunc) wait_10_seconds = _stepfunctions.Wait(self, "Wait for 10 seconds", time=_stepfunctions.WaitTime.duration(Duration.seconds(10)) ) self._start_state = wait_10_seconds self._end_states = [lambda_invoked.end_states] def start_state(self): return self._start_state def end_states(self): return self._end_states class StepfunctionsClasStack(Stack): def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: super().__init__(scope, construct_id, **kwargs) test_lambda_1 = SubMachine(self, "SubMachine1", jobTypeParam="one") state_machine = _stepfunctions.StateMachine(self, "TestStateMachine", definition=test_lambda_1, # role=marketo_role ) ``` When I try and deploy this code, I get the following error: ``` jsii.errors.JSIIError: Cannot read properties of undefined (reading 'bindToGraph') ``` I am not sure where I am going wrong. Thoughts? Thanks
1
answers
0
votes
7
views
tkansara
asked 15 days ago

DyanamoDB connection issue - aws-sdk for NodeJs

While connecting to DynamoDB via aws-sdk for NodeJs, i'm seeing below issue: > InvalidSignatureException: Credential should be scoped to a valid region, not 'eu-east-2'. at Request.extractError (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/protocol/json.js:52:27) at Request.callListeners (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/sequential_executor.js:106:20) at Request.emit (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/sequential_executor.js:78:10) at Request.emit (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/request.js:686:14) at Request.transition (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/request.js:22:10) at AcceptorStateMachine.runTo (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/state_machine.js:14:12) at /Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/state_machine.js:26:10 at Request.<anonymous> (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/request.js:38:9) at Request.<anonymous> (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/request.js:688:12) at Request.callListeners (/Users/saravanan_vij/Documents/Personal/GitHub/Backtester/node_modules/aws-sdk/lib/sequential_executor.js:116:18) { code: 'InvalidSignatureException', time: 2021-12-30T17:22:35.364Z, requestId: '3CT4LSCT0395369U3SVVNMV4B3VV4KQNSO5AEMVJF66Q9ASUAAJG', statusCode: 400, retryable: false, retryDelay: 24.638717702395528 I cannot believe that I'm not able to resolve this issue still after spending several hours on it and its weird that nobody else has faced or logged it. Or maybe I'm doing something wrong. Here's the config code: > AWS.config.update({ region: DynamoDBConfig.region, endpoint: DynamoDBConfig.endpoint, accessKeyId: DynamoDBConfig.accessKeyId,//process.env.AWS_ACCESS_KEY_ID, secretAccessKey: DynamoDBConfig.secretAccessKey//process.env.AWS_SECRET_ACCESS_KEY }); > const dynamodb = new AWS.DynamoDB({region: DynamoDBConfig.region}); Note: The same code is working fine on the localhost DynamoDB.
4
answers
1
votes
8
views
Saru
asked 17 days ago

EKS Network Load Balancer Service

Hello, I have an EKS cluster (terraform code see below) and follow the guide to set up the Load Balancer Controller (https://docs.aws.amazon.com/eks/latest/userguide/aws-load-balancer-controller.html). But when I deploy the service (terraform code see below) and want to expose it via "LoadBalancer" it keeps in a pending state and no external adr. is available. The Load Balancer controller gives the following error: Log Error from eksckubectl logs pod/aws-load-balancer-controller-5b57cdc6cc-dtjbg -n kube-system {"level":"error","ts":1640857282.2362676,"logger":"controller-runtime.manager.controller.service","msg":"Reconciler error","name":"terraform-example","namespace":"default","error":"AccessDenied: User: arn:aws:sts::009661972061:assumed-role/my-cluster2021123008214425030000000b/i-0a40de3c4e8541004 is not authorized to perform: elasticloadbalancing:CreateTargetGroup on resource: arn:aws:elasticloadbalancing:eu-central-1:009661972061:targetgroup/k8s-default-terrafor-630f67813d/* because no identity-based policy allows the elasticloadbalancing:CreateTargetGroup action\n\tstatus code: 403, request id: 2491099a-a6fd-4e6f-bab8-3c758eda0d0b"} If I add the AWSLoadBalancerControllerIAMPolicy to the my-cluster2021123008214425030000000b role manually it works. But as far as I read the documentation the AWSLoadBalancerControllerIAMPolicy is for the controller in the kube-system namespace and not the worker nodes. Is there anything missing from the documentation? Or what is the intended way of solving this? best regards rene Terraform EKS: ``` terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 3.27" } } required_version = ">= 0.14.9" } provider "aws" { profile = "default" region = "eu-central-1" } data "aws_eks_cluster" "eks" { name = module.eks.cluster_id } data "aws_eks_cluster_auth" "eks" { name = module.eks.cluster_id } provider "kubernetes" { host = data.aws_eks_cluster.eks.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.eks.certificate_authority[0].data) token = data.aws_eks_cluster_auth.eks.token } module "eks" { source = "terraform-aws-modules/eks/aws" cluster_version = "1.21" cluster_name = "my-cluster" vpc_id = "vpc-xx" subnets = ["subnet-xx", "subnet-xx", "subnet-xx"] worker_groups = [ { instance_type = "t3.medium" asg_max_size = 5 role_arn = "arn:aws:iam::xxx:role/worker-node-example" } ] } ``` Terraform service: ``` terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 3.27" } kubernetes = { source = "hashicorp/kubernetes" version = ">= 2.0.1" } } required_version = ">= 0.14.9" } provider "kubernetes" { host = "xxx" cluster_ca_certificate = base64decode("xxx") exec { api_version = "client.authentication.k8s.io/v1alpha1" command = "aws" args = [ "eks", "get-token", "--cluster-name", "my-cluster" ] } } provider "aws" { profile = "default" region = "eu-central-1" } resource "aws_sqs_queue" "gdpr_queue" { name = "terraform-example-queue.fifo" fifo_queue = true content_based_deduplication = true sqs_managed_sse_enabled = true } resource "aws_sqs_queue" "private_data_queue" { name = "terraform-example-queue.fifo" fifo_queue = true content_based_deduplication = true sqs_managed_sse_enabled = true } resource "aws_db_instance" "database" { allocated_storage = 10 engine = "postgres" engine_version = "13.3" instance_class = "db.t3.micro" name = "mydb" username = "foo" password = "foobarbaz" skip_final_snapshot = true vpc_security_group_ids = [aws_security_group.basic_security_group.id] } resource "aws_security_group" "basic_security_group" { name = "allow rds connection" description = "Allow rds traffic" vpc_id = "vpc-xxx" ingress { description = "postgres" from_port = 5432 to_port = 5432 protocol = "all" cidr_blocks = ["0.0.0.0/0"] ipv6_cidr_blocks = ["::/0"] } } resource "kubernetes_service" "gdpr-hub-service" { metadata { name = "terraform-example" annotations = { "service.beta.kubernetes.io/aws-load-balancer-type" = "external" "service.beta.kubernetes.io/aws-load-balancer-nlb-target-type" = "ip" "service.beta.kubernetes.io/aws-load-balancer-scheme" : "internet-facing" } } spec { selector = { App = kubernetes_deployment.gdpr-hub-service-deployment.spec.0.template.0.metadata.0.labels.App } session_affinity = "ClientIP" port { port = 80 target_port = 8080 } type = "LoadBalancer" } } resource "kubernetes_deployment" "gdpr-hub-service-deployment" { depends_on = [ aws_db_instance.database, aws_sqs_queue.gdpr_queue, aws_sqs_queue.private_data_queue ] metadata { name = "gdpr-hub-service" labels = { App = "gdpr-hub-service" } } spec { replicas = 2 selector { match_labels = { App = "gdpr-hub-service" } } template { metadata { labels = { App = "gdpr-hub-service" } } spec { container { image = "xxxx" name = "gdpr-hub-service" port { container_port = 8080 } resources { limits = { cpu = "2" memory = "1024Mi" } requests = { cpu = "250m" memory = "50Mi" } } } } } } } ```
2
answers
0
votes
9
views
renes
asked 17 days ago

How to upload video files using rest API after receiving an "upload URL"

I'm working with ShotGrid (an AutoDesk service) who make it possible to upload media to their S3 buckets The basic idea: Developer sends a request to ShotGrid for an AWS S3 "upload URL" [ShotGrid's upload documentation](https://developer.shotgridsoftware.com/rest-api/?shell#requesting-an-upload-url) explains how to make the request for the "upload URL", and it seems to work just, but then there's no documentation explaining how to actually execute the upload after receiving it. So far I'm getting errors, the most promising of which shows "SignatureDoesNotMatch / The request signature we calculated does not match the signature you provided. Check your key and signing method." More detail below... I've tried the following: Request for 'upload URL' is ``` curl -X GET https//myshow.shotgrid.autodesk.com/api/v1/entity/Version/{VersionId}/_upload?\filename={FileName} \ -H 'Authorization: Bearer {BearerToken} \ -H 'Accept: application/json' ``` Result is ``` { "UrlRequest": { "data": {"timestamp": "[timestsamp]", "upload_type": "Attachment", "upload_id": null, "storage_service": "s3", "original_filename": "[FileName]", "multipart_upload": false }, "links": { "upload": "https://[s3domain].amazonaws.com/[longstring1]/[longstring2]/[FileName] ?X-Amz-Algorithm=[Alg] &X-Amz-Credential=[Creds] &X-Amz-Date=[Date] &X-Amz-Expires=900 &X-Amz-SignedHeaders=host &X-Amz-Security-Token=[Token] &X-Amz-Signature=[Signature]", "complete_upload": "/api/v1/entity/versions/{VersionId}/_upload" } } ``` Then the upload request... ``` curl -X PUT -H 'x-amz-signature=[Signature-See-Above]' -d '@/Volumes/Path/To/Upload/Media' 'https://[uploadUrlFromAbove]' ``` And get the following error... ``` <Error> <Code>SignatureDoesNotMatch</Code> <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message> </Error> ```
3
answers
0
votes
8
views
Trln
asked 19 days ago

Popular users

see all
1/18