By using AWS re:Post, you agree to the Terms of Use
/DevOps/

DevOps

AWS provides a set of flexible services designed to enable companies to more rapidly and reliably build and deliver products using AWS and DevOps practices. These services simplify provisioning and managing infrastructure, deploying application code, automating software release processes, and monitoring your application and infrastructure performance.

Recent questions

see all
1/18

Is it possible to use a non-default bridge network when running CodeBuild locally?

When I run codebuild locally it creates the default docker network and volumes as in the below output. Is there a way to use different bridge network and volumes instead of these default ones? I tried modifying the docker command in [codebuild_build.sh](https://github.com/aws/aws-codebuild-docker-images/blob/master/local_builds/codebuild_build.sh) to add a network (below build command output shows `--network mylocaltestingnetwork`) but that didn't help as such. The reason I am trying to use a different network and volumes is because I am using [localstack](https://localstack.cloud/) along with it, and the bridge network and volumes need to be accessible from the codebuild local container. If I configure my localstack to use the `agent-resources_default` network created by local codebuild then codebuild is able to access localstack. But, I would like to keep the dependency external to both codebuild and localstack by using a separate bridge network. ``` $ ./codebuild_build.sh -i public.ecr.aws/codebuild/amazonlinux2-x86_64-standard:3.0 -a codebuild-output/ -b buildspec-local.yml -c -p localstack Build Command: docker run -it -v /var/run/docker.sock:/var/run/docker.sock -e "IMAGE_NAME=public.ecr.aws/codebuild/amazonlinux2-x86_64-standard:3.0" -e "ARTIFACTS=<repo path>/codebuild-output/" --network mylocaltestingnetwork -e "SOURCE=<repo path>" -e "BUILDSPEC=<repo path>/buildspec-local.yml" -e "AWS_CONFIGURATION=<homedir>/.aws" -e "AWS_PROFILE=localstack" -e "INITIATOR=<user>" public.ecr.aws/codebuild/local-builds:latest Removing network agent-resources_default Removing volume agent-resources_source_volume Removing volume agent-resources_user_volume Creating network "agent-resources_default" with the default driver Creating volume "agent-resources_source_volume" with local driver Creating volume "agent-resources_user_volume" with local driver Creating agent-resources_agent_1 ... done Creating agent-resources_build_1 ... done Attaching to agent-resources_agent_1, agent-resources_build_1 agent_1 | [Container] 2022/01/14 15:57:15 Waiting for agent ping agent_1 | [Container] 2022/01/14 15:57:17 Waiting for DOWNLOAD_SOURCE agent_1 | [Container] 2022/01/14 15:57:23 Phase is DOWNLOAD_SOURCE ```
0
answers
0
votes
1
views
bbideep
asked 2 days ago

Using AWS Lambda with RDS Proxy: Socket Ended Error

**Some background:** I have a Node 12 Lambda running an Apollo GraphQL server. It connects to a MySQL RDS database via RDS Proxy using IAM authentication. **Problem:** Since switching from direct DB connection to RDS Proxy via IAM auth, we get intermittent socket ended errors, seen below: ``` Error: This socket has been ended by the other party at TLSSocket.writeAfterFIN [as write] (net.js:456:14) at PoolConnection.write (/var/task/node_modules/mysql2/lib/connection.js:363:20) at PoolConnection.writePacket (/var/task/node_modules/mysql2/lib/connection.js:294:12) at Query.start (/var/task/node_modules/mysql2/lib/commands/query.js:60:16) at Query.execute (/var/task/node_modules/mysql2/lib/commands/command.js:45:22) at PoolConnection.handlePacket (/var/task/node_modules/mysql2/lib/connection.js:456:32) at PoolConnection.addCommand (/var/task/node_modules/mysql2/lib/connection.js:478:12) at PoolConnection.query (/var/task/node_modules/mysql2/lib/connection.js:546:17) at MysqlQueryRunner.<anonymous> (/var/task/node_modules/typeorm/driver/mysql/MysqlQueryRunner.js:184:56) at step (/var/task/node_modules/typeorm/node_modules/tslib/tslib.js:143:27) { code: 'EPIPE', fatal: true } ``` I know that this is far from enough info to debug, but in lieu of throwing up all possible config surrounding these services, I'll wait for more pointed questions. Happy to post settings, connection info and any other config as needed to debug. Thanks in advance for any help!
2
answers
0
votes
6
views
fdizdar
asked 9 days ago

Docker push doesn't work even docker login succeeded during AWS CodePipeline Build stage

Hello, I'm preparing CI/CD using AWS CodePipeline. Unfortunatelly I have an error during build stage. Below there is content of my buildspec.yml file, where: AWS_DEFAULT_REGION = eu-central-1 CONTAINER_NAME=cicd-1-app REPOSITORY_URI = <ACCOUNT_ID>.dkr.ecr.eu-central-1.amazonaws.com/cicd-1-app ``` version: 0.2 phases: install: runtime-versions: java: corretto11 pre_build: commands: - echo Logging in to Amazon ECR... - aws --version - TAG="$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | head -c 8)" - IMAGE_URI=${REPOSITORY_URI}:${TAG} build: commands: - echo Build started on `date` - echo $IMAGE_URI - mvn clean package -Ddockerfile.skip - docker build --tag $IMAGE_URI . post_build: commands: - printenv - echo Build completed on `date` - echo $(docker images) - echo Pushing docker image - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin <ACCOUNT_ID>.dkr.ecr.eu-central-1.amazonaws.com - docker push $IMAGE_URI - echo push completed - printf '[{"name":"%s","imageUri":"%s"}]' $CONTAINER_NAME $IMAGE_URI > imagedefinitions.json artifacts: files: - imagedefinitions.json ``` I got error: ``` [Container] 2022/01/06 19:57:36 Running command aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin <ACCOUNT_ID>.dkr.ecr.eu-central-1.amazonaws.com WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [Container] 2022/01/06 19:57:37 Running command docker push $IMAGE_URI The push refers to repository [<ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/cicd-1-app] 37256fb2fd27: Preparing fe6c1ddaab26: Preparing d4dfab969171: Preparing no basic auth credentials [Container] 2022/01/06 19:57:37 Command did not exit successfully docker push $IMAGE_URI exit status 1 [Container] 2022/01/06 19:57:37 Phase complete: POST_BUILD State: FAILED [Container] 2022/01/06 19:57:37 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker push $IMAGE_URI. Reason: exit status 1 ``` Even docker logged in successfully there is "no basic auth credentials" error. Do you know what could be a problem? Best regards.
2
answers
0
votes
8
views
KM
asked 10 days ago

LocalDynamoDb intermittently generates HTTP 500 errors

For a big project we use DynamoLocalDb in our unit tests. Most of the times, these tests pass. The code also works as expected on our production environments, where we use the "real" dynamodb that's part of the VPC. However, sometimes the unit tests fail. Particularly when calling `putItem()` we sometimes get the following exception: ```txt The request processing has failed because of an unknown error, exception or failure. (Service: DynamoDb, Status Code: 500, Request ID: db23be5e-ae96-417b-b268-5a1433c8c125, Extended Request ID: null) software.amazon.awssdk.services.dynamodb.model.DynamoDbException: The request processing has failed because of an unknown error, exception or failure. (Service: DynamoDb, Status Code: 500, Request ID: db23be5e-ae96-417b-b268-5a1433c8c125, Extended Request ID: null) at software.amazon.awssdk.services.dynamodb.model.DynamoDbException$BuilderImpl.build(DynamoDbException.java:95) at software.amazon.awssdk.services.dynamodb.model.DynamoDbException$BuilderImpl.build(DynamoDbException.java:55) at software.amazon.awssdk.protocols.json.internal.unmarshall.AwsJsonProtocolErrorUnmarshaller.unmarshall(AwsJsonProtocolErrorUnmarshaller.java:89) at software.amazon.awssdk.protocols.json.internal.unmarshall.AwsJsonProtocolErrorUnmarshaller.handle(AwsJsonProtocolErrorUnmarshaller.java:63) at software.amazon.awssdk.protocols.json.internal.unmarshall.AwsJsonProtocolErrorUnmarshaller.handle(AwsJsonProtocolErrorUnmarshaller.java:42) at software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.lambda$handle$0(MetricCollectingHttpResponseHandler.java:52) at software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:64) at software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.handle(MetricCollectingHttpResponseHandler.java:52) at software.amazon.awssdk.core.internal.http.async.AsyncResponseHandler.lambda$prepare$0(AsyncResponseHandler.java:89) at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) at software.amazon.awssdk.core.internal.http.async.AsyncResponseHandler$BaosSubscriber.onComplete(AsyncResponseHandler.java:132) at java.base/java.util.Optional.ifPresent(Optional.java:183) at software.amazon.awssdk.http.crt.internal.AwsCrtResponseBodyPublisher.completeSubscriptionExactlyOnce(AwsCrtResponseBodyPublisher.java:216) at software.amazon.awssdk.http.crt.internal.AwsCrtResponseBodyPublisher.publishToSubscribers(AwsCrtResponseBodyPublisher.java:281) at software.amazon.awssdk.http.crt.internal.AwsCrtAsyncHttpStreamAdapter.onResponseComplete(AwsCrtAsyncHttpStreamAdapter.java:114) at software.amazon.awssdk.crt.http.HttpStreamResponseHandlerNativeAdapter.onResponseComplete(HttpStreamResponseHandlerNativeAdapter.java:33) ``` Our software is written in Kotlin, version 1.5.31 and the project is build with maven. The DynamoDbLocal version we use is 1.16.0. We use amazon sdk 2.16.67. Our DynamoLocalDb is spun up inside our unit tests as follows: ```kotlin val url: String by lazy { System.setProperty("sqlite4java.library.path", "target/dynamo-native-libs") System.setProperty("aws.accessKeyId", "test-access-key") System.setProperty("aws.secretAccessKey", "test-secret-key") val port = randomFreePort() logger.info { "Creating local in-memory Dynamo server on port $port" } val instance = ServerRunner.createServerFromCommandLineArgs(arrayOf("-inMemory", "-port", port.toString())) try { instance.safeStart() } catch (e: Exception) { instance.stop() fail("Could not start Local Dynamo Server on port $port.", e) } Runtime.getRuntime().addShutdownHook(object : Thread() { override fun run() { logger.debug("Stopping Local Dynamo Server on port $port") instance.stop() } }) "http://localhost:$port" } ``` Our client is created with: ```kotlin val client: DynamoDbAsyncClientWrapper by lazy { DynamoDbAsyncClientWrapper( DynamoDbAsyncClient.builder() .region(Region.EU_WEST_1) .credentialsProvider(DefaultCredentialsProvider.builder().build()) .endpointOverride(URI.create(url)) .httpClientBuilder(AwsCrtAsyncHttpClient.builder()) .build() ) } ``` The code for our Kotlin Dynamo Wrapper DSL is open sourced an available here: https://github.com/ximedes/kotlin-dynamodb-wrapper The information in the stacktrace thrown by DynamoLocalDb is uninformative, and the asynchronous nature of the code also does not give a good hint as to where this error originated. We have tried several changes to our code, but we are running out of options. We are looking for a possible cause of this intermittent problem, or a way to reliably reproduce it.
1
answers
0
votes
6
views
Human-7858696
asked 11 days ago

jsii.errors.JSIIError: Cannot read properties of undefined (reading 'bindToGraph')

HI All This is my first implementation of StateMachineFragment. Goal: Attempting to create a class for re-usable lambda state. This class can take a parameter and pass this as payload to Lambda and the lambda will execute the right query based on the payload. Below is my POC code to 'classs-ify' the lambda and the call to statemachine. ``` from aws_cdk import ( Duration, Stack, # aws_sqs as sqs, aws_stepfunctions as _stepfunctions, aws_stepfunctions as sfn, aws_stepfunctions_tasks as _stepfunctions_tasks, aws_lambda as _lambda, ) from constructs import Construct class SubMachine(_stepfunctions.StateMachineFragment): def __init__(self, parent, id, *, jobTypeParam): super().__init__(parent, id) existingFunc = _lambda.Function.from_function_arn(self, "ExistingLambdaFunc", function_arn="arn:aws:lambda:us-east-1:958$#$#$#$:function:dummyFunction") lambda_invoked = _stepfunctions_tasks.LambdaInvoke(self, "someID", lambda_function=existingFunc) wait_10_seconds = _stepfunctions.Wait(self, "Wait for 10 seconds", time=_stepfunctions.WaitTime.duration(Duration.seconds(10)) ) self._start_state = wait_10_seconds self._end_states = [lambda_invoked.end_states] def start_state(self): return self._start_state def end_states(self): return self._end_states class StepfunctionsClasStack(Stack): def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: super().__init__(scope, construct_id, **kwargs) test_lambda_1 = SubMachine(self, "SubMachine1", jobTypeParam="one") state_machine = _stepfunctions.StateMachine(self, "TestStateMachine", definition=test_lambda_1, # role=marketo_role ) ``` When I try and deploy this code, I get the following error: ``` jsii.errors.JSIIError: Cannot read properties of undefined (reading 'bindToGraph') ``` I am not sure where I am going wrong. Thoughts? Thanks
1
answers
0
votes
7
views
tkansara
asked 15 days ago

Start_state and End_states for step functions

Hi All All I am trying to do is to create a reusable ambda component where I can pass parameters to the class so that the Lambda can do different things, based on input param. I am using CDK in python to deploy the stack. I would like to create a parallel stepfunction, where I can pass the same lambda using different param/payload so they can be branches. I am running the following code: ``` from aws_cdk import ( # Duration, Stack, # aws_sqs as sqs, aws_stepfunctions as _stepfunctions, aws_stepfunctions_tasks as _stepfunctions_tasks, aws_lambda as _lambda, ) from constructs import Construct class LambdaJob(_stepfunctions.StateMachineFragment): def __init__(self, parent, id, *, jobTypeParam): super().__init__(parent, id) existingFunc = _lambda.Function.from_function_arn(self, "ExistingLambdaFunc", function_arn="arn:aws:lambda:us-east-1:95842$$$$$:function:dummyFunction") lambda_invoked = _stepfunctions_tasks.LambdaInvoke(self, "someID", lambda_function=existingFunc) wait_10_seconds = _stepfunctions.Wait(self, "Wait for 10 seconds", time=_stepfunctions.WaitTime.duration(Duration.seconds(10)) ) self.start_state = wait_10_seconds class StepfunctionsClasStack(Stack): def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: super().__init__(scope, construct_id, **kwargs) test_lambda_1 = LambdaJob(self, "Quick") #state_machine = _stepfunctions.StateMachine(self, "TestStateMachine", # definition=LambdaJob(self, "Quick"), # # role=marketo_role # ) ``` However i keep getting the following error: ``` TypeError: Can't instantiate abstract class LambdaJob with abstract methods end_states, start_state ``` Any thoughts on what I am doing wrong ? Thanks
1
answers
0
votes
7
views
tkansara
asked 15 days ago

Popular users

see all
1/18