AWS Codebuild, run builds in parallel
0
Hi,
I am using this config in a buildspec, but builds are still run sequentially:
version: 0.2
phases:
pre_build:
commands:
- pwd
- ls -la
build:
commands:
- echo "hi world" && sleep 50
- echo "hey" && sleep 10
- echo "hey" && sleep 20
batch:
fast-fail: true
build-list:
- identifier: build1
- identifier: build2
- identifier: build3
What have I missed?
Thank you
Language
English
asked 2 years ago887 views
2 Answers
- Newest
- Most votes
- Most comments
1
OK, I found an answer. I need to use build matrix instead of build list. Then every echo will be running in parallel.
1
Hi,
Batch cannot be defined in the build section. You might see something like this in the build logs
In the section BUILD
The following keys cannot be identified:
batch
You build spec needs to be like below (some env variables added to verify the build stages)
version: 0.2
batch:
fast-fail: true
build-list:
- identifier: build1
env:
variables:
STAGE: build1
- identifier: build2
env:
variables:
STAGE: build2
- identifier: build3
env:
variables:
STAGE: build3
phases:
pre_build:
commands:
- pwd
- ls -la
build:
commands:
- echo "hi world" && sleep 50
- echo "hey" && sleep 10
- echo "$STAGE" && sleep 20
- echo "$AWS_DEFAULT_REGION"
Also you need to setup the batch configuration details for the build project and / or enable concurrency as well ( I could not verify the concurrency part). You would then have the details shown in the Batch history tab
--Syd
answered 2 years ago
Relevant content
- asked 5 years ago
- asked a year ago
AWS OFFICIALUpdated 2 years ago- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated 2 years ago
In this case I will have 3 batches, that will run the same commands: - echo "hi world" && sleep 50 - echo "hey" && sleep 10 - echo "$STAGE" && sleep 20 - echo "$AWS_DEFAULT_REGION"
But I want, that one batch run - echo "hey" && sleep 10 and another run - echo "$STAGE" && sleep 20 at the same time, is it possible?
About concurrency part, I need to contact AWS support to increase the concurrent build limit, as default is 1.
Also, batch can be defined in the build section. I just use my first script above and I don't choose any batch configuration detail. There are no errors, but commands are running in sequence.
i dont think batch is designed for that. If you have mutiple environments / environment variables for which you need to run identical commands you use batch.
It wont throw up any errors but within the build logs you would see the log snippet i mentioned in the previous comment. That's what i got in my logs. It's essentially ignoring the batch section.
"i dont think batch is designed for that. If you have mutiple environments / environment variables for which you need to run identical commands you use batch"
I have a few docker build commands in my buildspec. I have docker build -t 1 and docker build -t 2. My goal was to run them in parallel just to reduce the build time. Is that possible?