Amazon Linux 2023 - issue with installing packages with cloud-init

Question

I am attempting to install packages via cloud-init in an Amazon Linux 2023 instance.  Sometimes it works, sometimes it fails with:
"RPM: error: can't create transaction lock on /var/lib/rpm/.rpm.lock (Resource temporarily unavailable)"
This issue occurs when attempting pkg installs in both a cloudconfig and a script.
The most annoying part is that it fails most of the time but not always.  And it doesn't always fail in the same place.  Sometimes it get through cloudconfig pkgs and fails in the pkgs being installed by the script, sometimes it doesn't get that far.

Austin M · Accepted Answer

Based on my testing with the amazon linux 2023 image and SSM enabled, I see the following processes utilizing yum during the cloud-init phase of the EC2 initialization

From ~17 to ~39 seconds
`rpm -U --force ./amazon-cloudwatch-agent.rpm`

From ~70 to ~86 seconds
`rpm -U amazon-ssm-agent.rpm`

I tested this using the following in my user-data file:
```bash
timeout=0
while [ -f /var/lib/rpm/.rpm.lock ];
do
  echo "$timeout seconds"
  ls -a /var/lib/rpm/
  file /var/lib/rpm/.rpm.lock
  ps -ef|grep rpm
  # When the timeout is equal to zero, show an error and leave the loop.
  if [ "$timeout" == 120 ]; then
    break
  fi
  sleep 1
  ((timeout++))
done
```

Instead of performing an install after a sleep or when the lock is released, you will have better luck if you just wrap your yum installs in retry logic. Here is an example:

```bash
max_attempts=5
attempt_num=1
success=false
while [ $success = false ] && [ $attempt_num -le $max_attempts ]; do
  echo "Trying yum install"
  yum update -y
  yum install java-1.8.0 java-17-amazon-corretto-devel.x86_64 wget telnet -y
  # Check the exit code of the command
  if [ $? -eq 0 ]; then
    echo "Yum install succeeded"
    success=true
  else
    echo "Attempt $attempt_num failed. Sleeping for 3 seconds and trying again..."
    sleep 3
    ((attempt_num++))
  fi
done
```

AWS-User-6683498 · Answer

Hello,

Failing to get a transaction lock is due to something else is accessing the resource and doesn’t want anything else to access to make changes. You can try adding a sleep.

dxc-a3s-sbx · Answer

This is still happening one year later. The latest Amazon Linux 2023 AMI as of June 26th 2024 has a dangling /var/lib/rpm/.rpm.lock file dated June 20th, probably coming from the AMI creation. Strangely enough, it does not interfere with installing rpm packages, by itself:
```
Cloud-init v. 22.2.2 running 'modules:config' at Wed, 26 Jun 2024 13:46:22 +0000. Up 11.15 seconds.
Cloud-init v. 22.2.2 running 'modules:final' at Wed, 26 Jun 2024 13:46:23 +0000. Up 12.47 seconds.
total 15940
drwxr-xr-x.  2 root root       91 Jun 20 23:10 .
drwxr-xr-x. 26 root root    16384 Jun 20 23:10 ..
-rw-r--r--.  1 root root        0 Jun 20 23:10 .rpm.lock
-rw-r--r--.  1 root root 16273408 Jun 20 23:10 rpmdb.sqlite
-rw-r--r--.  1 root root    32768 Jun 20 23:11 rpmdb.sqlite-shm
-rw-r--r--.  1 root root        0 Jun 20 23:10 rpmdb.sqlite-wal
Amazon Linux 2023 repository                     33 MB/s |  25 MB     00:00
Amazon Linux 2023 Kernel Livepatch repository    32 kB/s |  10 kB     00:00
mysql84-community-release-el9-1.noarch.rpm       38 kB/s |  13 kB     00:00
Dependencies resolved.
================================================================================
 Package                        Arch        Version     Repository         Size
================================================================================
Installing:
 mysql84-community-release      noarch      el9-1       @commandline       13 k

Transaction Summary
================================================================================
Install  1 Package

Total size: 13 k
Installed size: 14 k
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                        1/1
  Installing       : mysql84-community-release-el9-1.noarch                 1/1
  Verifying        : mysql84-community-release-el9-1.noarch                 1/1

Installed:
  mysql84-community-release-el9-1.noarch

Complete!
```

But with a longer user data that basically does the same after some checks, it always fails with the  error referenced here.
The solution above succeds on the third try.

Jose Guay · Answer

Hello Himmel.

Can you confirm you are running the rpm install with elevated permissions? i.e. sudo [command] or sudo su?

Thank you.

Amazon Linux 2023 - issue with installing packages with cloud-init

Add your answer

Relevant content