By using AWS re:Post, you agree to the Terms of Use
/Linux Provisioning/

Questions tagged with Linux Provisioning

Sort by most recent
  • 1
  • 90 / page

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Using cloud init to configure an Amazon Linux 2 on-premise VM

I'm developing Ansible playbooks to configure our software and service to install and run in EC2 instances running Amazon Linux 2. Our software works very well within AL2, and the Ansible playbooks are coming along. I want to test them very thoroughly, which means destroying and re-creating the VM over and over. I don't really want to keep doing that within AWS itself, because it's slow to keep transferring things over the network, and I would prefer not to be billed for it, and I have plenty of host machines available here in my office. I read the instructions for deploying the on-premise qcow2 images. They say to create a seed.iso, and then go through the VM manager wizard on the host machine, and define the seed.iso image as the CD image, and then start it. Honestly Amazon, you really didn't think this through. Our VM hostmachines don't have GUIs installed. And I want this to be a process I can automate, so I don't want to have to open up a GUI wizard and click around on things every time I do a test. I want a command line process I can run that gets my VM started and configures network and a SSH key on it so I can then hit it with Ansible. Why would I be automating everything except the one task of creating the VM itself, which will require clicking around in a GUI? That's what we need to avoid. I can see that the VM would try to do a cloud init process but there's no documentation anywhere about how that actually happens. The best I can find is that cloud init will try to access config files on a magic IP, but it's not clear what that IP is or what I need to do on my VM host to make that possible. Does anyone have ideas on this? It's really disappointing that Amazon expects people to use their on-premise images with a GUI tool. Someone didn't think that through. Thank you!
0
answers
0
votes
1
views
AWS-User-0086891
asked a month ago

Setting MKL_NUM_THREADS to be more than 16 for m5 instances

Hey, I have a 32-core EC2 linux m5 instance. My python installed via anaconda. I notice that my numpy cannot use more than 16 cores. Looks like my numpy uses libmkl_rt.so: ``` [2]: np.show_config() blas_mkl_info: libraries = ['mkl_rt', 'pthread'] library_dirs = ['/home/ec2-user/anaconda3/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/ec2-user/anaconda3/include'] blas_opt_info: libraries = ['mkl_rt', 'pthread'] library_dirs = ['/home/ec2-user/anaconda3/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/ec2-user/anaconda3/include'] lapack_mkl_info: libraries = ['mkl_rt', 'pthread'] library_dirs = ['/home/ec2-user/anaconda3/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/ec2-user/anaconda3/include'] lapack_opt_info: libraries = ['mkl_rt', 'pthread'] library_dirs = ['/home/ec2-user/anaconda3/lib'] define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)] include_dirs = ['/home/ec2-user/anaconda3/include'] ``` When I tried to set MKL_NUM_THREADS below 16, it works ``` (base) ec2-user@ip-172-31-18-3:~$ export MKL_NUM_THREADS=12 && python -c "import ctypes; mkl_rt = ctypes.CDLL('libmkl_rt.so'); print (mkl_rt.mkl_get_max_threads())" 12 ``` When I tried to set it to 24, it stops at 16 ``` (base) ec2-user@ip-172-31-18-3:~$ export MKL_NUM_THREADS=24 && python -c "import ctypes; mkl_rt = ctypes.CDLL('libmkl_rt.so'); print (mkl_rt.mkl_get_max_threads())" 16 ``` But I do have 32 cores ``` In [2]: os.cpu_count() Out[2]: 32 ``` Is there any other settings I need to check? Thanks, Bill
3
answers
0
votes
4
views
AWS-User-2549579
asked 3 months ago

Is there a way to identify an EBS Volume inside a Linux EC2 instance using its volume ID ?

We are working on a use case where we need to map the disk label within the instance to the corresponding Volume ID in EBS. While performing some validations on some AMIs, we found that there is a difference between the behavior for Windows and Linux We have observed that the requirement we need is present in case of Windows (AMI Used: Windows_Server-2016-English-Full-Containers-2022.01.19) The following query yields the required result. Here the serial number of the disk is mapping to the EBS volume id The device driver for this instance was the AWS PV Storage Host Adapter ``` PS C:\Users\Administrator> Get-WmiObject Win32_DiskDrive | select-object -property serialnumber,index serialnumber index ------------ ----- vol0b44250cf530aa7f3 0 vol0f38be626e3137975 1 vol0bdc570ca980fb5fb 2 ``` However in case of Linux instances (AMI Used: amzn2-ami-kernel-5.10-hvm-2.0.20220121.0-x86_64-gp2) we are seeing that the EBS volume ID is not present within the disk metadata. We checked the following points inside the Linux: 1. Directories within /dev/disk: For the above AMI, the disk serial number is not being exposed in the /dev/disk/by-id directory. In the /dev/disk/by-path directory, there are entries present in the following format xen-vbd-51712 -> ../../xvda . Is it possible to map the string xen-vbd-51712 to the EBS volume ? 2. udevadm info <disk_label>: This is yielding the following information attached below, however the volume id is not present in the below. ``` P: /devices/vbd-51712/block/xvda N: xvda S: disk/by-path/xen-vbd-51712 S: sda E: DEVLINKS=/dev/disk/by-path/xen-vbd-51712 /dev/sda E: DEVNAME=/dev/xvda E: DEVPATH=/devices/vbd-51712/block/xvda E: DEVTYPE=disk E: ID_PART_TABLE_TYPE=gpt E: ID_PART_TABLE_UUID=08cf25fb-6b18-47c3-b4cb-fea548b3a3a2 E: ID_PATH=xen-vbd-51712 E: ID_PATH_TAG=xen-vbd-51712 E: MAJOR=202 E: MINOR=0 E: SUBSYSTEM=block E: TAGS=:systemd: E: USEC_INITIALIZED=34430 ``` As per https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/device_naming.html, the device name which is provided when the EBS volume is attached to the instance is not guaranteed to be the same which is visible inside the instance. ``` "When you attach a volume to your instance, you include a device name for the volume. This device name is used by Amazon EC2. The block device driver for the instance assigns the actual volume name when mounting the volume, and the name assigned can be different from the name that Amazon EC2 uses" ``` Since our use case can involve frequent addition/removal of EBS volumes on an instance, we wanted to find a deterministic method to identify a volume inside a Linux instance. Could you please let us know that if there is a route by which we can relate the disk within EC2 instance with the corresponding EBS volume id ?
1
answers
0
votes
7
views
AWS-User-3460534
asked 3 months ago

FreeIPA server installation fails at pki-tomcatd start on alinux2

Hi there I come from China I'm trying to install FreeIPA on a fresh Amazon Linux 2 instance (Amazon Linux 2 AMI (HVM) - Kernel 4.14, SSD Volume Type) I have the exact same problem as in this link: https://forums.aws.amazon.com/thread.jspa?messageID=997191&tstart=0 Hope to get an answer,Thanks! ______________________________________________________________________ I'm trying to install FreeIPA on a fresh Amazon Linux 2 instance (ami-087c17d1fe0178315), reproducing steps that were successful a few months ago, but the installation fails during pki-tomcat setup. Details below, but a very similar bug is described here: https://bodhi.stg.fedoraproject.org/updates/FEDORA-2021-e55a8d7545 Has anyone found a workaround for this? Or do the RPMs in alinux2 need to be updated? Details: Configure Route53 for group-ipa.groupdev.local, ipa-ca.groupdev.local to resolve to instance's IP address. hostnamectl set-hostname group-ipa.groupdev.local yum update -y yum install freeipa-server ipa-server-install ... enter config info ... ... installation proceeds for quite a while, then ... Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes 1/30: configuring certificate server instance ipaserver.install.dogtaginstance: CRITICAL Failed to configure CA instance: Command '/usr/sbin/pkispawn -s CA -f /tmp/tmpLuET89' returned non-zero exit status 1 ipaserver.install.dogtaginstance: CRITICAL See the installation logs and the following files/directories for more information: ipaserver.install.dogtaginstance: CRITICAL /var/log/pki/pki-tomcat error RuntimeError: CA configuration failed. ipapython.admintool: ERROR CA configuration failed. ipapython.admintool: ERROR The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information pkispawn logs note a connecton failure after: 2021-10-04 16:36:50 pkispawn : INFO ....... executing 'systemctl daemon-reload' 2021-10-04 16:36:50 pkispawn : INFO ....... executing 'systemctl start pki-tomcatd@pki-tomcat.service' and the pki-tomcatd logs suggest an authentication problem between tomcat and the LDAP server: $ journalctl -u pki-tomcatd@pki-tomcat.service ... Oct 04 13:54:10 group-ipa.groupdev.local server: CMSEngine.initializePasswordStore() begins Oct 04 13:54:10 group-ipa.groupdev.local server: CMSEngine.initializePasswordStore(): tag=internaldb Oct 04 13:54:10 group-ipa.groupdev.local server: testLDAPConnection connecting to group-ipa.groupdev.local:389 Oct 04 13:54:11 group-ipa.groupdev.local server: CMSEngine.initializePasswordStore(): tag=replicationdb Oct 04 13:54:11 group-ipa.groupdev.local server: testLDAPConnection connecting to group-ipa.groupdev.local:389 Oct 04 13:54:11 group-ipa.groupdev.local server: testLDAPConnection: Invalid Password Oct 04 13:54:11 group-ipa.groupdev.local server: testLDAPConnection connecting to group-ipa.groupdev.local:389 Oct 04 13:54:11 group-ipa.groupdev.local server: testLDAPConnection: Invalid Password Oct 04 13:54:11 group-ipa.groupdev.local server: testLDAPConnection connecting to group-ipa.groupdev.local:389 Oct 04 13:54:11 group-ipa.groupdev.local server: testLDAPConnection: Invalid Password Oct 04 13:54:11 group-ipa.groupdev.local server: CMSEngine: init(): password test execution failed: 2 Oct 04 13:54:11 group-ipa.groupdev.local server: Password test execution failed. Is the database up? Oct 04 13:54:11 group-ipa.groupdev.local server: Password test execution failed. Is the database up? Oct 04 13:54:11 group-ipa.groupdev.local server: at com.netscape.cmscore.apps.CMSEngine.initializePasswordStore(CMSEngine.java:467) Oct 04 13:54:11 group-ipa.groupdev.local server: at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:535) Oct 04 13:54:11 group-ipa.groupdev.local server: at com.netscape.certsrv.apps.CMS.init(CMS.java:191) Oct 04 13:54:11 group-ipa.groupdev.local server: at com.netscape.certsrv.apps.CMS.start(CMS.java:1458) Oct 04 13:54:11 group-ipa.groupdev.local server: at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:117) /var/log/dirsrv/slapd-GROUPDEV-LOCAL/access shows the successful bind for cn=Directory Manager, then three "Entry does not exist" results for "cn=Replication Manager mas ter Agreement1-...", which appear to be interpreted as "Invalid Password" on tomcat's side.
1
answers
0
votes
0
views
AWS-User-4856351
asked 3 months ago

Initializing 2 NICs Automatically in Ubuntu AMI (Cloud-init issue)

I'm trying to standup a Ubuntu 20.04 based AMI with 2 NICs connected to 2 different subnets and have those NICs configured automatically. I've read the guidance about creating file in /etc/netplan to define the NIC interface but have been reading through cloud-init python scripts, since EC2 is using it to init the NICs, to figure out why this isn't done automatically and I think I found the root cause ... it seems cloud-init starts out properly configuring both NICs but then deletes the config for the primary NIC meaning that in order to SSH I would need to go over the secondary NIC. I managed to get cloud-init to initialize both NICs correctly by editing the following file and commenting out the following lines of code ``` /usr/lib/python3/dist-packages/cloudinit/net/__init__.py ``` ``` def _bringup_device(self): .... #self.cleanup_cmds.append( # ['ip', '-family', 'inet', 'link', 'set', 'dev', self.interface, # 'down']) #self.cleanup_cmds.append( # ['ip', '-family', 'inet', 'addr', 'del', cidr, 'dev', # self.interface]) def _bringup_router(self): ... #self.cleanup_cmds.insert( # 0, # ['ip', '-4', 'route', 'del', self.router, 'dev', self.interface, # 'src', self.ip]) .... #self.cleanup_cmds.insert( # 0, ['ip', '-4', 'route', 'del', 'default', 'dev', self.interface]) ``` To confirm that the solution works I delete all the cloud-init collateral and reboot to force it to restart. I also created a new custom AMI from the EC2 instance after deleting the files and confirmed that a new EC2 comes up with both NICs properly configured too ``` rm -rf /var/lib/cloud/* ``` Having to modify + maintain a custom copy of cloud-init is probably not a sustainable long term solution ... would this be a cloud-init issue or an EC2 cloud-init config issue? What's the best course of action to have this as an available long term solution? I think having cloud-init handle the NIC initialization vs. having to hand craft a file in /etc/netplan/ is probably a more sustainable long term solution. Thanks
0
answers
0
votes
5
views
alfayez
asked 3 months ago

DLAMI does not have CUDA/NVIDIA (and cannot access cuda from pytorch)

I running on :Deep Learning AMI (Ubuntu 18.04) Version 56.0 - ami-083abc80c473f5d88, but I have tried several similar DLAMI. I am unable to access CUDA from pytorch to train my models. See here: ``` $ apt list --installed | grep -i "nvidia" ``` ``` WARNING: apt does not have a stable CLI interface. Use with caution in scripts. libnvidia-compute-460-server/bionic-updates,bionic-security,now 460.106.00-0ubuntu0.18.04.2 amd64 [installed,automatic] libnvidia-container-tools/bionic,now 1.7.0-1 amd64 [installed,automatic] libnvidia-container1/bionic,now 1.7.0-1 amd64 [installed,automatic] nvidia-container-toolkit/bionic,now 1.7.0-1 amd64 [installed] nvidia-cuda-dev/bionic,now 9.1.85-3ubuntu1 amd64 [installed,automatic] nvidia-cuda-doc/bionic,now 9.1.85-3ubuntu1 all [installed,automatic] nvidia-cuda-gdb/bionic,now 9.1.85-3ubuntu1 amd64 [installed,automatic] nvidia-cuda-toolkit/bionic,now 9.1.85-3ubuntu1 amd64 [installed] nvidia-docker2/bionic,now 2.8.0-1 all [installed] nvidia-fabricmanager-450/now 450.142.00-1 amd64 [installed,upgradable to: 450.156.00-0ubuntu0.18.04.1] nvidia-opencl-dev/bionic,now 9.1.85-3ubuntu1 amd64 [installed,automatic] nvidia-profiler/bionic,now 9.1.85-3ubuntu1 amd64 [installed,automatic] nvidia-visual-profiler/bionic,now 9.1.85-3ubuntu1 amd64 [installed,automatic] ``` And it shows I have Nvidia. However, when I run python: ``` ~$ bpython bpython version 0.22.1 on top of Python 3.8.12 /home/ubuntu/anaconda3/envs/pytorch_p38/bin/python3.8 >>> import torch.nn as nn >>> import torch >>> torch.cuda.is_available() False ``` Even after I re-install nvidia ``` sudo apt install nvidia-driver-455 ``` I get this: ``` (pytorch_p38) ubuntu@ip-172-31-95-17:~$ nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Mon_Oct_12_20:09:46_PDT_2020 Cuda compilation tools, release 11.1, V11.1.105 Build cuda_11.1.TC455_06.29190527_0 (pytorch_p38) ubuntu@ip-172-31-95-17:~$ bpython bpython version 0.22.1 on top of Python 3.8.12 /home/ubuntu/anaconda3/envs/pytorch_p38/bin/python3.8 >>> import torch >>> torch.cuda.is_available() False ``` Does anyone know how to get pytorch to be able to access cuda? Any help is greatly appreciated
0
answers
0
votes
3
views
AWS-User-9348549
asked 4 months ago
1
answers
0
votes
4
views
oliveirafilipe
asked 4 months ago

[EC2 FPGA] XDMA transfers fail on f1.16xlarge

Hello, I've been using EC2, FPGAs for about a 15 months now. I've always been using f1.2xlarge instances, with Ubuntu OS installed, and it worked as expected. Now, due to the amount of CPU intensive work I need to do, I've decided to try using a more robust f1.16xlarge. However, I ran into problems here. I've done all the steps: Loaded the AGFI, checked via `lspci` is it available, and then tried some simple XDMA read/write tests, just to make sure the connection is still there. Sadly, I get no communication with the PCIe FPGA board. Below is the dmesg output, that reports that the "magic" error in the descriptor happened. Again, I'm using the same driver, same AGFI, and the same Python wrappers around C invocation of kernel. ``` [ 1683.589120] xdma:engine_service_final_transfer: engine 0-H2C0-MM, status error 0x80010. [ 1683.589123] xdma:engine_status_dump: SG engine 0-H2C0-MM status: 0x00080010: MAGIC_STOPPED,DESC_ERR:UNSUPP_REQ [ 1683.589126] 0-H2C0-MM, s 0x80010, aborted xfer 0x00000000e19d64e9, cmpl 0/1 [ 1683.589136] xdma:xdma_xfer_submit: xfer 0x00000000e19d64e9,1024, failed, ep 0x0. ``` EDIT: I've figured out the problem, after looking more closely at the `dmesg` output, I figured out that the AGFI was loaded on a different FPGA slot. I've loaded the AGFI as I always do: `sudo fpga-load-local-image -S 0 -I $MY_AGFI_ID -H`. I don't see how it could end up on Slot 8? When I try to adjust my test and run it on Slot #8, all works as expected! To be honest `dmesg` shows this pretty straightforward: ``` [ 183.021774] xdma:remove_one: pdev 0x00000000999de0ae, xdev 0x0000000076ff6236, 0x00000000ca7f10f1. [ 183.021777] xdma:xpdev_free: xpdev 0x0000000076ff6236, destroy_interfaces, xdev 0x00000000ca7f10f1. [ 183.024133] xdma:xpdev_free: xpdev 0x0000000076ff6236, xdev 0x00000000ca7f10f1 xdma_device_close. [ 186.066065] pci 0000:00:0f.0: [1d0f:f000] type 00 class 0x058000 [ 186.066817] pci 0000:00:0f.0: reg 0x10: [mem 0x86000000-0x87ffffff] [ 186.067206] pci 0000:00:0f.0: reg 0x14: [mem 0x85200000-0x853fffff] [ 186.067855] pci 0000:00:0f.0: reg 0x18: [mem 0x5e000410000-0x5e00041ffff 64bit pref] [ 186.068493] pci 0000:00:0f.0: reg 0x20: [mem 0x5c000000000-0x5dfffffffff 64bit pref] [ 186.083784] pci 0000:00:0f.0: BAR 4: assigned [mem 0x5c000000000-0x5dfffffffff 64bit pref] [ 186.084214] pci 0000:00:0f.0: BAR 0: assigned [mem 0x86000000-0x87ffffff] [ 186.084317] pci 0000:00:0f.0: BAR 1: assigned [mem 0x85200000-0x853fffff] [ 186.084421] pci 0000:00:0f.0: BAR 2: assigned [mem 0x5e000410000-0x5e00041ffff 64bit pref] [ 186.084996] xdma:xdma_device_open: xdma device 0000:00:0f.0, 0x000000006c4610d7. [ 186.086074] xdma:map_single_bar: BAR0 at 0x86000000 mapped at 0x00000000f2cc5fa3, length=33554432(/33554432) [ 186.086088] xdma:map_single_bar: BAR1 at 0x85200000 mapped at 0x00000000caa22a31, length=2097152(/2097152) [ 186.086106] xdma:map_single_bar: BAR2 at 0x5e000410000 mapped at 0x000000003049f8eb, length=65536(/65536) [ 186.086109] xdma:map_bars: config bar 2, pos 2. [ 186.086110] xdma:map_single_bar: Limit BAR 4 mapping from 137438953472 to 2147483647 bytes [ 186.086115] xdma:map_single_bar: BAR4 at 0x5c000000000 mapped at 0x00000000bfd17001, length=2147483647(/137438953472) [ 186.086116] xdma:identify_bars: 4 BARs: config 2, user 0, bypass 4. [ 186.095983] xdma:pci_keep_intx_enabled: 0000:00:0f.0: clear INTX_DISABLE, 0x406 -> 0x6. [ 186.096158] xdma:irq_msix_channel_setup: engine 8-H2C0-MM, irq#572. [ 186.096193] xdma:irq_msix_channel_setup: engine 8-H2C1-MM, irq#573. [ 186.096225] xdma:irq_msix_channel_setup: engine 8-H2C2-MM, irq#574. [ 186.096270] xdma:irq_msix_channel_setup: engine 8-H2C3-MM, irq#575. [ 186.096301] xdma:irq_msix_channel_setup: engine 8-C2H0-MM, irq#576. [ 186.096334] xdma:irq_msix_channel_setup: engine 8-C2H1-MM, irq#577. [ 186.096366] xdma:irq_msix_channel_setup: engine 8-C2H2-MM, irq#578. [ 186.096397] xdma:irq_msix_channel_setup: engine 8-C2H3-MM, irq#579. [ 186.096431] xdma:irq_msix_user_setup: 8-USR-0, IRQ#580 with 0x000000000932b671 [ 186.096463] xdma:irq_msix_user_setup: 8-USR-1, IRQ#581 with 0x000000005edcc121 [ 186.096511] xdma:irq_msix_user_setup: 8-USR-2, IRQ#582 with 0x00000000249674d9 [ 186.096560] xdma:irq_msix_user_setup: 8-USR-3, IRQ#583 with 0x00000000d26d07c5 [ 186.096594] xdma:irq_msix_user_setup: 8-USR-4, IRQ#584 with 0x00000000c940ac79 [ 186.096627] xdma:irq_msix_user_setup: 8-USR-5, IRQ#585 with 0x000000001fccab2f [ 186.096666] xdma:irq_msix_user_setup: 8-USR-6, IRQ#586 with 0x0000000009c457eb [ 186.096699] xdma:irq_msix_user_setup: 8-USR-7, IRQ#587 with 0x000000002bedefd1 [ 186.096732] xdma:irq_msix_user_setup: 8-USR-8, IRQ#588 with 0x000000004ca712de [ 186.096765] xdma:irq_msix_user_setup: 8-USR-9, IRQ#589 with 0x00000000e191ad7b [ 186.096799] xdma:irq_msix_user_setup: 8-USR-10, IRQ#590 with 0x00000000026a9f8b [ 186.096833] xdma:irq_msix_user_setup: 8-USR-11, IRQ#591 with 0x00000000a7138ee8 [ 186.096868] xdma:irq_msix_user_setup: 8-USR-12, IRQ#592 with 0x00000000b0c4b138 [ 186.096902] xdma:irq_msix_user_setup: 8-USR-13, IRQ#593 with 0x000000007f7aa664 [ 186.096934] xdma:irq_msix_user_setup: 8-USR-14, IRQ#594 with 0x0000000070f6c0f6 [ 186.096970] xdma:irq_msix_user_setup: 8-USR-15, IRQ#595 with 0x000000009aed6be9 [ 186.096978] xdma:probe_one: 0000:00:0f.0 xdma8, pdev 0x000000006c4610d7, xdev 0x0000000044888d47, 0x0000000058d876e9, usr 16, ch 4,4. ``` Is this a bug of some kind? How to protect myself? Thank you in advance.
1
answers
0
votes
2
views
jelicicm
asked 4 months ago

SSM Agent update failure

Hi all. We're just starting with SSM and hoping to use this quite extensively moving forward. I have a small handful of devices online using it but I just noticed that one of my devices has gone offline. Luckily it is not in the field yet so I was able to access it locally. What appears to have happened was SSM Agent was attempting to update but failed and never came back. I'm not so concerned that it didn't update and more concerned that it didn't start again. Going through the ssm agent logs I came across this line ``` "standardError": "E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\nWARNING: Could not install the python3-apt, this may cause the patching operation to fail.\nfailed to run commands: exit status 1 ``` The SSM update log itself ends like this ``` 2022-01-20 21:47:09 DEBUG UpdateInstanceInformation Response{ } 2022-01-20 21:47:09 INFO initiating cleanup of other versions in amazon-ssm-agent and amazon-ssm-agent-updater folder 2022-01-20 21:47:09 INFO removing artifacts in the folder: /var/lib/amazon/ssm/update/amazon-ssm-agent 2022-01-20 21:47:09 INFO removed files and folders: 3.1.821.0 2022-01-20 21:47:09 INFO removing artifacts in the folder: /var/lib/amazon/ssm/update/amazon-ssm-agent-updater 2022-01-20 21:47:09 INFO removed files and folders: 3.1.715.0 2022-01-20 21:47:09 INFO initiating cleanup of files in update download folder 2022-01-20 21:47:09 INFO Successfully downloaded manifest Successfully downloaded updater version 3.1.821.0 Updating amazon-ssm-agent from 3.1.715.0 to 3.1.821.0 Successfully downloaded https://s3.us-east-2.amazonaws.com/amazon-ssm-us-east-2/amazon-ssm-agent/3.1.715.0/amazon-ssm-agent-ubuntu-amd64.tar.gz Successfully downloaded https://s3.us-east-2.amazonaws.com/amazon-ssm-us-east-2/amazon-ssm-agent/3.1.821.0/amazon-ssm-agent-ubuntu-amd64.tar.gz Initiating amazon-ssm-agent update to 3.1.821.0 failed to install amazon-ssm-agent 3.1.821.0, ErrorMessage=The execution of command returned Exit Status: 125 exit status 125 Initiating rollback amazon-ssm-agent to 3.1.715.0 failed to uninstall amazon-ssm-agent 3.1.821.0, ErrorMessage=The execution of command returned Exit Status: 2 exit status 2 Failed to update amazon-ssm-agent to 3.1.821.0 ``` Then the error log gives me this ``` 2022-01-20 16:29:54 ERROR [Submit @ processor.go.140] [ssm-agent-worker] [MessagingDeliveryService] [Association] [associationId=5752f0d0-1f57-492e-83f7-740484b81d73] Document Submission failed: Job with id 5752f0d0-1f57-492e-83f7-740484b81d73 already exists 2022-01-20 21:47:02 ERROR [AppendError @ context.go.129] failed to install amazon-ssm-agent 3.1.821.0, ErrorMessage=The execution of command returned Exit Status: 125 exit status 125 ``` Could anyone help me out here? I cannot have these fail like this when they go out into the wild. The OS is Ubuntu Server 18 on this box, 20 on others we have.
0
answers
0
votes
5
views
AWS-User-7038179
asked 4 months ago

Launched EC2 instance UNREACHABLE for Ubuntu 20.04 AMI with python 3.9 upgrade

I am using **EC2 Ubuntu 20.04 VM**. Due to **[CVE-2021-3177][1]**, Python needs to be upgraded to the latest version of Python3.9 which would be 3.9.5 currently. I did that using the `apt install` option as per the steps mentioned below: sudo apt update sudo apt upgrade -y sudo apt install python3.9 The above ensures that Python3.9.5 is now available. But now python3.8 & python3.9 is available. So next we will use the update-alternatives command to make python3.9 as the default version. sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1 sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 2 Now that alternatives are defined, we will switch to Option 2 as the default option i.e. Python3.9 sudo update-alternatives --config python3 Once done, the following command would point to the latest version. sudo python3 -V However, if you use the `sudo apt update` command, you will see an error stating that Traceback (most recent call last): File "/usr/lib/cnf-update-db", line 8, in <module> from CommandNotFound.db.creator import DbCreator File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 11, in <module> import apt_pkg ModuleNotFoundError: No module named 'apt_pkg' Reading package lists... Done E: Problem executing scripts APT::Update::Post-Invoke-Success 'if /usr/bin/test -w /var/lib/command-not-found/ -a -e /usr/lib/cnf-update-db; then /usr/lib/cnf-update-db > /dev/null; fi' E: Sub-process returned an error code To fix this we will have to add a link using the following command cd /usr/lib/python3/dist-packages/ sudo ln -s apt-pkg.cpython-{38m,39m}-x86_64-linux-gnu.so Next, I tried used the following commands apt purge python3-apt apt install python3-apt sudo apt install python3.9-distutils python3.9-dev Once done following command will now not result in any errors sudo apt update This means that the issue is fixed. **I can use this machine & use it after reboot too. ** **But for some reason, If I create an AMI and launch an instance then that instance is unreachable.** Appreciate your help. [1]: https://nvd.nist.gov/vuln/detail/CVE-2021-3177
1
answers
0
votes
2
views
awswiki
asked 4 months ago

Multiple EBS volumes to gain perfomance

Hi, I want to run "HCL Domino Server 12" on an EC2 instance, Domino is a server specialized on collaboration applications and includes a mail server, we can see it also as web servers that includes a Non-sql database behind as the engine for the email and the apps . During server setup, I can specify different paths for transactional logging, view indexes, mail&database applications, etc. I was thinking on create different filesystems for each route, and assign a different EBS volumen to each path/filesystem, but I do have several concerns / question on it: - EBS baseline: I am aware that t3 family EC2 instances have a baseline for CPU on 30%.... What about baselines for EBS ? Does t3 also have a baseline for EBS and credits towards the use of them ? I did not see clearly that info. - EBS and IOPs / Troughtput: I guess that if I have 3 EBS disk, and each of them have a performance base-level of 3000 IOPS and 125 MiB/s throughput. Does that means that using 3, I will have 9000 IOPS in total and 375 Mibs ? I am not sure if I have a previous bottleneck on the EC2 (ie, EC2 having a maximum on total for all the disk of 300 Mibs, so even I have multiple EBS volumes the maximum troughput is the one obtained by the EC2 machine) - Root Volumes: When you create and EC2 machine on a t3.large instance, how is by default created the Root Volume ? Is using an EBS gp2 or an EBS gp3 volume ? - NVMe SSD volumes: I saw EC2 images (ie, m5ad.large) that insted using "normal" EBS SSD volumes on the rrot, they provide you directly with 1x75 NVMe SSD volumes, and higher. I am confused there, since when I mounted additional SSD volumes on my linux systems, they always appeared also as a "NVMe" device. Are not normal gp2 / gp3 volumes NVMe based ? Can someone explained the difference and the value of this 1x75 VNMe SSD volumes offered by the m5ad.large image ?
1
answers
0
votes
12
views
AndresG
asked 4 months ago

Can and Can't install php7.4 on Amazon Linux 2 with user-data

I'm trying to install wordpress(latest version) with php7.4 on Amazon linux 2 by using following user-data. ``` #!/bin/bash yum update -y amazon-linux-extras enable php7.4 yum clean metadata yum install -y httpd mysql php php-{pear,cgi,common,curl,mbstring,gd,mysqlnd,gettext,bcmath,json,xml,fpm,intl,zip,imap,devel} wget http://wordpress.org/latest.tar.gz -P /tmp/ tar zxvf /tmp/latest.tar.gz -C /tmp cp -r /tmp/wordpress/* /var/www/html/ chown apache:apache -R /var/www/html systemctl enable httpd.service systemctl start httpd.service ``` However, I got issue about php7.4 installation. Some EC2 instances can't enable amzn2extra-php7.4 repo, then use amzn2-core. php 5.4 will be installed. ``` ================================================================================ Dec 24 17:42:04 ip-10-0-0-43 cloud-init: Package Arch Version Repository Size Dec 24 17:42:04 ip-10-0-0-43 cloud-init: ================================================================================ Dec 24 17:42:04 ip-10-0-0-43 cloud-init: Installing: ~~ Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php x86_64 5.4.16-46.amzn2.0.2 amzn2-core 1.4 M Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-bcmath x86_64 5.4.16-46.amzn2.0.2 amzn2-core 58 k Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-cli x86_64 5.4.16-46.amzn2.0.2 amzn2-core 2.8 M Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-common x86_64 5.4.16-46.amzn2.0.2 amzn2-core 563 k Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-devel x86_64 5.4.16-46.amzn2.0.2 amzn2-core 602 k Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-fpm x86_64 5.4.16-46.amzn2.0.2 amzn2-core 1.4 M Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-gd x86_64 5.4.16-46.amzn2.0.2 amzn2-core 127 k Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-intl x86_64 5.4.16-46.amzn2.0.2 amzn2-core 98 k Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-mbstring x86_64 5.4.16-46.amzn2.0.2 amzn2-core 505 k Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-mysqlnd x86_64 5.4.16-46.amzn2.0.2 amzn2-core 172 k Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-pear noarch 1:1.10.12-9.amzn2 amzn2-core 359 k Dec 24 17:42:04 ip-10-0-0-43 cloud-init: php-xml x86_64 5.4.16-46.amzn2.0.2 amzn2-core 125 k ~~ ~~~ [root@ip-10-0-0-43 ~]# php -v PHP 5.4.16 (cli) (built: Oct 31 2019 18:34:05) Copyright (c) 1997-2013 The PHP Group Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies ``` Some EC2 instances can enable amzn2extra-php7.4 repo, php 7.4 will be installed. ``` Dec 24 17:42:07 ip-10-0-0-147 cloud-init: ================================================================================ Dec 24 17:42:07 ip-10-0-0-147 cloud-init: Package Arch Version Repository Size Dec 24 17:42:07 ip-10-0-0-147 cloud-init: ================================================================================ Dec 24 17:42:07 ip-10-0-0-147 cloud-init: Installing: ~ Dec 24 17:42:07 ip-10-0-0-147 cloud-init: php x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 3.2 M Dec 24 17:42:07 ip-10-0-0-147 cloud-init: php-bcmath x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 68 k Dec 24 17:42:07 ip-10-0-0-147 cloud-init: php-cli x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 4.9 M Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-common x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 1.1 M Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-devel x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 1.3 M Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-fpm x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 1.7 M Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-gd x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 182 k Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-intl x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 220 k Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-json x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 71 k Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-mbstring x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 514 k Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-mysqlnd x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 241 k Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-pear noarch 1:1.10.12-9.amzn2 amzn2-core 359 k Dec 24 17:42:08 ip-10-0-0-147 cloud-init: php-xml x86_64 7.4.21-1.amzn2 amzn2extra-php7.4 199 k ~~ [root@ip-10-0-0-147 ~]# php -v PHP 7.4.21 (cli) (built: Jul 7 2021 17:35:08) ( NTS ) Copyright (c) The PHP Group Zend Engine v3.4.0, Copyright (c) Zend Technologies ``` I launched 10 EC2 instances with the "same" configuration as test. 7x EC2 instances: php5.4 was installed 3x EC2 instances: php7.4 was installed I'm using default Amazon linux AMI (Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type ) ami-0218d08a1f9dac831 ( region: ap-northeast-1) I have no idea why this happens..
1
answers
0
votes
8
views
AWS-User-6711472
asked 5 months ago

Root User for Linux

I am not very fluent in using Linux at the command line. With that in mind, I set up Plesk on my Lightsail instance, and it is working great. What I am trying to do is migrate some websites from a colleague's AWS Lightsail instance which is also running Plesk. I have used this migration tool before, and it works very well once you get the servers authenticated and a connection established. Unfortunately, to do this, I have the option of providing the root user and password for the source server, or I can use SSH keys as credentials. We tried a few different combinations of credentials he uses to sign in to his AWS account, and to sign in to his Plesk dashboard, but nothing will work. I do not have enough familiarity with SSH keys to make a very educated guess on how to proceed, so I am hoping to simply figure out how to get or change the password for his root user. I do not think this is the same as the "root" user credentials I use to sign in to my AWS console. Why does this seem so complicated? When I have had a VPS with other providers in the past, I could usually get a root user password and this migration tool just works. I know this will require him (my colleague) to connect to his server, which I do not have credentials for at this point. He is willing to give me control for the period required to complete the migration, then he will change passwords again. What I am hoping for is a simple method for changing the root user password, and whether this would cause any issues by changing it, as long as he has it. Would this cause any issues between the VPS and the AWS console as far as administration goes? Any help or suggestions would be greatly appreciated. I have been working on this for over a week with no real progress. I am running out of time.
1
answers
0
votes
12
views
AWS-User-1242566
asked 5 months ago
  • 1
  • 90 / page