My AMI not working as expected

Question

I have an EC2 running and it's a web server with nginx + FastAPI.

I want to form auto-scaling with this EC2.

So I made AMI from this EC2 & Launch Template.

But it's status failed at 1/2 and keep restarting.

so I tried to see system logs from EC2 generated by Auto-scaling

There were certain errors in init-local

The System log is below

```
[    7.581985] cloud-init[341]: Cloud-init v. 23.4.4-0ubuntu0~22.04.1 running 'init-local' at Thu, 18 Apr 2024 21:18:05 +0000. Up 7.56 seconds.
[    7.957874] cloud-init[341]: 2024-04-18 21:18:06,101 - util.py[WARNING]: failed stage init-local
[    7.984474] cloud-init[341]: failed run of stage init-local
[    7.985372] cloud-init[341]: ------------------------------------------------------------
[    7.986838] cloud-init[341]: Traceback (most recent call last):
[    7.987794] cloud-init[341]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 781, in status_wrapper
[    7.989185] cloud-init[341]:     ret = functor(name, args)
[    7.989952] cloud-init[341]:           ^^^^^^^^^^^^^^^^^^^
[    7.990695] cloud-init[341]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 442, in main_init
[    7.991989] cloud-init[341]:     init.apply_network_config(bring_up=bring_up_interfaces)
[    7.993040] cloud-init[341]:   File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 1048, in apply_network_config
[    7.994448] cloud-init[341]:     return self.distro.apply_network_config(
[    7.995338] cloud-init[341]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[    7.996267] cloud-init[341]:   File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 413, in apply_network_config
[    7.997749] cloud-init[341]:     self._write_network_state(network_state, renderer)
[    7.998726] cloud-init[341]:   File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 124, in _write_network_state
[    8.000191] cloud-init[341]:     return super()._write_network_state(*args, **kwargs)
[    8.001222] cloud-init[341]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[    8.002298] cloud-init[341]:   File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 293, in _write_network_state
[    8.003795] cloud-init[341]:     renderer.render_network_state(network_state)
[    8.004798] cloud-init[341]:   File "/usr/lib/python3/dist-packages/cloudinit/net/netplan.py", line 309, in render_network_state
[    8.006257] cloud-init[341]:     self._netplan_generate(run=self._postcmds, same_content=same_content)
[    8.007431] cloud-init[341]:   File "/usr/lib/python3/dist-packages/cloudinit/net/netplan.py", line 322, in _netplan_generate
[    8.008849] cloud-init[341]:     subp.subp(self.NETPLAN_GENERATE, capture=True)
[    8.009838] cloud-init[341]:   File "/usr/lib/python3/dist-packages/cloudinit/subp.py", line 322, in subp
[    8.011066] cloud-init[341]:     raise ProcessExecutionError(
[    8.011853] cloud-init[341]: cloudinit.subp.ProcessExecutionError: Unexpected error while running command.
[    8.013099] cloud-init[341]: Command: ['netplan', 'generate']
[    8.013941] cloud-init[341]: Exit code: 1
[    8.014523] cloud-init[341]: Reason: -
[    8.015121] cloud-init[341]: Stdout:
[    8.015668] cloud-init[341]: Stderr: Traceback (most recent call last):
[    8.016536] cloud-init[341]:           File "/usr/sbin/netplan", line 20, in 
[    8.017554] cloud-init[341]:             from netplan import Netplan
[    8.018412] cloud-init[341]:           File "/usr/share/netplan/netplan/__init__.py", line 18, in 
[    8.019644] cloud-init[341]:             from netplan.cli.core import Netplan
[    8.020616] cloud-init[341]:           File "/usr/share/netplan/netplan/cli/core.py", line 24, in 
[    8.021893] cloud-init[341]:             import netplan.cli.utils as utils
[    8.022819] cloud-init[341]:           File "/usr/share/netplan/netplan/cli/utils.py", line 23, in 
[    8.024093] cloud-init[341]:             import netifaces
[    8.024866] cloud-init[341]:         ModuleNotFoundError: No module named 'netifaces'
[    8.025922] cloud-init[341]: ------------------------------------------------------------
[[0;1;31mFAILED[0m] Failed to start [0;1;39mInitial cloud-init job (pre-networking)[0m.
See 'systemctl status cloud-init-local.service' for details.
```

any helps would be appreciated!!!!

Accepted Answer

```
[ 7.581985] cloud-init[341]: Cloud-init v. 23.4.4-0ubuntu0~22.04.1 running 'init-local' at Thu, 18 Apr 2024 21:18:05 +0000. Up 7.56 seconds.
```
So it's the User Data script where its failing. Could you post the user Data Script here?

This looks like the important bit:
```
[ 8.024093] cloud-init[341]: import netifaces 
[ 8.024866] cloud-init[341]: ModuleNotFoundError: No module named 'netifaces'
```
The User Data script runs as root, and not as whatever user you usually use to install the python modules & run the script usually. And that the current directory, and the $HOME directory, will be the `root` user's home directory, and not the homedir of the `ubuntu` or `ec2-user` (or whatever) user.

Also, on the face of it this problem looks similar https://stackoverflow.com/questions/65248471/modulenotfounderror-how-do-i-install-python3-packages-so-my-my-user-data-scrip (although your error is for `netifaces` and the one in the stackoverflow post is for `boto3`).

My AMI not working as expected

Relevant content