Directory Error when running SageMaker backup-ebs lifecycle for Amazon Linux 2 transition

Question

I'm following [this guide](https://aws.amazon.com/blogs/machine-learning/migrate-your-work-to-amazon-sagemaker-notebook-instance-with-amazon-linux-2/) for transitioning to Amazon Linux 2 provided by AWS

I've set up the two needed lifecycle configurations and created a new S3 Bucket to store the backup. I've also ensured the IAM roles have the required S3 permissions and updated the notebook with the ebs-backup-bucket tag per the instructions.

When I run the notebook with the new configuration I get the following error: 
"Notebook Instance Lifecycle Config [LIFECYCLE ARN] for Notebook Instance [NOTEBOOK ARN] took longer than 5 minutes. Please check your CloudWatch logs for more details if your Notebook Instance has Internet access.

Looking at the logs I get the error: 
`/bin/bash: /tmp/OnStart_2022-11-09-01-51ontlqcqt: /bin/bash^M: bad interpreter: No such file or directory`

Any thoughts on how to resolve this issue? The code for the backup lifecycle configuration can be found [here](https://github.com/aws-samples/amazon-sagemaker-notebook-instance-lifecycle-config-samples/blob/master/scripts/migrate-ebs-data-backup/on-start.sh)

Accepted Answer

The extra `^M` symbol (i.e. Ctrl-M) stopped the whole scrip from being interpreted properly.

This issue is normally seen in scripts prepared in MSDOS/Windows based system but used in Linux system due to difference of line endings.

In Unix based OS, lines end with `
` but MSDOS/Win based system ends with `
`

In Linux based system, you could show your prepared scripts by running
```
cat -e some-script.sh 
``` 
The results would be something similar to
```
#!/bin/bash^M$
... ...^M$
```
`$` is normal Unix end-of-line symbol. Windows uses an extra one `^M` and this symbol is not recognized by Unix system. That's why, in SageMaker Notebook Lifecycle Configuration, which is running Linux, your script was interpreted as `/bin/bash^M`

To mitigate the issue, please convert the scripts to Unix based ending and update life cycle configuration. To achieve this, you could use `Notepad++` in Windows. You can go to the `Edit` menu, select the `EOL Conversion` submenu, and from the options that come up select `UNIX/OSX Format`. The next time you save the file, its line endings will, all going well, be saved with UNIX-style line endings.

Alternatively, you could put the script in a Linux environment, e.g. EC2 instance with Amazon Linux 2 and install `dos2unix` via `sudo yum install dos2unix`. After installation, you could convert your files via
```
dos2unix -n file.sh  output.sh 
```
After the conversion, please update LCC with the new scripts. You could verify that `^M` has been removed via 
```
cat -e your_script.sh
```
The output will print all special characters directly without hiding.

Directory Error when running SageMaker backup-ebs lifecycle for Amazon Linux 2 transition

Relevant content