Directory Error when running SageMaker backup-ebs lifecycle for Amazon Linux 2 transition

The extra ^M symbol (i.e. Ctrl-M) stopped the whole scrip from being interpreted properly.

This issue is normally seen in scripts prepared in MSDOS/Windows based system but used in Linux system due to difference of line endings.

In Unix based OS, lines end with \n but MSDOS/Win based system ends with \r\n

In Linux based system, you could show your prepared scripts by running

cat -e some-script.sh 

The results would be something similar to

#!/bin/bash^M$
... ...^M$

$ is normal Unix end-of-line symbol. Windows uses an extra one ^M and this symbol is not recognized by Unix system. That's why, in SageMaker Notebook Lifecycle Configuration, which is running Linux, your script was interpreted as /bin/bash^M

To mitigate the issue, please convert the scripts to Unix based ending and update life cycle configuration. To achieve this, you could use Notepad++ in Windows. You can go to the Edit menu, select the EOL Conversion submenu, and from the options that come up select UNIX/OSX Format. The next time you save the file, its line endings will, all going well, be saved with UNIX-style line endings.

Alternatively, you could put the script in a Linux environment, e.g. EC2 instance with Amazon Linux 2 and install dos2unix via sudo yum install dos2unix. After installation, you could convert your files via

dos2unix -n file.sh  output.sh 

After the conversion, please update LCC with the new scripts. You could verify that ^M has been removed via

cat -e your_script.sh

The output will print all special characters directly without hiding.