S3: Sync only files within folders with specific filenames.

Question

Hi,

I have on my root server a directory with sql dumps (created once a day and rotating every 7 days)

Example folder structure:
```
2022-06-27/db_name_1/db_name_1-2022-06-27-22-35.sql.gz
                     /db_name_2/db_name_2-2022-06-27-22-35.sql.gz
                     /db_name_3/db_name_3-2022-06-27-22-35.sql.gz
2022-06-28/db_name_1/db_name_1-2022-06-28-22-35.sql.gz
                     /db_name_2/db_name_2-2022-06-28-22-35.sql.gz
                     /db_name_3/db_name_3-2022-06-28-22-35.sql.gz
2022-06-29/db_name_1/db_name_1-2022-06-29-22-35.sql.gz
                     /db_name_2/db_name_2-2022-06-29-22-35.sql.gz
                     /db_name_3/db_name_3-2022-06-29-22-35.sql.gz
```
I my case I would like only to backup db_name_1.

So in my S3 bucket I would like to have only the sql dumps of the db_name_1

```
...
db_name_1-2022-06-27-22-35.sql.gz
db_name_1-2022-06-28-22-35.sql.gz
db_name_1-2022-06-29-22-35.sql.gz
...
```

So basically I would like to tell aws cli to just look at db_name_1-*.sql.gz

Is this possible?

Answer

Kindly first refer to the use of include & exclude filters as referred in link below...
https://docs.aws.amazon.com/cli/latest/reference/s3/index.html#use-of-exclude-and-include-filters

Not sure if formatting is showing the folder structure is showing up correctly, but if it is like following ..

```
2022-06-27/db_name_1/db_name_1-2022-06-27-22-35.sql.gz
2022-06-27/db_name_2/db_name_2-2022-06-27-22-35.sql.gz
2022-06-27/db_name_3/db_name_3-2022-06-27-22-35.sql.gz
2022-06-28/db_name_1/db_name_1-2022-06-28-22-35.sql.gz
2022-06-28/db_name_2/db_name_2-2022-06-28-22-35.sql.gz
2022-06-28/db_name_3/db_name_3-2022-06-28-22-35.sql.gz
```

then you can try this

```
aws s3 cp . s3:/// --recursive --exclude "*" --include "*/db_name_1/db_name_*.sql.gz"
```

You can try with specific database name as above or asterick if you want them all. As always try with small set of files to test the code to meet your needs.

S3: Sync only files within folders with specific filenames.

関連するコンテンツ