Transfer Files from Google Drive to EC2
Hi Dears, I am trying to transfer a machine learning model file (500 MB) from google drive to EC2, for inferencing.
I tried different commands, and I see that wget works better, but when I do it :
The result was :
Connecting to drive.google.com (drive.google.com)|172.293.62.138|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: 'uc?id=1-Dqk6fZzDiFKTqnnQ2yqW48uJk-CPqrB' 2022-09-20 18:26:40 (37.2 MB/s) - 'uc?id=1-Dqk6fZzDiFKTqnnQ2yqW48uJk-CPqrB' saved [2241] [2]- Done wget https://drive.google.com/uc?id=1-Dqk6fZzDiFKTqnnQ2yqW48uJk-CPqrB
But, I cant see the file that I need to transfer, I see file with neme " uc?id=1-Dqk6fZzDiFKTqnnQ2yqW48uJk-CPqrB "... Can you please help with that step? many
Thanks in advance
Best
Basem
- Newest
- Most votes
- Most comments
Hi THere
Take a look at this article showing the steps to download a file from Google Drive using wget. You need to make it publicly shareable first. Pay attention to the later part of the article, there is special handling for files greater than 100mb.
https://medium.com/@acpanjan/download-google-drive-files-using-wget-3c2c025a8b99
You may also want to take a look at gdown, available here: https://github.com/wkentaro/gdown.
Downloading files can be done either via command line, or from within Python code.
Relevant content
- Accepted Answerasked 2 years ago
- asked 5 months ago
AWS OFFICIALUpdated 3 years ago- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 months ago
- AWS OFFICIALUpdated a year ago
Thanks for your kind response dear matt! But can you please, tell me what things must be changed? as I have a google drive link not google docs as mentioned in the articel, Where I change fileid and file name, but ask specifically for the links included in the command.
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=FILEID' -O- | sed -rn 's/.confirm=([0-9A-Za-z_]+)./\1\n/p')&id=FILEID" -O FILENAME && rm -rf /tmp/cookies.txt
If you look in the comments of the article there are some other tips. One was to use this command and replace FILEID and FILENAME
wget --no-check-certificate --load-cookies /tmp/cookies.txt "https://drive.google.com/uc?export=download&confirm=true$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://drive.google.com/uc?export=download&id=FILEID' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=FILEID" -O FILENAME && rm -rf /tmp/cookies.txt