AWS S3 Get object not working. Getting corrupt file
I have a .zip file on an S3 bucket. When I try to get it and save it as a file it is corrupt and can't be opened. I don't know what to do, is really hard to get that S3 zip file. I am using this code as a guide https://docs.aws.amazon.com/AmazonS3/latest/userguide/download-objects.html. I'm currently on Unity, but the SDK for .NET should work. The resulted file has the first 7.4 kb all NULL. This is the code that gets the zip file, returns the buffer, and saves it with File.WriteAllBytes:
/// <summary>
/// Get Object from S3 Bucket
/// </summary>
public async Task<byte[]> GetZip(string pFile)
{
try
{
Debug.Log("KEY: " + pFile);
GetObjectRequest request = new GetObjectRequest
{
BucketName = S3Bucket,
Key = pFile
};
using (GetObjectResponse response = await S3Client.GetObjectAsync(request))
using (Stream responseStream = response.ResponseStream)
{
Debug.Log("Response stream");
if (responseStream != null)
{
byte[] buffer = new byte[(int)response.ResponseStream.Length];
int result = await responseStream.ReadAsync(buffer, 0, (int)response.ResponseStream.Length);
Debug.Log("Stream result: " + result);
File.WriteAllBytes(songPath, buffer);
Debug.Log("Readed all bytes: " + buffer.Length + " - " + ((int)response.ResponseStream.Length));
return buffer;
}
else
{
Debug.Log("Response is null");
}
}
}
catch (AmazonS3Exception e)
{
// If bucket or object does not exist
Debug.Log("Error encountered ***. Message:"+ e.Message + " when reading object");
}
catch (Exception e)
{
Debug.Log("Unknown encountered on server. Message:"+ e.Message + " when reading object");
}
return null;
}
C# Directly supports writing the stream to a file:
using (var response = await S3Client.GetObjectAsync(request)) using (var responseStream = response.ResponseStream) using (var outputFileStream = new FileStream(songPath, FileMode.Create)) { await responseStream.CopyToAsync(outputFileStream); }
Ok. Fiddled with your code a bit. And got NoSupported
errors on responseStream.Length.
I changed to code to using response.ContentLength
and it started working for me.
This is the code I ended up with:
async Task<byte[]> GetZip(string pFile) { try { Console.WriteLine("KEY: " + pFile); Amazon.S3.Model.GetObjectRequest request = new Amazon.S3.Model.GetObjectRequest { BucketName = S3Bucket, Key = pFile }; using (Amazon.S3.Model.GetObjectResponse response = await S3Client.GetObjectAsync(request)) using (Stream responseStream = response.ResponseStream) { Console.WriteLine("Response stream"); if (responseStream != null) { byte[] buffer = new byte[(int)response.ContentLength]; int result = await responseStream.ReadAsync(buffer, 0, (int)response.ContentLength); Console.WriteLine("Stream result: " + result); File.WriteAllBytes(songPath, buffer); Console.WriteLine("Readed all bytes: " + buffer.Length + " - " + ((int)response.ContentLength)); return buffer; } else { Console.WriteLine("Response is null"); } } } catch (Amazon.S3.AmazonS3Exception e) { // If bucket or object does not exist Console.WriteLine("Error encountered ***. Message:"+ e.Message + " when reading object"); } catch (Exception e) { Console.WriteLine("Unknown encountered on server. Message:"+ e.Message + " when reading object"); } return new byte[0]; } GetZip(pFile).Wait();
I tried it but stills returns a corrupted zip file. I'm using the same sdk but in Unity. The zip file I get has the first 7.4 kb all at NULL.
With these versions?
<ItemGroup> <PackageReference Include="AWSSDK.Core" Version="3.7.11.13" /> <PackageReference Include="AWSSDK.S3" Version="3.7.9.13" /> </ItemGroup>
On that page I still can find many different versions.
aws-sdk-netstandard2.0.zip aws-sdk-net45.zip aws-sdk-net35.zip
I am not sure. I think you have the right one. Not sure though if I can help any further on my Mac :-)
In Unity I just imported the .dll files. I downloaded the files from here: https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/unity-special.html. My version for core is 3.3.0.0 and for S3 is also 3.3.0.0. I downloaded the aws-sdk-netstandard2.0.zip. I'm going to try downloading the aws-sdk-net45.zip instead and see if it works
Is the file truly corrupt or has it been unzipped and you didn't expect it to be? From memory the Content Encoding metadata on the object in S3 will influence whether a zip file is transparently unzipped on the fly.
In the bucket, the file is a zip file. I didn't know it could be unzipped on the fly. If that's the case how can I save the folder with its content? Or how can I change it so it downloads as a zip file? Please can you be more specific, the content metadata of the file in the bucket is application/zip
Relevant questions
S3 Resource Owner and default Bucket/Object Privileges
Accepted Answerasked 3 years agoSharing file with URL link in S3
asked 6 months ago"Bucket name already exists", but it is not listed on the S3 Mgmt Console
asked 4 years agoIs it possible to use artifacts in an S3 Bucket From a Different Region To CodeDeploy Application
asked 4 months agoIs there a way to not change an S3 object's Last-Modified date when copying it to a new bucket?
asked 6 months agoAWS S3 Get object not working. Getting corrupt file
asked a month agoUploading a file I downloaded from Sharepoint to S3 Bucket
Accepted Answerasked a month agoError Msg: "This XML file does not appear to have any style information.."
asked 3 years agoHow do we unzip a file in S3 bucket using C#.NET
asked 2 months agoWhat needs to be done to make event bridge invoke a fargate task when file added to s3
asked 2 months ago
It worked but I had to use this:
byte[] buffer = new byte[(int)response.ContentLength]; using (var outputFileStream = new MemoryStream(buffer)) { await responseStream.CopyToAsync(outputFileStream) } File.WriteAllBytes(CustomSongsManager.CustomFolder + pFile, buffer);
Thanks