AWS S3 Get object not working. Getting corrupt file

1

I have a .zip file on an S3 bucket. When I try to get it and save it as a file it is corrupt and can't be opened. I don't know what to do, is really hard to get that S3 zip file. I am using this code as a guide https://docs.aws.amazon.com/AmazonS3/latest/userguide/download-objects.html. I'm currently on Unity, but the SDK for .NET should work. The resulted file has the first 7.4 kb all NULL. This is the code that gets the zip file, returns the buffer, and saves it with File.WriteAllBytes:

/// <summary>
    /// Get Object from S3 Bucket
    /// </summary>
    public async Task<byte[]> GetZip(string pFile)
    {
        try
        {
            Debug.Log("KEY: " + pFile);
            GetObjectRequest request = new GetObjectRequest
            {
                BucketName = S3Bucket,
                Key = pFile
            };
            using (GetObjectResponse response = await S3Client.GetObjectAsync(request))
            using (Stream responseStream = response.ResponseStream)
            {
                Debug.Log("Response stream");
                if (responseStream != null)
                {
                    byte[] buffer = new byte[(int)response.ResponseStream.Length];
                    int result = await responseStream.ReadAsync(buffer, 0, (int)response.ResponseStream.Length);
                    Debug.Log("Stream result: " + result);
                    File.WriteAllBytes(songPath, buffer);
                    Debug.Log("Readed all bytes: " + buffer.Length + " - " + ((int)response.ResponseStream.Length));
                    return buffer;
                }
                else
                {
                    Debug.Log("Response is null");
                }
            }
            
        }
        catch (AmazonS3Exception e)
        {
            // If bucket or object does not exist
            Debug.Log("Error encountered ***. Message:"+ e.Message + " when reading object");
        }
        catch (Exception e)
        {
            Debug.Log("Unknown encountered on server. Message:"+ e.Message + " when reading object");
        }
        return null;
    }
질문됨 2년 전1248회 조회
3개 답변
1

C# Directly supports writing the stream to a file:

using (var response = await S3Client.GetObjectAsync(request))
using (var responseStream = response.ResponseStream)
using (var outputFileStream = new FileStream(songPath, FileMode.Create))
{
    await responseStream.CopyToAsync(outputFileStream);
}
wparad
답변함 2년 전
  • It worked but I had to use this:

    byte[] buffer = new byte[(int)response.ContentLength]; using (var outputFileStream = new MemoryStream(buffer)) { await responseStream.CopyToAsync(outputFileStream) } File.WriteAllBytes(CustomSongsManager.CustomFolder + pFile, buffer);

    Thanks

0

Ok. Fiddled with your code a bit. And got NoSupported errors on responseStream.Length.

I changed to code to using response.ContentLength and it started working for me.

This is the code I ended up with:

async Task<byte[]> GetZip(string pFile)
{
    try
    {
        Console.WriteLine("KEY: " + pFile);
        Amazon.S3.Model.GetObjectRequest request = new Amazon.S3.Model.GetObjectRequest
        {
            BucketName = S3Bucket,
            Key = pFile
        };
        using (Amazon.S3.Model.GetObjectResponse response = await S3Client.GetObjectAsync(request))
        using (Stream responseStream = response.ResponseStream)
        {
            Console.WriteLine("Response stream");
            if (responseStream != null)
            {
                byte[] buffer = new byte[(int)response.ContentLength];
                int result = await responseStream.ReadAsync(buffer, 0, (int)response.ContentLength);
                Console.WriteLine("Stream result: " + result);
                File.WriteAllBytes(songPath, buffer);
                Console.WriteLine("Readed all bytes: " + buffer.Length + " - " + ((int)response.ContentLength));
                return buffer;
            }
            else
            {
                Console.WriteLine("Response is null");
            }
        }
    }
    catch (Amazon.S3.AmazonS3Exception e)
    {
        // If bucket or object does not exist
        Console.WriteLine("Error encountered ***. Message:"+ e.Message + " when reading object");
    }
    catch (Exception e)
    {
        Console.WriteLine("Unknown encountered on server. Message:"+ e.Message + " when reading object");
    }
    return new byte[0];
}

GetZip(pFile).Wait();
profile picture
JaccoPK
답변함 2년 전
  • I tried it but stills returns a corrupted zip file. I'm using the same sdk but in Unity. The zip file I get has the first 7.4 kb all at NULL.

  • With these versions?

      <ItemGroup>
        <PackageReference Include="AWSSDK.Core" Version="3.7.11.13" />
        <PackageReference Include="AWSSDK.S3" Version="3.7.9.13" />
      </ItemGroup>
  • On that page I still can find many different versions.

    aws-sdk-netstandard2.0.zip
    
    aws-sdk-net45.zip
    
    aws-sdk-net35.zip
    
  • I am not sure. I think you have the right one. Not sure though if I can help any further on my Mac :-)

  • In Unity I just imported the .dll files. I downloaded the files from here: https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/unity-special.html. My version for core is 3.3.0.0 and for S3 is also 3.3.0.0. I downloaded the aws-sdk-netstandard2.0.zip. I'm going to try downloading the aws-sdk-net45.zip instead and see if it works

0

Is the file truly corrupt or has it been unzipped and you didn't expect it to be? From memory the Content Encoding metadata on the object in S3 will influence whether a zip file is transparently unzipped on the fly.

전문가
답변함 2년 전
  • In the bucket, the file is a zip file. I didn't know it could be unzipped on the fly. If that's the case how can I save the folder with its content? Or how can I change it so it downloads as a zip file? Please can you be more specific, the content metadata of the file in the bucket is application/zip

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠