By using AWS re:Post, you agree to the Terms of Use
/Encryption/

Questions tagged with Encryption

Sort by most recent
  • 1
  • 90 / page

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Enabling S3 Encryption-at-rest on a go-forward basis with s3fs

Hi, We have some buckets (have been around for a while, approx 200GB+ data) and we want to **turn on** encryption-at-rest using SSE-S3 (the most "transparent" way) https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-encryption.html The S3 buckets are mounted to our Linux VMs using S3FS https://github.com/s3fs-fuse/s3fs-fuse which has support for this (seems fairly transparent) So, it seems like the way this works is that you can only enable this on files on a go-forward basis so the older files that already exist will not be in encrypted-at-rest (which is ok, we can backfill this later) Has anybody tried to do this before using this combo? If we mount the bucket using s3fs with `-o use_sse` option, what will happen as the files will be half-and-half? Will it "just work"? s3fs will be mounted with `-o use_see` and it will be able to handle files that are BOTH the old way (not encrypted-at-rest) and the newer files (encrypted-at-rest) ... we can then start backfilling the older files and we have time or will this fail catastrophically the minute we mount the s3 bucket :( Is the solution to just start a new bucket and do the SSE-S3 and then just start moving the files over (we have done this before in terms of having code in our application check for a file in multiple buckets before giving up) Of course, we will test all this stuff, just wanted to ask a quick question in case we are worried about this too much and if this is a "no big deal" or "be very careful" Thanks!
0
answers
0
votes
7
views
asked 2 days ago

Using aws s3api put-object --sse-customer-key-md5 fails with CLI

I'm trying to use aws s3api put-object/get-object with server side encryption with customer keys. I'm using Powershell, but I don't believe that is the source of my issue. On the surface, sse-customer-key-md5 appears to be a pretty simple input: https://docs.aws.amazon.com/cli/latest/reference/s3api/put-object.html Specifies the 128-bit MD5 digest of the encryption key according to RFC 1321. Amazon S3 uses this header for a message integrity check to ensure that the encryption key was transmitted without error. put-object works when I don't use --sse-customer-key-md5: >aws s3api put-object ` --bucket abc ` --sse-customer-algorithm AES256 ` --sse-customer-key "testaes256testaes256testaes25612" ` --region us-east-1 ` --key test.pdf ` --body C:\test.pdf > { "SSECustomerKeyMD5": "ezatpv/Yg0KkjX+5ZcsxdQ==", "SSECustomerAlgorithm": "AES256", "ETag": "\"0d44c3df058c4e190bd7b2e6d227be73\"" } I agree with the SSECustomerKeyMD5 result: >$key = "testaes256testaes256testaes25612" $md5 = new-object -TypeName System.Security.Cryptography.MD5CryptoServiceProvider $utf8 = new-object -TypeName System.Text.UTF8Encoding $hash = $md5.ComputeHash($utf8.GetBytes($key)) $EncodedString =[Convert]::ToBase64String($hash) Write-Host "Base64 Encoded String: " $EncodedString Base64 Encoded String: ezatpv/Yg0KkjX+5ZcsxdQ== Now I resubmit my put request with the --sse-customer-key-md5 option. Before anyone jumps on the base64 encoding, I've tried submitting the MD5 hash in Base64, Hexidecimal (With and without delimiters), JSON of the MD5 hash result, and upper case and lower case versions of the aforementioned. None work. Has anyone gotten this to work and, if so, format did you use? >aws s3api put-object ` --bucket abc ` --sse-customer-algorithm AES256 ` --sse-customer-key "testaes256testaes256testaes25612" ` --sse-customer-key-md5 "ezatpv/Yg0KkjX+5ZcsxdQ==" ` --region us-east-1 ` --key test.pdf ` --body C:\test.pdf > aws : At line:1 char:1 + aws s3api put-object ` + ~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:String) [], RemoteException + FullyQualifiedErrorId : NativeCommandError An error occurred (InvalidArgument) when calling the PutObject operation: The calculated MD5 hash of the key did not match the hash that was provided. Thanks
2
answers
0
votes
3
views
asked a month ago
1
answers
0
votes
9
views
asked a month ago

Application side data protection with FIPS 140-2 Level 3 : what to use out of Encryption SDK, KMS or Cloud HSM?

Hello there, I do have a requirement in my application to encrypt and decrypt data using a symmetric key algorithm (mostly AES/CBC/PKCS5Padding). CONSTRAINT and Requirements are 1. I need to use FIPS 140-2 Level 3 compliant key storage solution 2. This is an existing encrypted data and hence I should be able to import my existing keys (plain keys) to whatever solution I use. 3. Even in the future, keys should be open for EXPORT so that encrypted data with this new solution WILL NOT require another re-encryption with new keys. Keeping the above points in mind, I came across below solutions so far and need guidance and help if someone finds that not a good solution or it will break any of the above requirements I listed. 1. I can use AWS Encryption SDK with AWS KMS using a custom key store where the custom key store would be my own Cloud HSM. 2. I can directly use Cloud HSM by leveraging standard Cloud HSM integration using Cloud HSM JCE provider and client SDK. 3. I can AWS KMS with KMS API with a custom key store where the custom key store would be my own Cloud HSM. I knew #2 will work without breaking any of my requirement and compliance list but I want to see if I can use Encryption SDK and/or KMS for my use case as I can get help of SDK to choose best industry practices to write cryptography code instead of I write whole code (in case of Cloud HSM integration) but below points will stop me. 1. Custom key stores can not work with imported keys so it will break my requirement #2. 2. I can use AWS Encryption SDK with KMS but as import does not work for custom key stores, it's not usable any more. Can I use AWS Encryption SDK somehow to help me with data encryption directly with Cloud HSM? 3. Data enveloper protection (by AWS Encryption SDK) is really more secure for symmetric key encryption. If I use that today and later want to move to Cloud HSM, will it break the decryption flow? Any suggestion/experience learning/insights or architectural direction is greatly appreciated.
1
answers
0
votes
7
views
asked 2 months ago

S3 bucket creation with encryption is failing because of AWSSamples::S3BucketEncrypt::Hook

Hi, I have activated **AWSSamples::S3BucketEncrypt::Hook** with the following configuration but S3 bucket creation with encryption enabled seems to be failing because of the hook. It works when I disable the hook. Could this be an issue? ``` { "CloudFormationConfiguration": { "HookConfiguration": { "TargetStacks": "ALL", "FailureMode": "FAIL", "Properties": { "minBuckets": "1", "encryptionAlgorithm": "AES256" } } } } ``` ``` { "CloudFormationConfiguration": { "HookConfiguration": { "TargetStacks": "ALL", "FailureMode": "FAIL", "Properties": { "minBuckets": "1", "encryptionAlgorithm": "aws:kms" } } } } ``` [AWSSamples::S3BucketEncrypt::Hook configuration](https://imgur.com/w9NnjEP) [AWSSamples::S3BucketEncrypt::Hook](https://imgur.com/OsETMvV) **CloudFormation for S3 bucket with AES256 encryption** - Expected to Pass ``` AWSTemplateFormatVersion: 2010-09-09 Description: S3 bucket with default encryption Resources: EncryptedS3Bucket: Type: 'AWS::S3::Bucket' Properties: BucketName: !Sub 'encryptedbucket-${AWS::Region}-${AWS::AccountId}' BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: 'AES256' DeletionPolicy: Delete ``` **CloudFormation for S3 bucket with KMS encryption** - Expected to Pass ``` AWSTemplateFormatVersion: "2010-09-09" Description: This CloudFormation template provisions an encrypted S3 Bucket Resources: EncryptedS3Bucket: Type: 'AWS::S3::Bucket' Properties: BucketName: !Sub 'encryptedbucket-${AWS::Region}-${AWS::AccountId}' BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: 'aws:kms' KMSMasterKeyID: !Ref EncryptionKey BucketKeyEnabled: true Tags: - Key: "keyname1" Value: "value1" EncryptionKey: Type: AWS::KMS::Key Properties: Description: KMS key used to encrypt the resource type artifacts EnableKeyRotation: true KeyPolicy: Version: "2012-10-17" Statement: - Sid: Enable full access for owning account Effect: Allow Principal: AWS: !Ref "AWS::AccountId" Action: kms:* Resource: "*" Outputs: EncryptedBucketName: Value: !Ref EncryptedS3Bucket ```
0
answers
0
votes
9
views
asked 3 months ago

Python lambda failing to initialize RSA public key occasionally

I'm trying to create a custom request authorizer working with several user pools, in Python. So to validate tokens I tried first with pyjwk/cryptography ``` claims = jwt.decode(token, options={"verify_signature": False, "require":["iss"]}) issuer = claims['iss'] jwks_client = PyJWKClient(issuer+"/.well-known/jwks.json",False) signing_key = jwks_client.get_signing_key_from_jwt(token) ``` Occasionally, about 5% of the time, lambda instance will just timeout on this last line, even with 30 second run time. Thought maybe it was network, rewrote it to get the JWK through requests and initialize the key with RSAAlgorithm.from_jwk, nope - the JWK is retrieved, but it's initializing the key that fails. Called RSAAlgorithm.from_jwk outside the handle method with dummy hardcoded JWK to move initialization of cryptography to init stage; handler method works smoother now, instead of being slow on the first invocation, but the random failure still happens. Thought maybe it was cryptography or pyjwk, switched to python-jose and it's different backends. Nope - still fails in loading the key, now written as jwk.construct(). What is causing this strange and random behavior? An instance that failed once stays permanently broken and doesn't recover in the next request. On the logs there's nothing, although such broken instances drop the memory usage. Here are first two requests from broken and working instances running the same image, same time, for same user pool key. Broken: 2022-02-14T17:38:17.185+02:00 START RequestId: d9d92287-ccae-4aa4-8f94-6e8ed8c276a4 Version: $LATEST 2022-02-14T17:38:17.205+02:00 [DEBUG] 2022-02-14T15:38:17.205Z d9d92287-ccae-4aa4-8f94-6e8ed8c276a4 Starting new HTTPS connection (1): cognito-idp.us-east-1.amazonaws.com:443 2022-02-14T17:38:20.190+02:00 END RequestId: d9d92287-ccae-4aa4-8f94-6e8ed8c276a4 2022-02-14T17:38:20.190+02:00 REPORT RequestId: d9d92287-ccae-4aa4-8f94-6e8ed8c276a4 Duration: 3003.51 ms Billed Duration: 3000 ms Memory Size: 128 MB Max Memory Used: 53 MB Init Duration: 467.93 ms 2022-02-14T17:38:20.190+02:00 2022-02-14T15:38:20.189Z d9d92287-ccae-4aa4-8f94-6e8ed8c276a4 Task timed out after 3.00 seconds 2022-02-14T17:38:20.706+02:00 START RequestId: a5242265-c13d-4015-9b7d-2699f0b26efe Version: $LATEST 2022-02-14T17:38:20.709+02:00 [DEBUG] 2022-02-14T15:38:20.709Z a5242265-c13d-4015-9b7d-2699f0b26efe Starting new HTTPS connection (1): cognito-idp.us-east-1.amazonaws.com:443 2022-02-14T17:38:23.712+02:00 END RequestId: a5242265-c13d-4015-9b7d-2699f0b26efe 2022-02-14T17:38:23.712+02:00 REPORT RequestId: a5242265-c13d-4015-9b7d-2699f0b26efe Duration: 3004.51 ms Billed Duration: 3000 ms Memory Size: 128 MB Max Memory Used: 23 MB 2022-02-14T17:38:23.712+02:00 2022-02-14T15:38:23.711Z a5242265-c13d-4015-9b7d-2699f0b26efe Task timed out after 3.00 seconds Working: 2022-02-14T17:38:23.733+02:00 START RequestId: 2ea5db18-c9b5-4df8-b3ef-dfc01f9ede00 Version: $LATEST 2022-02-14T17:38:23.740+02:00 [DEBUG] 2022-02-14T15:38:23.739Z 2ea5db18-c9b5-4df8-b3ef-dfc01f9ede00 Starting new HTTPS connection (1): cognito-idp.us-east-1.amazonaws.com:443 2022-02-14T17:38:23.926+02:00 [DEBUG] 2022-02-14T15:38:23.926Z 2ea5db18-c9b5-4df8-b3ef-dfc01f9ede00 https://cognito-idp.us-east-1.amazonaws.com:443 "GET /us-east-1_.../.well-known/jwks.json HTTP/1.1" 200 916 2022-02-14T17:38:23.942+02:00 [DEBUG] 2022-02-14T15:38:23.941Z 2ea5db18-c9b5-4df8-b3ef-dfc01f9ede00 Got the key a2PUhJTqMTiNysvmY+RfUPARHESV35jOMXWXJ4mAa/A= in 0.20495343208312988 seconds 2022-02-14T17:38:23.960+02:00 [INFO] 2022-02-14T15:38:23.960Z 2ea5db18-c9b5-4df8-b3ef-dfc01f9ede00 response {'principalId': '...', 'policyDocument': {...}} 2022-02-14T17:38:23.980+02:00 END RequestId: 2ea5db18-c9b5-4df8-b3ef-dfc01f9ede00 2022-02-14T17:38:23.980+02:00 REPORT RequestId: 2ea5db18-c9b5-4df8-b3ef-dfc01f9ede00 Duration: 244.45 ms Billed Duration: 245 ms Memory Size: 128 MB Max Memory Used: 55 MB Init Duration: 447.66 ms 2022-02-14T17:38:24.149+02:00 START RequestId: 1cca0b7a-0fa4-477d-9ddd-95d97db113b2 Version: $LATEST 2022-02-14T17:38:24.154+02:00 [DEBUG] 2022-02-14T15:38:24.154Z 1cca0b7a-0fa4-477d-9ddd-95d97db113b2 Got the cached key a2PUhJTqMTiNysvmY+RfUPARHESV35jOMXWXJ4mAa/A= 2022-02-14T17:38:24.155+02:00 [INFO] 2022-02-14T15:38:24.155Z 1cca0b7a-0fa4-477d-9ddd-95d97db113b2 response {'principalId': '...', 'policyDocument': {...}} 2022-02-14T17:38:24.156+02:00 END RequestId: 1cca0b7a-0fa4-477d-9ddd-95d97db113b2 2022-02-14T17:38:24.156+02:00 END RequestId: 1cca0b7a-0fa4-477d-9ddd-95d97db113b2 2022-02-14T17:38:24.156+02:00 REPORT RequestId: 1cca0b7a-0fa4-477d-9ddd-95d97db113b2 Duration: 2.64 ms Billed Duration: 3 ms Memory Size: 128 MB Max Memory Used: 55 MB
1
answers
0
votes
6
views
asked 3 months ago
  • 1
  • 90 / page