- 最新
- 最多得票
- 最多評論
💡 You can add the desired metadata fields to the
fileSchema
section of the JSON file. For example, to include a customx-amz-meta-x-version
field, you would update thefileSchema
to include that field, and then populate the corresponding column in the CSV data file with the appropriate version IDs.
Example:
{ "sourceBucket" : "dev-djool-xyubd4", "destinationBucket" : "arn:aws:s3:::dev-djool-s3-reports", "version" : "2016-11-30", "creationTimestamp" : "1712797200000", "fileFormat" : "CSV", "fileSchema" : "Bucket, Key, VersionId, IsLatest, IsDeleteMarker, LastModifiedDate, ETag, x-amz-meta-x-version", "files" : [ { "key" : "dev-djool-xyubd4/dev-djool-xyubd4-copy-job/data/4dba9155-e423-4352-b031-191c26e01cae.csv.gz", "size" : 29959, "MD5checksum" : "a50e27f226b1a68a7496466c9303bd6a" } ] }
Key Source:
Hi
Please follow the steps outlined in this re:Post thread and let me know if you face any difficulty.
I'd be more than happy to assist if there are any additional questions or challenges you see while following the steps.
Edit:
Earlier I missed the x-amz-meta-x-version part, which you mentioned. To better understand, I would prefer to see the inventory configuration settings and also will see if this metadata is tied to all the objects in your s3 bucket/prefix. It'd be tedious to remove the metadata from each object and also it'd create a new version of the object if you modify/add/remove the metadata. Hence, I'd prefer to remove "x-amz-meta-x-version" from manifest.json fileschema, save this and get a new checksum and size of manifest.json and update those in manifest.checksum file. Upload these files in the respective lcation and then run your s3 batch job. Long term, if metadata of objects are supposed to be that way only, then you may need to figure out the purpose of this user defined metadata and is it really required.
Let me know if it doesn't work(I'm sure it'd solve your problem), happy t o help further.
Abhishek
Hey Abhishek, Unknown task fields are present in the schema: [x-amz-meta-x-version] AWS doesnt understood x-amz-meta-x-version field. Could you please help for this issue.
Hello Dnyaneshwar, Please refer edit section in my answer and do the needful. It would solve the problem.
So, I dont know how I can share my Inventory Management configuration for your observation. But other your point is about does it really needed ? so, answer is yes, since I want to keep the reference of old versionId somewhere in newly copied items. And for that thought to use Object Meta Data fields with custom defined attribute. So, keeping x-amz-meta-x-version is might be not essential but keeping old respective versionId is what intention is to achieve. By the way without x-amz-meta-x-version this all scenario works perfectly fine, but this is not meeting the ultimate goal
相關內容
- AWS 官方已更新 8 個月前
- AWS 官方已更新 2 年前
- AWS 官方已更新 3 年前
Every time for any small changes in CSV or either manifesto file with hard coded checksum, I am getting this error.
"Manifest checksum mismatch occurred. Expected: df8485b1964c44b3af09599e984b0b70. Calculated: b1495a57cddb893113a63bc108294ceb" Since this is manually alteration in CSV file then how I can handle checksum problem ?
@secondabhi_aws appears to have the solution you need. Please review their suggestion at Solution Manifest checksum mismatch and let us know if your issue persists. We're here to help!
@Osvaldo Marte, checksum issue is clear now. But AWS manifest.json file doesnt understood
x-amz-meta-x-version
field.Reasons for failure: Unknown task fields are present in the schema: [x-amz-meta-x-version] I tried exact same way you mentioned about CSV & manifest files. but no luck yet.
I was digging more and I think the main issue is that AWS S3 Batch Operations do not support directly applying custom metadata from the manifest file; instead, you need to use an AWS Lambda function to dynamically set metadata like
x-amz-meta-x-version
during the operation. The operations that can be configured (copy, tag, delete, etc.) do not include direct manipulation of object metadata based on entries in the manifest file.One possible solution is to establish an AWS Lambda Function that will be triggered by your S3 Batch Operation. Its purpose will be to copy each object and explicitly set the
x-amz-meta-x-version
metadata to the old version ID as specified in the operation's input.