How do you deploy OSDU M18 on AWS?

OSDU is a Data Platform for the Energy Industry developed by the OSDU Forum, which AWS is a member of.

AWS has the largest community of active OSDU Data Platform application partners. Once deployed on AWS, OSDU unlocks the art and science of upstream workflows powered by the broadest and deepest set of cloud capabilities.

What's new in OSDU M18 on AWS

On July 2023, release M18 was made available to the public by AWS. For common features, enhancements, bug fixes and known issues related to the OSDU forum, please refer to the OSDU M18 Release Notes.

Additional features and enhancements delivered by as part of M18 include:

• Re-architecture of OSDU on AWS infrastructure, to enable multi-tenancy
• Full adoption of OSDU on AWS deployment through Terraform
• Notification service fix to enable notifications for storage change event
• Enabled token refresh when running re-indexing API
• Adoption of Amazon Managed Workflows for Apache Airflow (MWAA) for DAGs
• Upgraded to use IMDSv2 for all compute instances
• Support Spot Instances for OSDU on AWS
• Key technologies version upgrade, such as EKS 1.26, Istio 1.17
• Enhanced infrastructure security and adoption of best practice from Trusted Advisor
• Validation of seismic sub-project name to avoid hyphen
• Adoption of Amazon RDS for PostgreSQL database
• Configured AWS Lambda function to validate code-signing
• Fixed issue with Terraform roles having too much IAM permission
• Modularized the OSDU bootstrapping scripts so that it can be run idempotently
• Resolved key issues reported by EKS Best Practices scan on OSDU
• Improved pod and instance autoscaling
• Adoption of Amazon Managed Service for Prometheus for metrics

Deployment artifacts

For M18, two artifacts are available to be downloaded:

• osdu.zip – the artifact to deploy OSDU on AWS
• aws-osdu-migration-tool.zip – the artifact to be deployed to perform data migration from M16 to M18

Implementation guide

A comprehensive OSDU Data Platform on AWS implementation guide detailing the installation procedure is available. The implementation guide contains:

• an overview of the different services that make up OSDU on AWS, as well as how they interact with each other
• step by step instructions on how to deploy the solution using Terraform to deploy the infrastructure and Helm chart for services deployments
• explanation of post-deployment activities such as managing multi-tenancy
• instructions on how to run data loading utilities to ingest data from different formats like CSV, WITSML, and SEG-Y
• instructions to migrate data from a previous version (M16) to a new deployment of OSDU M18
• troubleshooting tips for the issues most commonly found during deployment or while running the solution
• instructions to uninstall the solution, destroying the infrastructure and core services deployments, or destroying a tenant

Recommendations

AWS customers are encouraged to upgrade to M18 by October 11, 2023 in order not to be affected by End of Support for Amazon EKS 1.23