"NewSQL" AWS database service for OLAP and OLTP combined?


Does AWS currently, or does it have planned for the future, an equivalent service to "NewSQL" services such as TiDB, Vitess, Yugabyte, and Oracle Heatwave? My understanding is that these services allow OLAP and OLTP within the same database by combining transactional and columnar technologies, eliminating much ETL between a data warehouse and consumer facing db and providing real time analytics. I understand some of these services offer hosting on AWS, but I am wondering if there is an AWS native solution currently or planned.

asked 2 years ago1268 views
1 Answer

AWS constantly listens to customers to deliver what they need, and roughly 90-95% of the AWS roadmap is driven by customer input. There is no public information currently on whether a NewSQL or HTAP offering is on the roadmap. If you have a specific requirement, please contact your AWS account manager or solutions architect so that they can take note of your input.

NewSQL offerings may offer OLTP scale-out, HTAP, or both. Some of these offerings provide OLTP scale-out (but not OLAP); for example, YugabyteDB and Vitess. While TiDB and Heatwave do provide HTAP.

To address HTAP use cases, you can consider using Amazon Aurora as the OLTP data store, and continuously replicate that data to S3 for querying with Amazon Athena or Redshift Spectrum. Please refer to this blog post - https://aws.amazon.com/blogs/big-data/combine-transactional-and-analytical-data-using-amazon-aurora-and-amazon-redshift/

You could also use AWS Database Migration Service to stream data from Amazon Aurora to Amazon Redshift - https://d1.awsstatic.com/events/reinvent/2019/REPEAT_1_Analytical_use_cases_with_AWS_Database_Migration_Service_(AWS_DMS)_DAT360-R1.pdf

Note that the above approach (replicate from an OLTP data store to OLAP) is basically what Heatwave does. This pattern is also used by Azure Synapse Connector for CosmosDB and SQL Server.

If your requirement is for OLTP scale-out, Aurora Multi-Master (https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-multi-master.html) provides some degree of this; however, the amount of scaling is not comparable to what Vitess, YugabyteDB, or Citus PostgreSQL are capable of (Aurora MM is 4 nodes maximum, versus hundreds with NewSQL offerings).

It is important to note, however, that Amazon Aurora on a single writer node is capable of high throughput (for example see this excellent comparison of Amazon Aurora, YugabyteDB, and CockroachDB - https://blog.yugabyte.com/comparing-distributed-sql-performance-yugabyte-db-vs-amazon-aurora-postgresql-vs-cockroachdb/ - see the conclusion at the bottom "This means that Aurora PostgreSQL is a great choice if your database instance would never need to handle more than 120K transactions/sec"). Hence for most use cases that do not require >>100K transactions/second, Amazon Aurora is an excellent choice..

One or more of the NewSQL offerings mentioned in the question (e.g. Yugabyte) are available as a fully-managed service on AWS, and are available on the AWS Marketplace, e.g. https://aws.amazon.com/marketplace/pp/prodview-d6flojaonhhzg

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions