Skip to content

Add pgvectorscale extension to Amazon RDS / Aurora PostgreSQL for DiskANN/HNSW hybrid vector search

1

Hi AWS RDS team,

We're running large-scale GraphRAG workloads on RDS PostgreSQL with pgvector, and roadmapping our growth, can you please add pgvectorscale to RDS?

pgvectorscale provides:

  • DiskANN/HNSW hybrid indexing for massive disk-based search
  • Up to 10x memory reduction vs pgvector's HNSW
  • Sub-10ms QPS at scale with recall >0.95

Timescale has requested this in https://github.com/timescale/pgvectorscale/issues/113#issuecomment-4077697461 and confirmed it's feasible for RDS.

Request: Bundle pgvectorscale 0.7+ into the next RDS PostgreSQL 16.x / Aurora PG 16.x minor release.

Impact: Critical for cost-effective vector DB at enterprise scale without self-managing EC2 Postgres.

Alternatives tried: pgvector alone OOMs; self-managed too ops-heavy.

Thanks for considering!

2 Answers
0

I appreciate your detailed feedback about pgvectorscale for Amazon RDS and Aurora PostgreSQL. While I can't make commitments about future feature releases or roadmap items, I can share information about the current pgvector capabilities available in these services.

Amazon RDS for PostgreSQL and Aurora PostgreSQL currently support pgvector, with recent versions including pgvector 0.8.0. This version brings several performance improvements including enhanced query planner index selection when filters are present, better data filtering capabilities in WHERE clauses and joins, iterative index scans to prevent overfiltering, and performance improvements for searching and building HNSW indexes.

For pgvector 0.7.0 and earlier versions, support includes features like halfvec and sparsevec data types, binary vector indexing, scalar and binary quantization support, and the ability to index up to 4,000 dimensions for halfvec and 64,000 for binary vectors. The extension also supports approximate nearest neighbor (ANN) indexing methods including IVFFLAT and HNSW for efficient similarity search.

For feature requests like adding pgvectorscale, I'd recommend engaging with AWS through official channels such as AWS Support, your AWS account team, or AWS forums where product teams actively review customer feedback. These channels ensure your requirements are properly documented and considered for future development priorities.

In the meantime, you might explore optimizing your current pgvector implementation using the quantization features and indexing improvements available in the latest versions to help manage memory usage and query performance at scale.
Sources
Community | AWS open source newsletter, #198
Announcing pgvector 0.8.0 support in Aurora PostgreSQL - AWS
Amazon RDS for PostgreSQL supports pgvector 0.8.0 - AWS
Community | Vector Embeddings and RAG Demystified: Leveraging Amazon Bedrock, Aurora, and LangChain - Part 2

answered 2 months ago
EXPERT
reviewed 2 months ago
0

Hello.

If your AWS account has a paid support contract (AWS Business Support+), try opening a support case and submitting a feature request.
https://docs.aws.amazon.com/awssupport/latest/user/case-management.html

You have a general development question, or you want to request a feature. (AWS Business Support+, AWS Enterprise Support, or AWS Unified Operations plan)

EXPERT
answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.