How metadata filter selectivity affects recall in Amazon S3 Vectors

Amazon S3 Vectors lets you attach metadata to vectors and filter on it at query time. The filtering looks simple: pass a filter expression, get back matching vectors ranked by similarity. But there's a behavior that surprises people the first time they hit it. As a filter gets more selective, query quality can degrade. You still get results back. They're just less likely to be the right results.

The official documentation mentions this in one line: queries with filters "may return fewer than top K results when the vector index contains very few matching results." That's true, but it's only half the story, and it describes a symptom rather than the cause. This article explains the cause, shows it measured on a million vectors, and uses what was learned to provide guidance for designing indexes, metadata, and queries. Every number here comes from synthetic vectors built to expose the effect, so treat the shapes and relative gaps as the findings, not the absolute values (full methodology at the end of this article). I pushed things to extremes to show the effects and my results are correct at time of writing. But services change, always consult the latest documentation.

How S3 Vectors filtering works

The key fact is in the AWS documentation, it's easy to miss: S3 Vectors performs the similarity search and the filter evaluation in tandem, not in sequence. It doesn't find the nearest vectors and then discard the ones that fail your filter. It scans candidate vectors and checks similarity and the filter condition together.

This is the better design for most cases (the docs note it's "more likely to find matching results" than filtering after the search), but it has a consequence worth understanding.

Similarity search over a large-scale index is approximate (ANN). To stay fast, the engine explores a bounded set of candidate vectors near your query rather than scanning everything. Think of it like searching a library by walking the nearby shelves rather than checking every book in the building. Now add a filter. Every candidate that fails the filter is a candidate that didn't count toward your K results. If your filter rejects most of what's in the neighborhood the engine explores, it can exhaust its candidate budget before finding K vectors that both match the filter and are close to your query.

Two things follow, and they're distinct:

1. If matching vectors are rare in the whole index, you get back fewer than K results. This is the case the docs mention.
2. Long before that, you can get a full K results back that are worse than the true top K, because the close-and-matching vectors fell outside the explored candidate set. This one is less obvious from the documentation, and it's the one worth understanding.

The severity depends on whether the vectors that match your filter are correlated with similarity to your query. If matches are spread evenly through the vector space, the filter behaves well. If the matching vectors happen to sit far from where your queries land, filtering works against you. It's like fishing in a lake where the species you want only lives on the far shore: your net works fine, but you're casting it in the wrong spot.

Measuring it

To show this cleanly I built two synthetic corpora, each one million 768-dimensional vectors, L2-normalised, stored in S3 Vectors with the cosine metric. I ran 200 queries and compared every result against exact, brute-force top-10 computed offline with NumPy. That exact set is the recall denominator. Recall@10 is the fraction of the true top-10 that a filtered query returned.

I controlled two variables independently:

• Selectivity: what fraction of the corpus the filter keeps. I swept it on a log scale (50%, 10%, 1%, 0.1%, 0.01%). By assigning filter buckets by similarity rank, the selectivity stays identical across layouts, so the only thing that changes is which vectors match.
• Layout: how matches relate to query position. Uniform means the filter field is assigned independently of position (the benign case). Adversarial means matches are placed away from where queries land, a constructed worst case rather than a proven ceiling. Real workloads should fall between these two.

I also built two corpora with different geometries, for an important reason. Approximate search has its own baseline recall gap even with no filter at all. To attribute a recall drop to filtering, you need to know the no-filter baseline:

• Separable corpus (4,096 well-separated clusters): unfiltered recall@10 = 0.86. A clean control where most true neighbors are recoverable before any filter.
• Realistic corpus (64 broad, overlapping clusters): unfiltered recall@10 = 0.55. Closer to real embeddings, where ANN already misses some true neighbors and filtering stacks on top.

Selectivity drives recall down

Recall@10 as the filter tightens, for both corpora and both layouts:

Selectivity	Separable, uniform	Separable, adversarial	Realistic, uniform	Realistic, adversarial
No filter (baseline)	0.86	0.86	0.55	0.55
50%	0.86	0.64	0.52	0.21
10%	0.83	0.39	0.48	0.19
1%	0.42	0.27	0.39	0.16
0.1%	0.23	0.19	0.28	0.14
0.01%	0.15	0.14	0.15	0.12

On the separable corpus, the uniform filter is almost free at loose settings: recall holds at 0.86 at 50% selectivity and 0.83 at 10%. Then it drops. 0.42 at 1%. 0.23 at 0.1%. 0.15 at 0.01%. The adversarial layout never gets an easy ride: 0.64 at 50%, 0.39 at 10%, down to 0.14. Same selectivity, same K, same vectors. The only difference is whether matches correlate with query position.

The realistic corpus tells the same story from a lower starting point. Because the no-filter baseline is already 0.55, the filtered numbers are lower throughout, and the adversarial layout sits at roughly half the uniform layout at every selectivity. The shape is identical; the effect stacks on top of the ANN baseline gap.

The takeaway isn't the absolute numbers (those depend on my synthetic construction). It's the shape: recall is roughly flat for loose filters and falls as the filter tightens, and how fast it falls depends on whether your filter correlates with similarity.

Result count and recall fail separately

This is the part the documentation we are going deeper on. The table below shows the average number of results returned alongside recall@10 for the same queries, on the separable corpus with a uniform layout. The most favorable test case:

Selectivity	Avg results returned	Recall@10
50%	10.0	0.86
10%	10.0	0.83
1%	10.0	0.42
0.1%	10.0	0.23
0.01%	7.7	0.15

The result count stays pinned at the full 10 across every selectivity until the most extreme setting (0.01%), where matching vectors become rarer than 10 in the entire corpus and the count dips to around 8. Recall, meanwhile, has been trending down the entire time.

The consequence is easy to miss. A filtered query can return a full set of K results that are mostly the wrong results. If you use "did I get K results back?" as a health check, everything looks fine while quality has already degraded. A result count tells you whether enough vectors matched the filter. It tells you nothing about whether they were the right ones.

Partitioning recovers what filtering loses

If a selective filter on a field hurts recall, what if that field stopped being a runtime filter and became the index itself? I tested this with a low-cardinality field (16 values). Two designs for the same logical query "find similar vectors where category = c":

• Filter: one shared index, query with a category = c filter.
• Partition: one index per category value, query the relevant index with no filter.

Across four representative categories, the dedicated index beat the runtime filter on recall every time, by 15 to 31 points:

Category	Filter recall@10	Partition recall@10	Filter latency	Partition latency
0 (far band)	0.35	0.66	62 ms	105 ms
5	0.50	0.74	63 ms	107 ms
10	0.68	0.84	63 ms	107 ms
15 (near band)	0.76	0.91	63 ms	102 ms

The gap is largest for category 0 (the band placed farthest from queries, the adversarial case) and smallest for the near band. That's the mechanism again: filtering hurts most when matches are far from the query, and partitioning removes the filter entirely so that pressure disappears.

It isn't free. The partition queries ran about 40 ms slower because each hits a separate, less-warmed index: the per-index fixed cost and cold cache outweigh any benefit from the smaller index size. That's the trade: recall for latency, plus the operational cost of more indexes to manage.

Guidance: getting the most out of S3 Vectors filtering

Everything above leads to a handful of design rules. I could have posted these as a tl;dr right at the top but you needed to read about my hard work.

Design your metadata schema carefully

• Make a field filterable only if you'll filter on it. Filterable metadata is capped at 2 KB per vector; non-filterable metadata counts toward the total 40 KB budget per vector. Store source text, chunks, URLs, and descriptions as non-filterable. Reserve filterable metadata for the handful of fields you query on.
• Decide non-filterable keys at index creation. Non-filterable keys are immutable once the index exists; you can't convert a field later. Plan the schema before you create the index.

Construct queries with the right expectations

• Expect recall to fall as filters get more selective. A filter that keeps half the corpus barely moves recall; one that keeps a fraction of a percent can halve it or worse. Budget for it when you choose how aggressively to filter.
• Don't trust result count as a quality signal. A filtered query can return a full K results that are mostly wrong. Validate recall against a known-good sample rather than assuming a full result set means a good one.
• Prefer coarse filters when you can. If a broad filter and a narrow one both satisfy your need, the broad one preserves more recall. Push fine-grained distinctions into re-ranking after retrieval rather than into the filter itself.

Lay out indexes around your filters

This is the strongest signal that came out of my testing.

• Promote a low-cardinality, almost-always-present filter field to an index boundary. If a field has natural, low cardinality (language, region, tenant, product line), queries nearly always constrain on it, and recall matters, give each value its own index and skip the runtime filter. I measured 15–31 points of recall recovered this way.
• Apply the rule; don't overuse it. Partition when the field is low-cardinality, queries reliably constrain on it, and the recall gain outweighs the latency cost I saw (~40 ms per query). Don't partition on high-cardinality fields: a vector bucket allows up to 10,000 indexes, and over-partitioning fragments your vectors across many small indexes, which can hurt ANN quality and operational simplicity.
• Use multi-tenancy as the easy case. If you already query each tenant independently, a per-tenant index is the same pattern and gives you data isolation for free via IAM.

The two query shapes look like this with boto3. A filtered query on a shared index, versus an unfiltered query against a per-value index:

import boto3
client = boto3.client("s3vectors", region_name="eu-west-1")

# Filter: one shared index, category applied as a runtime filter
resp = client.query_vectors(
    vectorBucketName="my-vectors",
    indexName="documents",
    queryVector={"float32": embedding},
    topK=10,
    filter={"category": {"$eq": "finance"}},   # filtering requires QueryVectors + GetVectors
    returnMetadata=True,
)

# Partition: a dedicated index per category value, no filter needed
resp = client.query_vectors(
    vectorBucketName="my-vectors",
    indexName="documents-finance",            # route to the right index
    queryVector={"float32": embedding},
    topK=10,
    returnMetadata=True,
)

One IAM note from that first call: using filter (or returnMetadata) requires both s3vectors:QueryVectors and s3vectors:GetVectors. Without GetVectors, a filtered query returns 403.

Operational notes

A few things that bit me while running these tests. They are in the docs but I skimmed over them too quickly. First, keep ingestion batched at 500 vectors per request and at or under the 2,500 vectors/s-per-index limit (I sustained ~2,480/s, so budget roughly 400 seconds per million vectors per index). Second, index names reject underscores: documents_finance fails with "Invalid index name", but documents-finance works fine. Third, the 2 KB filterable-metadata cap is a hard limit I confirmed; 1,811 bytes went in fine, 2,211 bytes was rejected with ValidationException: Filterable metadata must have at most 2048 bytes, while a 20 KB text chunk stored as non-filterable metadata on the same vector succeeded without complaint (the overall cap is 40 KB per vector). The two metadata classes exist for exactly this split.

Closing

S3 Vectors gives you cost-effective vector storage with metadata filtering built in. The filtering is powerful, but it isn't a free post-processing step bolted onto the search. It's evaluated inside the approximate search, and that's why selective filters cost recall. Once you understand that, the design choices flow naturally: keep filterable metadata lean, expect and measure recall under filtering, and when a filter field has natural low cardinality, turn it into an index boundary rather than a runtime filter condition.

This was an interesting set of tests to run, and I hope the results save you time when integrating S3 Vectors into your workflow.

Methodology

Results from two synthetic corpora of 1,000,000 L2-normalised 768-dimensional vectors each, cosine distance, 200 queries, recall@10 measured against exact brute-force top-10. Selectivity was controlled by similarity rank so it stays identical across layouts; the "adversarial" layout is a constructed worst case and the "uniform" layout is the benign case, with real workloads expected to fall between them. Absolute recall values are artifacts of the synthetic construction; the shapes and relative gaps are the transferable findings. Limits referenced (2 KB filterable metadata, 40 KB total, 2,500 vectors/s per index, 10,000 indexes per bucket) are from the S3 Vectors limitations page; filtering behavior from the metadata filtering page.