AWS CLI filter bug - filters being OR'ed, not AND'ed

0

When using the CLI to query subnets, filters are being OR'ed, while the Filter documentation clearly states that they are supposed to be AND'ed:

If you specify multiple filters, the filters are joined with an AND, and the request returns only results that match all of the specified filters.

For example (VPC and subnet ids have been anonymized):

aws --region eu-west-1 ec2 describe-subnets  --filter Name=vpc-id,Values=vpc-1 --filter Name=tag:Type,Values=private | jq  '[ .Subnets[] | select(.Tags[].Key == "Name") | select(.Tags[].Value | contains("default")) | {id: .SubnetId, vpc: .VpcId, name: (.Tags[]|select(.Key=="Name")|.Value), type: (.Tags[]|select(.Key=="Type")|.Value) } ]'

This is supposed to give back subnets that have VpcId "vpc-1" AND tag:Type "private", but instead I get a list of subnets from 3 different VPCs

[
  { "id": "subnet-1", "vpc": "vpc-1", "name": "private/default/a", "type": "private" },
  { "id": "subnet-2", "vpc": "vpc-1", "name": "private/default/b", "type": "private" },
  { "id": "subnet-3", "vpc": "vpc-1", "name": "private/default/c", "type": "private" },
  { "id": "subnet-4", "vpc": "vpc-2", "name": "private/default/a", "type": "private" },
  { "id": "subnet-5", "vpc": "vpc-2", "name": "private/default/b", "type": "private" },
  { "id": "subnet-6", "vpc": "vpc-2", "name": "private/default/c", "type": "private" },
  { "id": "subnet-7", "vpc": "vpc-3", "name": "private/default/a", "type": "private" },
  { "id": "subnet-8", "vpc": "vpc-3", "name": "private/default/b", "type": "private" },
  { "id": "subnet-9", "vpc": "vpc-3", "name": "private/default/c", "type": "private" }
]

I saw this behavior on CLI v.2.8.13, and then upgraded to 2.11.11 and am still seeing it.

I tried the exact same query in Python using boto3, and got the correct behavior:

>>> client = boto3.client(service_name='ec2', region_name='eu-west-1')
>>> for subnet in client.describe_subnets(Filters=[{'Name': 'vpc-id', 'Values': ['vpc-1']}, {'Name': 'tag:Type', 'Values': ['private']}])['Subnets']:
...     name = [tag['Value'] for tag in subnet['Tags'] if tag['Key'] == 'Name']
...     type = [tag['Value'] for tag in subnet['Tags'] if tag['Key'] == 'Type']
...     print(f"id={subnet['SubnetId']}, vpc={subnet['VpcId']}, {name=}, {type=}")
...
id=subnet-1, vpc=vpc-1, name=['private/default/a'], type=['private']
id=subnet-2, vpc=vpc-1, name=['private/default/b'], type=['private']
id=subnet-3, vpc=vpc-1, name=['private/default/c'], type=['private']
asked a year ago376 views
2 Answers
0
Accepted Answer

Put them under one filter to get the AND:

aws --region eu-west-1 ec2 describe-subnets  \
--filter Name=vpc-id,Values=vpc-1 Name=tag:Type,Values=private \
| jq  '[ .Subnets[] | select(.Tags[].Key == "Name") | select(.Tags[].Value | contains("default")) | {id: .SubnetId, vpc: .VpcId, name: (.Tags[]|select(.Key=="Name")|.Value), type: (.Tags[]|select(.Key=="Type")|.Value) } ]'

The AND behavior is documented and the reference to multiple filters makes sense in the context of a list:

If you specify multiple filters, the filters are joined with an AND , and the request returns only results that match all of the specified filters.

What seems to be undocumented is the OR with multiple --filters.

profile pictureAWS
EXPERT
kentrad
answered a year ago
  • Thanks! That's not at all clear from the documentation. To my mind, --filter <filter1> --filter <filter2> should either give an error or should work exactly the same as --filter <filter1> <filter2>. Most shell commands I'm used to take the former approach (e.g., sed -e 'script1' -e 'script2', not sed -e 'script1' 'script2'), so it actually never occurred to me to just put multiple filters after one --filter.

0

I think the last filter takes precedence, out of multiple --filter options. Instead of

--filter Name=vpc-id,Values=vpc-1 --filter Name=tag:Type,Values=private

Try

--filter Name=vpc-id,Values=vpc-1 Name=tag:Type,Values=private
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions