Hi Everyone,
I'm hitting a wall with AWS IoT MQTT-over-WebSockets and would love a second pair of eyes. Every WSS handshake against my account's IoT data endpoint comes back with HTTP 403 Forbidden and the body {"message":"Forbidden","traceId":"..."}. The header is x-amzn-ErrorType: ForbiddenException. This happens for every caller I've tried, including my own root-account IAM user that has AdministratorAccess attached. Because the WS upgrade never succeeds, the broker never sees the CONNECT packet, so I get nothing in CloudWatch v2 logs to look at.
Endpoint: aiqroq3b5zlot-ats.iot.us-east-2.amazonaws.com (returned by aws iot describe-endpoint --endpoint-type iot:Data-ATS)
Region: us-east-2
The weird part is that REST publishing works fine with the same credentials. aws iot-data publish to the same endpoint succeeds and lands a Publish-In Success line in AWSIotLogsV2 every time. So the credentials, the endpoint, the network path - all good for the data plane. Only the WSS handshake on /mqtt is rejected. A plain HTTPS GET on /mqtt with the same presigned URL returns 404 (which is expected since /mqtt is upgrade-only), so I know SigV4 is being validated correctly - it's not a signature problem.
Things I've ruled out:
IAM - aws iam simulate-principal-policy says iot:Connect is Allowed for both my admin user and a Cognito-Identity-Pool-authenticated identity, matched by AdministratorAccess in one case and a permissive inline policy in the other.
AWS IoT policy - I attached an IoT policy with iot:* on * to the Cognito identity. aws iot list-attached-policies --target <identityId> confirms it's there. Same 403.
Custom authorizer - aws iot describe-default-authorizer returns ResourceNotFoundException. aws iot list-authorizers returns an empty array.
Domain configurations - only the default iot:Data-ATS and iot:CredentialProvider are present.
SCP / permissions boundary - REST publish works with the same role, so nothing in the permission tree is blocking iot:* wholesale.
Signing code - I reproduced this from CloudShell using boto3 with my admin credentials, completely bypassing my app's SigV4 code. Same 403. So it isn't a bug in my client signing.
Header variations - I tried Sec-WebSocket-Protocol with mqtt, mqttv3.1.1, and no value. With and without the Origin header. Both the well-known sample Sec-WebSocket-Key and a fresh openssl rand -base64 16 value. All combinations produce the same 403.
Reproduction (CloudShell, admin user):
A short Python script using boto3 produces a SigV4 presigned URL for service iotdevicegateway, region us-east-2, on https://<endpoint>/mqtt with X-Amz-SignedHeaders=host. Then curl -i against that URL with Connection: Upgrade, Upgrade: websocket, Sec-WebSocket-Version: 13, a random Sec-WebSocket-Key, and Sec-WebSocket-Protocol: mqtt returns 403 every time.
Here are some traceIds from my recent failed handshakes if anyone from the IoT team can pull the broker-side denial reason:
8fd55d51-d4f2-dd1e-973a-02d33a34cca2 (admin user, random WS key, around 2026-04-30 08:06 UTC)
f85c44e4-5191-d615-02e3-f99479b6213a (admin user, around 2026-04-30 07:53 UTC)
7d192525-1db5-564e-350a-13f894d79037 (Cognito identity with permissive IoT policy and permissive IAM policy, around 2026-04-30 07:29 UTC)
57c89eb6-c80f-09bb-625c-461809aa6844 (same Cognito identity, mqttv3.1.1 subprotocol, around 2026-04-30 07:47 UTC)
v2 logging is set up correctly - aws iot get-v2-logging-options shows a roleArn, defaultLogLevel DEBUG, disableAllLogs false. I can see Publish-In events from the REST publishes, so the logging pipe works. I just never see any Connect events, which lines up with the broker never being reached.
So my question: with no custom authorizer, no SCP block, valid SigV4, and iot:Connect allowed by IAM, what could be causing iotdevicegateway to return 403 ForbiddenException on the WS upgrade specifically? Is there an account-level switch or hidden state I'm missing? Has anyone seen this pattern before?
Happy to provide more traceIds or run any specific test that helps narrow it down.
Thank you!
