Aws MSK security behaviour when both IAM and SCRAM enabled

Question

What is the Authn and Authz mechanism when both IAM as well as SASL/SCRAM are enabled?

| **Authn & Authz mech** | MSK authn | MSK authz | Kafka client authn | Kafka client authz | Kafka ACL behaviour | Property allow.everyone.if.no.acl.found |
| --- | --- | --- | --- | --- | --- |
| SASL/IAM clients | IAM | IAM | SASL/IAM | IAM | No effect | No effect |
| SASL/SCRAM clients | IAM | IAM | SASL/SCRAM | **?** | **?** | **?** |

Can someone please clarify about the question marks in the table above

Amazon MSK allows using both IAM as well as SASL/SCRAM together in the same cluster. 
The behaviour of Authn and Authz for Kafka API is not clear when both IAM role-based and SASL/SCRAM are enabled.

1. For SASL/SCRAM clients, Does that mean we can we use SASL/SCRAM for authn and IAM for authorisations at topic level? (The documentation says Kafka ACLS have no effect when IAM authentication is enabled for an MSK cluster)

2. Or does that mean (that for SASL/SCRAM clients) we use SASL/SCRAM for authn but there is no authz at topic level (because Kafka ACLs do not work)?

3. Is it a bug that aws-msk allows both mechanisms and should really allow only one of them in a cluster?

These are not clear from the documentation unfortunately.

Accepted Answer

Hi there, when multi authentication is enabled on MSK cluster authorization depends on which of those access control methods a client is using to access MSK cluster.

Let's take the above example where both IAM and SASL/SCRAM is enabled and say 'client A' is accessing MSK cluster via IAM auth and 'client B' is accessing cluster via SASL/SCRAM. You can still invoke Apache Kafka ACL APIs and add ACLs for an MSK cluster that uses IAM access control but ACLs stored in Apache ZooKeeper have no effect on authorization for IAM roles. So access/authorization for 'client A' which uses IAM auth will be controlled by the IAM policy as ACLs have no effect in the case of 'client A' even though those are added.

But when a client is using  non-IAM auth, these added ACLs (and allow.everyone.if.no.acl.found) will have effect i.e authorization in that case will be controlled by ACLs. So when 'client B' which uses SASL/SCRAM tries to perform any operations it validates against the ACLs that were added.

In short, to fill in the gaps in above table you mentioned -

| Authn & Authz mech | Kafka client authn | Kafka client authz | Kafka ACL behaviour | Property allow.everyone.if.no.acl.found | 
| --- | --- | --- | --- | --- | --- |
| SASL/IAM clients | SASL/IAM | IAM | No effect | No effect | 
| SASL/SCRAM clients | SASL/SCRAM | ACLs | Applies/Does have an effect | Applies/Does have an effect |

Answer

For the followup question "What I am not clear about is Zookeeper authn and authz. Is that always TLS or is it possible to have SCRAM or IAM for Zookeeper?"

Currently zookeeper uses TLS only for encryption but not for authentication. Zookeeper doesn't support any access control methods at the moment. The best security practice is to place zookeeper nodes in a separate security group (1) and block/control access using inbound rules so that users can perform any kafka data plane actions like reading, writing, creating topics, listing topics etc using only brokers/bootstrap server. That way users/clients must be authenticated and authorized to perform those actions.

(1) https://docs.aws.amazon.com/msk/latest/developerguide/zookeeper-security.html

Aws MSK security behaviour when both IAM and SCRAM enabled

Relevant content