How to match responses to requests when multiple devices use the 'Device provisioning MQTT API' at same time?

Hi. Those topics are not regular MQTT pub sub topics. AWS IoT Core tracks the requester, and sends the response only on the same connection from which the request came. Other devices can't subscribe to receive the response. And they can't subscribe to receive a request from any other device (requests only route to the fleet provisioning service).

You've raised some important points about using the Device provisioning MQTT API, particularly when multiple devices are attempting to provision simultaneously. Let's address your concerns:

1. Matching responses to requests: When using the CreateKeysAndCertificate API, you're correct that there isn't a built-in request ID mechanism to easily match responses to specific requests. However, there are a few strategies you can employ:

a. Unique Client IDs: Ensure that each device uses a unique MQTT client ID when connecting to AWS IoT. This helps prevent connection conflicts and allows for better tracking of individual device communications.

b. Timing and Public Key Matching: While it may seem like extra work, matching the public key in the response to the one in the request is a reliable method to ensure the correct response is processed by each device. Given the security implications of provisioning, this additional step is justified.

2. Handling error responses: You're correct that the error responses on the rejected topic don't include identifying information. To mitigate this:

a. Implement a timeout mechanism: If a device doesn't receive a response on the accepted topic within a certain timeframe, it can check the rejected topic for any errors.

b. Retry with backoff: If an error is received or no response is detected, implement a retry mechanism with an exponential backoff to reduce the likelihood of collisions.

3. Scaling for industrial applications: For scenarios where hundreds of devices may attempt to provision simultaneously:

a. Implement a staggered provisioning approach: Program devices to attempt provisioning at slightly different times or with random delays to reduce simultaneous requests.

b. Use Fleet Provisioning Templates: These allow you to define the provisioning process more precisely and can help manage large-scale provisioning more effectively.

c. Consider using the RegisterThing API instead of CreateKeysAndCertificate if your use case allows. RegisterThing provides more structured provisioning and includes a templateName in the response, which can be used for correlation.

While the current API design may present some challenges for high-volume, simultaneous provisioning, these strategies should help manage the process more effectively. Remember to thoroughly test your provisioning process under various conditions to ensure reliability at scale.

It's worth noting that before publishing any request message, you should always subscribe to the corresponding response topics first. This ensures that you don't miss any responses due to timing issues.
Sources
Device provisioning MQTT API - AWS IoT Core
Device provisioning - IoT Lens
Provisioning identity in AWS IoT Core for device connections - Device Manufacturing and Provisioning with X.509 Certificates in AWS IoT Core
MQTT multiple subscribers to one topic not working | AWS re:Post

I erroneously wrote that I was using the CreateKeysAndCertificate API in my original post when I am actually using the CreateCertificateFromCsr API. I corrected my post, but the AI-generated response appeared before I made the edit. The core of my question also applies to the CreateKeysAndCertificate API, as both API calls suffer from the same problem.

The AI-generated response is what I was already considering (implementing), namely filtering on public keys and using timeouts. I am just curious if there are alternative approaches.