- Newest
- Most votes
- Most comments
Hi.
Our method was to create jobs on a target that excludes devices in a static group "banlist". Then, for any device that needs to execute the job again we would simply add them to the banlist and remove from banlist afterwards.
I don't understand what you mean here. If "banlist" defines the devices to exclude, then I don't follow why you add devices to that group to make them execute the job again. I presume there's some mistake in the statement. Can you please clarify?
Could this matter be resolved if I awaited 10 seconds before removing the devices from banlist?
Why would you wait a certain amount of time, instead of just waiting until the job execution completes? I think there's some gap in the information provided.
A job's targets are an inclusive list of things and thing groups that the job should be sent to. Presuming the job is Continuous, if you remove a thing from a targeted thing group, it's job execution will transition to Removed. If you then re-add the thing to the thing group, a new job execution will be created for the thing.
The execution of jobs on target devices in AWS IoT Core works in a specific way, and understanding this can help address your issue with re-registering devices for job execution.
When a job is created, AWS IoT Jobs publishes a message for each target device on a specific MQTT topic. This message contains information about the job, including its ID and initial status. The job execution for a target device is initially set to a 'QUEUED' state.
The job execution remains in the 'QUEUED' state until one of three things happens:
- The device receives the job execution and reports its status as 'IN_PROGRESS'.
- The job or job execution is cancelled.
- The device is removed from the target group.
Your method of using a "banlist" group to control job execution is creative, but it may not be the most reliable approach due to the way AWS IoT Jobs handles targets. When you remove a device from the target group (by adding it to the banlist), its job execution status may change to 'REMOVED'. Simply removing it from the banlist later doesn't guarantee that a new job execution will be created for that device.
Instead of using a banlist approach, you might consider using the AWS IoT Jobs API directly to manage job executions for specific devices. You can use the CreateJobExecution API to create a new job execution for a specific device, even if it has already had a failed or rejected execution.
Regarding your questions about timing and bulk operations: While introducing delays or processing devices one by one might help in some cases, it's not a guaranteed solution. The core issue is more likely related to how job targets are managed rather than timing or bulk processing.
To have more control over retrying executions, you could:
- Use the CreateJobExecution API to create new job executions for specific devices.
- Use the DescribeJobExecution API to check the current status of job executions for specific devices.
- Use the UpdateJobExecution API to update the status of job executions as needed.
These methods would give you more precise control over job executions for individual devices, allowing you to implement your own retry logic while still leveraging the existing AWS IoT Jobs infrastructure.
Sources
Jobs key concepts - AWS IoT Core
Jobs workflow - AWS IoT Core
Jobs and job execution states - AWS IoT Core
answered 2 years ago
Relevant content
asked 3 years ago
asked 2 years ago
You got it right in the last statement. Banlist hack is to create a new job execution for the device. If there is a better way of doing that, please let me know. Essentially I am forcing the device to be removed from the target and add it back sometime later so the job executes again. The problem is, sometimes this doesn't happen. A new execution isn't created. It's not easy to detect this nor understand why. I hoped somebody could shed a light on this
In general, job executions shouldn't often fail or be rejected. Why are they rejected? What are the failure modes? I think you should try to reduce the incidence of both, if possible, and complete job executions within the maximum retries. If you do in fact exhaust retries, then remove and add is appropriate, and should work for a continuous job.
There are situations where the creation of the job execution may not occur right away. See the guidance here: https://docs.aws.amazon.com/iot/latest/developerguide/jobs-troubleshooting.html at "A job execution was not created for my thing or thing group". Are you perhaps adding it to the thing group while the continuous job is still in the concurrent (rollout) phase?