I want to add periodic tasks to the worker tier of my AWS Elastic Beanstalk environment. I also want to troubleshoot issues with periodic tasks that don't appear.
Resolution
Note: Elastic Beanstalk uses the aws-elasticbeanstalk-ec2-role Amazon Elastic Compute Cloud (Amazon EC2) instance profile that includes the AWSElasticBeanstalkWebTier, AWSElasticBeanstalkWorkerTier, and AWSElasticBeanstalkMulticontainerDocker managed permissions. AWSElasticBeanstalkWorkerTier is required for periodic tasks to work.
Add periodic tasks to a worker tier
To add a periodic task to the worker tier of your Elastic Beanstalk environment, include a cron.yaml file at the root of your source code.
Example:
version: 1
cron:
- name: "schedule"
url: "/schedule"
schedule: "0 */12 * * *"
Note: In the preceding example, the schedule task periodically sends an Amazon Simple Queue Service (Amazon SQS) message that describes the task. The schedule task sends the message to the beginning of the queue every 12th hour. After the message crosses the queue, an instance picks up the task to send an HTTP POST request to the "/schedule" endpoint.
Troubleshoot issues with periodic tasks
If your periodic tasks don't appear in the worker tier after you add them, then complete the following steps:
-
In your /var/log/aws-sqsd/default.log file, check for a log that's similar to the following:
schedule-parser: Successfully loaded 1 scheduled tasks from file /var/app/current/cron.yaml .
-
In your /var/log/aws-sqsd/default.log file, confirm that the Amazon SQS daemon is running and polling the correct queue.
If the Amazon SQS daemon starts correctly, then you see logs that are similar to the following:
2023-01-17T03:28:31Z init: initializing aws-sqsd 3.0.4 (2022-03-18)
2023-01-17T03:28:31Z schedule-parser: Successfully loaded 1 scheduled tasks from file /var/app/current/cron.yaml .
2023-01-17T03:28:31Z leader-election: initialized leader election
2023-01-17T03:28:31Z scheduler: initialized 1 job's pending time
2023-01-17T03:28:31Z pollers: start initializting poller timer...
2023-01-17T03:28:31Z pollers: start auto running poller...
2023-01-17T03:28:31Z leader-election: Starting leader election
2023-01-17T03:28:31Z leader-election: current role: worker
2023-01-17T03:28:31Z scheduler: Starting scheduler
2023-01-17T03:28:31Z start: polling https://sqs.us-east-1.amazonaws.com/111122223333/yourWorkerQueue
-
In your /var/log/aws-sqsd/default.log file, check for the HTTP POST request that the Amazon SQS daemon sends to http://localhost/ on port 80. The daemon sends the request when it polls the queue item.
You see a message that's similar to the following:
2019-07-05T13:54:52Z message: sent to http://localhost:80/
-
To confirm that the SQS queue receives the messages, check the NumberOfMessagesReceived Amazon CloudWatch metric for the queue that's associated with the worker environment.
-
To manually invoke the scheduled task, run the following command:
curl -d param1=value1 -d param2=value2 -H "Content-Type: application/json" -X POST http://localhost/scheduled-task-path
The -d (--data) flag can hold the data that you're trying to pass to the periodic task.
Note: If you use Amazon SQS FIFO queues to configure your worker environment, then periodic tasks don't work.
Related information
How do I create cron jobs on Amazon EC2 instances in Elastic Beanstalk environments?