How do I add periodic tasks to the worker tier of my Elastic Beanstalk environment and troubleshoot issues when periodic tasks don't appear?

3 minute read
0

I want to add periodic tasks to the worker tier of my AWS Elastic Beanstalk environment. I also want to troubleshoot issues with periodic tasks that don't appear.

Resolution

Note: Elastic Beanstalk uses the aws-elasticbeanstalk-ec2-role Amazon Elastic Compute Cloud (Amazon EC2) instance profile that includes the AWSElasticBeanstalkWebTier, AWSElasticBeanstalkWorkerTier, and AWSElasticBeanstalkMulticontainerDocker managed permissions. AWSElasticBeanstalkWorkerTier is required for periodic tasks to work.

Add periodic tasks to a worker tier

To add a periodic task to the worker tier of your Elastic Beanstalk environment, include a cron.yaml file at the root of your source code.

Example:

version: 1
cron:
 - name: "schedule"
   url: "/schedule"
   schedule: "0 */12 * * *"

Note: In the preceding example, the schedule task periodically sends an Amazon Simple Queue Service (Amazon SQS) message that describes the task. The schedule task sends the message to the beginning of the queue every 12th hour. After the message crosses the queue, an instance picks up the task to send an HTTP POST request to the "/schedule" endpoint.

Troubleshoot issues with periodic tasks

If your periodic tasks don't appear in the worker tier after you add them, then complete the following steps:

  1. In your /var/log/aws-sqsd/default.log file, check for a log that's similar to the following:

    schedule-parser: Successfully loaded 1 scheduled tasks from file /var/app/current/cron.yaml .
  2. In your /var/log/aws-sqsd/default.log file, confirm that the Amazon SQS daemon is running and polling the correct queue.
    If the Amazon SQS daemon starts correctly, then you see logs that are similar to the following:

    2023-01-17T03:28:31Z init: initializing aws-sqsd 3.0.4 (2022-03-18)
    2023-01-17T03:28:31Z schedule-parser: Successfully loaded 1 scheduled tasks from file /var/app/current/cron.yaml .
    2023-01-17T03:28:31Z leader-election:  initialized leader election
    2023-01-17T03:28:31Z scheduler: initialized 1 job's pending time
    2023-01-17T03:28:31Z pollers: start initializting poller timer...
    2023-01-17T03:28:31Z pollers: start auto running poller...
    2023-01-17T03:28:31Z leader-election:  Starting leader election
    2023-01-17T03:28:31Z leader-election:  current role: worker
    2023-01-17T03:28:31Z scheduler: Starting scheduler
    2023-01-17T03:28:31Z start: polling https://sqs.us-east-1.amazonaws.com/111122223333/yourWorkerQueue
  3. In your /var/log/aws-sqsd/default.log file, check for the HTTP POST request that the Amazon SQS daemon sends to http://localhost/ on port 80. The daemon sends the request when it polls the queue item.
    You see a message that's similar to the following:

    2019-07-05T13:54:52Z message: sent to http://localhost:80/
  4. To confirm that the SQS queue receives the messages, check the NumberOfMessagesReceived Amazon CloudWatch metric for the queue that's associated with the worker environment.

  5. To manually invoke the scheduled task, run the following command:

    curl -d param1=value1 -d param2=value2 -H "Content-Type: application/json" -X POST http://localhost/scheduled-task-path

    The -d (--data) flag can hold the data that you're trying to pass to the periodic task.

Note: If you use Amazon SQS FIFO queues to configure your worker environment, then periodic tasks don't work.

Related information

How do I create cron jobs on Amazon EC2 instances in Elastic Beanstalk environments?

AWS OFFICIAL
AWS OFFICIALUpdated 4 months ago
3 Comments

2024-06-24T07:02:16Z schedule-parser: Successfully loaded 1 scheduled tasks from file /var/app/current/cron.yaml . 2024-06-24T07:02:16Z init: missing required parameter params[:table_name] (ArgumentError) at /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/3.1.0/gems/aws-sdk-core-2.11.632/lib/aws-sdk-core/param_validator.rb:28:in `validate!'

what is the reason i get this error? the Sqld in the first place loads a scheduled task from the cron.yaml but instantly it fails the cronjob leader election. When i check the dynamodb, no tables are created (which results in the missing required params error) for a test app with only one router (which logs hello world) works well but the actual express.js app that i deploy to EB worker sends me this error and fails its deploy process. Could it be a memory issue? the exact same code that i deploy to eb web tier (without cron.yaml) cause no error. Why is dyanmodb table for leader election not creating?? (already satisfies all IAM roles to the instance)

replied 8 months ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied 8 months ago