We’re reviewing how SQL Server Agent jobs should be scheduled in systems where workload is intermittent.
In some environments, jobs are configured to run at very frequent intervals (for example every 1–5 minutes) regardless of whether there is actual work to process. As the number of processes grows, this can lead to redundant executions, unnecessary CPU/memory usage, and occasional resource spikes that affect scalability.
Has anyone implemented a pattern where SQL Agent jobs are dynamically triggered or conditionally enabled only when work is present, instead of relying on fixed schedules?
Curious what approaches people are using — control tables, queues, event-driven triggers, external orchestration, or something else.
We explored one such approach while troubleshooting performance during a migration, and documented the experience here if useful for context:
https://medium.com/towards-data-engineering/why-migrating-ssis-from-ec2-to-amazon-rds-is-harder-than-it-looks-525ea6593c87
