SageMaker Canvas - Auto Update

Based on the information provided, there are several potential issues that could be affecting your SageMaker Canvas Auto Update and Automatic Prediction features:

1. Login Requirement: SageMaker Canvas automations (including automatic dataset updates and batch predictions) only run while you're logged into the Canvas application. They pause when you log out. This could explain why no jobs are being triggered if no one is actively logged into Canvas.

2. Job History Visibility Delay: The job history for automatic dataset updates is not updated in real-time. There's typically a delay of approximately 5-10 minutes between when a job runs and when it appears in the interface. If you're checking immediately after the expected run time, you might not see the job in the history yet.

3. S3 Permissions: While you've set up a VPC endpoint for S3 to allow your Lambda function to write to the bucket, Canvas might need specific permissions to read from that S3 location. Ensure that the SageMaker execution role has proper permissions to access the S3 bucket where your data is stored.

4. File Schema Issues: For automatic dataset updates, Canvas doesn't perform schema validation. If the schema of your new files doesn't match the schema of the existing dataset, you might encounter errors when jobs run. Check that your Lambda function is generating CSV files with consistent schema.

5. File Overwriting Behavior: When Canvas imports files during an automatic update, it overwrites old files with new files if they have the same names. If your Lambda function is not generating files with new names or modifying existing files, Canvas might not detect changes.

To troubleshoot:

• Verify someone is logged into Canvas when updates should occur
• Check the job history in both the dataset's "Auto updates" tab and in the "ML Ops" section under "Automations" tab and "All jobs" sub-tab
• Review your S3 bucket permissions to ensure Canvas has access
• Confirm your CSV files maintain a consistent schema
• Verify that new data is actually being written to the S3 location at the expected times

Remember that when a dataset is automatically updated, models do not automatically rebuild with the new data. You would need to manually rebuild your model, though automatic batch predictions can still run on the updated dataset using your existing model.
Sources
Configure automatic updates for a dataset - Amazon SageMaker
sagemaker canvas dataset auto update, when is job history updated? | AWS re:Post
if sagemaker dataset gets a new version automatically, does the model automatically rebuild? | AWS re:Post
View your automatic dataset update jobs - Amazon SageMaker