Forwarding log messages to log file using step function

Hi there,

We are migrating from datapipeline to step function, and on datapipeline I am able to forward the log messages for individual steps to a log file, but this doesn't work on step function. Am I missing something in the configuration?

On datapipeline I have /home/hadoop/general/ProfileData.sh $1 > >(tee -a /home/hadoop/myLogFile.log) -- Worked

On step function I have "Args.$": "States.Array('bash', '/home/hadoop/general/ProfileData.sh', $.argument1, ' > >(tee -a /home/hadoop/myLogFile.log)' " --Not works

トピック

分析サーバーレスアプリケーション統合

タグ

AWS Data Pipeline AWS Step Functions

言語

English

rePost-User-9449669

質問済み 1年前421ビュー

1回答

新しい順
投票が多い順
コメントが多い順

What integration are you calling from Step Functions?

Usually, in distributed systems like this, it is best to send the logs outside the host that generated them because you may not have access to it later. In AWS, we send logs to CloudWatch Logs.

You can then create a CloudWatch Logs subscription that will send the logs to a file in S3.

エキスパート

Uri

回答済み 1年前

Justin Callison
1年前
You can find documentation to enable CloudWatch Logs for Step Functions below. When you enable this, each History Event for your workflow executions will be logged to the log group you have specified.

https://docs.aws.amazon.com/step-functions/latest/dg/cw-logs.html

Forwarding log messages to log file using step function

関連するコンテンツ