Forwarding log messages to log file using step function

Hi there,

We are migrating from datapipeline to step function, and on datapipeline I am able to forward the log messages for individual steps to a log file, but this doesn't work on step function. Am I missing something in the configuration?

On datapipeline I have /home/hadoop/general/ProfileData.sh $1 > >(tee -a /home/hadoop/myLogFile.log) -- Worked

On step function I have "Args.$": "States.Array('bash', '/home/hadoop/general/ProfileData.sh', $.argument1, ' > >(tee -a /home/hadoop/myLogFile.log)' " --Not works

主题

分析无服务器应用程序集成

标签

AWS 数据管道 AWS Step Functions

语言

English

rePost-User-9449669

已提问 1 年前421 查看次数

1 回答

最新
投票最多
评论最多

What integration are you calling from Step Functions?

Usually, in distributed systems like this, it is best to send the logs outside the host that generated them because you may not have access to it later. In AWS, we send logs to CloudWatch Logs.

You can then create a CloudWatch Logs subscription that will send the logs to a file in S3.

专家

Uri

已回答 1 年前

Justin Callison
1 年前
You can find documentation to enable CloudWatch Logs for Step Functions below. When you enable this, each History Event for your workflow executions will be logged to the log group you have specified.

https://docs.aws.amazon.com/step-functions/latest/dg/cw-logs.html

Forwarding log messages to log file using step function

相关内容