I am using Appium and NodeJS in a custom test environment and am finding that when a test run fails it gets wrongly reported as a pass.
According to the docs for Custom Test Environments, "If any command in the test phase fails, the test is marked as failed.". But this doesn't seem to be true. My test command is node index.js $DEVICEFARM_APP_PATH which installs the app on the device etc etc, and when the test fails that node process will exit with code 1, i.e. a fail (any non-zero process exit code is a failure, right?). But it gets reported in the AWS console as a pass.
Is there some other way of reporting test fails that I've completely missed?
