Calabash android custom TestSpec unavailable


I've been trying to execute my calabash-android tests on Amazon Device Cloud for a while now but I've not been able to successfully reproduce my local setup.
I read in that:

Device Farm provides a default test spec for each supported test type
but when I create a TestSpec for my calabash android tests and try to upload it, I can't select the calabash testSpec because the option just isn't available.

I've uploaded it (From project settings > uploads) as external data but this doesn't allow me to select it from the jenkins AWS Device Farm plugin.

In conclusion: I would need to execute some calabash-android tests from Device Farm but I'm presented with a lot of unknowns, such as the ruby gems available in the system, the ruby version, project path and other variables I would expect.

If this is not the intended way to execute the calabash-android tests in Device Farm, can someone point me in the right direction?

asked 4 years ago9 views
3 Answers
Accepted Answer


DeviceFarm as of today does not support custom environment for calabash based tests.
That why you are not able to upload a test spec file.
Can you try running your tests in standard mode and let us know if that works

answered 4 years ago


Device Farm has launched support for running Appium ruby based tests. Please check our documentation for more details

Fo running your calabash tests, you can use the APPIUM_RUBY teststype.
U can customize the yaml file with the exact same set of commands as you do to run your tests locally.

Also , which gems and dependencies to install are all controlled by you.

Let me know if you need more help on this

answered 4 years ago

Thank you very much for your prompt response.
I will look into that solution and will get back to you if I have any other issues
Thank you very much!

answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions