Android – Continous Integration

Configuring continuous integration for an Android application, including UI tests, can prove to be a challenge. Not only you will have to set up the standard JDK, but also download, create and start the Android Emulator. It can take a lot of time, especially if it's your first time. Even after a successful build, you might find that your tests take ages, while everything runs smoothly using a local emulator or a device. The exact configuration might differ depending on the CI service and e.g. docker image. Though, the basic steps for setting up an Android CI are usually the same:

1. Install Java JDK
2. Install Android SDK (accepting the licenses)
3. Download your code
4. Build the code
5. Run unit tests
6. Download Android Emulator
7. Create the emulator instance
8. Start and wait for the emulator (unlock the screen)
9. Run instrumentation tests
10. Gather the results (code coverage/errors/screenshots)

Some of these steps would need to be run only once depending on the choice of tools for the CI. Somewhere between these steps, it is also a good idea to stop for a moment and think about hardware acceleration.

Travis and CircleCI

Travis and CircleCI have quite nice options to run CI for your Android projects. It works flawlessly for unit tests. As for instrumentation tests, it may vary. A proper configuration might not be that trivial. Currently, we can read that:

[Tavis] Warning: At the moment, these steps are not fully supported by Travis CI Android builder.
[CircleCI] Note: Running the Android emulator is not supported by the type of virtualization CircleCI uses on Linux.

However, it's still possible to run the emulator using some workarounds, without having to rely on third-party services like Firebase Test Lab. For CircleCI there is a setup made by Dogu Deniz Ugur which facilitates the MacOS environment to run instrumentation tests. In the case of Travis, I haven't been too lucky with the default configuration from the docs nor any other from random articles.

If you encounter some block which you can't handle alone, it's a good idea to take a look at open-source projects which use the service of your choice. One of the examples is a quite popular app AnkiDroid with CI setup on Travis. In fact, a few times when my jobs started failing without any change in the code, I was able to pinpoint the problem by following Mike Hardy – one of the contributors to the AnkiDroid. In most of the cases, the problems were with the emulator channel or version upgrade.

Where to go next

After the initial configuration of Android CI you can further improve your setup:

1. Create test suites and parallelize tests if you have a lot of UI tests and they take a lot of time.
2. Implement test repetition to overcome test flakiness and stabilize the build.
3. Capture and upload screenshots during test failure to get additional information from the server on what needs to be fixed.

The last step may be setting Continuous Deployment (CD), e.g. automatic uploading of the application for internal testing on Google Play. In this case, Fastlane is the tool which can significantly facilitate this process.