Efficient UI testing series

• Introduction
• Local and Instrumentation tests in Android
• Android Studio and Android SDK tools
• UI Testing Frameworks
• Exploring "Espresso Test Recorder"
• First test case with Kakao framework
• Testing Android Fragment in isolation
• Exploring test application and first test cases
• A domain-specific language for testing
• Introducing UI tests with mocking
• Flaky tests
• Achieving efficiency in tests

Testing allows us to improve the quality of the product because after any changes you can be sure that everything that is covered by test cases works correctly.

We can test different parts of applications, like business logic, User Interface, and so forth. Let us start with an overview of two categories of tests which are specific for the Android platform. It's local and instrumentation test cases.

Local tests can be run on local JVM(Java Virtual Machine). It means that local tests cannot be run on different devices. These tests are mostly used for business logic verification with or without interaction with Android components. In the case when we have them, and we would like to use local tests we can use a Robolectric framework.

Instrumentation tests require a device or emulator. These tests are much slower than local ones because we should interact with a real device. In practice, developers often use this category of tests for verification User Interface in isolation or E2E(End-To-End) tests.

All instrumentation tests cases can be split into UI and Non-UI instrumentation tests.

UI instrumentation tests are usually used for verification:

• User Interface
• Interaction with screens (Activities, Fragments)
• Activity/Fragment in isolation
• Navigation between Activities/Fragments

Non-UI instrumentation tests are usually used for verification:

• Interaction with a database on device(s)
• Interaction with services on device(s)
• Interaction with resources on device(s)
• Interaction with files on device(s)

In practice, we should combine them for efficient testing of Android applications.

One of the main specifications of Android is fragmentation. Even nowadays you can find many devices with different versions of Android, and the main reason is that many vendors produce devices with their software and many devices look the other way. Fortunately, lastly, Google introduced "Android One" program which allows installing default Android without any modification of core to devices and few vendors join to this program, and we can see devices with pure Android from Google, Nokia, some devices from Xiaomi, Motorola, etc.

Let us take a look at the Android Dashboard.

More information you can find on an Android Dashboards page.

As you can see we have a lot of devices with different versions of Android OS with more than 10% on market:

• Lollipop (5.1, API: 21) – 14.4%
• Marshmallow (6.0, API: 23) – 21.3%
• Nougat (7.0, API: 24) – 18.1%
• Nougat (7.1, API: 25) – 10.1%
• Oreo (8.0, API: 26) – 14.0%

It means that we should support many different Android versions and verify applications on different devices with other versions of Android OS. In addition to it, you can face a situation when you have issues applicable only for devices from specific vendors, like Samsung, Xiaomi, and others.

As a result, different versions of low-level components depend on a version of OS. I mean SQLite database and OpenGL. Different Android versions use other versions of SQLite. Fortunately, even old version of SQLite is stable, but a newer version can have some feature which can improve your development process. OpenGL also connected with hardware, and for some rendering process you can have limitation from hardware and don’t support devices with a low version of OpenGL.

Instrumentation UI and Non-UI tests allow you to run test cases on different devices simultaneously. It can give you many benefits from a testing perspective.

In this series of articles, we will mostly talk about UI tests and archiving efficiency for out test cases. The whole series built with a fail-safe approach; it means that we will add efficiency after building “not ideal” test cases.

I had a presentation about "Efficient UI testing Android apps" and video is available.

In this series, I would like to add more information about this topic.
However, if you have any question connected with an article or a video,
please let me know.