Stable Android UI Testing

For the better part of the last year or so I have been working on a native mobile app for my LLC, Bytonomy, and so far it’s only included one complete rewrite (migrating from Xamarin.Forms to native iOS/Android). With a rewrite, one tends to learn a lot from their mistakes and look to migrate to an improved solution. For us, it was a mix of moving away from the limitations of Xamarin.Forms but also learn something new.

In the latest phase of development, we’re going through and applying some polish and stability. This meant adding new unit tests, finalizing user experiences, and testing these experiences. And as we all know testing is essential in uncovering potholes in business logic, and ensuring you have a stable product.

One of the greatest assets in any unit test framework and/or library is the ability to mock dependencies. Mocking allows the tester to dictate what a consumer is to receive, allowing for very robust and strong testing. But what happens when you have a dependency you can’t mock, i.e. your app’s network requests?

Integration and user interface (UI) tests, however, shouldn’t use mocks, but instead concrete implementations of dependencies. But what happens if the component needs to make a network request? What if the test device is having connectivity issues? What if the development or test is down? What if the API functionality just hasn’t been written yet?

I first encountered this issue when I was developing the UI tests for our iOS mobile app. Even though I have direct access to our backend services and API and can guarantee that they’re running (at least for testing :P), waiting for network requests to resolve can make my tests run slower, and can create instability. Luckily, I stumbled across this great article on Medium, which handed me a great solution. With Embassy and Ambassador, I was able to stand up a local server on the test device, which runs in parallel with my tests. The benefit of all this being that I can guarantee the API will be responsive, I can directly dictate what the API returns to test edge cases, and my tests aren’t failing due to bad network requests.

What you actually came to read

When it came to testing our Android counterpart, the same questions came to mind, how can we write clean, stable, and robust UI tests. And I thought why not do what we did in iOS and stand up a server in our tests?

This tutorial is possible due to folks behind Javalin, an excellent library for running an HTTP server with Java and Kotlin.

Setup

Let’s add Javalin to our project’s build.gradle dependencies.

dependencies {
...
    implementation 'io.javalin:javalin:3.8.0'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
    androidTestImplementation 'androidx.test:runner:1.2.0'
    androidTestImplementation 'androidx.test:rules:1.2.0'
...
}

When it comes to testing, I put all my testing utilities in a separate shared module, this is something I recommend so you don’t have repeated non-production code scattered throughout your application

Server Implementation

package com.bytonomy.tech.nyteout.testutils

import io.javalin.Javalin

abstract class MockServer {
    protected var server: Javalin = Javalin.create()

    open fun start(){
        server.start(8080)
    }

    fun stop() = server.stop()
}

Our app connects to several different REST endpoints as you’ll see shortly below, so I wanted to create a base class each controller can extend from. With this thing shim over Javalin, we can easily increase complexity as needed by our controllers or here in the base if that complexity id shared.

package com.bytonomy.tech.nyteout.testutils

import kotlin.Any
import com.google.gson.Gson

class AuthenticationController : MockServer() {
    companion object AuthenticationRouteResults{
        var loginResult: Any = 200
    }

    override fun start() {
        super.start()
        initializeRoutes()
    }

    private fun initializeRoutes(){
        server.post("api/authentication/login") { ctx ->
            if(loginResult is Int){
                ctx.status(loginResult as Int)
            }else{
                ctx.status(200)
                ctx.result(Gson().toJson(loginResult)).contentType("application/json")
            }
        }
    }
}

I decided to leave out the other endpoints from this implementation, as we are only going to test the login functionality. In the class you can that we boot up the server and define our login route. While not as elegantly done in the above described Embassy article, we make loginResult globally accessible so we can change the value in our tests.

Time to test!

For brevity sake, I tried to cut out all the nonsense and show you only the good stuff.

...

    @get:Rule
    var loginActivityRule: IntentsTestRule<LoginActivity> = IntentsTestRule(LoginActivity::class.java)
    private val authenticationController: AuthenticationController = AuthenticationController()

    @Before
    fun startUp() {
        authenticationController.start()
        AuthenticationController.loginResult = 401
    }

    @After
    fun tearDown() {
        authenticationController.stop()
    }


...

To start things off, we set up the activity that’s under test and instantiate our controller we want to mock. Using the @Before annotation we start our server before every test and set the loginResult to send us back 401. Likewise, we want to tear down or stop our server after each test

...
    @Test
    fun whenLoginInfoIsInvalid_ErrorSnackbar_IsDisplayed() {
        onView(withId(R.id.usernameEntry))
                .perform(typeText(Any.string()))
        onView(withId(R.id.passwordEntry))
                .perform(typeText(Any.string()))
                .perform(closeSoftKeyboard())
        onView(withId(R.id.signInButton))
                .perform(click())
        onView(withId(com.google.android.material.R.id.snackbar_text))
                .check(matches(withText(R.string.failed_to_login)))
    }

    @Test
    fun whenLoginInfoIsValid_NavigateToMainActivity() {
        val apiResults = HashMap<String, String>()
        apiResults["token"] = TestConstants.validAuthToken
        AuthenticationController.loginResult = apiResults

        onView(withId(R.id.usernameEntry))
                .perform(typeText(Any.string()))
        onView(withId(R.id.passwordEntry))
                .perform(typeText(Any.string()))
                .perform(closeSoftKeyboard())
        onView(withId(R.id.signInButton))
                .perform(click())


        intended(allOf(
                hasComponent(hasShortClassName(".${MainActivity::class.java.simpleName}")),
                toPackage(MainActivity::class.java.`package`!!.name)))
    }
    
    ...

Because of the setup we did in the previous step, it makes our first test pretty clean and straight forward. All we have to do is enter our credentials and validate our app responds correctly to the 401.

While I would have loved to use Spek or Kotest, to design better BDD styled spec tests, limitations with the frameworks don’t allow me to run them with AndroidJUnitRunner. Because of this, it leaves our second test a little messy, where we first have to set the API response to return a token, then we can test that user can successfully login to the application.