Advanced test setup

The pages Test in Android Studio and Test from the command line explain how to set up and run basic test configurations. However, when your application and its test requirements get more advanced, you may need to adapt your test configurations further. For example, you might need advanced test setup when you want to do the following:

• Run instrumented tests only for a specific build variant, or override its manifest settings.
• Change the build type your tests run against, or configure its Gradle options.
• Extract your instrumented tests into their own test module.

This page describes various ways to configure your tests when the default settings don't fit the use case at hand.

Create an instrumented test for a build variant

If your project includes build variants with unique source sets, you might want to include instrumented tests that correspond to those source sets. This keeps your test code organized and allows you to run only tests that apply to a given build variant.

You can link instrumented tests to a build variant by placing them in their own source set, which is located at src/androidTestVariantName.

Instrumented tests in the src/androidTest/ source set are shared by all build variants. When building a test APK for the "MyFlavor" variant of your app, Gradle combines the src/androidTest/ and src/androidTestMyFlavor/ source sets.

To add a testing source set for your build variant in Android Studio, follow these steps:

1. In the Project window on the left, click the drop-down menu and select the Project view.
2. Within the appropriate module folder, right-click the src folder and click New > Directory.
3. For the directory name, enter "androidTestVariantName." For example, if you have a build variant called "MyFlavor" then the directory name should be "androidTestMyFlavor." Then click OK.
4. Right-click on the new directory and click New > Directory.
5. Enter "java" as the directory name, and then click OK.

Now you can add tests to this new source set by following the steps to add a new test. When you reach the Choose Destination Directory dialog, select the new variant test source set.

The following table shows an example of how instrumentation test files could reside in source sets that correspond to the app's code source sets.

Table 1. App source code and corresponding instrumentation test files.

Just as it does for your app source sets, the Gradle build merges and overrides files from different test source sets. In this case, the AndroidFooTest.java file in the androidTestMyFlavor source set overrides the version in the androidTest source set. The reason behind this is that the product flavor source set has priority over the main source set. For more information about how source sets are merged, see Configure your build.

Configure instrumentation manifest settings

Instrumented tests are built into a separate APK, with their own AndroidManifest.xml file. When Gradle builds your test APK, it automatically generates the AndroidManifest.xml file and configures it with the <instrumentation> node. One of the reasons Gradle configures this node for you is to make sure that the targetPackage property specifies the correct package name of the app under test. You can change some of the other settings for this node by either creating another manifest file in the test source set or configuring your module-level build.gradle file as shown in the following code sample. The full list of options can be found in the BaseFlavor API reference.

android {

    ...

    // Each product flavor you configure can override properties in the
    // defaultConfig {} block. To learn more, go to Configure product flavors.

    defaultConfig {

        ...

        // Specifies the application ID for the test APK.
        testApplicationId = "com.test.foo"

        // Specifies the fully-qualified class name of the test instrumentation
        // runner.
        testInstrumentationRunner = "android.test.InstrumentationTestRunner"

        // If set to true, enables the instrumentation class to start and stop profiling.
        // If set to false (default), profiling occurs the entire time the instrumentation
        // class is running.
        testHandleProfiling = true

        // If set to true, indicates that the Android system should run the instrumentation
        // class as a functional test. The default value is false.
        testFunctionalTest = true

    }
}

Change the test build type

By default, all instrumentation tests run against the debug build type. You can change this to another build type by using the testBuildType property in your module-level build.gradle file. For example, if you want to run your tests against your staging build type, edit the file as shown in the following snippet.

android {

    ...

    testBuildType "staging"

}

Configure Gradle test options

The Android Gradle plugin allows you to specify certain options for all or just some of your tests. In the module-level build.gradle file, use the testOptions{} block to specify options that change how Gradle runs all your tests.

android {
    ...
    // Encapsulates options for running tests.
    testOptions {
        // Changes the directory where Gradle saves test reports. By default,
        // Gradle saves test reports in the
        // path_to_your_project/module_name/build/outputs/reports/ directory.
        // '$rootDir' sets the path relative to the root directory of the
        // current project.
        reportDir "$rootDir/test-reports"
        // Changes the directory where Gradle saves test results. By default,
        // Gradle saves test results in the
        // path_to_your_project/module_name/build/outputs/test-results/ directory.
        // '$rootDir' sets the path relative to the root directory of the
        // current project.
        resultsDir "$rootDir/test-results"
    }
}

android {
    ...
    // Encapsulates options for running tests.
    testOptions {
        // Changes the directory where Gradle saves test reports. By default,
        // Gradle saves test reports in the
        // path_to_your_project/module_name/build/outputs/reports/ directory.
        // '$rootDir' sets the path relative to the root directory of the
        // current project.
        reportDir "$rootDir/test-reports"
        // Changes the directory where Gradle saves test results. By default,
        // Gradle saves test results in the
        // path_to_your_project/module_name/build/outputs/test-results/ directory.
        // '$rootDir' sets the path relative to the root directory of the
        // current project.
        resultsDir = "$rootDir/test-results"
    }
}

To specify options for only local unit tests, configure the unitTests{} block inside testOptions{}.

android {
    ...
    testOptions {
        ...
        // Encapsulates options for local unit tests.
        unitTests {
            // By default, local unit tests throw an exception any time the code
            // you are testing tries to access Android platform APIs (unless you
            /// mock Android dependencies yourself or with a testing framework like Mockito).
            // However, you can enable the following property so that the test
            // returns either null or zero when accessing platform APIs, rather
            // than throwing an exception.
            returnDefaultValues true

            // Encapsulates options for controlling how Gradle executes local
            // unit tests. For a list of all the options you can specify, read
            // Gradle's reference documentation.
            all {
                // Sets JVM argument(s) for the test JVM(s).
                jvmArgs '-XX:MaxPermSize=256m'

                // You can also check the task name to apply options to only the
                // tests you specify.
                if (it.name == 'testDebugUnitTest') {
                    systemProperty 'debug', 'true'
                }
                ...
            }
        }
    }
}

android {
    ...
    testOptions {
        ...
        // Encapsulates options for local unit tests.
        unitTests {
            // By default, local unit tests throw an exception any time the code
            // you are testing tries to access Android platform APIs (unless you
            /// mock Android dependencies yourself or with a testing framework like Mockito).
            // However, you can enable the following property so that the test
            // returns either null or zero when accessing platform APIs, rather
            // than throwing an exception.
            returnDefaultValues = true

            // Encapsulates options for controlling how Gradle executes local
            // unit tests. For a list of all the options you can specify, read
            // Gradle's reference documentation.
            all {
                // Sets JVM argument(s) for the test JVM(s).
                jvmArgs = listOf("-XX:MaxPermSize=256m")

                // You can also check the task name to apply options to only the
                // tests you specify.
                if (it.name == "testDebugUnitTest") {
                    systemProperty = mapOf("debug" to "true")
                }
                ...
            }
        }
    }
}

Use separate test modules for instrumented tests

If you want to have a module dedicated for instrumented tests and isolate the rest of your code from your tests, you can create a separate test module and configure its build similar to that of a library module. To create a test module, proceed as follows:

1. Create a library module.
2. In the module-level build.gradle file, apply the com.android.test plugin instead of com.android.library.
3. Sync your project by clicking Sync Project

After you create your test module, you can include your test code in the main or variant source set (for example, src/main/java or src/variant/java). If your app module defines multiple product flavors, you can recreate those flavors in your test module, and, using variant aware dependency management, the test module attempts to test the matching flavor in the target module.

By default, test modules contain and test only a debug variant. However, you can create new build types to match the tested app project. To make the test module test a different build type, and not the debug one, use VariantFilter to disable the debug variant in the test project, as shown below:

android {
    variantFilter { variant ->
        if (variant.buildType.name.equals('debug')) {
            variant.setIgnore(true);
        }
    }
}

android {
    variantFilter {
        if (buildType.name == "debug") {
            ignore = true
        }
    }
}

If you want a test module to target only certain flavors or build types of an app, you can use the matchingFallbacks property to target only the variants you want to test. This also prevents the test module from having to configure those variants for itself.