Add instrumentation arguments

Configure the behavior of the library with the following instrumentation arguments. You can either add these to your Gradle configuration or apply them directly when running instrumentation from the command line. To set these arguments for all Android Studio and command line test runs, add them to testInstrumentationRunnerArguments:

android {
    defaultConfig {
        // ...
        testInstrumentationRunnerArguments["androidx.benchmark.dryRunMode.enable"] = "true"
    }
}

You can also set up instrumentation arguments when running the benchmarks from Android Studio. To change the arguments, do the following:

  1. Edit the run configuration by clicking Edit and then clicking the configuration.
    edit run configuration
    Figure 1. Edit the run configuration.
  2. Edit the instrumentation arguments by clicking More by Instrumentation arguments.
    edit the instrumentation arguments
    Figure 2. Edit the instrumentation arguments.
  3. Add the required instrumentation argument by clicking Add under Instrumentation Extra Params.
    add required instrumentation argument
    Figure 3. Add the required instrumentation argument.

If you're calling directly from the command line, use -P android.testInstrumentationRunnerArguments.[name of the argument].

./gradlew :benchmark:connectedAndroidTest -P android.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile

androidx.benchmark.compilation.enabled

Lets you disable compilation between each iteration of the benchmark. By default, the target application is re-installed and re-compiled between each benchmark, to respect the CompilationMode passed into measureRepeated. Disabling this lets you to skip both reinstall and compilation if, for example, you want to fully compile the target app once before running the test suite and run all benchmarks against that fully compiled target.

  • Argument type: boolean
  • Defaults to: true

androidx.benchmark.dryRunMode.enable

Lets you run benchmarks in a single loop to verify whether they work properly. You can use it with regular tests as part of verification.

  • Argument type: boolean
  • Defaults to: false

androidx.benchmark.enabledRules

Allows filtering runs to just one type of test: Baseline Profile generation or Macrobenchmark test. Comma-separated lists are also supported.

  • Argument type: string
  • Available options:
    • Macrobenchmark
    • BaselineProfile
  • Defaults to: Not specified

The Macrobenchmark run for $packageName has method tracing enabled. This makes the VM run slower than usual, so only consider the metrics from the trace files in relative terms—for example, comparing how fast the first run is to the second run.

androidx.benchmark.perfettoSdkTracing.enable

Enables androidx.tracing.perfetto tracepoints such as Jetpack Compose composition tracing. This only has an effect when StartupMode.COLD isn't used, since enabling the tracepoints wakes the target process.

  • Argument type: boolean
  • Defaults to: false

androidx.benchmark.startupProfiles.enable

Lets you disable the generation of startup profiles during benchmarking.

  • Argument type: boolean
  • Defaults to: true

androidx.benchmark.suppressErrors

Accepts comma-separated list of errors to turn into warnings.

  • Argument type: list of strings

  • Available options:

    • DEBUGGABLE

      The DEBUGGABLE error indicates that the target package is running with debuggable=true in its manifest, which drastically reduces runtime performance to support debugging features. To avoid this error, run benchmarks with debuggable=false. The debuggable argument affects execution speed in ways that mean benchmark improvements might not carry over to a real user's experience or might regress release performance.

    • LOW-BATTERY

      When battery is low, devices often reduce performance to save remaining battery, for example by disabling big cores. This occurs even when the devices are plugged in. Only suppress this error if you are deliberately profiling the app with reduced performance.

    • EMULATOR

      The EMULATOR error tells you that the benchmark is running on an emulator, which isn't representative of real user devices. Emulator benchmark improvements might not carry over to a real user's experience or might regress real device performance. You should use a physical device to benchmark instead. Suppress this error with extreme caution.

    • NOT-PROFILEABLE

      Target package $packageName is running without <profileable shell=true>. Profileable is required on Android 10 and 11 to let Macrobenchmark capture detailed trace information from the target process, such as system tracing sections defined in the app or libraries. Suppress this error with extreme caution.

    • METHOD-TRACING-ENABLED

      The Macrobenchmark run for the app being benchmarked has method tracing enabled. This causes the VM to run slower than usual, so only consider the metrics from the trace files in relative terms—for example, comparing how fast the first run is to the second run. Suppressing this error can result in inaccurate results if you compare benchmarks for builds with different method tracing options.

  • Defaults to: an empty list

additionalTestOutputDir

Configures where JSON benchmark reports and profiling results are saved on device.

  • Argument type: path string
  • Defaults to: test APK's external directory
Previous
Control your app