Benchmark Baseline Profiles with Macrobenchmark library

We recommend using Jetpack Macrobenchmark to test how an app performs when Baseline Profiles are enabled, and then compare those results to a benchmark with Baseline Profiles disabled. With this approach, you can measure app startup time—both time to initial and full display—or runtime rendering performance to see if the frames produced can cause jank.

Macrobenchmarks let you control pre-measurement compilation using the CompilationMode API. To measure the results, set the compilationMode parameter to the correct value as shown in the following snippet:

@RunWith(AndroidJUnit4ClassRunner::class)
class ColdStartupBenchmark {
    @get:Rule
    val benchmarkRule = MacrobenchmarkRule()

    @Test
    fun startupNoCompilation() = startup(CompilationMode.None())

    @Test
    fun startupPartialWithBaselineProfiles() =
        startup(CompilationMode.Partial(baselineProfileMode = BaselineProfileMode.Require))

    @Test
    fun startupPartialCompilation = startup(
        CompilationMode.Partial(
            baselineProfileMode = BaselineProfileMode.Disable,
            warmupIteration = 3
        )
    )

    @Test
    fun startupFullCompilation() = startup(CompilationMode.Full())

    private fun startup(compilationMode: CompilationMode) = benchmarkRule.measureRepeated(
        packageName = "com.example.macrobenchmark.target",
        metrics = listOf(StartupTimingMetric()),
        compilationMode = compilationMode,
        iterations = 10,
        startupMode = StartupMode.COLD,
        setupBlock = {
            pressHome()
        }
    ) {
        // Waits for the first rendered frame, which represents time to initial display.
        startActivityAndWait()

        // Waits for content to be visible, which represents time to fully drawn.
        device.wait(Until.hasObject(By.res("my-content")), 5_000)
    }
}

In the following screenshot, you can see the results directly in Android Studio for the Now in Android sample app ran on Google Pixel 7. The results show that app startup is fastest when using Baseline Profiles (229.0ms) in contrast with no compilation (324.8ms).

While the previous example shows app startup results captured with StartupTimingMetric, there are other important metrics worth considering, such as FrameTimingMetric. For more information about all the types of metrics, see Capture Macrobenchmark metrics.

Time to full display

The previous example measures the time to initial display (TTID), which is the time taken by the app to produce its first frame. However, this doesn't necessarily reflect the time until the user can start interacting with your app. The time to full display (TTFD) metric is more useful in measuring and optimizing the code paths necessary to have a fully useable app state.

We recommend optimizing for both TTID and TTFD, as both are important. A low TTID helps the user see that the app is actually launching. Keeping the TTFD short is important to help ensure that the user can interact with the app quickly.

For strategies on reporting when the app UI is fully drawn, see Improve startup timing accuracy.

Recommended for you

• Note: link text is displayed when JavaScript is off
• [Write a Macrobenchmark][11]
• [Capture Macrobenchmark metrics][12]
• [App startup analysis and optimization {:#app-startup-analysis-optimization}][13]