AndroidBenchmarkRunner

public class AndroidBenchmarkRunner extends AndroidJUnitRunner


Instrumentation runner for benchmarks, used to increase stability of measurements and minimize interference.

To use this runner, put the following in your module level build.gradle:

android {
defaultConfig {
testInstrumentationRunner "androidx.benchmark.junit4.AndroidBenchmarkRunner"
}
}

Minimizing Interference

This runner launches a simple opaque activity used to reduce benchmark interference from other windows. Launching other activities is supported e.g. via ActivityTestRule and ActivityScenario - the opaque activity will be relaunched if not actively running before each test, and after each test's cleanup is complete.

For example, sources of potential interference:

  • live wallpaper rendering

  • homescreen widget updates

  • hotword detection

  • status bar repaints

  • running in background (some cores may be foreground-app only)

Clock Stability

While it is better for performance stability to lock clocks with the ./gradlew lockClocks task provided by the gradle plugin, this is not possible on most devices. The runner provides a fallback mode for preventing thermal throttling.

On devices that support android.view.Window.setSustainedPerformanceMode, the runner will set this mode on the window of every Activity launched (including the opaque Activity mentioned above). The runner will also launch a continuously spinning Thread. Together, these ensure that the app runs in the multithreaded stable performance mode, which locks the maximum clock frequency to prevent thermal throttling. This ensures stable clock levels across all benchmarks, even if a continuous suite of benchmarks runs for many minutes on end.

Summary

Public constructors

Public methods

void

Protected methods

void

Ensures we've onStopped() all activities which were onStarted().

Inherited methods

From androidx.test.runner.AndroidJUnitRunner
@NonNull Application
newApplication(
    @NonNull ClassLoader cl,
    @NonNull String className,
    @NonNull Context context
)

Does initialization before creating the application.

void
onCreate(@NonNull Bundle arguments)

Sets up lifecycle monitoring, and argument registry.

boolean
void

This implementation of onStart() will guarantee that the Application's onCreate method has completed when it returns.

void
sendStatus(int resultCode, @NonNull Bundle results)
From android.app.Instrumentation
@NonNull TestLooperManager
void
@NonNull Instrumentation.ActivityMonitor
addMonitor(
    @NonNull String cls,
    @NonNull Instrumentation.ActivityResult result,
    boolean block
)
@NonNull Instrumentation.ActivityMonitor
addMonitor(
    @NonNull IntentFilter filter,
    @NonNull Instrumentation.ActivityResult result,
    boolean block
)
void
void
callActivityOnCreate(
    @NonNull Activity activity,
    @NonNull Bundle icicle,
    @NonNull PersistableBundle persistentState
)
void
void
void
callActivityOnPostCreate(
    @NonNull Activity activity,
    @Nullable Bundle savedInstanceState
)
void
callActivityOnPostCreate(
    @NonNull Activity activity,
    @Nullable Bundle savedInstanceState,
    @Nullable PersistableBundle persistentState
)
void
callActivityOnRestoreInstanceState(
    @NonNull Activity activity,
    @NonNull Bundle savedInstanceState
)
void
callActivityOnRestoreInstanceState(
    @NonNull Activity activity,
    @Nullable Bundle savedInstanceState,
    @Nullable PersistableBundle persistentState
)
void
callActivityOnSaveInstanceState(
    @NonNull Activity activity,
    @NonNull Bundle outState
)
void
callActivityOnSaveInstanceState(
    @NonNull Activity activity,
    @NonNull Bundle outState,
    @NonNull PersistableBundle outPersistentState
)
void
boolean
checkMonitorHit(
    @NonNull Instrumentation.ActivityMonitor monitor,
    int minHits
)
void
@NonNull Bundle
@NonNull Bundle
@NonNull ComponentName
@NonNull Context
@NonNull String
@NonNull Context
@NonNull UiAutomation
@NonNull UiAutomation
getUiAutomation(int flags)
boolean
invokeContextMenuAction(
    @NonNull Activity targetActivity,
    int id,
    int flag
)
boolean
invokeMenuActionSync(@NonNull Activity targetActivity, int id, int flag)
boolean
void
void
void
sendCharacterSync(int keyCode)
void
sendKeyDownUpSync(int keyCode)
void
void
void
void
void
void
setInTouchMode(boolean inTouch)
void
@NonNull Activity
void

This method is deprecated.

void
void
void

This method is deprecated.

void
void
void
@NonNull Activity
@NonNull Activity
waitForMonitorWithTimeout(
    @NonNull Instrumentation.ActivityMonitor monitor,
    long timeOut
)
From androidx.test.runner.MonitoringInstrumentation
void
void
void
void
void
void
void
void
void
void
execStartActivities(
    @NonNull Context who,
    @NonNull IBinder contextThread,
    @NonNull IBinder token,
    @NonNull Activity target,
    @NonNull Intent[] intents,
    @NonNull Bundle options
)
@NonNull Instrumentation.ActivityResult
execStartActivity(
    @NonNull Context who,
    @NonNull IBinder contextThread,
    @NonNull IBinder token,
    @NonNull Activity target,
    @NonNull Intent intent,
    int requestCode
)
@NonNull Instrumentation.ActivityResult
execStartActivity(
    @NonNull Context who,
    @NonNull IBinder contextThread,
    @NonNull IBinder token,
    @NonNull Activity target,
    @NonNull Intent intent,
    int requestCode,
    @NonNull Bundle options
)
@NonNull Instrumentation.ActivityResult
execStartActivity(
    @NonNull Context who,
    @NonNull IBinder contextThread,
    @NonNull IBinder token,
    @NonNull Fragment target,
    @NonNull Intent intent,
    int requestCode,
    @NonNull Bundle options
)
@NonNull Instrumentation.ActivityResult
execStartActivity(
    @NonNull Context who,
    @NonNull IBinder contextThread,
    @NonNull IBinder token,
    @NonNull String target,
    @NonNull Intent intent,
    int requestCode,
    @NonNull Bundle options
)

This API was added in Android API 23 (M)

@NonNull Instrumentation.ActivityResult
execStartActivity(
    @NonNull Context who,
    @NonNull IBinder contextThread,
    @NonNull IBinder token,
    @NonNull Activity target,
    @NonNull Intent intent,
    int requestCode,
    @NonNull Bundle options,
    @NonNull UserHandle user
)

This API was added in Android API 17 (JELLY_BEAN_MR1)

void
finish(int resultCode, @NonNull Bundle results)
@NonNull String
void
void

Perform application MultiDex installation only when instrumentation installation is not available.

void
interceptActivityUsing(
    @NonNull InterceptingActivityFactory interceptingActivityFactory
)

Use the given InterceptingActivityFactory to create Activity instance in newActivity.

final boolean

Checks whether this instance of instrumentation should be considered as a primary instrumentation process.

boolean

This method is deprecated.

use isPrimaryInstrProcess()

@NonNull Activity
newActivity(
    @NonNull ClassLoader cl,
    @NonNull String className,
    @NonNull Intent intent
)
@NonNull Activity
newActivity(
    @NonNull Class<@NonNull ?> clazz,
    @NonNull Context context,
    @NonNull IBinder token,
    @NonNull Application application,
    @NonNull Intent intent,
    @NonNull ActivityInfo info,
    @NonNull CharSequence title,
    @NonNull Activity parent,
    @NonNull String id,
    @NonNull Object lastNonConfigurationInstance
)
void
void
final void
boolean
void
@NonNull Activity
@NonNull Throwable

Unwraps an exception from certain wrapper types, e.g. RuntimeException.

void

Use default mechanism of creating activity instance in newActivity

Public constructors

AndroidBenchmarkRunner

Added in 1.0.0
public AndroidBenchmarkRunner()

Public methods

onDestroy

@CallSuper
public void onDestroy()

Protected methods

waitForActivitiesToComplete

@CallSuper
protected void waitForActivitiesToComplete()

Ensures we've onStopped() all activities which were onStarted().

According to Activity's contract, the process is not killable between onStart and onStop. Breaking this contract (which finish() will if you let it) can cause bad behaviour (including a full restart of system_server).

We give the app 2 seconds to stop all its activities, then we proceed.

This should never be run on the main thread.