Эта страница переведена с помощью Cloud Translation API.

Контрольный показатель

Руководство пользователя Пример кода Codelab

Справочник по API
androidx.benchmark
androidx.benchmark.junit4
androidx.benchmark.macro
androidx.benchmark.macro.junit4

Точно измеряйте производительность вашего кода в Android Studio.

Последнее обновление	Стабильный выпуск	Кандидат на выпуск	Бета-версия	Альфа-релиз
9 апреля 2025 г.	1.3.4	-	-	1.4.0-альфа11

Объявление зависимостей

Чтобы добавить зависимость от Benchmark, вам необходимо добавить в свой проект репозиторий Google Maven. Для получения дополнительной информации прочтите репозиторий Google Maven .

Макробенчмарк

Чтобы использовать Macrobenchmark в своем проекте, добавьте следующие зависимости в файл build.gradle для вашего модуля макробенчмарка :

классный

dependencies {
  androidTestImplementation "androidx.benchmark:benchmark-macro-junit4:1.3.4"
}

Котлин

dependencies {
  androidTestImplementation("androidx.benchmark:benchmark-macro-junit4:1.3.4")
}

Микробенчмарк

Чтобы использовать Microbenchmark в своем проекте, добавьте следующие зависимости в файл build.gradle для вашего модуля microbenchmark :

классный

dependencies {
    androidTestImplementation "androidx.benchmark:benchmark-junit4:1.3.4"
}

android {
    ...
    defaultConfig {
        ...
        testInstrumentationRunner "androidx.benchmark.junit4.AndroidBenchmarkRunner"
    }
}

Котлин

dependencies {
    androidTestImplementation("androidx.benchmark:benchmark-junit4:1.3.4")
}

android {
    ...
    defaultConfig {
        ...
        testInstrumentationRunner = "androidx.benchmark.junit4.AndroidBenchmarkRunner"
    }
}

Библиотека Microbenchmark также предоставляет плагин Gradle для использования с вашим модулем microbenchmark. Этот плагин устанавливает настройки конфигурации сборки по умолчанию для модуля, настраивает копию выходных данных теста на хост и предоставляет задачу ./gradlew lockClocks .

Чтобы использовать плагин, включите следующую строку в блок `plugins` вашего файла build.gradle верхнего уровня:

классный

plugins {
  id 'androidx.benchmark' version '1.3.4' apply false
}

Котлин

plugins {
  id("androidx.benchmark") version "1.3.4" apply false
}

Затем примените плагин к файлу build.gradle вашего тестового модуля.

классный

plugins {
  id 'androidx.benchmark'
}

Котлин

plugins {
    id("androidx.benchmark")
}

Обратная связь

Ваши отзывы помогают сделать Jetpack лучше. Дайте нам знать, если вы обнаружите новые проблемы или у вас есть идеи по улучшению этой библиотеки. Пожалуйста, ознакомьтесь с существующими проблемами в этой библиотеке, прежде чем создавать новую. Вы можете добавить свой голос к существующей проблеме, нажав кнопку со звездочкой.

Создать новую задачу

Дополнительную информацию см. в документации по системе отслеживания проблем .

Версия 1.4

Версия 1.4.0-альфа11

9 апреля 2025 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha11 . Версия 1.4.0-alpha11 содержит эти коммиты .

Изменения API

Изменен таймаут аргумента TraceProcessor : Duration на timeoutMs long для удобства использования вызывающей программы Java. ( I9fbb5 )
Пометить конструктор TraceProcessor как внутренний. Вызывающие стороны должны использовать TraceProcessor.startServer или TraceProcessor.runServer . ( Ia8c5b )

Исправления ошибок

При завершении приложения с помощью MacrobenchmarkScope.killProcess проверьте результаты команды уничтожения, чтобы предотвратить сбой в автоматическом режиме и ошибку тайм-аута. ( I84555 )

Обновления зависимостей

Эта библиотека теперь ориентирована на уровень языка Kotlin 2.0 и требует KGP 2.0.0 или новее. ( Idb6b5 )

Версия 1.4.0-альфа10

26 марта 2025 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha10 . Версия 1.4.0-alpha10 содержит эти коммиты .

Изменения API

Увеличено время ожидания по умолчанию для загрузки и запроса сервера TraceProcessor до 120 секунд (с 60/30 ранее), а также сделано возможность настройки обоих параметров с помощью одного параметра времени ожидания. ( Ифек87 )

Исправления ошибок

Исправлено несколько проблем, возникавших при тестировании или захвате профилей приложения без profileinstaller , включая BroadcastReciever . Это влияет только на запуск на корневых устройствах. ( Ied308 )

Версия 1.4.0-альфа09

12 марта 2025 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha09 . Версия 1.4.0-alpha09 содержит эти коммиты .

Изменения API

Добавлен TraceProcessor и Session API с закрывающимися дескрипторами для упрощения использования с настраиваемыми жизненными циклами. Это также шаг к упрощению использования Coroutine и Java API. Функции расширения TraceProcessor.runServer {} теперь помечены как экспериментальные, так как они, скорее всего, будут перемещены и в будущем станут неэкспериментальными. ( I358b4 )

Исправления ошибок

Исправлена ошибка, из-за которой захват контрольного и базового профиля не работал с API 36 из-за изменения в pgrep toybox, который теперь требует -a для печати полной командной строки. ( Idc991 )
Отфильтруйте конфигурацию трассировки по умолчанию, чтобы снизить риск потери данных в трассировках на новых уровнях API. ( I54e8a )
Добавлен экспериментальный аргумент инструментария androidx.benchmark.killExistingPerfettoRecordings , для которого можно установить значение false , чтобы продолжить захват ранее существующей трассировки perfetto. По умолчанию существующие записи трассировки perfetto на устройстве уничтожаются во избежание помех. ( I02a3c )
Поле JSON context.osCodenameAbbreviated теперь будет иметь REL для выпущенных версий ОС с API 35 и выше, поскольку нечисловые кодовые имена больше не поддерживаются базовой платформой. ( Ib17fd )
Исправлен сбой в FrameTimingMetric при повторной синхронизации кадров. ( I7c6f4 , б/394610806 )
Больше не предполагайте, что Choreographer#doFrame является вершиной кадра стека в основном потоке для FrameTimingQuery . ( Iee0e0 , б/340206285 )

Версия 1.4.0-альфа08

12 февраля 2025 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha08 . Версия 1.4.0-alpha08 содержит эти коммиты .

Изменения API

API-интерфейсы расширения TraceProcessor.runSession() перенесены в экспериментальный режим, поскольку в конечном итоге они, скорее всего, станут конкретными конструкторами в Android. ( Ib0528 , б/393640753 )
Большая часть реализации Startup Insights теперь общедоступна/экспериментальна и перенесена в артефакт TraceProcessor . См. StartupInsights . ( I0aa00 )
Устаревший BenchmarkRule.runWithTimingDisabled {} в пользу BenchmarkRule.runWithMeasurementDisabled {} , который более четко описывает поведение — все метрики приостанавливаются. Кроме того, откройте суперкласс MicrobenchmarkScope , поскольку переобъявление функции runWithMeasurementDisabled для открытого доступа невозможно, поскольку она встроена. ( I9e23b , б / 389149423 , б / 149979716 )
Библиотеки тестов производительности перешли на Kotlin 2.0. ( I9d1e0 )
Удален аргумент инструментария androidx.benchmark.startupProfiles.enable . Он больше не полезен, так как им можно управлять с помощью аргумента includeInStartupProfile в BaselineProfileRule.collect() . ( I39eb4 )

Исправления ошибок

Уменьшено количество внутренних функций Microbenchmark, вызываемых во время профилирования, чтобы сделать, например, трассировку метода более четкой ( Ifaed8 ).
Спекулятивное исправление сбоев: «Не удалось остановить [ ProcessPid(processName=perfetto, pid=...) ]». Теперь Benchmark будет регистрировать сообщение, а не аварийно завершать работу, если фоновый процесс Perfetto не может быть остановлен перед запуском теста. ( I37d3e , б/323601788 )
Исправлены исключения IllegalStateExceptions с меткой «Ожидаемый стандартный вывод pm dump-profiles », вызванные слишком строгой проверкой выходного формата. ( I358dc )

Версия 1.4.0-альфа07

29 января 2025 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha07 . Версия 1.4.0-alpha07 содержит эти коммиты .

Новые возможности

BaselineProfileRule теперь представляет API collectWithResults(...) который включает список путей к вычисляемым профилям. ( I056f8 )
Добавлен аргумент инструментария androidx.benchmark.measureRepeatedOnMainThread.throwOnDeadline , для которого можно установить значение false, чтобы отключить генерацию по крайнему сроку для measureRepeatedOnMainThread для локального тестирования. Иное не рекомендуется, так как это увеличивает вероятность ошибок ANR во время тестов. ( Идбек , б/353226476 )

Изменения API

Добавлен @JvmOverloads в конструктор MicrobenchmarkConfig . ( I13fd3 )
Переработано правило BenchmarkRule , которое будет построено поверх сопрограмм и будет поддерживать лучшее поведение yield() . В ходе этого рефакторинга было удалено несколько экспериментальных API BenchmarkState , но при необходимости последуют замены. Кроме того, для пояснения поведения добавлен runWithMeasurementDisabled (все измерения приостанавливаются). В будущем runWithTimingDisabled станет устаревшим. ( I19837 , б/389149423 , б/311242861 )
Переместите PerfettoTraceProcessor в TraceProcessor в новом артефакте androidx.benchmark:benchmark-traceprocessor и сделайте большую часть его API неэкспериментальным. Любой пользовательский TraceMetric или что-либо, читающее трассировки, необходимо будет обновить до нового импорта TraceProcessor . Новый API TraceProcessor работает точно так же, как старый, но представляет собой отдельную библиотеку интерфейса (несколько аналогичную слою androidx.sqlite из Room) со специфичной для Android реализацией, встроенной в макротест. Новый артефакт также можно использовать в JVM, но сейчас вам необходимо запустить собственную копию двоичного файла TraceProcessor и предложить порт для подключения к нему. ( I3a767 , I62563 , б/381134564 )

Исправления ошибок

Выдавать более четкое сообщение об ошибке, когда MacrobenchmarkScope.startActivityAndWait не удается запустить целевой процесс (возможно, из-за сбоя в целевом процессе), вместо более двусмысленного сообщения «Невозможно подтвердить завершение запуска действия» ( I3539b ).
Исправлено несколько синтаксических ошибок в примерах Kotlin и подсветка синтаксиса в нескольких примерах Java/build.gradle. ( Ib3808 )
Уточнена документация по параметрам ArtMetric и CaptureInfo . ( I96e60 )

Версия 1.4.0-альфа06

11 декабря 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha06 . Версия 1.4.0-alpha06 содержит эти коммиты .

Изменения API

Удалено использование @Language("sql") в PerfettoTraceProcessor.Session.query() , так как подсветка/анализ Studio нарушен. ( Idc2fa , б/377733398 )

Исправления ошибок

Эта библиотека теперь использует аннотации NULL NULL JSpecify , которые являются типизированными. Разработчики Kotlin должны использовать следующие аргументы компилятора для обеспечения правильного использования: -Xjspecify-annotations=strict , -Xtype-enhancement-improvements-strict-mode ( I46810 , b/326456246 ).
Исправлен ArtMetric для отчета о загрузке классов (не инициализации) и улучшена документация для пояснения поведения во время выполнения. ( I9915c )
В многопользовательском режиме Android выполняйте команды от имени пользователя root только на устройствах с root-доступом. ( I88b44 )

Версия 1.4.0-альфа05

13 ноября 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha05 . Версия 1.4.0-alpha05 содержит эти коммиты .

Исправления ошибок

Исправлена проблема в API 34+, из-за которой CompilationMode.None() имела нестабильную производительность, не отражающую начальную производительность в худшем случае. Это связано с изменением платформы, которое позволяет verify состояния компиляции ART частично компилировать приложения (влияя только на загрузку классов) вскоре после первого запуска. ( Ie48d0 )
Исправлена проблема, при которой (особенно короткие) трассировки могли быть записаны, но не сообщали об измерениях из встроенных метрик Macrobenchmark из-за усечения имени процесса в трассировке Perfetto. Теперь макросбенчмарк решает эту проблему, ища во всех встроенных запросах усеченное имя пакета в дополнение к ожидаемому имени пакета. Обратите внимание, что пользовательские реализации TraceMetric или другие прямые вызовы PerfettoSession.query могут реализовать то же самое поведение, изменив process.name LIKE "$packageName" в запросе Perfetto на (process.name LIKE "$packageName" OR process.name LIKE "$(packageName.takeLast(15))") . ( I5bf01 , б/377565760 )

Версия 1.4.0-альфа04

30 октября 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha04 . Версия 1.4.0-alpha04 содержит эти коммиты .

Новые возможности

(Экспериментальный вариант) Включите создание базового профиля и сравнительный анализ приложений, установленных дополнительным пользователем, например любого приложения на автономных устройствах Android Auto. Эта поддержка была протестирована в некоторых сценариях, но если она вам не подходит, сообщите нам об ошибке. ( I9fcbe , б/356684617 , б/373641155 )

Исправления ошибок

isProfileable теперь всегда переопределяется в тестовых сборках, а isDebuggable теперь всегда переопределяется как в тестовых, так и в nonMinified сборках (захват базового профиля). ( I487fa , б/369213505 )
Исправляет обнаружение компиляции на некоторых физических устройствах до API 28 - влияет на json context.compilationMode , а также на поведение androidx.benchmark.requireAot=true (который больше не выдает некорректно) ( Ic3e08 , b/374362482 )
В метриках CpuEventCounter выдать исключение, если наблюдаются недопустимые измерения (например, инструкции/cpucycles==0) ( I8c503 ).

Версия 1.4.0-альфа03

16 октября 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha03 . Версия 1.4.0-alpha03 содержит эти коммиты .

Изменения API

Macrobenchmark : добавляет ArtMetric , который можно использовать для проверки покрытия профиля или общей производительности среды выполнения Android. Регистрирует количество и общую продолжительность JIT, инициализации класса (если доступно) и проверки класса. Кроме того, CaptureInfo изменено, чтобы включить дополнительную основную версию ART по умолчанию. ( I930f7 )
Добавьте coefficientOfVariation в выходные данные Benchmark JSON, чтобы продемонстрировать стабильность в рамках данного теста. ( Ib14ea )

Исправления ошибок

Исправлена задача CollectBaselineProfileTask , когда в устройстве AVD есть пробелы. ( Ia0225 , б/371642809 )
Спекулятивное исправление ошибок из исключений StartupMode.COLD : Package <packagename> must not be running prior to cold start! . Теперь MacrobenchmarkScope.killProcess() (включая тот, который запускается перед каждой итерацией и используется для реализации поведения StartupMode.COLD ) будет ждать, чтобы убедиться, что все процессы приложения остановились. ( I60aa6 , б/351582215 )
Исправлена ошибка, из-за которой ошибка UNLOCKED_ появлялась на некоторых корневых эмуляторах. ( Ic5117 )
Эта библиотека теперь использует аннотации NULL JSpecify , которые являются типизированными. Разработчики Kotlin должны использовать следующие аргументы компилятора для обеспечения правильного использования: -Xjspecify-annotations=strict , -Xtype-enhancement-improvements-strict-mode ( I7104f , b/326456246 ).

Версия 1.4.0-альфа02

2 октября 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha02 . Версия 1.4.0-alpha02 содержит эти коммиты .

Изменения API

Задачи Gradle lockClocks и unlockClocks перемещены в тестовые проекты, а не на верхний уровень. Это изменение было необходимо, поскольку, к сожалению, невозможно зарегистрировать их как действия верхнего уровня, не нарушая изоляции проекта. ( I02b8f , б/363325823 )

Исправления ошибок

BaselineProfileRule теперь собирает профили для многопроцессных приложений, сигнализируя каждому запущенному процессу в конце блока о необходимости создания дампа профилей. Если компиляция на основе профиля никогда не находит процесс для трансляции, компиляция завершится неудачно, поскольку в нем неожиданно присутствуют данные профиля. Кроме того, добавлен аргумент инструмента для управления продолжительностью ожидания дампа: androidx.benchmark.saveProfileWaitMillis ( I0f519 , b/366231469 ).
Из Benchmark 1.3.2 : Исправлена ошибка, из-за которой Firebase Test Lab (FTL) не могла извлечь файлы результатов базового профиля или макротеста из плагина Gradle базового профиля. ( I2f678 , б / 285187547 )

Чтобы использовать FTL, примените плагин к модулю базового профиля в блоке плагинов, используя:

  plugins {
      ...
      id("com.google.firebase.testlab")
  }

а затем настройте тестовую лабораторию Firebase с помощью:

  firebaseTestLab {

      // Credentials for FTL service
      serviceAccountCredentials.set(file("credentials.json"))

      // Creates one or more managed devices to run the tests on.
      managedDevices {
          "ftlDeviceShiba34" {
              device = "shiba"
              apiLevel = 34
          }
      }

      // Ensures the baseline profile is pulled from the device.
      // Note that this will be automated as well later with aosp/3272935.
      testOptions {
          results {
              directoriesToPull.addAll("/storage/emulated/0/Android/media/${android.namespace}")
          }
      }
  }

Также созданное устройство FTL необходимо добавить в расширение базового профиля:

  baselineProfile {
      managedDevices += "ftlDeviceShiba34"
      useConnectedDevices = false
  }

Версия 1.4.0-альфа01

18 сентября 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.4.0-alpha01 . Версия 1.4.0-alpha01 содержит эти коммиты .

Новая функция — статистика запуска приложений

Первоначальную версию статистики запуска приложений можно включить в Macrobenchmark. ( 09fae38 )

Чтобы включить в тесте запуска:

  @Test
  fun startup {
      macrobenchmarkRule.measureRepeated(
          …
          packageName = "com.example.my.application.id"
          metrics = listOf(StartupTimingMetric()),
          iterations = 5,
          startupMode = StartupMode.COLD,
          compilationMode = CompilationMode.None(),
          experimentalConfig = ExperimentalConfig(startupInsightsConfig = StartupInsightsConfig(isEnabled = true))
          ) {
          scope.startActivityAndWait(...)
      }
  }

Затем при запуске теста запуска будет анализироваться трассировка для поиска распространенных проблем и выводиться после метрик в выходные данные теста Studio на вкладке тестов, например:

StartupBenchmark_startup[startup=COLD,compilationMode=None]
├── Metrics
│   ├──   timeToFullDisplayMs                min  1,147.2,   median  1,208.8,   max  1,307.4
│   └──   timeToInitialDisplayMs             min  1,147.2,   median  1,208.8,   max  1,307.4
├── App Startup Insights
│   ├── App in debuggable mode (expected: false)
│   │   └── seen in iterations: 0(true) 1(true) 2(true) 3(true) 4(true) 5(true) 6(true) 7(true) 8(true) 9(true)
│   ├── Potential CPU contention with another process (expected: < 100000000ns)
│   │   └── seen in iterations: 4(105022546ns)
│   └── Main Thread - Binder transactions blocked (expected: false)
│       └── seen in iterations: 7(true)
└── Traces
    └── Iteration 0 1 2 3 4 5 6 7 8 9

Эта функция все еще находится в стадии разработки, ожидается улучшение документации и расширяемость, но отзывы приветствуются.

Новые возможности

Добавлено свойство gradle androidx.baselineprofile.suppressWarnings для подавления всех предупреждений базового профиля. ( 314153а )
Метрики Microbench теперь отображаются в кривых Perfetto в виде счетчиков. ( 3214854 )
Добавьте экспериментальные сценарии для отключения jit (требуется root/перезапуск во время выполнения) и сброса состояния производительности/тестирования устройства. В настоящее время они не публикуются как задачи Gradle. ( 7c3732b )
Добавлен аргумент теста для пропуска тестов при работе на эмуляторе. Если включена automaticGenerationDuring сборкаGenerationDuring, тесты также запускают создание базового профиля. Это не удастся, если используются эмуляторы. С новым аргументом skipBenchmarksOnEmulator мы можем пропустить тест. ( 0c2ddcd )
Изменение логики события perf для запуска на API 23+ ( 2550048 )

Изменения API

Существующий экспериментальный аргумент PerfettoConfig для MacrobenchmarkRule.measureRepeated() перемещен в новый объект ExperimentalConfig .

Исправления ошибок

Увеличьте количество повторов lockClocks.sh ( 99e9dac ).
Не создавайте nonMinified и тестовые типы сборок, если они существуют. Из-за ошибки, даже если существовали nonMinified и эталонные типы сборок, они должны были быть воссозданы. ( е75f0a5 )
Игнорировать непрерывные фрагменты результатов TraceSectionMetric . ( а927d20 )
Улучшена проверка эмулятора с учетом префикса sdk_ . ( 1587de8 )
Обрабатывать неработающие пакеты, как указано в FrameTimingGfxInfoMetric . ( 35cc79c )
Исправлена ошибка, из-за которой androidx.benchmark.cpuEventCounter создавал поврежденные значения для событий, не связанных с инструкциями. ( 06edd59 )
Исправьте resumeTiming/runWithTimingDisabled чтобы соблюдать порядок приоритетов метрик и значительно уменьшить влияние приостановки/возобновления метрик с более низким приоритетом на результаты метрик с более высоким приоритетом. Например, если использовать счетчики производительности процессора через аргумент инструментария cpuEventCounter.enable , timeNs больше не будет значительно уменьшаться при возникновении паузы или возобновления. ( 5de0968 )

Версия 1.3

Версия 1.3.4

26 марта 2025 г.

Выпущен androidx.benchmark:benchmark-*:1.3.4 . Версия 1.3.4 содержит эти коммиты .

Исправления ошибок

Исправлена несовместимость изоляции проекта Gradle в плагине Benchmark Baseline Gradle. ( б/404523257 )

Версия 1.3.3

16 октября 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.3 . Версия 1.3.3 содержит эти коммиты .

Исправления ошибок

Исправлена CollectBaselineProfileTask , когда в устройстве AVD есть пробелы ( Ia0225 , b/371642809 ).

Версия 1.3.2

2 октября 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.2 . Версия 1.3.2 содержит эти коммиты .

Исправления ошибок

Исправлена ошибка, из-за которой Firebase Test Lab (FTL) не могла извлечь файлы результатов базового профиля или макробенчмарка из плагина Gradle базового профиля. ( I2f678 , б / 285187547 )

Чтобы использовать FTL, примените плагин к модулю базового профиля в блоке плагинов, используя:

  plugins {
      ...
      id("com.google.firebase.testlab")
  }

а затем настройте тестовую лабораторию Firebase с помощью:

  firebaseTestLab {

      // Credentials for FTL service
      serviceAccountCredentials.set(file("credentials.json"))

      // Creates one or more managed devices to run the tests on.
      managedDevices {
          "ftlDeviceShiba34" {
              device = "shiba"
              apiLevel = 34
          }
      }

      // Ensures the baseline profile is pulled from the device.
      // Note that this will be automated as well later with aosp/3272935.
      testOptions {
          results {
              directoriesToPull.addAll("/storage/emulated/0/Android/media/${android.namespace}")
          }
      }
  }

Также созданное устройство FTL необходимо добавить в расширение базового профиля:

  baselineProfile {
      managedDevices += "ftlDeviceShiba34"
      useConnectedDevices = false
  }

Версия 1.3.1

18 сентября 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.1 . Версия 1.3.1 содержит эти коммиты .

Исправления ошибок

Добавлено свойство gradle androidx.baselineprofile.suppressWarnings для подавления всех предупреждений базового профиля ( I7c36e , b/349646646 ).
Исправлен плагин Gradle Baseline Profile, позволяющий использовать уже существующие nonMinified… и benchmark… если они созданы приложением, вместо создания оболочек. ( Я8934 , б/361370179 )
Исправлена java.lang.AssertionError: ERRORS (not suppressed): EMULATOR , когда на эмуляторах включено automaticGenerationDuringBuild . Вместо этого используется новый аргумент, чтобы пропустить тест. ( If3f51 , б/355515798 )
Минимизация микробенчмарка — сохраните подклассы org.junit.runner.notification.RunListener в библиотеке тестов proguard ( Ic8ed5 , b/354264743 ).
Исправьте TraceSectionMetric , чтобы игнорировать незавершающие фрагменты. Раньше считалось, что они имеют продолжительность -1, например, при суммировании или поиске минимальной продолжительности. ( Если74b7 )
Исправлена проблема в FrameTimingGfxInfoMetric , из-за которой запуск метрики приводил к сбою, если процесс еще не запущен. ( I6e412 )

Версия 1.3.0

21 августа 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.0 . Версия 1.3.0 содержит эти коммиты .

Изменения в микробенчмарке с версии 1.2.0

Трассировка методов включена по умолчанию в микротестах при работе на большинстве устройств.
- Трассировка метода выполняется как отдельный этап после измерений — это позволяет получать точные измерения и трассировки метода в результате одного запуска теста.
- Трассировка метода в некоторых версиях ОС Android и ART повлияет на последующие этапы измерения — в этих версиях трассировка метода отключена по умолчанию, и на выходе Studio выводится предупреждение.
Тесты основных потоков и ошибки ANR
- Добавлена measureRepeatedOnMainThread для тестов потоков пользовательского интерфейса (например, тех, которые взаимодействуют с пользовательскими интерфейсами Compose/View), чтобы избежать ошибок ANR при работе в течение многих секунд.
- Трассировки метода пропускаются, если ожидается, что они превысят крайний срок предотвращения ошибок ANR. Установите androidx.benchmark.profiling.skipWhenDurationRisksAnr значение false, чтобы отключить это поведение (не рекомендуется для запусков CI, поскольку ошибки ANR могут вызвать проблемы при длительных запусках CI).
Минимизация
- Встроенные правила proguard для улучшения микробенчмаркинга с включенной минификацией.
- Минимизация/R8 в библиотечном модуле требует AGP 8.3 и может быть включена через android.buildTypes.release.androidTest.enableMinification в вашем build.gradle
- Добавлен экспериментальный API BlackHole.consume() для предотвращения устранения неработающего кода ( If6812 , b/286091643 )
Метрики
- Экспериментальная функция счетчика событий процессора (метрики из perf_event_open , для которой требуется root на большинстве версий платформы), доступ через InstrumentationArgument androidx.benchmark.cpuEventCounter.enable (можно установить в true ), а androidx.benchmark.cpuEventCounter.events можно установить, например, в ( Instructions,CpuCycles ). Это должно поддерживаться в некоторых эмуляторах пользовательской отладки, но поддержка не тестировалась на всех доступных эмуляторах.

Изменения в MACRObenchmark с версии 1.2.0

Переработка метода трассировки макротестов.
- Теперь трассировки метода ограничены длительностью measureBlock и могут захватывать несколько сеансов, если процесс запускается несколько раз.
- Раньше трассировка метода работала только для тестов StartupMode.COLD и ничего не фиксировала для measureBlocks , которые не перезапустили целевой процесс.
- Исправлена очистка трассировок методов в макробенчмарке, поэтому трассировки методов должны быть полностью записаны и действительны даже на более медленных устройствах. ( I6349a , б/329904950 )
Корректно выгружать профиль ART во время отдельных итераций warmUp , когда процесс завершается, чтобы измерения CompilationMode.Partial(warmup=N) были более точными. ( I17923 )
Сообщение об ошибке трансляции Drop Shader
- Добавлены предложения по отладке для удаления сообщения об ошибке трансляции шейдера.
- Добавьте два аргумента инструментария для переопределения поведения отбрасывания шейдера для обхода сбоев при тестировании приложений без ProfileInstaller 1.3:
  - androidx.benchmark.dropShaders.enable=true/false : можно использовать для пропуска всех отбрасываний шейдеров (включая то, что происходит в StartupMode.Cold запусках), особенно при тестировании приложений, которые еще не используют Profileinstaller 1.3.
  - androidx.benchmark.dropShaders.throwOnFailure=true/false : может использоваться для допуска сбоев при попытке отбросить шейдеры, например, при тестировании приложений без Profileinstaller 1.3 ( I4f573 ).
Добавлен экспериментальный вариант MacrobenchmarkRule#measureRepeated , который использует специальный PerfettoConfig для полностью настраиваемой записи трассировки Perfetto. Обратите внимание, что неправильно настроенные конфигурации могут привести к сбою встроенных классов метрик. ( Idfd3d , б/309841164 , б/304038384 )
Отмените фоновые задания dexopt перед запуском Macrobenchmark, чтобы уменьшить помехи. ( I989ed )
Macrobenchmark теперь ожидает 1 секунду, пока целевое приложение сбросит профиль ART (ранее оно ждало 500 мс). ( И85а50 , б/316082056 )
TraceSectionMetric капитальный ремонт
- Примечание . Изменения TraceSectionMetric , приведенные ниже, могут повлиять на выходные данные при использовании CI и могут привести к разрывам или нарушению синтаксического анализа.
- Сумма теперь используется по умолчанию, поскольку эта метрика чаще всего используется для повторяющихся событий, и в этих случаях сначала данные отбрасываются.
- Изменено, чтобы сделать его более настраиваемым, с большим количеством доступных режимов.
- Имена режимов теперь встроены в имя выходного показателя метрики (в Studio и JSON).
- Теперь поддерживает фрагменты, созданные с помощью Trace.{begin|end}AsyncSection .
Метрики
- Питание — добавлены PowerMetric.deviceSupportsHighPrecisionTracking , PowerMetric.deviceBatteryHasMinimumCharge() и PowerMetric.deviceSupportsPowerEnergy()
- Metric.getResult переименован в getMeasurements в соответствии с типом возвращаемого значения.
- Добавлены метки log.w/Exception для всех ошибок обнаружения запуска. Это не меняет текущего поведения (поэтому некоторые ошибки выдаются, а другие молча не обнаруживают запуск), просто делает его более понятным. Обычно те, которые Log.w() не сообщают о показателях запуска, — это те, в которых отсутствуют события, не связанные с кадрами, исключения выдаются при обнаружении запуска, за исключением информации о синхронизации кадра (из срезов UI/RT). ( Id240f , б/329145809 )
- В FrameTimingMetric добавлено frameCount , чтобы помочь обнаружить сценарии, в которых измерения изменяются из-за изменения количества созданных кадров (добавлены новые анимации, исправлены проблемы с недействительностью). ( I1e5aa )
- Уточнено, что frameOverrunMs является предпочтительной метрикой для отслеживания, когда она доступна в документах, и почему. ( I18749 , б/329478323 )
- Устранена проблема, из-за которой незавершенные кадры в начале и конце трассировки могли быть объединены в пары, что ошибочно регистрировалось как один очень длинный кадр. ( I39353 , б/322232828 )
- Устраните ошибку FrameTimingMetric , когда кадры не создаются, и всегда выводите ссылку для отслеживания в случае сбоя анализа метрики, чтобы помочь в диагностике проблемы. ( I956b9 )
- Исправлен сбой в FrameTimingMetric , из-за которого не удавалось проанализировать идентификатор кадра, особенно на некоторых OEM-устройствах. ( Ia24bc , б/303823815 , б/306235276 )
- Ослаблена строгость проверок в FrameMetrics и добавлена более подробная информация о сообщениях об ошибках. ( Иадеде )

Захват базового профиля / изменения плагина Gradle с версии 1.2.0

Максимальная рекомендуемая версия AGP увеличена до 9.0.0-alpha01.
Убедитесь, что задачи mergeArtProfile и mergeStartupProfile всегда ожидают создания базового профиля. ( I623d6 , б/343086054 )
При успешном создании базового профиля будет выведена сводка того, что изменилось ( I824c8 , b/269484510 ).
Добавлен DSL для отключения предупреждений ( Ic4deb , b/331237001 ).
Исправлено, чтобы гарантировать, что тесты используют сгенерированные базовые профили, когда automaticGenerationDuringBuild отключен ( Ic144f , b/333024280 ).
Исправлены переопределения свойств плагина BaselineProfile gradle, чтобы включить генерацию базового профиля и тестирование при настройке типа сборки nonMinified или эталонного теста. ( Ib8f05 , b/324837887 )
Исправлено включение базовых профилей библиотеки в AAR до AGP 8.3.0-alpha15. ( I1d2af , б/313992099 )
Исправлен исходный URL-адрес базового и стартового профиля в конце задачи генерации. ( I802e5 , б/313976958 )

Другие существенные изменения с версии 1.2.0

Захват трассировки
- Уменьшена ошибка EXITCODE 2 при запуске perfetto с ошибки на зарегистрированное предупреждение.
- Включить трассировку AIDL по умолчанию в тестах (требуется API 28) ( Ia0af2 , b/341852305 )
- Включите трассировку тегов портера по умолчанию в тестах. Это фиксирует, например, точки трассировки блокировки. ( Icfe44 , р/286551983 )
- Увеличено время ожидания начала захвата трассировки, чтобы избежать сбоев при запуске трассировки на более медленных устройствах ( I98841 , b/329145808 ).
- Добавлены общедоступные API PerfettoTraceProcessor.Session.queryMetrics с вариантами JSON, textproto и proto двоичными (некодированными). Они позволяют запрашивать метрики, встроенные в TraceProcessor ( I54d7f , b/304038382 ).
- Включите блокировку начала записи трассировки Perfetto, чтобы снизить риск потери данных в начале трассировки. Поддерживается только в API 33+. ( Ie6e41 , b/310760059 )
Вывод JSON
- Добавлена дополнительная информация в контексте теста в вывод JSON:
  - context.artMainlineVersion — целочисленная версия основного модуля Art (если он присутствует на устройстве, в противном случае -1 )
  - context.build.id – равен android.os.Build.ID.
  - context.build.version.codename — равно android.os.Build.VERSION.CODENAME
  - context.build.version.abbreviatedCodename — соответствует первой букве кодового имени предварительной версии (в том числе в сборках выпуска) ( Ie5020 ).
- Добавлен список profilerOutput в выходные данные JSON для упрощения работы с трассировками профилирования (например, трассировки Perfetto, Method) ( I05ddd , b/332604449 ).
- Добавлено предупреждение при использовании Android Test Orchestrator в модулях тестирования, так как это приведет к многократной перезаписи выходных файлов JSON для каждого модуля. ( Ia1af6 , б/286899049 )
- Выдавайте, когда имена файлов длиннее 200 символов, чтобы избежать неясных сбоев при записи или постобработке файлов. ( I4a5ab )

Версия 1.3.0-rc01

7 августа 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.0-rc01 . Версия 1.3.0-rc01 содержит эти коммиты .

Исправления ошибок

Исправлена ошибка, из-за которой androidx.benchmark.cpuEventCounter создавал поврежденные значения для событий, не связанных с инструкциями ( I7386a , b/286306579 ).
Исправьте resumeTiming / runWithTimingDisabled , чтобы соблюдать порядок приоритетов метрик и значительно уменьшить влияние приостановки/возобновления метрик с более низким приоритетом на результаты метрик с более высоким приоритетом. Например, если использовать счетчики производительности процессора через аргумент инструментария cpuEventCounter.enable , timeNs больше не будет значительно уменьшаться при возникновении паузы или возобновления. ( I39c2e , б/286306579 , б/307445225 )
Уменьшена вероятность того, что выборка стека приведет к тому, что measureRepeatedOnMainThread достигнет жесткого тайм-аута основного потока, за счет перемещения преобразования выборки стека за пределы основного потока. ( I487a8 , б/342237318 )
Убрано ручное определение доступа к API новой платформы, поскольку это происходит автоматически посредством моделирования API при использовании R8 с AGP 7.3 или новее (например, R8 версии 3.3) и для всех сборок при использовании AGP 8.1 или новее (например, D8 версии 8.1). Клиентам, не использующим AGP, рекомендуется выполнить обновление до D8 версии 8.1 или более поздней. Более подробную информацию можно найти в этой статье . ( I9496c , б/345472586 )
Добавлена проверка версии agp для отправки имени пакета в качестве instr arg. До версии AGP 8.4.0 имя пакета целевого приложения нельзя было отправить в приложение инструментирования через аргументы инструментария. ( 0c72a3f )

Версия 1.3.0-beta02

10 июля 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.0-beta02 . Версия 1.3.0-beta02 содержит эти коммиты .

Исправления ошибок

Аккуратно обрабатывайте EXITCODE 2 при запуске Perfetto, чтобы записать предупреждение, но продолжайте.

Версия 1.3.0-бета01

12 июня 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.0-beta01 . Версия 1.3.0-beta01 содержит эти коммиты .

Изменения API

Для обеспечения единообразия MethodTracing.affectsMeasurementOnThisDevice переименован в AFFECTS_MEASUREMENT_ON_THIS_DEVICE . ( I1bdfa )
Добавлен экспериментальный API BlackHole.consume() для предотвращения устранения неработающего кода в микробенчмарках. ( Если6812 , б/286091643 )
Микробенчмарк теперь будет корректно генерировать результаты, чтобы трассировка метода не мешала измерениям. Это происходит на некоторых устройствах, когда принудительно включена трассировка метода (через аргументы инструмента или MicrobenchmarkConfig ), а также если предпринимается попытка измерения после трассировки метода. Затронутые устройства используют API 26–30 или некоторые версии основного модуля ART, на которые влияет это вмешательство, и могут быть обнаружены во время выполнения с помощью ProfilerConfig.MethodTracing.affectsMeasurementOnThisDevice . ( Iafb92 , б/303660864 )

Исправления ошибок

Рекомендуемая максимальная версия agp увеличена до 9.0.0-alpha01. ( I5bbb0 )
Добавлен режим компиляции в контекст тестирования ( If5612 , b/325512900 ).
Включить трассировку AIDL по умолчанию (требуется API 28) ( Ia0af2 , b/341852305 )
Добавлена дополнительная информация в контексте теста в выводе JSON:
- context.artMainlineVersion — целочисленная версия основного модуля Art (если он присутствует на устройстве, в противном случае -1)
- context.build.id – равен android.os.Build.ID
- context.build.version.codename — равно android.os.Build.VERSION.CODENAME
- context.build.version.abbreviatedCodename — соответствует первой букве кодового имени предварительной версии (даже в сборках выпуска) ( Ie5020 ).
Исправляет StackSampling для соблюдения androidx.benchmark.profiling.sampleDurationSeconds ( Ib1d53 ).
Измените зависимость macro->common на api() , чтобы ее было проще использовать, например, PerfettoTrace и PerfettoConfig . ( Icdae3 , б/341851833 )
Убедитесь, что задачи mergeArtProfile и mergeStartupProfile всегда ожидают создания базового профиля. ( I623d6 , б/343086054 )
Учитывайте состояние включения варианта при принятии решения о том, следует ли включать вариант. ( I5d19e , б/343249144 )
Увеличено время ожидания запуска по умолчанию для процессора трассировки perfetto. ( I87e8c , б/329145808 )

Версия 1.3.0-альфа05

14 мая 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.0-alpha05 . Версия 1.3.0-alpha05 содержит эти коммиты .

Исправления ошибок

Выдавать более четкое исключение, когда метрика Macrobench возвращает нулевые значения для всех итераций ( Iab58f , b/314931695 )
В правила proguard microbench добавлены дополнительные правила обхода, включая поддержку правил прослушивателя и других наблюдаемых предупреждений/ошибок. ( I14d8f , б / 329126308 , б / 339085669 )
Трассировка метода выполняется как отдельный этап макротестирования и больше не влияет на измерения. ( If9a50 , б/285912360 , б/336588271 )
Добавлены дополнительные предложения по отладке для удаления сообщения об ошибке трансляции шейдера. ( I5efa6 , б/325502725 )

Версия 1.3.0-альфа04

1 мая 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.0-alpha04 . Версия 1.3.0-alpha04 содержит эти коммиты .

Изменения API

Добавлен экспериментальный вариант MacrobenchmarkRule#measureRepeated , который использует специальный PerfettoConfig для полностью настраиваемой записи трассировки Perfetto. Обратите внимание, что неправильно настроенные конфигурации могут привести к сбою встроенных классов метрик. ( Idfd3d , б/309841164 , б/304038384 )
Переименуйте PowerMetric.deviceSupportsPowerEnergy в PowerMetric.deviceSupportsHighPrecisionTracking для ясности ( I5b82f ).
Добавлены PowerMetric.deviceBatteryHasMinimumCharge() и PowerMetric.deviceSupportsPowerEnergy() позволяющие изменять или пропускать тесты на основе возможностей измерения мощности устройства. ( I6a591 , б/322121218 )

Исправления ошибок

Добавлено сравнение с предыдущим базовым профилем ( I824c8 , b/269484510 ).
Добавлен DSL для отключения предупреждений ( Ic4deb , b/331237001 ).
Изменено исключение в журнале информации, когда варианты тестов отключены ( I8a517 , b/332772491 ).
Упростите сбор трассировок методов для макробенчмарка, ограниченного продолжительностью фактического measureBlock() . Раньше он запускался при запуске целевого процесса и поддерживал только холодный запуск ( Iee85a , b/300651094 ).
Избегайте сбоев, когда процессор трассировки perfetto запускается медленно ( I98841 , b/329145808 ).

Версия 1.3.0-альфа03

17 апреля 2024 г.

Выпущен androidx.benchmark:benchmark-*:1.3.0-alpha03 . Версия 1.3.0-alpha03 содержит эти коммиты .

Новые возможности

Добавляет общедоступные API PerfettoTraceProcessor.Session.queryMetrics с вариантами JSON, textproto и proto двоичными (некодированными) вариантами. Они позволяют запрашивать метрики, встроенные в TraceProcessor ( I54d7f , b/304038382 ).
Добавлено profilerOutput к выходу JSON для более легкого инструмента для профилирования (например, перфетто, методов). ( I05DDD , B/332604449 )
Добавлена тег питания к конфигурации Perfetto. Это захватывает, например, Wakelock Tracepoints. ( ICFE44 , B/286551983 )
Добавлен аргумент ind androidx.benchmark.profiling.skipWhenDurationRisksAnr , может быть установлен на false, чтобы не пропустить трассировки метода, когда ожидаемая продолжительность может вызвать ANR - настоятельно рекомендуется избегать в прогонах CI.
Добавлен экспериментальный аргумент androidx.benchmark.profiling.perfCompare.enable , установите это на true для выполнения времени сравнения между фазами измерения и профилирования. Полезно, например, оценка накладных расходов методов. ( I61fb4 , b/329146942 )

Изменения API

Изменено TraceSectionMetric.Mode на герметичный класс, чтобы обеспечить будущее расширение без исчерпывания, когда операторы ( i71f7b )
Добавлены TraceSectionMetric.Mode.Average и .Count , и переупорядоченные ARG, поэтому более распространенный аргумент (режим) был ранее в списке ARG, снижая необходимость определения имен параметров. ( IBF0B0 , B/315830077 , B/322167531 )
Переименованная Metric.getResult для getMeasurements в соответствии с типом возврата ( i42595 )

Исправления ошибок

Исправление для обеспечения использования контрольных показателей сгенерированных базовых профилей, когда выключено automaticGenerationDuringBuild ( IC144F , B/333024280 )
Исправьте переопределение свойств плагина BaselineProfile Gradle, чтобы включить базовую генерацию профиля и сравнительный анализ при настройке nonMinified или сравнительного типа сборки. ( IB8F05 , B/324837887 )
Фиксированные следы метода промывки в Macrobenchmark, поэтому следы метода должны быть полностью захвачены и действительными даже на более медленных устройствах. ( I6349a , b/329904950 )
Включите начало блокировки на перфетто, чтобы снизить риск отсутствия данных в начале трассировки. Поддерживается только на API 33+. ( IE6E41 , B/310760059 )
Добавлено предупреждение, когда тестовый оркестратор Android используется в эталонных модулях, так как это приведет к повторному перезаписыванию файлов json для каждого модуля. ( IA1AF6 , B/286899049 )
Force ',' (Comma) тысячи сепараторов для последовательности при выводе студии, игнорируя локали устройства ( i3e921 , b/313496656 )
TraceSectionMetric теперь поддерживает срезы, созданные с использованием Trace.{begin|end}AsyncSection . ( I91b32 , b/300434906 )
Добавлены метки log.w / исключения для всех сбоев обнаружения запуска. Это не меняет текущее поведение (поэтому некоторые ошибки бросают, а другие молча не смогут обнаружить запуск), просто делает его более понятным. Как правило, те, которые Log.w() и не сообщают о метриках запуска, являются теми, где отсутствуют некадровые события, исключения выявляются при обнаружении запуска, за исключением информации о времени рамки (из срезов пользовательского интерфейса/RT). ( ID240F , B/329145809 )
Отмените фон Dexopt задания перед запуска Macrobenchmark, чтобы уменьшить помехи. ( I989ed )
Добавлено измерение frameCount в FrameTimingMetric для помощи в обнаружении сценариев, где измерения изменяются, потому что количество произведенных кадров изменилось (добавлены новые анимации, проблемы с недействительностью фиксированы). ( I1e5aa )
Уточнил, что frameOverrunMs является предпочтительной метрикой для отслеживания, когда она доступна в документах, и почему. ( I18749 , B/329478323 )

Версия 1.3.0-Alpha02

20 марта 2024 г.

androidx.benchmark:benchmark-*:1.3.0-alpha02 выпускается. Версия 1.3.0-Alpha02 содержит эти коммиты .

Новые возможности

Экспериментальная поддержка R8 в микрофенде через встроенные правила прогиба. Обратите внимание, что эта поддержка является экспериментальной, и требует AGP 8.3 для министерства библиотечных модульных тестов. Используйте следующее, чтобы включить Minifice/Optimization/оптимизацию R8 в build.gradle вашего эталонного модуля, что должно привести к значительному повышению производительности, в зависимости от рабочей нагрузки. ( I738a3 , b/184378053 )
```
android {
    buildTypes.release.androidTest.enableMinification = true
}
```

Исправления ошибок

Исправляет предупреждение о отслеживании метода, чтобы быть на отдельной линии от выхода микрофенга. ( I0455C , B/328308833 )

Версия 1.3.0-Alpha01

21 февраля 2024 года

androidx.benchmark:benchmark-*:1.3.0-alpha01 выпускается. Версия 1.3.0-Alpha01 содержит эти коммиты.

Изменения API

Переименованные в MicrobenchmarkConfig Boolean Parameters, чтобы избежать ненужного слова «должно» ( IA8F00 , B/303387299 )
Добавлены BenchmarkRule.measureRepeatedOnMainThread , поэтому основные резьбы (например, касаются видов или сочинения UI) могли избежать запуска ANR, особенно во время больших люксов в CI. ( I5c86d )
Добавлена FrameTimingGfxInfoMetric , экспериментальная альтернативная реализация FrameTimingMetric с измерениями, поступающими непосредственно с платформы, а не извлечена из трассировки Perfetto. ( I457CB , B/322232828 )
Добавьте возможность сбросить художественный профиль во время индивидуальных итераций warmUp . ( I17923 )
Несколько изменений в TraceSectionMetric API:
- Добавить Mode.Min , Mode.Max
- Добавить аргумент метки, чтобы переопределить имя раздела как метрическое метрическое
- Добавлено имя режима для вывода, чтобы прояснить значение метрики
- Изменено по умолчанию на сумму, так как большинство использования этой метрики предназначено для повторных событий, будьте в курсе этих изменений в использовании CI, поскольку это может создавать разрывы или разорение. ( IC1E82 , B/301892382 , B/301955938 )

Исправления ошибок

Улучшенное сообщение об ошибке в базовом плагине Gradle Profile, когда указанное управляемое устройство не существует ( IDEA2B , B/313803289 )
Исправление для включения базовых профилей библиотеки в AAR до AGP 8.3.0-ALPHA15 ( I1D2AF , B/313992099 )
Исключенный URL -адрес профиля и профиль запуска в конце задачи генерации ( I802E5 , B/313976958 )
Скорректированные тайм -ауты источника данных, чтобы попытаться исправить java.lang.IllegalStateException: Failed to stop [ProcessPid(processName=perfetto, pid=...)] ( i8dc7d , b/323601788 )
Добавьте два аргумента инструментальных приборов для переоценки поведения с падением шейдеров в обходные сбои, когда сравнительные приложения без ProfileInstaller 1.3:
- androidx.benchmark.dropShaders.enable=true/false : можно использовать для пропуска всех падений шейдеров (включая то, что выполняется в StartupMode.Cold Launches), ESP при сравнительном анализе, которые еще не используют ProfileInstaller 1.3
- androidx.benchmark.dropShaders.throwOnFailure=true/false : можно использовать для переноса отказов при попытке сбросить шейдеры, например, когда приложения сравнительного анализа без профиля INSTALLER 1.3 ( I4F573 )
Пропустить метод отслеживания на нити пользовательского интерфейса, когда ожидается, что при броске займет больше времени, чем несколько секунд, а метод очистки. ( I6e768 )
Бросьте, когда имена файлов длиннее 200 Chars, чтобы избежать неясных сбоев при написании или пост-обработке. ( I4a5ab )
Исправляет проблему, в которой непосмертные рамки в начале и в конце трассировки могут быть соединены вместе, что неправильно сообщало бы как единственный чрезвычайно длинный кадр. ( I39353 , B/322232828 )
Используйте --skip verification на API 30+ при переустановке пакета на API 30-33 для очистки художественных профилей на сборках пользователей. Это помогает обойти пьесу защиту предупреждений, которые вызывают неудачи на некоторых классах устройств. ( IC9E36 )
Используйте am force-stop для убийства приложений, когда не системное приложение, такое как системный пользовательский интерфейс или пусковая установка. ( I5e028 )
Macrobenchmark теперь ожидает 1 second , когда целевое приложение пропустит профиль искусства (ранее он ждал 500 ms ). ( I85A50 , B/316082056 )
Улучшение FrameTimingMetric Error, когда кадры не производятся, и всегда выводит ссылку на трассировку при сбое метрического анализа, чтобы помочь в диагностике проблем. ( I956b9 )
Фиксированный сбой в FrameTimingMetric сбое для анализа идентификатора рамки, особенно на определенных устройствах OEM. ( IA24BC , B/303823815 , B/306235276 )
Расслабленная строгость чеков в FrameMetrics и добавил больше деталей в сообщения об ошибках. ( Iadede )

Версия 1.2

Версия 1.2.4

17 апреля 2024 года

androidx.benchmark:benchmark-*:1.2.4 выпускается. Версия 1.2.4 содержит эти коммиты .

Исправления ошибок

Исправляет базовый профиль SRCSET, который не настраивается в эталонных вариантах. Также фиксирует automaticGenerationDuringBuild в библиотеках, вызывая круговую зависимость. ( I28ab7 , b/333024280 )
Используйте am force-stop для убийства приложений, когда не системное приложение, такое как системный пользовательский интерфейс или пусковая установка. Это исправляет StartupMode.COLD Clackmarks, выходящие из пакета "Package $ Package не должны работать до холодного запуска!" Из -за процесса убийства не полностью успешно. ( I5e028 )

Версия 1.2.3

24 января 2024 года

androidx.benchmark:benchmark-*:1.2.3 выпущен. Версия 1.2.3 содержит эти коммиты.

Исправления ошибок

Удаленное исключение из плагина базового профиля Gradle, когда версия AGP составляет 8.3.0 или выше.
Исправление для включения базовых профилей библиотеки в AAR до AGP 8.3.0-Alpha15.

Версия 1.2.2

1 декабря 2023 года

androidx.benchmark:benchmark-*:1.2.2 выпущен. Версия 1.2.2 содержит эти коммиты.

Базовые профили

В журналах выполнения покажут выходной путь базового профиля в виде локального URI файла ( AOSP/2843918 , AOSP/2853665 , B/313976958 )

Версия 1.2.1

15 ноября 2023 года

androidx.benchmark:benchmark-*:1.2.1 выпускается. Версия 1.2.1 содержит эти коммиты.

Новые возможности

Улучшенное сообщение об ошибке, когда пользователь отключает варианты тестирования ( B/307478189 )
Добавлены свойства для поддержки в качестве интеграции тестового прогона ( B/309805233 ), ( B/309116324 )

Версия 1.2.0

18 октября 2023 года

androidx.benchmark:benchmark-*:1.2.0 выпускается. Версия 1.2.0 содержит эти коммиты.

Важные изменения с 1.1.0

Базовые профили

Новый плагин Gradle Baseline Profile автоматизирует захват и включение базовых профилей в ваш рабочий процесс тестирования и строительства.
BaselineProfileRule.collect теперь Stable, оптимизированная и упрощенная версия предыдущего экспериментального BaselineProfileRule.collectBaselineProfile
- Просто укажите packageName и приведите свое приложение
Для библиотек, генерирующих базовые профили, теперь вы можете отфильтровать правила, сгенерированные либо в коде ( BaselineProfileRule.collect Argemt), либо даже просто в плагине Gradle
Исправления
- Фиксированный базовый сбор профилей на Android U+ ( ID1392 , B/277645214 )

Macrobenchmark

Сборник
- MacRobenchmark теперь правильно полностью сбрасывает состояние компиляции для каждого компиляции - это требует переустановки APK перед Android 14, поэтому настойчиво рекомендуется сравнение на Android 14+, если вы хотите сохранить состояние (например, вход пользователя) в том, что измеряется.
- Вы также можете обойти это, контролируя компиляцию приложений отдельно и пропустив компиляцию с помощью CompilationMode.Ignore() или аргумента инструментации
Аргументы в области инструментов
- Поддержка androidx.benchmark.dryRunMode.enable Инструментария аргумента (уже доступна в MicroBenchmark) для более быстрых пробелов валидации (например, при создании эталона или в Presubmit)
- Поддержка androidx.benchmark.profiling.mode=StackSampling и MethodTracing .
- Добавлено androidx.benchmark.enabledRules , чтобы разрешить фильтрацию времени выполнения.
- Добавлен androidx.benchmark.perfettoSdkTracing.enable аргумент, чтобы обеспечить отслеживание с трассировкой-перфетто, например, Compose Cose Cose Cose Cose Cosepose. Обратите внимание, что при использовании с StartupMode.COLD время будет значительно затронуто, так как библиотека трассировки загружается и включена во время запуска приложения.
Требования
- Macrobenchmark теперь требует ProfileInstaller 1.3.0 или более в целевом приложении, чтобы включить захват / сброс профиля и очистку кеша шейдера.
Новые экспериментальные метрические API
- Добавлен экспериментальный TraceSectionMetric , который позволяет извлечь простое время из блоков trace("") {} в вашем приложении, или Tracemetric для использования полной возможности запроса TraceProcessor .
- Добавлен экспериментальный PowerMetric для захвата информации об использовании энергии
- Добавлено экспериментальные MemoryCountersMetric чтобы считать разломы страниц.
- Добавлен экспериментальный PerfettoTraceProcessor API, который внутри используется для извлечения метрик из системных следов (так как Perfetto Traces)
Исправления
- Фиксированные сбои при установке или извлечении профилей из приложения, установленного из нескольких APK (например, из пакета приложений).
- Фиксированные FrameTimingMetric игнорирующие рамки с непоследовательными идентификаторами рамки (как правило, кадры во время RIPI на API 31+) ( I747D2 , B/279088460 )
- Фиксированные ошибки анализа на следах> 64 МБ ( IEF831 , B/269949822 )
- Уточненные ошибки, когда устройство ОС Device (особенно эмулятор) неправильно настроено для отслеживания или компиляции
- Проверка проверки уровня батареи на устройства без батареи (микро и макро)
- Улучшенный выход файлов, с более четкими ошибками для неверных каталогов вывода и более безопасных по умолчанию
- Улучшенная стабильность StartupMode.COLD , последовательно сбрасывая кэш шейдеров (также выявленную через MacrobenchmarkScope.dropShaderCache )
- Фиксированная запасная сторона Leanback для startActivityAndWait .

Microbenchmark

Функции
- Профилирование было перемещено на отдельную фазу после других показателей, поэтому один тестовый прогон может отображать как точные результаты времени , так и результаты профилирования.
Экспериментальные API
- Добавлен экспериментальный API MicrobenchmarkConfig для определения пользовательских метрик и настройки трассировки и профилирования. Может использоваться для захвата трассов метода или захвата трацепоинтов (но имейте в виду отслеживание накладных расходов).
- Добавлены экспериментальные API для контроля BenchmarkState отдельно от BenchmarkRule , без Junit
- Добавлена экспериментальная запись PerfettoTrace , чтобы обеспечить захват трассы перфетто, с пользовательской конфигурацией, отделенной от контрольных API.
Исправления
- Обходной путь отсутствует ведущие пробелы в выходе на эталона Android Studio.
- Исправлена проблема, где предупреждения могут не напечатать в результате вывода Android Studio.
- Исправлено фиксированное сбоев SampledProfiling на Android 13 (API 33) и выше.
- Массовая улучшенная производительность dryRunMode , пропустив IsolationActivity и трассировку перфетто (в 10 раз быстрее сухой режим пробега в старых версиях ОС).

Версия 1.2.0-RC02

6 октября 2023 года

androidx.benchmark:benchmark-*:1.2.0-rc02 выпущен. Версия 1.2.0-RC02 содержит эти коммиты.

Исправления ошибок

Исправьте результат Benchmark File, чтобы больше не сломать BaselineProfile плагин. Файлы были сгенерированы и скопированы с устройства, но были переименованы так, чтобы плагин Gradle не видел их. ( I8DBCC , B/303034735 , B/296453339 )
Уточненная tracing-perfetto Загрузка сообщений об ошибках при введении из модуля Macrobenchmark в целевое приложение.

Версия 1.2.0-RC01

20 сентября 2023 года

androidx.benchmark:benchmark-*:1.2.0-rc01 выпущен. Версия 1.2.0-RC01 содержит эти коммиты.

Исправления ошибок

Исключение (с инструкциями по лекарствам) теперь бросается, когда перфетто SDK Трассировка не может инициализироваться в эталонном. ( I6c878 , b/286228781 )
Исправить сбой OOM при преобразовании метода Art Trace -> Perfetto Format. ( I106BD , B/296905344 )
(Macrobenchmark) Уточненная метка трассировки метода при связи в выходе из студийного теста, а фиксированные имена файлов с фиксированным методом были уникальными на устройстве/хосте, поэтому они не будут перезаписаны, когда запускается более одного эталона. ( I08E65 , B/285912360 )
Гарантирует, что устройство бодрствует при захвате базового профиля. ( I503fc )

Версия 1.2.0-бета05

30 августа 2023 года

androidx.benchmark:benchmark-*:1.2.0-beta05 выпускается. Версия 1.2.0-бета05 содержит эти коммиты.

Новые возможности

Плагин базового профиля Gradle теперь поддерживает плагин Android Gradle 8.3. ( AOSP/2715214 )

Версия 1.2.0-бета04

23 августа 2023 года

androidx.benchmark:benchmark-*:1.2.0-beta04 выпускается. Версия 1.2.0-бета04 содержит эти коммиты.

Новые возможности

Базовый плагин Gradle теперь поддерживает плагин Android Gradle 8.3. ( AOSP/2715214 )

Исправления ошибок

Исправьте сбои в письменной / перемещении и вытягивании файлов (особенно из параметризованных тестов) путем дезинфицирующегося выходных имен файлов, избегая '=' и ':' в именах выходных файлов. ( I759d8 )

Версия 1.2.0-бета03

9 августа 2023 года

androidx.benchmark:benchmark-*:1.2.0-beta03 выпускается. Версия 1.2.0-бета03 содержит эти коммиты.

Изменения API

Добавлен аргумент в TraceSectionMetric только к целевому пакету, по умолчанию ( IA219B , B/292208786 )

Исправления ошибок

Переименованное fullTracing.enable perfettoSdkTracing.enable fullTracing.enable будет продолжать работать в качестве запасного. ( I7cc00 )
Внутренние трацепоинты библиотеки (включая петлю/фазовое отслеживание MicroBenchmark) теперь будут отображаться в Studio System System Trace Seecter и гнездятся под правильным процессом в перфетто. ( I6b2e7 , b/293510459 )
Удаленная ошибка MacRobenchmark на API 31+, а также пропустить проверку на устройствах, корневых eng/userdebug. ( I2abac , b/291722507 )
При использовании оптимизации макета DEX правила профиля запуска также теперь рассматриваются как базовые правила профиля. ( AOSP/2684246 , B/293889189 )

Версия 1.2.0-бета02

26 июля 2023 года

androidx.benchmark:benchmark-*:1.2.0-beta02 выпускается. Версия 1.2.0-бета02 содержит эти коммиты.

Изменения API

Добавлены экспериментальные API для пользовательских метрик и конфигурации MicroBench (например, профилировщик и трассировка). ( I86101 , B/291820856 )

Исправления ошибок

Ошибка отчета в Macrobench, когда ОС неправильно настроена для отслеживания, как это было недавно исправлено в эмуляторах API 26/28 ARM64. ( I0A328 , B/282191686 )
Добавлены подробные для сброса компиляции, сбросить неспособность предложить обновление эмулятора, так как некоторые эмуляторы недавно исправили эту проблему. ( I8c815 , b/282191686 )
Сделайте androidx.test.uiautomator:uiautomator:2.2.0 api вместо зависимости implementation . ( I1981E )

Версия 1.2.0-бета01

18 июля 2023 года

androidx.benchmark:benchmark-*:1.2.0-beta01 выпускается. Версия 1.2.0-бета01 содержит эти коммиты.

Исправления ошибок

Исправить предупреждения, которые иногда подавляются в результате вывода в студии, и ведущие обходные пробелы из базовых выводов, не отображаемых в студии ( IA61D0 , B/227205461 , B/286306579 , B/285912360 )
Фиксированный комментарий для FrameTimingMetric . Субметрия называется frameDurationCpuMs . ( IB097F , B/288830934 ).

Версия 1.2.0-Альфа16

21 июня 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha16 выпускается. Версия 1.2.0-Alpha16 содержит эти коммиты.

Изменения API

BaselineProfileRule.collectBaselineProfile() API был переименован в BaselineProfileRule.collect() . ( I4b665 )

Исправления ошибок

Поддержка Macrobenchmark для androidx.benchmark.profiling.mode = MethodTracing . ( I7ad37 , b/285912360 )
Профилирование MicroBenchmark перемещалось в отдельную фазу, поэтому оно происходит в последовательности после измерения, вместо того, чтобы заменить его. Разделы трассировки MethodTracing и теперь также включены в захваченную трассу перфетто, если присутствуют. ( I9f657 , b/285014599 )
Добавьте измерение счета в TraceSectionMetric с Mode.Sum . ( IC121A , B/264398606 )

Версия 1.2.0-Альфа15

7 июня 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha15 выпускается. Версия 1.2.0-Alpha15 содержит эти коммиты.

Новые возможности

Добавлена экспериментальная MemoryUsageMetric , чтобы отслеживать использование памяти целевого приложения. ( I56453 , b/133147125 , b/281749311 )
Добавьте поддержку для полностью настраиваемых конфигураций перфетто с помощью PerfettoTrace.record ( if9d75 , b/280460183 )
Добавлено свойство, чтобы пропустить базовый профиль. Использование: ./gradlew assemble -Pandroidx.baselineprofile.skipgeneration . ( I37FDA , B/283447020 )

Изменения API

API collectBaselineProfile всегда генерирует стабильные базовые профили. API collectStableBaselineProfile был удален, и вместо этого следует использовать collectBaselineProfile . ( I17262 , B/281078707 )
Изменено BaselineProfileRule filterPredicate Arg на NON-NULL, с эквивалентным значением по умолчанию, так что поведение фильтра по умолчанию более ясное в DOCS. ( I3816e )

Исправления ошибок

Отключить IsolationActivity и отслеживание перфетто в dryRunMode чтобы значительно повысить производительность, так как это было большинство времени выполнения. ( IE4F7D )
Поддержка выборки стека вызовов в MacRobenchMarks с использованием аргументов испытаний на инструмент androidx.benchmark.profiling.mode=StackSampling и androidx.benchmark.profiling.sampleFrequency . ( I1d13b , b/282188489 )
Исправляет сбой при сброске шейдеров на Android U (API 34), а также на эмуляторы. ( I031CA , B/274314544 )

Версия 1.2.0-Альфа14

3 мая 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha14 выпускается. Версия 1.2.0-Alpha14 содержит эти коммиты.

Исправления ошибок

Исправьте FrameTimingMetric игнорирующие рамки с непоследовательными идентификаторами кадра. Это приведет к тому, что некоторые анимации в недавних версиях платформы (API 31+) игнорируют многие кадры, в то время как RenderThread анимировал (например, во время Ripple). ( I747d2 , b/279088460 )
Фиксированный анализ процессора трассировки для следов более 64 МБ. ( IEF831 , B/269949822 )
Фиксированная базовая генерация профиля на Android U сбой из-за различных выводов команды pm dump-profiles . ( ID1392 , B/277645214 )
Исправить сценарий блокировки часов GPU для правильного сравнения строк ( i53e54 , b/213935715 )

Версия 1.2.0-Альфа13

5 апреля 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha13 выпускается. Версия 1.2.0-Alpha13 содержит эти коммиты.

Изменения API

Добавлен параметр типа профиля при генерации базовых профилей для поддержки предстоящей функции профиля запуска ( IE20D7 , B/275093123 )
Добавлен новый экспериментальный TraceMetric API для определения полностью индивидуальных метрик на основе содержания перфетто. ( I4ce31 , b/219851406 )
Добавьте экспериментальную метрику, чтобы определить количество разломов страниц во время эталона. ( I48db0 )

Версия 1.2.0-Альфа12

22 марта 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha12 выпускается. Версия 1.2.0-Alpha12 содержит эти коммиты.

Новые возможности

Новый плагин Gradle Baseline Profile выпущен в версии Alpha, что облегчает генерацию базового профиля и упрощает рабочий процесс разработчика.

Изменения API

Удаленная поддержка трассировки перфетто на API 21 и 22, которая включает в себя как микробенчки, так и экспериментальные API PerfettoTrace . До этой версии соединения UiAutomation были ненадежными на некоторых устройствах. ( I78e8c )
Добавлен общедоступный экспериментальный API для PerfettoTraceProcessor , чтобы обеспечить содержание трассировки. Это шаг к полностью пользовательским показателям на основе данных трассировки перфетто. ( I2659e , b/219851406 )

Версия 1.2.0-Альфа11

8 марта 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha11 выпускается. Версия 1.2.0-альфа11 содержит эти коммиты.

Исправления ошибок

Фиксированные сбои в MacrobenchmarkRule и BaselineProfileRule при переустановке или извлечении профилей из пакета приложений с несколькими APK. ( I0d8c8 , b/270587281 )

Версия 1.2.0-Альфа10

22 февраля 2023 г.

androidx.benchmark:benchmark-*:1.2.0-alpha10 выпускается. Версия 1.2.0-Alpha10 содержит эти коммиты.

Новые возможности

На Android 14+ Macrobenchmark больше не переустанавливает целевые приложения для сброса состояния компиляции благодаря новой функции платформы. Ранее необходимо было иметь корневое устройство или иметь дело со всем состоянием приложения (например, вход пользователя), который удаляется перед каждым бланком. ( I9b08c , b/249143766 )

Исправления ошибок

Исправьте DryRunMode , чтобы больше не сбой с пустым профилем, из -за пропуска компиляции. Вместо этого он запускает одну итерацию и извлекает профиль, чтобы гарантировать, что что -то запечатлело. ( I2f05d , b/266403227 )
Исправьте PowerMetric Crash при проверке присутствия PowerStats на старых уровнях API. ( 5FAAF9 , B/268253898 )

Версия 1.2.0-Alpha09

11 января 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha09 выпускается. Версия 1.2.0-Alpha09 содержит эти коммиты.

Исправления ошибок

Включено передавать None в androidx.benchmark.enabledRules Инструментация Arg, чтобы отключить все тесты / генерацию базового профиля. ( I3d7fd , b/258671856 )
Исправить захват PerfettoTrace в модулях приложений (т.е. не взыскание испытательных APK) ( i12CFC )
Фиксированный базовый профиль ADB Порядок вытягивания аргумента на выходе студии ( I958D1 , B/261781624 )
Emulator Api 33 теперь правильно распознается как таковой при попытке запустить макробенч -маркирство и правильно распечатать предупреждение. ( 69133b , b/262209591 )
Проверка проверки уровня батареи на устройствах без батареи в Macrobenchmark ( FE4114 , B/232448937 )

Версия 1.2.0-Alpha08

7 декабря 2022 года

androidx.benchmark:benchmark-*:1.2.0-alpha08 выпускается. Версия 1.2.0-Alpha08 содержит эти коммиты.

Изменения API

Добавлено экспериментальное новое APIS PerfettoTrace.record {} и PerfettoTraceRule для захвата трассов перфетто (также известных как системные следы) как часть теста, чтобы осмотреть поведение и производительность тестирования. ( I3ba16 )
BaselineProfileRule теперь принимает предикат фильтра вместо списка префиксов пакетов. Это дает тест полный контроль на фильтрации. ( I93240 )
Добавьте экспериментальный API BaselineProfileRule.collectStableBaselineProfile , который ожидает, пока базовый профиль не станет стабильным для n итераций. ( I923f3 )
Добавьте возможность указать префикс имени выходного файла при генерации базовых профилей с использованием BaselineProfileRule . ( I7b59f , b/260318655 )

Исправления ошибок

Повышение безопасности вывода файлов, что должно предотвратить тихие не писать / добавленные выходные файлы, особенно на API 21/22. ( If8c44 , b/227510293 )
Исправьте simpleperf Trace Toplow, чтобы правильно создать и поместить файл. Это также должно более широко исправлять проблемы, когда Файл безуспешно вытягивается Gradle. ( I12A1C , B/259424099 )
Улучшить сообщение об ошибке ProfileInstaller, напечатанное, когда ProfileInstaller слишком старый. Теперь это говорит вам обновить версию ProfileInstaller (1.2.1) для измерения базовых профилей на API с 31 по 33, вместо того, чтобы сказать, что она не поддерживается. ( IA517F , B/253519888 )
Исправьте несколько сбоев команды оболочки Onerror Print Print Nead API <= 23, включая неудачную бинарную настройку перфетто и сбои захвата трассировки ( IB6B87 , B/258863685 )
Автоматически сортируют сгенерированные правила профиля, чтобы минимизировать количество изменений при изменении с течением времени (при регистрации правил профиля в управление источником). ( IE2509 )
Исправлено сбой на непредубежденных сборках ниже Android 13 (API 33) с сообщением Expected no stderr from echo 3 > /proc/sys/vm/drop_caches ( i6c245 , b/259508183 )

Известные проблемы - MacrobenchmarkScope.dropShaderCache() может сбои из -за отсутствующего реестра вещания в Manifest ProfileInstaller, который еще не был выпущен. ( I5c728 , b/258619948 ) для profileinstaller:1.3.0-alpha02

  <!-- workaround bug in profileinstaller 1.3.0-alpha02, remove when updating to alpha03+ -->
  <receiver
    android:name="androidx.profileinstaller.ProfileInstallReceiver"
    android:permission="android.permission.DUMP"
    android:exported="true">
    <intent-filter>
        <action android:name="androidx.profileinstaller.action.BENCHMARK_OPERATION" />
    </intent-filter>
  </receiver>

Версия 1.2.0-Alpha07

9 ноября 2022 г.

androidx.benchmark:benchmark-*:1.2.0-alpha07 выпускается. Версия 1.2.0-Alpha07 содержит эти коммиты.

Изменения API

Добавляет PowerMetric API для измерения энергии и мощности в макробенчах. ( IFE601 , B/220183779 )
Исправлено MacrobenchmarkScope.dropShaderCache() , чтобы фактически отбросить шейдерный кэш. Это удаляет примерно 20 мс шума из StartupMode.COLD Clardmars, поскольку шейды теперь последовательно очищают каждую итерацию. Раньше Partial компиляция с использованием итераций разминки сообщала о неправильно быстрых числах, так как кэширование шейдера с большей вероятностью произошло во время разминки. Это исправление требует либо корневого устройства, либо для использования profileinstaller:1.3.0-alpha02 в целевом приложении. Для изменений API библиотеки ProfileInstaller , пожалуйста, обратитесь к странице ProfileInstaller 1.30-ALPHA02 . ( IA5171 , B/231455742 )
Добавлен TraceSectionMode("label", Mode.Sum) , что позволяет измерять общее время, проведенное на нескольких сечениях трассировки с той же меткой. Например, TraceSectionMetric("inflate", Mode.Sum) сообщит о метрических inflateMs для общего времени в макробенчмарке, потраченном на инфляцию. Также удалено требование API 29, так как TraceSectionMetric работает вместе с androidx.tracing.Trace обратно к более низким уровням API с использованием forceEnableAppTracing в целевом приложении. ( ID7B68 , B/231455742 )

Исправления ошибок

Улучшенная безопасность всех внутренних команд оболочки путем проверки всех выходных/ошибок. ( I5984d , b/255402908 , b/253094958 )
Укажите устройство в базовом профиле adb pull -команде, чтобы команда Pull можно было просто скопировать, если подключены несколько устройств (до одного эмулятора) ( i6ac6c , b/223359380 )
Добавьте ошибку, если MacRobenchmark Test APK не настроен как самостоятельно. Эта ошибка предотвращает Macrobenchmarking из процесса целевого приложения. В процессе Macrobench не сможет скомпилировать/убить/запустить приложение или контролировать свои собственные разрешения ( i4279b )
Исправлена проблема в measureRepeated() , где StartupMode.COLD не убивает целевой процесс после setupBlock . Теперь setupBlock , взаимодействующий с приложением, не оставит процесс приложения, и неверное измерение холодного старта. ( I8ebb7 )

Версия 1.2.0-Alpha06

24 октября 2022 года

androidx.benchmark:benchmark-*:1.2.0-alpha06 выпускается. Версия 1.2.0-Alpha06 содержит эти коммиты.

Изменения API

BaselineProfileRule больше не требует корня на Android 13 (API 33) и больше не эксперименталь. ( IE0A7D , B/250083467 , B/253094958 )
- Это изменение также фиксирует, как профили из приложения промываются на диск на непрестанных устройствах, но требуют обновления зависимости профиля целевого приложения.
- Чтобы использовать BaselineProfileRule или CompilationMode.Partial(warmupIterations) на неровном устройстве, вы также должны обновить ваше целевое приложение для использования androidx.profileinstaller.profileinstaller:1.3.0-alpha01 . Это позволяет правильно промывать профиль на диск, чтобы его можно было собрать/извлечь.

Исправления ошибок

Исправляет сбой SampledProfiling на API 33+. ( I40743 , b/236109374 )

Версия 1.2.0-Alpha05

5 октября 2022 года

androidx.benchmark:benchmark-*:1.2.0-alpha05 выпускается. Версия 1.2.0-Alpha05 содержит эти коммиты.

Исправления ошибок

Fix Frame Breakdown в Studio System Trace Viewer для просмотра тестов .
Правильный FrameTimingMetric для перечисления FrameOverrun как требует API 31 вместо 29 ( i716dd , b/2207025554 )
Установите итерацию в BaselineProfileRule и четко выбросьте, если целевой пакет не установлен (уже был сделан для Macrobenchmarkrule). ( IC09A3 , B/227991471 )

Версия 1.2.0-Alpha04

21 сентября 2022 года

androidx.benchmark:benchmark-*:1.2.0-alpha04 выпускается. Версия 1.2.0-Alpha04 содержит эти коммиты.

Новые возможности

Добавьте поддержку dryRunMode.enable Amentleation Arcome в Macrobenchmark (уже доступный в Micro) для более быстрой локальной разработки, и проверка автоматизации приложений (например, в Presubmit). Это переопределяет итерации до 1, пропускает компиляцию, подавляет все ошибки конфигурации и отключает выход измерения .JSON. ( IB51B4 , B/175149857 )
На командной строке Gradle:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.dryRunMode.enable=true
```
В build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.dryRunMode.enable', 'true'
    }
}
```

Исправления ошибок

Фиксированные StartupTimingMetric , чтобы больше не требует измеренных действий, которые должны быть запущены через MacrobenchmarkScope.startActivityAndWait() . Это означает, что метрика может забрать запуска из EG-уведомлений, Context.startActivity() , навигации на основе активности в приложении или команд Shell. ( Ia2de6 , b/245414235 )
Исправить ошибку, где startActivityAndWait и Timeout, пытаясь дождаться завершения запуска на эмуляторах, снижая строгость обнаружения кадров. ( IBE2C6 , B/244594339 , B/228946895 )

Версия 1.2.0-Alpha03

7 сентября 2022 года

androidx.benchmark:benchmark-*:1.2.0-alpha03 выпускается. Версия 1.2.0-Alpha03 содержит эти коммиты.

Новые возможности

Добавлены экспериментальные API для использования BenchmarkState независимо, отдельно от BenchmarkRule / JUnit4 . ( ID478F , B/228489614 )

Исправления ошибок

Добавлен запасной запасной путь для startActivityAndWait . ( 01ed77 , b/2428999915 )

Версия 1.2.0-Alpha02

24 августа 2022 года

androidx.benchmark:benchmark-*:1.2.0-alpha02 выпускается. Версия 1.2.0-Alpha02 содержит эти коммиты.

Изменения API

По умолчанию am force stop для MacrobenchmarkScope.killProcess() , даже при укоренении, за исключением базового генерации профиля. Это может быть переопределено дополнительным логическим аргументом. ( 02cce9 , b/241214097 )

Исправления ошибок

Поддержка базового профиля для системных приложений. ( I900b8 , b/241214097 )
Поддержка проверки на метрики мощности ODPM на унылых устройствах. ( A38C78 , B/229623230 )

Версия 1.2.0-Alpha01

27 июля 2022 года

androidx.benchmark:benchmark-*:1.2.0-alpha01 выпускается. Версия 1.2.0-Alpha01 содержит эти коммиты.

Новые возможности

Новый компонент трассировки-перфетто-компонента, позволяющий инструментальности, чтобы обеспечить перфетто SDK-трассировку в приложении, которое его обнаруживает ( i2cc7f )
Добавлен androidx.benchmark.enabledRules Аргумент Инструментации, чтобы включить фильтрацию MacRobenchmark, чтобы пробежать только на контрольные показатели, или просто базовое образование профиля. Перейдите в «Macrobenchmark», или «BaselineProfile», чтобы просто запустить один тип теста, например, когда просто генерируя BaselineProfiles на эмуляторе. Список, разделенный запятыми, также поддерживается. ( I756b7 , b/230371561 )
Например, в Build.gradle вашего Macrobenchmark:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.enabledRules', 'BaselineProfile'
    }
}
```
Или из командной строки Gradle:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
```

Изменения API

Добавлен новый PowerMetric для измерения задач энергии и мощности в критериях. ( I9f39b , b/220183779 )
Добавлен новый компиляционный режим CompilationMode.Ignore . ( IBBCF8 , B/230453509 )
Добавлен новый параметр в BaselineProfileRule#collectBaselineProfile , чтобы фильтровать выходной файл по именам пакетов ( IF7338 , B/220146561 )
Позволяет разработчику разгрузку устройства для измерения источника питания. ( I6a6cb )
Добавлена возможность очистить кеш шейдеров в MacrobenchmarkScope . ( I32122 )
Позволяет разработчику настроить отображение типа метрического типа и деталей, желаемых категориями подсистемы. ( I810c9 )
Previously an UnsupportedOperationException was thrown in the benchmark if run on an unsupported device. Now UOE only occurs if the metric is used on the unsupported device (ie: PowerMetric.configure ). ( I5cf20 , b/227229375 )
Added TotalPowerMetric and TotalEnergyMetric for measuring total power and energy in each system category in macrobenchmarks. ( I3b26b , b/224557371 )

Исправления ошибок

Fixed an issue where compiled methods were not correctly being reset between each macrobenchmark on unrooted builds. This unfortunately requires reinstalling the apk each iteration, which will clear application data for each macrobenchmark. ( I31c74 , b/230665435 )
Fix trace recording crash on API 21/22 ( If7fd6 , b/227509388 , b/227510293 , b/227512788 )
Overhaul activity launch completion detection to fix 'Unable to read any metrics' exception in startup macrobenchmarks. ( Ia517c )

Version 1.1.1

9 ноября 2022 г.

androidx.benchmark:benchmark-*:1.1.1 is released. Version 1.1.1 contains these commits.

Исправления ошибок

Fixes android.system.ErrnoException: open failed: EACCES which would occur on some Android11 (API 30)+ devices. This is a cherry-pick of a fix from 1.2.0-alpha01 . ( aosp/2072249 )

Версия 1.1.0

June 15, 2022

androidx.benchmark:benchmark-*:1.1.0 is released. Version 1.1.0 contains these commits.

This version is identical to androidx.benchmark:benchmark-*:1.1.0-rc03 .

Important changes since 1.0.0

Support for Jetpack Macrobenchmarks, which allows you to measure whole-app interactions like startup and scrolling , provides the ability to capture traces & measure trace sections .
Support for Baseline Profiles
- CompilationMode.Partial to measure the effectiveness of Baseline Profiles.
- @BaselineProfileRule to automatically generate Baseline profiles for a given critical user journey.
Support for Allocation metrics & profiling during Microbenchmark runs.

Version 1.1.0-rc03

June 1, 2022

androidx.benchmark:benchmark-*:1.1.0-rc03 is released. Version 1.1.0-rc03 contains these commits.

Исправления ошибок

Avoid reinstalling the target package on every benchmark iteration. ( aosp/2093027 , b/231976084 )
Remove the 300ms delay from pressHome() . ( aosp/2086030 , b/231322975 )
Improve Macrobenchmark iteration speed by optimizing Shell commands used under the hood. ( aosp/2086023 , b/231323582 )
Support for Managed Gradle Devices when generating Baseline Profiles with Macrobenchmarks. ( aosp/2062228 , b/228926421 )

Version 1.1.0-rc02

May 11, 2022

androidx.benchmark:benchmark-*:1.1.0-rc02 is released. Version 1.1.0-rc02 contains these commits.

Note that this release includes a behavior change, as apps are now fully reinstalled in between each benchmark to ensure accurate measurements.

Bug Fixes/Behavior Changes

Fixed an issue where app compilation was not correctly reset between macrobenchmarks, and not reset at all on unrooted builds. This fixes many cases where running multiple tests would result in CompilationMode having little to no effect on measurements. To workaround this problem, the target app is now fully reinstalling each test method, which will clear application data between each macrobenchmark. ( I31c74 , b/230665435 )
As this prevents apps from setting up state before tests, it is now possible to skip compilation / reinstallation to enable working around this. You can for example fully compile the target with a shell command cmd package compile -f -m speed <package> , and then bypass macrobenchmark's compilation step.
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.compilation.enabled, 'false'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.compilation.enabled=false
```
Made it possible to share a module between macrobenchmarks and baseline profile generating tests by adding androidx.benchmark.enabledRules instrumentation argument. Pass in 'Macrobenchmark', or 'BaselineProfile' to just run one type of test, eg when generating BaselineProfiles on an emulator. ( I756b7 , b/230371561 )
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.enabledRules', 'BaselineProfile'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
```

Version 1.1.0-rc01

April 20, 2022

androidx.benchmark:benchmark-*:1.1.0-rc01 is released. Version 1.1.0-rc01 contains these commits.

Исправления ошибок

Baseline profile output links in Android Studio now use a unique file name. This way the output always reflects the latest results of using a BaselineProfileRule . ( aosp/2057008 , b/228203086 )

Version 1.1.0-beta06

April 6, 2022

androidx.benchmark:benchmark-*:1.1.0-beta06 is released. Version 1.1.0-beta06 contains these commits.

Исправления ошибок

Fix trace recording crash on API 21/22 ( If7fd6 , b/227509388 )
Overhaul activity launch completion detection to fix 'Unable to read any metrics' exception in startup macrobenchmarks. ( Ia517c )
Fix startup metrics for Macrobenchmarks when CompilationMode.None() is used. Before this change, CompilationMode.Partial() would appear to be slower than Compilation.None() . ( 611ac9 ).

Version 1.1.0-beta05

March 23, 2022

androidx.benchmark:benchmark-*:1.1.0-beta05 is released. Version 1.1.0-beta05 contains these commits.

Исправления ошибок

Kill package after skipping profile installation when using CompilationMode.None . ( aosp/1991373 )
Fixed an issue where Macrobenchmarks is unable to collect startup metrics when using StartupMode.COLD . ( aosp/2012227 b/218668335 )

Version 1.1.0-beta04

February 23, 2022

androidx.benchmark:benchmark-*:1.1.0-beta04 is released. Version 1.1.0-beta04 contains these commits.

Исправления ошибок

Fix missing metrics on Android 10, and NoSuchElementException caused by process names not being captured correctly in traces. ( Ib4c17 , b/218668335 )
Use PowerManager for thermal throttling detection on Q (API 29) and higher. This significantly reduces frequency of false positives in thermal throttling detection (benchmark retry after 90 second cooldown), and speeds up benchmarks significantly on user builds. It also provides throttle detection even when clocks are locked (if they're locked too high for the device's physical environment). ( I9c027 , b/217497678 , b/131755853 )
Filter simpleperf sampled profiling to measureRepeated thread only to simplify inspection ( Ic3e12 , b/217501939 )
Support metrics from named UI subprocesses in multi-process apps ( Ice6c0 , b/215988434 )
Filter Baseline Profile rules to target Android 9 (SDK 28). aosp/1980331 b/216508418
Skip Profile Installation when using Compilation.None() . Additionally, report warnings when the app is using an older version of androidx.profileinstaller and Android Gradle Plugin. aosp/1977029

Version 1.1.0-beta03

February 9, 2022

androidx.benchmark:benchmark-*:1.1.0-beta03 is released. Version 1.1.0-beta03 contains these commits.

Изменения API

Added AudioUnderrunMetric into macrobenchmark library under experimental flag to allow detection of audio underruns ( Ib5972 )

BaselineProfileRule no longer accepts a setup block as this functioned the same as the profileBlock . ( Ic7dfe , b/215536447 )

For eg

@Test
fun collectBaselineProfile() {
    baselineRule.collectBaselineProfile(
        packageName = PACKAGE_NAME,
        setupBlock = {
            startActivityAndWait()
        },
        profileBlock = {
            // ...
        }
    )
}

@Test
fun collectBaselineProfile() {
    baselineRule.collectBaselineProfile(
        packageName = PACKAGE_NAME,
        profileBlock = {
            startActivityAndWait()
            // ...
        }
    )
}

Исправления ошибок

Fixed issue where microbench profiler traces would fail to be updated in subsequent runs when linked in Studio output ( I5ae4d , b/214917025 )
Prevent compilation shell commands on API 23 ( Ice380 )
Renamed FrameCpuTime -> FrameDurationCpu , FrameUiTime -> FrameDurationUi to clarify these are durations, not timestamps, and to match prefixes. ( I0eba3 , b/216337830 )

Version 1.1.0-beta02

January 26, 2022

androidx.benchmark:benchmark-*:1.1.0-beta02 is released. Version 1.1.0-beta02 contains these commits.

Исправления ошибок

Microbenchmark Stack Sampling / Method Tracing Profile results are now linked in Studio output, similar to other profiling outputs, and do not suppress the allocation metric. ( Idcb65 , b/214440748 , b/214253245 )
BaselineProfileRule now prints the adb pull command in logcat and Studio output for pulling generated BaselineProfile text file. ( f08811 )

Version 1.1.0-beta01

January 12, 2022

androidx.benchmark:benchmark-*:1.1.0-beta01 is released. Version 1.1.0-beta01 contains these commits.

Исправления ошибок

Fixes profiler argument enable being ignored. ( I37373 , b/210619998 )
Removed deprecated CompliationModes ( I98186 , b/213467659 )
Switched baseline profile arg of CompilationMode.Partial to enum for clarity. ( Id67ea )

Version 1.1.0-alpha13

December 15, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha13 is released. Version 1.1.0-alpha13 contains these commits.

Изменения API

Add low-overhead System Tracing to microbench output on Android Q (API 29+). Note that this does not currently capture custom tracing (via android.os.Trace or androidx.tracing Jetpack APIs) to avoid affecting results. This tracing should be useful in diagnosing instability, especially from sources outside the benchmark. ( I298be , b/205636583 , b/145598917 )
Clarify CompilationModes into three classes - Full, None, Partial. Previously they were inconsistently named after compilation arguments (which we now treat as implementation details) and features. This makes the tradeoffs, potential combinations, and behavior across platform versions more clear. ( I3d7bf , b/207132597 )
Setup and measure are now always in pairs, in order. You can now query the package name and iteration (though the iteration may be null in certain warmup scenarios). ( Id3b68 , b/208357448 , b/208369635 )

Исправления ошибок

Fixed CompilationMode.Speed incorrectly treated as None ( I01137 )

Version 1.1.0-alpha12

November 17, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha12 is released. Version 1.1.0-alpha12 contains these commits.

Новые возможности

Add experimental TraceSectionMetric for custom trace-based timing measurements. ( I99db1 , b/204572664 )

Исправления ошибок

Wake device each iteration, to ensure UI can be tested - requires lockscreen is disabled. ( Ibfa28 , b/180963442 )
Fixes multiple crashes in StackSampling profiling mode on emulators and non-rooted devices ( Icdbda , b/202719335 )
Removed 0.5 second sleep at the end of each iteration - if you see missing metrics with this change, please file a bug. ( Iff6aa )
Reduce chances of dropped data, and lower memory overhead from tracing ( Id2544 , b/199324831 , b/204448861 )
Reduce trace size by ~40% by switching to compact sched storage format. ( Id5fb6 , b/199324831 )
Updated implementations of startup metrics to always end at end of renderthread. This will be more contistent across platform versions, and more closely map to in-app measurements. ( Ic6b55 )

Version 1.1.0-alpha11

November 3, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha11 is released. Version 1.1.0-alpha11 contains these commits.

Изменения API

Macrobenchmark now has a minSdkVersion of 23 . ( If2655 )
Adds a new experimental BaselineProfileRule which is capable of generating baseline profiles for app's critical user journey. Detailed documentation to follow. ( Ibbefa , b/203692160 )
Removes measureRepeated interface variant, which was added for java callers, as it caused ambiguity in completing/resolving the method. Java callers will again need to return Unit.Instance from measureRepeated. If this is an inconvenience, please file a bug, we can revisit this in a future version. ( Ifb23e , b/204331495 )

Version 1.1.0-alpha10

October 27, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha10 is released. Version 1.1.0-alpha10 contains these commits.

Изменения API

Backport StartupTimingMetric to work back to API 23. This new implementation also better handles reportFullyDrawn() to wait until corresponding content has been rendered. ( If3ac9 , b/183129298 )
Added JvmOverloads to multiple MacrobenchmarkScope methods for Java callers. ( I644fe , b/184546459 )
Provide alternative MacrobenchmarkRule.measureRepeated function that uses a Consumer<MacrobenchmarkScope> for idiomatic usage in Java language. ( If74ab , b/184546459 )

Исправления ошибок

Fix for traces not starting early enough, and missing metric data. This is expected to fix "Unable to read any metrics during benchmark" exceptions that were caused by the library itself. ( I6dfcb , b/193827052 , b/200302931 )
FrameNegativeSlack has been renamed to FrameOverrun to clarify its meaning - how much the frame went over its time budget. ( I6c2aa , b/203008701 )

Version 1.1.0-alpha09

October 13, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha09 is released. Version 1.1.0-alpha09 contains these commits.

Исправления ошибок

Support dropping Kernel page cache without root on API 31/S+, which will increase accuracy of StartupMode.COLD launches. ( Iecfdb , b/200160030 )

Version 1.1.0-alpha08

September 29, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha08 is released. Version 1.1.0-alpha08 contains these commits.

Изменения API

Enable scrolling macrobenchmarks to run back to API 23 ( If39c2 , b/183129298 )
Add new type of sampled metric to UI and JSON output, focused on percentiles of multiple samples per iteration. ( I56247 , b/199940612 )
Switch to floating point metrics throughout the benchmark libraries (truncated in the Studio UI). ( I69249 , b/197008210 )

Версия 1.1.0-альфа07

1 сентября 2021 г.

androidx.benchmark:benchmark-*:1.1.0-alpha07 is released. Version 1.1.0-alpha07 contains these commits.

Изменения API

Raised min API to 21 to reflect the intended lowest API level to be supported in the future. Current min API supported continues to be conveyed via RequiredApi(), and is currently 29 ( I440d6 , b/183129298 )

Исправления ошибок

Fixes ProfileInstaller to make it easier for apps using baseline profiles to run MacroBenchmarks using CompilationMode.BaselineProfile . ( I42657 , b/196074999 ) NOTE: requires also updating to androidx.profileinstaller:profileinstaller:1.1.0-alpha04 or greater.
StartupMode.COLD + CompilationMode.None benchmarks are now more stable. ( I770cd , b/196074999 )

Версия 1.1.0-альфа06

August 18, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha06 is released. Версия 1.1.0-alpha06 содержит эти коммиты.

Изменения API

Added androidx.benchmark.iterations instrumentation argument to allow manual overriding of iteration count when testing/profiling locally. ( 6188be , b/194137879 )

Исправления ошибок

Switched to Simpleperf as default sampling profiler on API 29+. ( Ic4b34 , b/158303822 )

Известные проблемы

CompilationMode.BaselineProfile is a work in progress. Avoid using it to determine how good a profile is for now.

Версия 1.1.0-альфа05

August 4, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha05 is released. Версия 1.1.0-alpha05 содержит эти коммиты.

1.1.0-alpha04 was cancelled before release due to a sporatic crash. b/193827052

Изменения API

Switched startActivityAndWait to invoke launch via am start , which reduces the time of each measurement iteration by approximately 5 seconds, at the cost of no longer supporting intent parcelables. ( I5a6f5 , b/192009149

Исправления ошибок

Reduce aggressiveness of thermal throttle detection, and recompute baseline if throttles are detected frequently. ( I7327b )
Fixes FrameTimingMetric to work on Android S beta ( Ib60cc , b/193260119 )
Use an EmptyActivity to bring the target app out of a force-stopped state to better support CompilationMode.BaselineProfile . ( Id7cac , b/192084204 )
Changed trace file extension to .perfetto-trace to match platform standard. ( I4c236 , b/174663039 )
StartupTimingMetric now outputs the "fullyDrawnMs" metric to measure time until your application has completed rendering. To define this metric for your app, call Activity.reportFullyDrawn when your initial content is ready, such as when your initial list items are loaded from DB or network. (reportFullyDrawn method available without build version checks on ComponentActivity). Note that your test must run long enough to capture the metric (startActivityAndWait doesn't wait for reportFullyDrawn). ( If1141 , b/179176560 )
Reduce cost of appending Ui metadata to traces by 50+ ms ( Ic8390 , b/193923003 )
Drastically increased polling frequency when stopping tracing, which can reduce eg startup benchmark runtime by 30+% ( Idfbc1 , b/193723768 )

Версия 1.1.0-альфа03

June 16, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha03 is released. Версия 1.1.0-alpha03 содержит эти коммиты.

Новые возможности

Added a new CompilationMode.BaselineProfile to support profiles installed using the Jetpack ProfileInstaller library . ( aosp/1720930 )

Исправления ошибок

The sample Gradle code for suppressing benchmark errors has been updated to use a non-deprecated API with a syntax that also supports .gradle.kts users.
Например,
```
testInstrumentationRunnerArguments["androidx.benchmark.suppressErrors"] = "EMULATOR,LOW-BATTERY"
```

Версия 1.1.0-альфа02

18 мая 2021 г.

Benchmark version 1.1.0-alpha02 brings a big component to benchmarking - Macrobenchmark. In addition to benchmark allowing you to measure CPU loops, macrobenchmark allows you to measure whole-app interactions like startup and scrolling, and capture traces. For more information see the library documentation .

androidx.benchmark:benchmark-*:1.1.0-alpha02 is released. Версия 1.1.0-alpha02 содержит эти коммиты.

Новые возможности

Macrobenchmark artifacts added ( androidx.benchmark:benchmark-macro-junit4 and androidx.benchmark:benchmark-macro )

Capture startup, scrolling/animation performance metrics from your app, locally or in CI
Capture and inspect traces from within Android Studio

Исправления ошибок

Workaround shell permissions issue with output directory on Android 12 (Note - may require updating Android Gradle Plugin to 7.0.0 canary and Android Studio to Arctic Fox (2020.3.1), to continue capturing output files on affected devices). ( Icb039 )
Support configuration caching in BenchmarkPlugin ( 6be1c1 , b/159804788 )
Simplified file output - on by default, in a directory that doesn't require requestLegacyExternalStorage=true ( 8b5a4d , b/172376362 )
Fixes library printing logcat warnings about not finding JIT thread on platform versions where it is not present. ( I9cc63 , b/161847393 )
Fix for reading device max frequency. ( I55c7a )

Версия 1.1.0-альфа01

10 июня 2020 г.

androidx.benchmark:benchmark-common:1.1.0-alpha01 , androidx.benchmark:benchmark-gradle-plugin:1.1.0-alpha01 , and androidx.benchmark:benchmark-junit4:1.1.0-alpha01 are released. Версия 1.1.0-alpha01 содержит эти коммиты.

New Features of 1.1

Allocation Metric - Benchmarks now run an additional phase after warmup and timing, capturing allocation counts. Allocations can cause performance problems on older versions of the platform (140ns in O became 8ns in M - measured on Nexus5X, with locked clocks). This metric is displayed in Android Studio console output, as well as in the
Profiling support - You can now capture profiling data for a benchmark run, to inspect why your code may be running slowly. Benchmark supports capturing either method tracing, or method sampling from ART. These files can be inspected with the Profiler inside Android Studio using File > Open .
The Benchmark Gradle plugin now provides defaults for simpler setup:
- testBuildType is set to release by default, to avoid using dependencies with code coverage built-in. The release buildType is also configured as the default buildType, which allows Android Studio to automatically select the correct build variant when opening a project for the first time. ( b/138808399 )
- signingConfig.debug is used as the default signing config ( b/153583269 )

** Bug Fixes **

Significantly reduced the warmup transition overhead, where the first measurement for each benchmark was artificially higher than others. This issue was more pronounced in very small benchmarks (1 microsecond or less). ( b/142058671 )
Fixed InstrumentationResultParser error printed for each benchmark when running from command line. ( I64988 , b/154248456 )

Известные проблемы

Command line, gradle invocations of Benchmark do not print out results directly. You can work around this by either running through Studio, or parsing the JSON output file for results.
Benchmark reporting fails to pull the report from devices that have an app installed with an applicationId ending with either “android” or “download” (case insensitive). Users hitting this issue should upgrade the Android Gradle Plugin to 4.2-alpha01 or later.

Версия 1.0.0

Benchmark Version 1.0.0

November 20, 2019

androidx.benchmark:benchmark-common:1.0.0 , androidx.benchmark:benchmark-gradle-plugin:1.0.0 , and androidx.benchmark:benchmark-junit4:1.0.0 are released with no changes from 1.0.0-rc01. Version 1.0.0 contains these commits .

Основные особенности версии 1.0.0

The Benchmark library allows you to write performance benchmarks of app code and get results quickly.

It prevents build and runtime configuration issues and stabilizes device performance to ensure that measurements are accurate and consistent. Run the benchmarks directly in Android Studio , or in Continuous Integration to observe code performance over time, and to prevent regressions.

Major features include:

Clock stabilization
Automatic thread prioritization
Support for UI performance testing, such as in the RecyclerView Sample
JIT-aware warmup and looping
JSON benchmark output for post-processing

Версия 1.0.0-rc01

October 23, 2019

androidx.benchmark:benchmark-common:1.0.0-rc01 , androidx.benchmark:benchmark-gradle-plugin:1.0.0-rc01 , and androidx.benchmark:benchmark-junit4:1.0.0-rc01 are released. Version 1.0.0-rc01 contains these commits .

Новые возможности

Added systrace tracing to benchmarks

Исправления ошибок

Fixed metric instability issue where JIT wouldn't finish before warm up due to deprioritization ( b/140773023 )
Unified JSON output directory across Android Gradle Plugin 3.5 and 3.6

Версия 1.0.0-бета01

October 9, 2019

androidx.benchmark:benchmark-common:1.0.0-beta01 , androidx.benchmark:benchmark-gradle-plugin:1.0.0-beta01 , and androidx.benchmark:benchmark-junit4:1.0.0-beta01 are released. Версия 1.0.0-beta01 содержит эти коммиты .

Новые возможности

Run garbage collection before each warmup to reduce memory pressure from one benchmark to leak to the next ( b/140895105 )

Исправления ошибок

Added androidx.annotation:android-experimental-lint dependency, so that Java code will correctly produce lint errors when experimental API is not used, similar to what is provided by the Kotlin experimental annotation for Kotlin callers.
Now correctly detects usage of additionalTestOutputDir instrumentation argument for output in Android Gradle Plugin 3.6, to know when AGP will handle data copy.
Fix undetected clock frequency in JSON to correctly print -1 ( b/141945670 ).

Version 1.0.0-alpha06

September 18, 2019

androidx.benchmark:benchmark-common:1.0.0-alpha06 , androidx.benchmark:benchmark-gradle-plugin:1.0.0-alpha06 , and androidx.benchmark:benchmark-junit4:1.0.0-alpha06 are released. Version 1.0.0-alpha06 contains these commits .

Новые возможности

Added a check for incorrectly using the old package for the test runner, which now provides a more-helpful error message

API меняется

The experimental annotation ExperimentalAnnotationReport is now correctly public. Usage of the experimental BenchmarkState#report API now requires this annotation

Версия 1.0.0-альфа05

September 5, 2019

androidx.benchmark:benchmark-common:1.0.0-alpha05 , androidx.benchmark:benchmark-gradle-plugin:1.0.0-alpha05 , and androidx.benchmark:benchmark-junit4:1.0.0-alpha05 are released. Коммиты, включенные в эту версию, можно найти здесь .

API меняется

BenchmarkState.reportData API is now marked experimental

Исправления ошибок

Fix for the clock-locking script, which would fail on devices that were either missing the cut or expr shell utilities.
Fixed an issue with ./gradlew lockClocks task that would hang on devices that were rooted with an older version of the su utility, which did not support the -c flag.

Версия 1.0.0-альфа04

August 7, 2019

androidx.benchmark:benchmark-common:1.0.0-alpha04 , androidx.benchmark:benchmark-gradle-plugin:1.0.0-alpha04 , and androidx.benchmark:benchmark-junit4:1.0.0-alpha04 are released. Коммиты, включенные в эту версию, можно найти здесь .

New documentation has also been added for how to use the Benchmark library without Gradle, both for usage with different build systems (such as Bazel or Buck), and when running in CI. For more information, see Build benchmarks without Gradle and Run benchmarks in Continuous Integration .

Новые возможности

Gradle plugin
- Now automatically disables test coverage, and sets the AndroidBenchmarkRunner by default ( b/138374050 )
- Added support for new AGP-based data copy, when running benchmarks and when using AGP 3.6+
JSON format additions
- Output total benchmark test run time ( b/133147694 )
- @Parameterized benchmarks that use a name string (for example @Parameters(name = "size={0},depth={1}") ) now output parameter names and values per benchmark in the JSON output ( b/132578772 )
Dry Run mode ( b/138785848 )
- Added a "dry run" mode for running each benchmark loop only once, to check for errors/crashes without capturing measurements. This can be useful eg for, for example, quickly running benchmarks in presubmit to check that they're not broken.

API меняется

Module structure has changed, splitting the library ( b/138451391 )
- benchmark:benchmark-junit4 contains classes with JUnit dependency: AndroidBenchmarkRunner , and BenchmarkRule , both of which have moved into the androidx.benchmark.junit4 package
- benchmark:benchmark-common contains the rest of the logic, including the BenchmarkState API
- This split will allow the library to support benchmarking without JUnit4 APIs in the future
Configuration warnings are now treated as errors, and will crash the test ( b/137653596 )
- This is done to further encourage accurate measurements, especially in CI
- These errors can be reduced back to warnings with an instrumentation argument. For example: -e androidx.benchmark.suppressErrors "DEBUGGABLE,LOW_BATTERY"

Исправления ошибок

Errors when writing to external storage on Q devices provide more-descriptive messages, with suggestions of how to resolve the issue
Screens are automatically turned on during benchmark runs, instead of failing when the screen is off

External contributions

Thanks to Sergey Zakharov for contributing JSON output improvements and the fix for screen off issues!

Версия 1.0.0-альфа03

July 2, 2019

androidx.benchmark:benchmark:1.0.0-alpha03 and androidx.benchmark:benchmark-gradle-plugin:1.0.0-alpha03 are released. Коммиты, включенные в эту версию, можно найти здесь .

Новые возможности

Expose sleep duration due to thermal throttling per benchmark in the full JSON report

Исправления ошибок

The Gradle plugin should no longer be required to be applied after Android plugins and the Android block
Adds support for benchmark reports on Android 10 devices using scoped storage

Версия 1.0.0-альфа02

June 6, 2019

androidx.benchmark:1.0.0-alpha02 and androidx.benchmark:benchmark-gradle-plugin:1.0.0-alpha02 are released. Коммиты, включенные в эту версию, можно найти здесь .

Note that we are treating the JSON schema as an API. We plan to follow the same stability constraints as other APIs: stable (with very rare exceptions) once in beta, and fixed in final release, with only additions in minor releases and changes/removals in major releases.

API меняется

Overhauled JSON schema. Further changes to the JSON schema are likely to be limited to additions:
- Reorganized the result object structure to support additional metric groups in the future ( b/132713021 )
- Added test run context information, such as device and build info and whether clocks are locked, to the top-level object ( b/132711920 )
- Time metric names now have 'ns' in their name ( b/132714527 )
- Additional stats added per reported metric (maximum, median, minimum), and removed simplified 'nanos' summary stat ( b/132713851 )
Removed XML output ( b/132714414 )
Thermal throttle detection removed from BenchmarkState.reportData API ( b/132887006 )

Исправления ошибок

Fixed ./gradlew lockClocks not sticking on some recent OS devices ( b/133424037 )
Throttling detection disabled for emulator ( b/132880807 )

Версия 1.0.0-альфа01

7 мая 2019 г.

androidx.benchmark:benchmark:1.0.0-alpha01 is released. The commits included in this version are available here .

Эталон

User Guide Code Sample Codelab

Справочник по API
androidx.benchmark
androidx.benchmark.junit4
androidx.benchmark.macro
androidx.benchmark.macro.junit4

Accurately measure your code's performance within Android Studio.

Latest Update	Стабильный выпуск	Кандидат на выпуск	Бета-версия	Альфа-релиз
9 апреля 2025 г.	1.3.4	-	-	1.4.0-alpha11

Объявление зависимостей

To add a dependency on Benchmark, you must add the Google Maven repository to your project. Для получения дополнительной информации прочтите репозиторий Google Maven .

Macrobenchmark

To use Macrobenchmark in your project, add the following dependencies to your build.gradle file for your macrobenchmark module :

классный

dependencies {
  androidTestImplementation "androidx.benchmark:benchmark-macro-junit4:1.3.4"
}

Котлин

dependencies {
  androidTestImplementation("androidx.benchmark:benchmark-macro-junit4:1.3.4")
}

Microbenchmark

To use Microbenchmark in your project, add the following dependencies to your build.gradle file for your microbenchmark module :

классный

dependencies {
    androidTestImplementation "androidx.benchmark:benchmark-junit4:1.3.4"
}

android {
    ...
    defaultConfig {
        ...
        testInstrumentationRunner "androidx.benchmark.junit4.AndroidBenchmarkRunner"
    }
}

Котлин

dependencies {
    androidTestImplementation("androidx.benchmark:benchmark-junit4:1.3.4")
}

android {
    ...
    defaultConfig {
        ...
        testInstrumentationRunner = "androidx.benchmark.junit4.AndroidBenchmarkRunner"
    }
}

Чтобы использовать плагин, включите следующую строку в блок `plugins` вашего файла build.gradle верхнего уровня:

классный

plugins {
  id 'androidx.benchmark' version '1.3.4' apply false
}

Котлин

plugins {
  id("androidx.benchmark") version "1.3.4" apply false
}

Затем примените плагин к файлу build.gradle вашего тестового модуля.

классный

plugins {
  id 'androidx.benchmark'
}

Котлин

plugins {
    id("androidx.benchmark")
}

Обратная связь

Создать новую задачу

Дополнительную информацию см. в документации по системе отслеживания проблем .

Version 1.4

Version 1.4.0-alpha11

9 апреля 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha11 is released. Version 1.4.0-alpha11 contains these commits .

Изменения API

Changed TraceProcessor 's argument timeout: Duration to timeoutMs long for Java caller usability. ( I9fbb5 )
Mark TraceProcessor constructor as internal. Callers should use TraceProcessor.startServer or TraceProcessor.runServer . ( Ia8c5b )

Исправления ошибок

When killing the application with MacrobenchmarkScope.killProcess , validate the kill command results to prevent silent failure, and timeout error. ( I84555 )

Dependency Updates

This library now targets Kotlin 2.0 language level and requires KGP 2.0.0 or newer. ( Idb6b5 )

Version 1.4.0-alpha10

26 марта 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha10 is released. Version 1.4.0-alpha10 contains these commits .

Изменения API

Increased default timeout for TraceProcessor server load and querying to 120 seconds (from 60/30 previously), and made both configurable with one timeout parameter. ( Ifec87 )

Исправления ошибок

Fixed a few issues that would occur when benchmarking or capturing profiles of an app without profileinstaller , and it's included BroadcastReciever . This only affects runs on rooted devices. ( Ied308 )

Version 1.4.0-alpha09

March 12, 2025

androidx.benchmark:benchmark-*:1.4.0-alpha09 is released. Version 1.4.0-alpha09 contains these commits .

Изменения API

Added TraceProcessor and Session API with closable Handles, for easier usage with custom lifecycles. This is also a step towards enabling easier Coroutine and Java API usage. The extension functions to TraceProcessor.runServer {} are now marked experimental, as they're likely to move, and be made non-experimental in the future. ( I358b4 )

Исправления ошибок

Fixed Benchmark and Baseline Profile capture not working with API 36 due to a change in pgrep toybox that now requires -a to print the full command line. ( Idc991 )
Filter default tracing config to reduce risk of data loss in traces on newer API levels. ( I54e8a )
Added experimental androidx.benchmark.killExistingPerfettoRecordings instrumentation argument, which can be set to false to allow pre existing perfetto trace capture to continue. By default pre existing perfetto trace captures on device are killed to prevent interference. ( I02a3c )
The JSON field context.osCodenameAbbreviated will now be REL for released OS versions at and above API 35, as non-numeric code names are no longer supported by the underlying platform. ( Ib17fd )
Fixes crash in FrameTimingMetric when resynced frames occur. ( I7c6f4 , b/394610806 )
No longer assume Choreographer#doFrame is the top of the stack frame on the main thread for FrameTimingQuery . ( Iee0e0 , b/340206285 )

Version 1.4.0-alpha08

12 февраля 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha08 is released. Version 1.4.0-alpha08 contains these commits .

Изменения API

Moved TraceProcessor.runSession() extension APIs to be experimental, as they are likely to move to be concrete constructors eventually on Android. ( Ib0528 , b/393640753 )
Most of the implementation of Startup Insights are now public/experimental, and move to the TraceProcessor artifact See StartupInsights . ( I0aa00 )
Deprecate BenchmarkRule.runWithTimingDisabled {} in favor of BenchmarkRule.runWithMeasurementDisabled {} , which more clearly describes the behavior - all metrics are paused. Additionally, expose the MicrobenchmarkScope superclass since redeclaring the runWithMeasurementDisabled function to open access isn't possible, since it's inline. ( I9e23b , b/389149423 , b/149979716 )
Benchmark libraries have moved to Kotlin 2.0. ( I9d1e0 )
Removed androidx.benchmark.startupProfiles.enable instrumentation argument. It is no longer useful, as it can be controlled by includeInStartupProfile arg in BaselineProfileRule.collect() . ( I39eb4 )

Исправления ошибок

Reduced the amount of Microbenchmark internal functions called during profiling to make eg Method traces more clear ( Ifaed8 )
Speculative fix for crashes: 'Failed to stop [ ProcessPid(processName=perfetto, pid=...) ]'. Now Benchmark will log a message instead of crashing when a background Perfetto process isn't able to be stopped before running the benchmark. ( I37d3e , b/323601788 )
Fix IllegalStateExceptions with 'Expected pm dump-profiles stdout' label that were caused by overly strict output format check. ( I358dc )

Version 1.4.0-alpha07

29 января 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha07 is released. Version 1.4.0-alpha07 contains these commits .

Новые возможности

BaselineProfileRule now introduces a collectWithResults(...) API which includes a list of paths to the computed profiles. ( I056f8 )
Added androidx.benchmark.measureRepeatedOnMainThread.throwOnDeadline instrumentation argument, which can be set to false to disable the throw on deadline behavior of measureRepeatedOnMainThread for local testing. Not otherwise recommended, as this increases the likelihood of ANRs during tests. ( Idbeec , b/353226476 )

Изменения API

Added @JvmOverloads to MicrobenchmarkConfig constructor. ( I13fd3 )
Refactored BenchmarkRule to be built on top of coroutines, and support better yield() behavior. This refactor removed several experimental BenchmarkState APIs, but will be followed by replacements as needed. Additionally, added runWithMeasurementDisabled to clarify behavior (all measurement is paused). In the future, runWithTimingDisabled will be deprecated. ( I19837 , b/389149423 , b/311242861 )
Move PerfettoTraceProcessor to TraceProcessor in a new androidx.benchmark:benchmark-traceprocessor artifact, and make most of its API non-experimental. Any custom TraceMetric or anything reading from traces will need to update to the new TraceProcessor import. The new TraceProcessor API works exactly like the old one, but is a standalone interface library (somewhat analogous to the androidx.sqlite layer from Room) with an Android-specific implementation built into macrobenchmark. The new artifact can be used on JVM as well, but currently you'll need to start your own copy of the TraceProcessor binary and offer a port to connect to it on. ( I3a767 , I62563 , b/381134564 )

Исправления ошибок

Throw a clearer error message when MacrobenchmarkScope.startActivityAndWait fails to launch the target process (potentially due to a crash in the target process), instead of the more ambiguous 'Unable to confirm activity launch completion' message ( I3539b )
Fixed several syntax errors in Kotlin samples, and syntax highlighting in several Java / build.gradle samples. ( Ib3808 )
Clarified ArtMetric and CaptureInfo parameter docs. ( I96e60 )

Version 1.4.0-alpha06

December 11, 2024

androidx.benchmark:benchmark-*:1.4.0-alpha06 is released. Version 1.4.0-alpha06 contains these commits .

Изменения API

Removed usage of @Language("sql") in PerfettoTraceProcessor.Session.query() , as Studio highlighting/parsing is broken. ( Idc2fa , b/377733398 )

Исправления ошибок

This library now uses JSpecify nullness annotations , which are type-use. Kotlin developers should use the following compiler arguments to enforce correct usage: -Xjspecify-annotations=strict , -Xtype-enhancement-improvements-strict-mode ( I46810 , b/326456246 )
Corrected ArtMetric to report class load (not init), and improved docs to clarify runtime behavior. ( I9915c )
On Android Multiuser, execute commands as root only on rooted devices. ( I88b44 )

Version 1.4.0-alpha05

November 13, 2024

androidx.benchmark:benchmark-*:1.4.0-alpha05 is released. Version 1.4.0-alpha05 contains these commits .

Исправления ошибок

Fixed an issue on API 34+ where CompilationMode.None() would have inconsistent performance not representative of initial, worst case performance. This works around a platform change which allows ART's compilation state verify to partially compile apps (only affecting class loading) shortly after first launch. ( Ie48d0 )
Fixed issue where (especially short) traces could be captured that wouldn't report measurement from built-in Macrobenchmark Metrics, due to the process name being truncated within the Perfetto trace. Now macrobenchmark works around this issue by looking for the truncated package name in all built-in queries, in addition to the expected package name. Note that custom TraceMetric implementations or other direct callers of PerfettoSession.query can implement this same behavior by changing process.name LIKE "$packageName" in a Perfetto query to instead be (process.name LIKE "$packageName" OR process.name LIKE "$(packageName.takeLast(15))") . ( I5bf01 , b/377565760 )

Version 1.4.0-alpha04

30 октября 2024 г.

androidx.benchmark:benchmark-*:1.4.0-alpha04 is released. Version 1.4.0-alpha04 contains these commits .

Новые возможности

(Experimental) Enable Baseline Profile generation, and benchmarking on apps installed to a secondary user, for example any app on headless Android Auto devices. This support has been tested in some scenarios, but let us know with a bug if it doesn't work for you. ( I9fcbe , b/356684617 , b/373641155 )

Исправления ошибок

isProfileable is now always overridden in benchmark builds, and isDebuggable is also now always overridden in both benchmark and nonMinified (baseline profile capture) builds. ( I487fa , b/369213505 )
Fixes compilation detection on some physical devices prior to API 28 - affects json context.compilationMode , as well as behavior of androidx.benchmark.requireAot=true (which no longer incorrectly throws) ( Ic3e08 , b/374362482 )
In CpuEventCounter metrics, throw if invalid measurements are observed (eg instructions/cpucycles==0) ( I8c503 )

Version 1.4.0-alpha03

16 октября 2024 года

androidx.benchmark:benchmark-*:1.4.0-alpha03 is released. Version 1.4.0-alpha03 contains these commits .

Изменения API

Macrobenchmark : Adds ArtMetric , which can be used to inspect profile coverage or general Android RunTime performance. Captures number and total duration of JIT, class init (where available), and class verification. Additionally, changes CaptureInfo to include optional ART mainline version with default. ( I930f7 )
Add coefficientOfVariation to Benchmark JSON output to show stability within a given benchmark run. ( Ib14ea )

Исправления ошибок

Fixed CollectBaselineProfileTask when AVD device has spaces in it. ( Ia0225 , b/371642809 )
Speculative fix for errors from StartupMode.COLD exceptions: Package <packagename> must not be running prior to cold start! . Now, MacrobenchmarkScope.killProcess() (including the one run before each iteration, used to implement StartupMode.COLD behavior) will wait to verify that the app's processes have all stopped running. ( I60aa6 , b/351582215 )
Fixed issue where UNLOCKED_ error would show up on some rooted emulators. ( Ic5117 )
This library now uses JSpecify nullness annotations , which are type-use. Kotlin developers should use the following compiler arguments to enforce correct usage: -Xjspecify-annotations=strict , -Xtype-enhancement-improvements-strict-mode ( I7104f , b/326456246 )

Version 1.4.0-alpha02

2 октября 2024 г.

androidx.benchmark:benchmark-*:1.4.0-alpha02 is released. Version 1.4.0-alpha02 contains these commits .

Изменения API

Moved Gradle tasks lockClocks and unlockClocks to be on benchmark projects, instead of available at the top level. This change was necessary as there is unfortunately no way to register these as top level actions without breaking project isolation. ( I02b8f , b/363325823 )

Исправления ошибок

BaselineProfileRule now collects profiles for multi-process apps by signaling each running process at the end of the block to dump profiles. If a profile based compilation never successfully finds a process to broadcast to, the compilation will fail, as it's unexpected to have profile data within. Additionally, added an instrumentation argument to control dump wait duration: androidx.benchmark.saveProfileWaitMillis ( I0f519 , b/366231469 )
From Benchmark 1.3.2 : Fixed Firebase Test Lab (FTL) being unable to pull Baseline Profile or Macrobenchmark result files from the Baseline Profile Gradle Plugin. ( I2f678 , b/285187547 )

To use FTL apply the plugin to the baseline profile module in the plugin block, with:

  plugins {
      ...
      id("com.google.firebase.testlab")
  }

and then configure firebase test lab with:

  firebaseTestLab {

      // Credentials for FTL service
      serviceAccountCredentials.set(file("credentials.json"))

      // Creates one or more managed devices to run the tests on.
      managedDevices {
          "ftlDeviceShiba34" {
              device = "shiba"
              apiLevel = 34
          }
      }

      // Ensures the baseline profile is pulled from the device.
      // Note that this will be automated as well later with aosp/3272935.
      testOptions {
          results {
              directoriesToPull.addAll("/storage/emulated/0/Android/media/${android.namespace}")
          }
      }
  }

Also the created FTL device needs to be added to the baseline profile extension:

  baselineProfile {
      managedDevices += "ftlDeviceShiba34"
      useConnectedDevices = false
  }

Version 1.4.0-alpha01

18 сентября 2024 года

androidx.benchmark:benchmark-*:1.4.0-alpha01 is released. Version 1.4.0-alpha01 contains these commits .

New Feature - App Startup Insights

Initial version of app startup insights can be enabled in Macrobenchmark. ( 09fae38 )

To enable in a startup benchmark:

  @Test
  fun startup {
      macrobenchmarkRule.measureRepeated(
          …
          packageName = "com.example.my.application.id"
          metrics = listOf(StartupTimingMetric()),
          iterations = 5,
          startupMode = StartupMode.COLD,
          compilationMode = CompilationMode.None(),
          experimentalConfig = ExperimentalConfig(startupInsightsConfig = StartupInsightsConfig(isEnabled = true))
          ) {
          scope.startActivityAndWait(...)
      }
  }

Then running your startup benchmark will analyze the trace to look for common problems, and print them after metrics to Studio test output in the benchmark tab, eg:

StartupBenchmark_startup[startup=COLD,compilationMode=None]
├── Metrics
│   ├──   timeToFullDisplayMs                min  1,147.2,   median  1,208.8,   max  1,307.4
│   └──   timeToInitialDisplayMs             min  1,147.2,   median  1,208.8,   max  1,307.4
├── App Startup Insights
│   ├── App in debuggable mode (expected: false)
│   │   └── seen in iterations: 0(true) 1(true) 2(true) 3(true) 4(true) 5(true) 6(true) 7(true) 8(true) 9(true)
│   ├── Potential CPU contention with another process (expected: < 100000000ns)
│   │   └── seen in iterations: 4(105022546ns)
│   └── Main Thread - Binder transactions blocked (expected: false)
│       └── seen in iterations: 7(true)
└── Traces
    └── Iteration 0 1 2 3 4 5 6 7 8 9

This feature is still a work-in-progress, with improvements to documentation and extensibility to follow, but feedback is welcome.

Новые возможности

Added gradle property androidx.baselineprofile.suppressWarnings to suppress all baseline profile warnings. ( 314153a )
Microbench metrics are now displayed in Perfetto traces as counters. ( 3214854 )
Add experimental scripts for disabling jit (requires root / runtime restart), and resetting device perf/test state. These are not currently published as gradle tasks. ( 7c3732b )
Added benchmark argument to skip tests when running on emulator. When automaticGenerationDuring build is enabled, benchmarks will also trigger baseline profile generation. This will fail, if emulators are used. With the new argument skipBenchmarksOnEmulator we can instead skip the test. ( 0c2ddcd )
Change perf event enable logic to run on API 23+ ( 2550048 )

Изменения API

Existing experimental PerfettoConfig argument to MacrobenchmarkRule.measureRepeated() moved to the new ExperimentalConfig object.

Исправления ошибок

Increase lockClocks.sh retry count ( 99e9dac )
Don't create nonMinified and benchmark build types if existing. Due to a bug, even if nonMinified and benchmark build types existed, they were going to be recreated. ( e75f0a5 )
Ignore non-terminating slices from TraceSectionMetric results. ( a927d20 )
Improved emulator check to consider sdk_ prefix. ( 1587de8 )
Treat non-running packages as cleared in FrameTimingGfxInfoMetric . ( 35cc79c )
Fix androidx.benchmark.cpuEventCounter producing corrupt values for non-Instruction events. ( 06edd59 )
Fix resumeTiming/runWithTimingDisabled to respect metric priority order, and significantly reduce impact of lower priority metric pause/resume on higher priority metric results. For example, if using cpu perf counters via cpuEventCounter.enable instrumentation argument, timeNs is no longer significantly reduced when pause/resume occur. ( 5de0968 )

Версия 1.3

Version 1.3.4

26 марта 2025 г.

androidx.benchmark:benchmark-*:1.3.4 is released. Version 1.3.4 contains these commits .

Исправления ошибок

Fixed Gradle Project Isolation incompatibilities in the Benchmark Baseline Gradle Plugin. ( b/404523257 )

Version 1.3.3

16 октября 2024 года

androidx.benchmark:benchmark-*:1.3.3 is released. Version 1.3.3 contains these commits .

Исправления ошибок

Fixed CollectBaselineProfileTask when AVD device has spaces in it ( Ia0225 , b/371642809 )

Version 1.3.2

2 октября 2024 г.

androidx.benchmark:benchmark-*:1.3.2 is released. Version 1.3.2 contains these commits .

Исправления ошибок

Fixed Firebase Test Lab (FTL) being unable to pull Baseline Profile or Macrobenchmark result files from the Baseline Profile Gradle Plugin. ( I2f678 , b/285187547 )

To use FTL apply the plugin to the baseline profile module in the plugin block, with:

  plugins {
      ...
      id("com.google.firebase.testlab")
  }

and then configure firebase test lab with:

  firebaseTestLab {

      // Credentials for FTL service
      serviceAccountCredentials.set(file("credentials.json"))

      // Creates one or more managed devices to run the tests on.
      managedDevices {
          "ftlDeviceShiba34" {
              device = "shiba"
              apiLevel = 34
          }
      }

      // Ensures the baseline profile is pulled from the device.
      // Note that this will be automated as well later with aosp/3272935.
      testOptions {
          results {
              directoriesToPull.addAll("/storage/emulated/0/Android/media/${android.namespace}")
          }
      }
  }

Also the created FTL device needs to be added to the baseline profile extension:

  baselineProfile {
      managedDevices += "ftlDeviceShiba34"
      useConnectedDevices = false
  }

Version 1.3.1

18 сентября 2024 года

androidx.benchmark:benchmark-*:1.3.1 is released. Version 1.3.1 contains these commits .

Исправления ошибок

Added gradle property androidx.baselineprofile.suppressWarnings to suppress all baseline profile warnings ( I7c36e , b/349646646 )
Fixed Baseline Profile Gradle Plugin to use pre-existing nonMinified… and benchmark… if created by the app instead of creating wrappers. ( Ia8934 , b/361370179 )
Fixed java.lang.AssertionError: ERRORS (not suppressed): EMULATOR when automaticGenerationDuringBuild is enabled on emulators. New argument is used to instead skip the test. ( If3f51 , b/355515798 )
Microbenchmark minification - keep subclasses of org.junit.runner.notification.RunListener in benchmark library proguard ( Ic8ed5 , b/354264743 )
Fix TraceSectionMetric to Ignore non-terminating slices. Previously these were considered to have -1 duration, eg during summation or finding minimum duration. ( If74b7 )
Fixed an issue in FrameTimingGfxInfoMetric where starting the metric would crash if the process wasn't already running. ( I6e412 )

Version 1.3.0

21 августа 2024 года

androidx.benchmark:benchmark-*:1.3.0 is released. Version 1.3.0 contains these commits .

Microbenchmark changes since 1.2.0

Method tracing is on by default in microbenchmarks when running on most devices
- Method tracing runs as a separate phase, after measurements - this enables accurate measurements and method traces to both be output from a single benchmark run
- Method tracing on some Android OS and ART versions will affect later measurement phases - on these versions, method tracing is off by default and a warning is printed to Studio output
Main thread benchmarks and ANRs
- Added measureRepeatedOnMainThread for UI thread benchmarks (eg those that interact with Compose/View UIs) to avoid ANRs when running for many seconds.
- Method traces are skipped if expected to overrun the ANR avoidance deadline. Set androidx.benchmark.profiling.skipWhenDurationRisksAnr to false to disable this behavior (not recommended for CI runs, as ANRs can cause problem in long CI runs).
Minification
- Embedded proguard rules to improve microbenchmarking with minification enabled
- Minification/R8 in a library module requires AGP 8.3, and can be enabled via android.buildTypes.release.androidTest.enableMinification in your build.gradle
- Experimental BlackHole.consume() API added to prevent dead code elimination ( If6812 , b/286091643 )
Метрики
- Experimental cpu event counter feature (metrics from perf_event_open , which requires root on most versions of the platform), access via InstrumentationArgument androidx.benchmark.cpuEventCounter.enable (can be set to true ), and androidx.benchmark.cpuEventCounter.events can be set eg to ( Instructions,CpuCycles ). This should be supported on some userdebug emulators, but support has not been tested across all available emulators

MACRObenchmark changes since 1.2.0

Method tracing overhaul for macrobenchmarks.
- Now method traces are scoped to the duration of the measureBlock , and can capture multiple sessions if the process starts multiple times.
- Previously, method tracing would only work for StartupMode.COLD benchmarks, and capture nothing for measureBlocks that didn't restart the target process
- Fixed method traces flush in macrobenchmark, so that method traces should be fully captured and valid, even on slower devices. ( I6349a , b/329904950 )
Correctly dump ART profile during individual warmUp iterations when process is killed so CompilationMode.Partial(warmup=N) measurements are more accurate. ( I17923 )
Drop Shader broadcast failure message
- Added debugging suggestions to drop shader broadcast failure message
- Add two instrumentation arguments for overriding shader dropping behavior to workaround crashes when benchmarking apps without ProfileInstaller 1.3:
  - androidx.benchmark.dropShaders.enable=true/false : can be used to skip all shader dropping (including that done in StartupMode.Cold launches), esp when benchmarking apps that don't yet use profileinstaller 1.3
  - androidx.benchmark.dropShaders.throwOnFailure=true/false : can be used to tolerate failures when trying to drop shaders, for example when benchmarking apps without profileinstaller 1.3 ( I4f573 )
Added experimental MacrobenchmarkRule#measureRepeated variant which takes a custom PerfettoConfig for fully customized Perfetto trace recording. Note that incorrectly configured configs may cause built in Metric classes to fail. ( Idfd3d , b/309841164 , b/304038384 )
Cancel background dexopt jobs before running a Macrobenchmark to reduce interference. ( I989ed )
Macrobenchmark now waits for 1 second for the target application to flush an ART profile (previously it waited for 500 ms). ( I85a50 , b/316082056 )
TraceSectionMetric overhaul
- Note : TraceSectionMetric changes below can affect outputs in CI usage, and may create discontinuities, or break parsing
- Sum is now the default, as most usage of this metric is for repeated events, and first would discard data in these cases
- Changed to be more customizable, with more available modes
- Mode names are now embedded in metric output name (in Studio and JSON)
- Now supports slices created using Trace.{begin|end}AsyncSection .
Метрики
- Power - Added PowerMetric.deviceSupportsHighPrecisionTracking , PowerMetric.deviceBatteryHasMinimumCharge() and PowerMetric.deviceSupportsPowerEnergy()
- Renamed Metric.getResult to getMeasurements to match return type
- Added log.w / exception labels to all startup detection failures. This does not change current behavior (so some errors throw, and others silently fail to detect the startup), just makes it more understandable. Generally the ones that Log.w() and fail to report startup metrics are those where non-frame events are missing, exceptions are thrown when startup is detected except for frame timing information (from UI/RT slices). ( Id240f , b/329145809 )
- Added frameCount measurement to FrameTimingMetric to aid in discovery of scenarios where measurements change because the number of frames produced changed (new animations added, invalidation issues fixed). ( I1e5aa )
- Clarified that frameOverrunMs is the preferred metric for tracking when available in docs, and why. ( I18749 , b/329478323 )
- Fixes issue where unterminated frames at the beginning and end of the trace could be paired together, which would incorrectly report as a single extremely long frame. ( I39353 , b/322232828 )
- Improve FrameTimingMetric error when frames aren't produced, and always output link to trace when failing metric parsing to assist in diagnosing problem. ( I956b9 )
- Fixed crash in FrameTimingMetric failing to parse frame id, especially on certain OEM devices. ( Ia24bc , b/303823815 , b/306235276 )
- Relaxed strictness of checks in FrameMetrics , and added more detail to error messages. ( Iadede )

Baseline Profile capture / Gradle plugin changes since 1.2.0

Increased max recommended version of AGP to 9.0.0-alpha01.
Ensure mergeArtProfile and mergeStartupProfile tasks always wait for baseline profile generation. ( I623d6 , b/343086054 )
Generating a baseline profile successfully will output a summary of what changed ( I824c8 , b/269484510 )
Added DSL to disable warnings ( Ic4deb , b/331237001 )
Fix to ensure benchmarks use generated baseline profiles when automaticGenerationDuringBuild is off ( Ic144f , b/333024280 )
Fix BaselineProfile gradle plugin property overrides to enable baseline profile generation and benchmarking when customizing a nonMinified or benchmark build type. ( Ib8f05 , b/324837887 )
Fix for including library baseline profiles in AAR prior to AGP 8.3.0-alpha15. ( I1d2af , b/313992099 )
Fixed baseline and startup profile output url at the end of generation task. ( I802e5 , b/313976958 )

Other significant changes since 1.2.0

Trace capture
- Reduced EXITCODE 2 error when starting perfetto from an error to logged warning
- Enable AIDL tracing by default in benchmarks(requires API 28) ( Ia0af2 , b/341852305 )
- Enable porter tag tracing by default in benchmarks. This captures, for example, wakelock tracepoints. ( Icfe44 , b/286551983 )
- Increased trace capture start timeout to avoid crashes when starting tracing on slower devices ( I98841 , b/329145808 )
- Added public API PerfettoTraceProcessor.Session.queryMetrics APIs with JSON, textproto, and proto binary (undecoded) variants. These allow you to query metrics built into TraceProcessor ( I54d7f , b/304038382 )
- Enable blocking start on Perfetto trace record to reduce risk of missing data at beginning of trace. Only supported on API 33+. ( Ie6e41 , b/310760059 )
JSON output
- Added additional information in benchmark context in JSON output:
  - context.artMainlineVersion - integer version of Art mainline module (if present on device, -1 otherwise)
  - context.build.id - Equals android.os.Build.ID
  - context.build.version.codename - Equals android.os.Build.VERSION.CODENAME
  - context.build.version.abbreviatedCodename - corresponds to first letter of pre-release codename (including on release builds) ( Ie5020 )
- Added profilerOutput list to JSON output for easier tooling around profiling traces (eg Perfetto, Method traces) ( I05ddd , b/332604449 )
- Added a warning when Android Test Orchestrator is used in benchmark modules, as this will cause per-module output JSON files to be repeatedly overwritten. ( Ia1af6 , b/286899049 )
- Throw when filenames are longer than 200 chars to avoid unclear crashes when writing or post-processing files. ( I4a5ab )

Version 1.3.0-rc01

7 августа 2024 г.

androidx.benchmark:benchmark-*:1.3.0-rc01 is released. Version 1.3.0-rc01 contains these commits .

Исправления ошибок

Fix androidx.benchmark.cpuEventCounter producing corrupt values for non-Instruction events ( I7386a , b/286306579 )
Fix resumeTiming / runWithTimingDisabled to respect metric priority order, and significantly reduce impact of lower priority metric pause/resume on higher priority metric results. For example, if using cpu perf counters via cpuEventCounter.enable instrumentation argument, timeNs is no longer significantly reduced when pause/resume occur. ( I39c2e , b/286306579 , b/307445225 )
Reduced chance of stack sampling causing measureRepeatedOnMainThread from hitting main thread hard timeout by moving stack sampling conversion off main thread. ( I487a8 , b/342237318 )
Removed manual outlining of access to new platform APIs since this happens automatically via API modeling when using R8 with AGP 7.3 or later (eg R8 version 3.3) and for all builds when using AGP 8.1 or later (eg D8 version 8.1). Clients who are not using AGP are advised to update to D8 version 8.1 or later. See this article for more details. ( I9496c , b/345472586 )
Added agp version check to send package name as instr arg. Previous to AGP 8.4.0 the target app package name cannot be send to the instrumentation app via instrumentation arguments. ( 0c72a3f )

Version 1.3.0-beta02

10 июля 2024 г.

androidx.benchmark:benchmark-*:1.3.0-beta02 is released. Version 1.3.0-beta02 contains these commits .

Исправления ошибок

Gracefully handle EXITCODE 2 when starting Perfetto to log a warning, but proceed.

Version 1.3.0-beta01

12 июня 2024 г.

androidx.benchmark:benchmark-*:1.3.0-beta01 is released. Version 1.3.0-beta01 contains these commits .

Изменения API

Renamed MethodTracing.affectsMeasurementOnThisDevice to AFFECTS_MEASUREMENT_ON_THIS_DEVICE for consistency. ( I1bdfa )
Added experimental BlackHole.consume() api to prevent dead code elimination in microbenchmarks. ( If6812 , b/286091643 )
Microbenchmark will now correctly throw to prevent method tracing from interfering with measurements. This occurs on certain devices when method tracing is forced on (via instrumentation args or MicrobenchmarkConfig ), and if a measurement is attempted after a method trace. Affected devices are running API 26-30 or certain ART mainline module versions affected by this interference, and can be detected at runtime via ProfilerConfig.MethodTracing.affectsMeasurementOnThisDevice . ( Iafb92 , b/303660864 )

Исправления ошибок

Bumped max agp version recommended to 9.0.0-alpha01. ( I5bbb0 )
Added compilation mode to benchmark context ( If5612 , b/325512900 )
Enable AIDL tracing by default (requires API 28) ( Ia0af2 , b/341852305 )
Added additional information in benchmark context in JSON output:
- context.artMainlineVersion - integer version of Art mainline module (if present on device, -1 otherwise)
- context.build.id - Equals android.os.Build.ID
- context.build.version.codename - Equals android.os.Build.VERSION.CODENAME
- context.build.version.abbreviatedCodename - corresponds to first letter of pre-release codename (even on release builds) ( Ie5020 )
Fixes StackSampling to respect androidx.benchmark.profiling.sampleDurationSeconds ( Ib1d53 )
Change macro->common dependency to be api() , so it's easier to use eg PerfettoTrace and PerfettoConfig . ( Icdae3 , b/341851833 )
Ensure mergeArtProfile and mergeStartupProfile tasks always wait for baseline profile generation. ( I623d6 , b/343086054 )
Consider variant enable state when deciding whether variant should be enabled. ( I5d19e , b/343249144 )
Increased default start timeout for perfetto trace processor. ( I87e8c , b/329145808 )

Version 1.3.0-alpha05

May 14, 2024

androidx.benchmark:benchmark-*:1.3.0-alpha05 is released. Version 1.3.0-alpha05 contains these commits .

Исправления ошибок

Throw clearer exception when macrobench metric returns zero values for all iterations ( Iab58f , b/314931695 )
Additional workaround rules added to microbench proguard rules, including support for listener rules and other observed warnings / errors. ( I14d8f , b/329126308 , b/339085669 )
Method tracing runs as a separate phase during a Macrobenchmark, and it no longer affects measurements. ( If9a50 , b/285912360 , b/336588271 )
Added extra debugging suggestions to drop shader broadcast failure message. ( I5efa6 , b/325502725 )

Version 1.3.0-alpha04

1 мая 2024 г.

androidx.benchmark:benchmark-*:1.3.0-alpha04 is released. Version 1.3.0-alpha04 contains these commits .

Изменения API

Added experimental MacrobenchmarkRule#measureRepeated variant which takes a custom PerfettoConfig for fully customized Perfetto trace recording. Note that incorrectly configured configs may cause built in Metric classes to fail. ( Idfd3d , b/309841164 , b/304038384 )
Rename PowerMetric.deviceSupportsPowerEnergy to PowerMetric.deviceSupportsHighPrecisionTracking for clarity ( I5b82f )
Added PowerMetric.deviceBatteryHasMinimumCharge() and PowerMetric.deviceSupportsPowerEnergy() to enable changing or skipping benchmarks based on device power measurement capability. ( I6a591 , b/322121218 )

Исправления ошибок

Added comparison with previous baseline profile ( I824c8 , b/269484510 )
Added DSL to disable warnings ( Ic4deb , b/331237001 )
Changed exception to info log when benchmark variants are disabled ( I8a517 , b/332772491 )
Make it simpler to capture method traces for a Macrobenchmark is scoped to the duration of the actual measureBlock() . Previously, it started at target process launch and only supported cold starts ( Iee85a , b/300651094 )
Avoid crashing when perfetto trace processor is slow to start ( I98841 , b/329145808 )

Version 1.3.0-alpha03

April 17, 2024

androidx.benchmark:benchmark-*:1.3.0-alpha03 is released. Version 1.3.0-alpha03 contains these commits .

Новые возможности

Adds public API PerfettoTraceProcessor.Session.queryMetrics APIs with JSON, textproto, and proto binary (undecoded) variants. These allow you to query metrics built into TraceProcessor ( I54d7f , b/304038382 )
Added profilerOutput to JSON output for easier tooling around profiling traces (eg perfetto, method traces). ( I05ddd , b/332604449 )
Added power tag to benchmark Perfetto Config. This captures, for example, wakelock tracepoints. ( Icfe44 , b/286551983 )
Added inst argument androidx.benchmark.profiling.skipWhenDurationRisksAnr , can be set to false to avoid skipping method traces when expected duration may cause an ANR - strongly recommended to avoid in CI runs.
Added experimental inst argument androidx.benchmark.profiling.perfCompare.enable , set this to true to run comparison timing between measurement and profiling phases. Useful in eg evaluating overhead of method tracing. ( I61fb4 , b/329146942 )

Изменения API

Changed TraceSectionMetric.Mode to sealed class to enable future expansion without breaking exhaustive when statements ( I71f7b )
Added TraceSectionMetric.Mode.Average and .Count , and reordered args so the more common argument (mode) was earlier in the arg list, reducing need for specifying parameter names. ( Ibf0b0 , b/315830077 , b/322167531 )
Renamed Metric.getResult to getMeasurements to match return type ( I42595 )

Исправления ошибок

Fix to ensure benchmarks use generated baseline profiles when automaticGenerationDuringBuild is off ( Ic144f , b/333024280 )
Fix BaselineProfile gradle plugin property overrides to enable baseline profile generation and benchmarking when customizing a nonMinified or benchmark build type. ( Ib8f05 , b/324837887 )
Fixed method traces flush in macrobenchmark, so that method traces should be fully captured and valid, even on slower devices. ( I6349a , b/329904950 )
Enable blocking start on Perfetto trace record to reduce risk of missing data at beginning of trace. Only supported on API 33+. ( Ie6e41 , b/310760059 )
Added a warning when Android Test Orchestrator is used in benchmark modules, as this will cause per-module output JSON files to be repeatedly overwritten. ( Ia1af6 , b/286899049 )
Force ',' (comma) thousands separators for consistency in Studio output, ignoring device locale ( I3e921 , b/313496656 )
TraceSectionMetric now supports slices created using Trace.{begin|end}AsyncSection . ( I91b32 , b/300434906 )
Added log.w / exception labels to all startup detection failures. This does not change current behavior (so some errors throw, and others silently fail to detect the startup), just makes it more understandable. Generally the ones that Log.w() and fail to report startup metrics are those where non-frame events are missing, exceptions are thrown when startup is detected except for frame timing information (from UI/RT slices). ( Id240f , b/329145809 )
Cancel background dexopt jobs before running a Macrobenchmark to reduce interference. ( I989ed )
Added frameCount measurement to FrameTimingMetric to aid in discovery of scenarios where measurements change because the number of frames produced changed (new animations added, invalidation issues fixed). ( I1e5aa )
Clarified that frameOverrunMs is the preferred metric for tracking when available in docs, and why. ( I18749 , b/329478323 )

Version 1.3.0-alpha02

20 марта 2024 г.

androidx.benchmark:benchmark-*:1.3.0-alpha02 is released. Version 1.3.0-alpha02 contains these commits .

Новые возможности

Experimental R8 support in microbench via embedded proguard rules. Note that this support is experimental, and requires AGP 8.3 for minification of library module tests. Use the following to enable R8 minification/optimization in your benchmark module's build.gradle , which should lead to a significant performance increase, depending on workload. ( I738a3 , b/184378053 )
```
android {
    buildTypes.release.androidTest.enableMinification = true
}
```

Исправления ошибок

Fixes method tracing warning to be on separate line from microbench output. ( I0455c , b/328308833 )

Version 1.3.0-alpha01

February 21, 2024

androidx.benchmark:benchmark-*:1.3.0-alpha01 is released. Version 1.3.0-alpha01 contains these commits.

Изменения API

Renamed MicrobenchmarkConfig boolean parameters to avoid unnecessary word 'should' ( Ia8f00 , b/303387299 )
Added BenchmarkRule.measureRepeatedOnMainThread so main thread benchmarks (eg ones touching Views or Compose UIs) can avoid triggering ANRs, especially during large suites in CI. ( I5c86d )
Added FrameTimingGfxInfoMetric , an experimental alternate implementation of FrameTimingMetric with measurements coming directly from the platform, rather than extracted from the Perfetto trace. ( I457cb , b/322232828 )
Add the ability to dump an ART profile during individual warmUp iterations. ( I17923 )
Several changes to TraceSectionMetric API:
- Add Mode.Min , Mode.Max
- Add label argument to override section name as metric label
- Added mode name to output to clarify metric meaning
- Changed default to sum, as most usage of this metric is for repeated events Be aware of this changes in CI usage, as it may create discontinuities or break parsing. ( Ic1e82 , b/301892382 , b/301955938 )

Исправления ошибок

Improved error message in baseline profile gradle plugin when specified managed device does not exist ( Idea2b , b/313803289 )
Fix for including library baseline profiles in AAR prior to AGP 8.3.0-alpha15 ( I1d2af , b/313992099 )
Fixed baseline and startup profile output url at the end of generation task ( I802e5 , b/313976958 )
Adjusted data source timeouts to attempt to fix java.lang.IllegalStateException: Failed to stop [ProcessPid(processName=perfetto, pid=...)] ( I8dc7d , b/323601788 )
Add two instrumentation arguments for overriding shader dropping behavior to workaround crashes when benchmarking apps without ProfileInstaller 1.3:
- androidx.benchmark.dropShaders.enable=true/false : can be used to skip all shader dropping (including that done in StartupMode.Cold launches), esp when benchmarking apps that don't yet use profileinstaller 1.3
- androidx.benchmark.dropShaders.throwOnFailure=true/false : can be used to tolerate failures when trying to drop shaders, for example when benchmarking apps without profileinstaller 1.3 ( I4f573 )
Skip method tracing on UI thread when expected to take longer than a few seconds, and cleanup method traces when throwing. ( I6e768 )
Throw when filenames are longer than 200 chars to avoid unclear crashes when writing or post-processing files. ( I4a5ab )
Fixes issue where unterminated frames at the beginning and end of the trace could be paired together, which would incorrectly report as a single extremely long frame. ( I39353 , b/322232828 )
Use --skip verification on API 30+ when reinstalling a package on API 30-33 to clear ART profiles on user builds. This helps bypass Play Protect warnings that cause failures on some class of devices. ( Ic9e36 )
Use am force-stop to kill apps when not a system app like System UI or Launcher. ( I5e028 )
Macrobenchmark now waits for 1 second for the target application to flush an ART profile (previously it waited for 500 ms ). ( I85a50 , b/316082056 )
Improve FrameTimingMetric error when frames aren't produced, and always output link to trace when failing metric parsing to assist in diagnosing problem. ( I956b9 )
Fixed crash in FrameTimingMetric failing to parse frame id, especially on certain OEM devices. ( Ia24bc , b/303823815 , b/306235276 )
Relaxed strictness of checks in FrameMetrics , and added more detail to error messages. ( Iadede )

Version 1.2

Version 1.2.4

April 17, 2024

androidx.benchmark:benchmark-*:1.2.4 is released. Version 1.2.4 contains these commits .

Исправления ошибок

Fixes baseline profile srcset not being set up in benchmark variants. Also fixes automaticGenerationDuringBuild in libraries causing a circular dependency. ( I28ab7 , b/333024280 )
Use am force-stop to kill apps when not a system app like System UI or Launcher. This fixes StartupMode.COLD benchmarks crashing from "Package $package must not be running prior to cold start!" due to process kill not fully succeeding. ( I5e028 )

Version 1.2.3

January 24, 2024

androidx.benchmark:benchmark-*:1.2.3 is released. Version 1.2.3 contains these commits.

Исправления ошибок

Removed exception from Baseline Profile Gradle Plugin when AGP version is 8.3.0 or higher.
Fix for including library baseline profiles in AAR prior to AGP 8.3.0-alpha15.

Version 1.2.2

1 декабря 2023 года

androidx.benchmark:benchmark-*:1.2.2 is released. Version 1.2.2 contains these commits.

Baseline Profiles

Execution logs will show the baseline profile output file path as a local file URI ( aosp/2843918 , aosp/2853665 , b/313976958 )

Версия 1.2.1

November 15, 2023

androidx.benchmark:benchmark-*:1.2.1 is released. Version 1.2.1 contains these commits.

Новые возможности

Improved error message when user disables test variants ( b/307478189 )
Added properties to support AS test run integration ( b/309805233 ), ( b/309116324 )

Version 1.2.0

October 18, 2023

androidx.benchmark:benchmark-*:1.2.0 is released. Version 1.2.0 contains these commits.

Important changes since 1.1.0

Baseline Profiles

New Baseline Profile Gradle Plugin automates capturing and including baseline profiles in your test and build workflow.
BaselineProfileRule.collect now stable, a streamlined and simplified version of the previous experimental BaselineProfileRule.collectBaselineProfile API
- Just specify packageName , and drive your app
For libraries generating baseline profiles, you can now filter the rules generated either in code ( BaselineProfileRule.collect argument), or even more simply in the gradle plugin
Исправления
- Fixed baseline profile collection on Android U+ ( Id1392 , b/277645214 )

Macrobenchmark

Сборник
- Macrobenchmark now correctly fully resets compilation state for each compile - this requires reinstalling the APK prior to Android 14, so benchmarking on Android 14+ is strongly recommended if you want to persist state (like user login) in what's being measured.
- You can also work around this by controlling app compilation separately, and skipping compilation with CompilationMode.Ignore() or instrumentation argument
Instrumentation Arguments
- Support for androidx.benchmark.dryRunMode.enable instrumentation argument, (already available in microbenchmark) for quicker validation runs (eg when creating the benchmark, or in presubmit)
- Support for androidx.benchmark.profiling.mode=StackSampling and MethodTracing .
- Added androidx.benchmark.enabledRules to allow runtime filtering baseline profile vs macrobenchmark rule tests
- Added androidx.benchmark.perfettoSdkTracing.enable argument to enable tracing with tracing-perfetto, eg Compose recomposition tracing. Note that when used with StartupMode.COLD , timing will be significantly affected as the tracing library is loaded and enabled during app startup.
Требования
- Macrobenchmark now requires ProfileInstaller 1.3.0 or greater in the target app, to enable profile capture / reset, and shader cache clearing.
New Experimental Metric APIs
- Added experimental TraceSectionMetric , which allows for extracting simple timing from trace("") {} blocks in your app, or TraceMetric for leveraging the full query capability of Perfetto TraceProcessor .
- Added experimental PowerMetric to capture power usage information
- Added experimental MemoryCountersMetric to count page faults
- Added experimental PerfettoTraceProcessor API, which is used internally to extract metrics from System traces (aka Perfetto traces)
Исправления
- Fixed crashes when installing or extracting profiles from an app installed from multiple APKs (eg from app bundle).
- Fixed FrameTimingMetric ignoring frames with inconsistent frame IDs (generally, frames during ripples on API 31+) ( I747d2 , b/279088460 )
- Fixed parsing errors on traces > 64MB ( Ief831 , b/269949822 )
- Clarified errors when device (especially emulator) OS image not correctly configured for tracing, or compilation
- Skip battery level check for devices without battery (micro and macro)
- Improved file output, with more clear errors for invalid output directories, and safer defaults
- Improved stability of StartupMode.COLD by consistently dropping the shader cache (also exposed via MacrobenchmarkScope.dropShaderCache )
- Fixed leanback fallback for startActivityAndWait .

Microbenchmark

Функции
- Profiling was moved to a separate phase, after other metrics, so one test run can display both accurate timing and profiling results.
Experimental APIs
- Added experimental MicrobenchmarkConfig API for defining custom metrics and configuring tracing and profiling. Can be used to capture method traces, or capture tracepoints (but be aware of tracing overhead).
- Added experimental APIs for controlling BenchmarkState separately from BenchmarkRule , without JUnit
- Added experimental PerfettoTrace record to enable capturing Perfetto traces, with custom configuration, separate from benchmark APIs.
Исправления
- Workaround missing leading whitespaces in Android Studio benchmark output.
- Fix issue where warnings could fail to print in Android Studio benchmark output.
- Fixed SampledProfiling crash on Android 13 (API 33) and higher.
- Massively improved performance of dryRunMode by skipping IsolationActivity and Perfetto tracing (Up to 10x faster dry run mode on older OS versions).

Version 1.2.0-rc02

October 6, 2023

androidx.benchmark:benchmark-*:1.2.0-rc02 is released. Version 1.2.0-rc02 contains these commits.

Исправления ошибок

Fix Benchmark file output to no longer break BaselineProfile Plugin file copying. Files were generated and copied off device, but had been renamed such that the gradle plugin wouldn't see them. ( I8dbcc , b/303034735 , b/296453339 )
Clarified tracing-perfetto loading error messages when injecting from macrobenchmark module into target application.

Version 1.2.0-rc01

20 сентября 2023 года

androidx.benchmark:benchmark-*:1.2.0-rc01 is released. Version 1.2.0-rc01 contains these commits.

Исправления ошибок

An exception (with remedy instructions) is now thrown when Perfetto SDK tracing fails to initialize in a Benchmark. ( I6c878 , b/286228781 )
Fix OOM crash when converting ART method trace -> perfetto format. ( I106bd , b/296905344 )
(Macrobenchmark) Clarified method tracing label when linked in Studio test output, and fixed method tracing filenames to be unique on device/host, so they won't be overwritten when more than one benchmark is run. ( I08e65 , b/285912360 )
Ensures that the device is awake when capturing a baseline profile. ( I503fc )

Version 1.2.0-beta05

August 30, 2023

androidx.benchmark:benchmark-*:1.2.0-beta05 is released. Version 1.2.0-beta05 contains these commits.

Новые возможности

The Baseline Profile Gradle Plugin now supports Android Gradle Plugin 8.3. ( aosp/2715214 )

Version 1.2.0-beta04

August 23, 2023

androidx.benchmark:benchmark-*:1.2.0-beta04 is released. Version 1.2.0-beta04 contains these commits.

Новые возможности

The Baseline Profiles Gradle plugin now supports Android Gradle Plugin 8.3. ( aosp/2715214 )

Исправления ошибок

Fix failures in writing / moving and pulling files (especially those from parameterized tests) by sanitizing output file names further, avoiding '=' and ':' in output file names. ( I759d8 )

Version 1.2.0-beta03

August 9, 2023

androidx.benchmark:benchmark-*:1.2.0-beta03 is released. Version 1.2.0-beta03 contains these commits.

Изменения API

Added argument to filter TraceSectionMetric to only the target package, on by default ( Ia219b , b/292208786 )

Исправления ошибок

Renamed fullTracing.enable instrumentation argument to perfettoSdkTracing.enable for consistency with artifact name, and other references. fullTracing.enable will continue to work as a fallback. ( I7cc00 )
Benchmark library internal tracepoints (including microbenchmark loop/phase tracing) will now show up in Studio system trace viewer, and nest under the correct process in Perfetto. ( I6b2e7 , b/293510459 )
Removed macrobenchmark NOT-PROFILEABLE error on API 31+, and skip profileable check on eng/userdebug rooted devices. ( I2abac , b/291722507 )
When using Dex Layout Optimizations, startup profile rules are also now considered as baseline profile rules. ( aosp/2684246 , b/293889189 )

Version 1.2.0-beta02

July 26, 2023

androidx.benchmark:benchmark-*:1.2.0-beta02 is released. Version 1.2.0-beta02 contains these commits.

Изменения API

Added experimental APIs for microbench custom metrics and configuration (eg profiler, and tracing). ( I86101 , b/291820856 )

Исправления ошибок

Report error in macrobench when OS is misconfigured for tracing, as was recently fixed in API 26/28 ARM64 emulators. ( I0a328 , b/282191686 )
Added detail to compilation reset failure to suggest updating emulator, as some emulators have recently fixed this issue. ( I8c815 , b/282191686 )
Make androidx.test.uiautomator:uiautomator:2.2.0 an api instead of an implementation dependency. ( I1981e )

Version 1.2.0-beta01

July 18, 2023

androidx.benchmark:benchmark-*:1.2.0-beta01 is released. Version 1.2.0-beta01 contains these commits.

Исправления ошибок

Fix warnings being sometimes suppressed in Benchmark output in Studio, and workaround leading whitespaces from Benchmark output not showing up in Studio ( Ia61d0 , b/227205461 , b/286306579 , b/285912360 )
Fixed comment for FrameTimingMetric . The submetric is named frameDurationCpuMs . ( Ib097f , b/288830934 ).

Version 1.2.0-alpha16

June 21, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha16 is released. Version 1.2.0-alpha16 contains these commits.

Изменения API

BaselineProfileRule.collectBaselineProfile() API has been renamed to BaselineProfileRule.collect() . ( I4b665 )

Исправления ошибок

Macrobenchmark support for androidx.benchmark.profiling.mode = MethodTracing . ( I7ad37 , b/285912360 )
Microbenchmark profiling moved to a separate phase, so it occurs in sequence after measurement, instead of replacing it. MethodTracing trace sections are also now included in the captured Perfetto trace, if present. ( I9f657 , b/285014599 )
Add count measurement to TraceSectionMetric with Mode.Sum . ( Ic121a , b/264398606 )

Version 1.2.0-alpha15

7 июня 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha15 is released. Version 1.2.0-alpha15 contains these commits.

Новые возможности

Added experimental MemoryUsageMetric for tracking memory usage of a target application. ( I56453 , b/133147125 , b/281749311 )
Add support for fully custom Perfetto configs with PerfettoTrace.record ( If9d75 , b/280460183 )
Added property to skip baseline profile generation. Usage: ./gradlew assemble -Pandroidx.baselineprofile.skipgeneration . ( I37fda , b/283447020 )

Изменения API

The collectBaselineProfile API always generates stable baseline profiles. The collectStableBaselineProfile API has been removed and collectBaselineProfile should be used instead. ( I17262 , b/281078707 )
Changed BaselineProfileRule 's filterPredicate arg to non-null, with a equivalent default value so that the default filter behavior is more clear in docs. ( I3816e )

Исправления ошибок

Disable IsolationActivity and Perfetto tracing in dryRunMode to significantly improve performance, as these were majority of runtime. ( Ie4f7d )
Support for call stack sampling in Macrobenchmarks using instrumentation test arguments androidx.benchmark.profiling.mode=StackSampling and androidx.benchmark.profiling.sampleFrequency . ( I1d13b , b/282188489 )
Fixes crash when dropping shaders on Android U (API 34), as well as on emulators. ( I031ca , b/274314544 )

Version 1.2.0-alpha14

May 3, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha14 is released. Version 1.2.0-alpha14 contains these commits.

Исправления ошибок

Fix FrameTimingMetric ignoring frames with inconsistent frame IDs. This would cause some animations on recent platform versions (API 31+) to ignore many frames while RenderThread was animating (eg during a ripple). ( I747d2 , b/279088460 )
Fixed trace processor parsing for traces larger than 64Mb. ( Ief831 , b/269949822 )
Fixed baseline profile generation on Android U failing because of the different output of pm dump-profiles command. ( Id1392 , b/277645214 )
Fix GPU clock locking script to compare strings correctly ( I53e54 , b/213935715 )

Version 1.2.0-alpha13

April 5, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha13 is released. Version 1.2.0-alpha13 contains these commits.

Изменения API

Added profile type parameter when generating baseline profiles to support upcoming startup profile feature ( Ie20d7 , b/275093123 )
Added new experimental TraceMetric API for defining fully custom metrics based on content of a Perfetto trace. ( I4ce31 , b/219851406 )
Add an experimental metric to determine the number of page faults during a benchmark. ( I48db0 )

Version 1.2.0-alpha12

March 22, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha12 is released. Version 1.2.0-alpha12 contains these commits.

Новые возможности

The new baseline profile gradle plugin is released in alpha version, making it easier to generate a baseline profile and simplifying the developer workflow.

Изменения API

Removed Perfetto tracing support on API 21 and 22, which includes both Microbenchmarks and the experimental PerfettoTrace APIs. Prior to this version, UiAutomation connections were unreliable on some devices. ( I78e8c )
Added public experimental API for PerfettoTraceProcessor to enable parsing trace content. This is a step toward fully custom metrics based on Perfetto trace data. ( I2659e , b/219851406 )

Version 1.2.0-alpha11

March 8, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha11 is released. Version 1.2.0-alpha11 contains these commits.

Исправления ошибок

Fixed crashes in MacrobenchmarkRule and BaselineProfileRule when reinstalling or extracting profiles from an app bundle with multiple APKs. ( I0d8c8 , b/270587281 )

Version 1.2.0-alpha10

22 февраля 2023 г.

androidx.benchmark:benchmark-*:1.2.0-alpha10 is released. Version 1.2.0-alpha10 contains these commits.

Новые возможности

On Android 14+, Macrobenchmark no longer reinstalls target applications to reset compilation state, thanks to a new platform feature. Previously it was necessary to have a rooted device, or to deal with all application state (eg user login) being removed before each benchmark runs. ( I9b08c , b/249143766 )

Исправления ошибок

Fix DryRunMode to no longer crash with empty profile, due to compilation skipping. Instead, it runs a single iteration and extracts the profile to ensure something is captured. ( I2f05d , b/266403227 )
Fix PowerMetric crash when checking for powerstats presence on old API levels. ( 5faaf9 , b/268253898 )

Version 1.2.0-alpha09

January 11, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha09 is released. Version 1.2.0-alpha09 contains these commits.

Исправления ошибок

Enabled passing None to androidx.benchmark.enabledRules instrumentation arg to disable all benchmarks / baseline profile generation. ( I3d7fd , b/258671856 )
Fix PerfettoTrace capture in app modules (ie non-self-instrumenting test APKs) ( I12cfc )
Fixed baseline profile adb pull argument order in Studio output ( I958d1 , b/261781624 )
Arm emulator api 33 is now correctly recognized as such when trying to run a macrobenchmark and will correctly print the warning. ( 69133b , b/262209591 )
Skip battery level check on devices without battery in Macrobenchmark ( fe4114 , b/232448937 )

Version 1.2.0-alpha08

December 7, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha08 is released. Version 1.2.0-alpha08 contains these commits.

Изменения API

Added experimental new APIs PerfettoTrace.record {} and PerfettoTraceRule to capture Perfetto traces (also known as System Traces) as part of a test, to inspect test behavior and performance. ( I3ba16 )
BaselineProfileRule now accepts a filter predicate instead of a list of package prefixes. This gives the test full control on filtering. ( I93240 )
Add an experimental API BaselineProfileRule.collectStableBaselineProfile which waits until a baseline profile is stable for N iterations. ( I923f3 )
Add the ability to specify an output file name prefix when generating baseline profiles using BaselineProfileRule . ( I7b59f , b/260318655 )

Исправления ошибок

Improve safety of file output writing, which should prevent output files from silently not being written / appended, especially on API 21/22. ( If8c44 , b/227510293 )
Fix simpleperf trace output to create and place the file correctly. This should also more generally fix issues where a file is unsuccessfully pulled by gradle. ( I12a1c , b/259424099 )
Improve profileinstaller error message printed when profileinstaller is too old. This now tells you to update profileinstaller version (1.2.1) for measuring baseline profiles on API 31 through 33, instead of saying it's not supported. ( Ia517f , b/253519888 )
Fix several shell command failures onerror message Print needed API <=23, including failed perfetto capture binary setup and trace capture failures ( Ib6b87 , b/258863685 )
Automatically sort generated profile rules to minimize the number of changes as they change over time (when checking-in profile rules into source control). ( Ie2509 )
Fixed crash on unrooted builds below Android 13 (API 33) with message Expected no stderr from echo 3 > /proc/sys/vm/drop_caches ( I6c245 , b/259508183 )

Known Issues - MacrobenchmarkScope.dropShaderCache() may crash due to a missing broadcast registry in profileinstaller manifest, which has not yet been released. ( I5c728 , b/258619948 ) To workaround the issue in profileinstaller:1.3.0-alpha02 , add the following to your application's (not your benchmark's) AndroidManifest.xml:

  <!-- workaround bug in profileinstaller 1.3.0-alpha02, remove when updating to alpha03+ -->
  <receiver
    android:name="androidx.profileinstaller.ProfileInstallReceiver"
    android:permission="android.permission.DUMP"
    android:exported="true">
    <intent-filter>
        <action android:name="androidx.profileinstaller.action.BENCHMARK_OPERATION" />
    </intent-filter>
  </receiver>

Version 1.2.0-alpha07

9 ноября 2022 г.

androidx.benchmark:benchmark-*:1.2.0-alpha07 is released. Version 1.2.0-alpha07 contains these commits.

Изменения API

Adds PowerMetric API for measuring energy and power in Macrobenchmarks. ( Ife601 , b/220183779 )
Fixed MacrobenchmarkScope.dropShaderCache() to actually drop the shader cache. This removes roughly 20ms of noise from StartupMode.COLD benchmarks, as shaders are now consistently cleared each iteration. Previously, Partial compilation using warmup iterations would report incorrectly fast numbers, as shader caching was more likely to happen during warmup. This fix requires either a rooted device, or using profileinstaller:1.3.0-alpha02 in the target app. For ProfileInstaller library's API changes, please refer to ProfileInstaller 1.30-alpha02 page. ( Ia5171 , b/231455742 )
Added TraceSectionMode("label", Mode.Sum) , allowing measurement of total time spent on multiple trace sections with the same label. For instance, TraceSectionMetric("inflate", Mode.Sum) will report a metric inflateMs for the total time in a macrobenchmark spent on inflation. Also removed API 29 requirement, as TraceSectionMetric works together with androidx.tracing.Trace back to lower API levels, with the use of forceEnableAppTracing within the target app. ( Id7b68 , b/231455742 )

Исправления ошибок

Improved safety of all internal shell commands by validating all output/errors. ( I5984d , b/255402908 , b/253094958 )
Specify device in baseline profile adb pull command, so the pull command can be simply copied if multiple devices are connected (up to one emulator) ( I6ac6c , b/223359380 )
Add error if macrobenchmark test apk isn't set up as self-instrumenting. This error prevents macrobenchmarking from within the target app's process. In process, macrobench wouldn't be able to compile/kill/cold start the app, or control its own permissions ( I4279b )
Fixed an issue in measureRepeated() where StartupMode.COLD wouldn't kill the target process after setupBlock . Now setupBlock interacting with the app will not leave the app process running, and an invalid cold start measurement. ( I8ebb7 )

Version 1.2.0-alpha06

October 24, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha06 is released. Version 1.2.0-alpha06 contains these commits.

Изменения API

BaselineProfileRule no longer requires root on Android 13 (API 33), and is no longer experimental. ( Ie0a7d , b/250083467 , b/253094958 )
- This change also fixes how profiles from an app are flushed to disk on unrooted devices, but requires updating the target app's profileinstaller dependency.
- To use BaselineProfileRule or CompilationMode.Partial(warmupIterations) on an unrooted device, you must also update your target app to use androidx.profileinstaller.profileinstaller:1.3.0-alpha01 . This enables flushing the profile to disk correctly, so that it can be compiled/extracted.

Исправления ошибок

Fixes SampledProfiling crash on API 33+. ( I40743 , b/236109374 )

Version 1.2.0-alpha05

October 5, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha05 is released. Version 1.2.0-alpha05 contains these commits.

Исправления ошибок

Fix frame breakdown in Studio system trace viewer for benchmark captured traces ( I3f3ae , b/239677443 )
Correct FrameTimingMetric to list FrameOverrun as requiring API 31 instead of 29 ( I716dd , b/220702554 )
Set iteration in BaselineProfileRule , and throw clearly if target package not installed (was already done for MacrobenchmarkRule). ( Ic09a3 , b/227991471 )

Version 1.2.0-alpha04

September 21, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha04 is released. Version 1.2.0-alpha04 contains these commits.

Новые возможности

Add support for dryRunMode.enable instrumentation argument to macrobenchmark (already available in micro) for faster local development, and validating app automation (eg in presubmit). This overrides iterations to 1, skips compilation, suppresses all configuration errors , and disables measurement .json file output. ( Ib51b4 , b/175149857 )
On Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.dryRunMode.enable=true
```
In build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.dryRunMode.enable', 'true'
    }
}
```

Исправления ошибок

Fixed StartupTimingMetric to no longer require measured Activities to be launched through MacrobenchmarkScope.startActivityAndWait() . This means the metric can pick up launches from eg notifications, Context.startActivity() , in-app Activity based navigation, or shell commands. ( Ia2de6 , b/245414235 )
Fix bug where startActivityAndWait would timeout trying to wait for launch completion on emulators by reducing strictness of frame detection. ( Ibe2c6 , b/244594339 , b/228946895 )

Version 1.2.0-alpha03

September 7, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha03 is released. Version 1.2.0-alpha03 contains these commits.

Новые возможности

Added experimental APIs for using BenchmarkState independently, separate from BenchmarkRule / JUnit4 . ( Id478f , b/228489614 )

Исправления ошибок

Added Leanback fallback for startActivityAndWait . ( 01ed77 , b/242899915 )

Version 1.2.0-alpha02

August 24, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha02 is released. Version 1.2.0-alpha02 contains these commits.

Изменения API

Default to am force stop for MacrobenchmarkScope.killProcess() , even when rooted, except during Baseline Profile generation. This can be overridden with an optional boolean argument. ( 02cce9 , b/241214097 )

Исправления ошибок

Support baseline profile generation for System apps. ( I900b8 , b/241214097 )
Support checking for ODPM power metrics on unrooted devices. ( a38c78 , b/229623230 )

Version 1.2.0-alpha01

July 27, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha01 is released. Version 1.2.0-alpha01 contains these commits.

Новые возможности

New tracing-perfetto-common component allowing tooling to enable Perfetto SDK tracing in an app that exposes it ( I2cc7f )
Added androidx.benchmark.enabledRules instrumentation argument to enable filtering macrobenchmark runs to just benchmarks, or just baseline profile generation. Pass in 'Macrobenchmark', or 'BaselineProfile' to just run one type of test, eg when just generating BaselineProfiles on an emulator. Comma-separated list also Supported. ( I756b7 , b/230371561 )
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.enabledRules', 'BaselineProfile'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
```

Изменения API

Added new PowerMetric for measuring energy and power tasks in benchmarks. ( I9f39b , b/220183779 )
Added a new compilation mode CompilationMode.Ignore to skip profile reset and compilation. ( Ibbcf8 , b/230453509 )
Added a new parameter to BaselineProfileRule#collectBaselineProfile to filter output file by package names ( If7338 , b/220146561 )
Enables developer to discharge device to measure power drain. ( I6a6cb )
Added the ability to clear shader cache in MacrobenchmarkScope . ( I32122 )
Enables developer to configure display of metric type and detail desired subsystem categories. ( I810c9 )
Previously an UnsupportedOperationException was thrown in the benchmark if run on an unsupported device. Now UOE only occurs if the metric is used on the unsupported device (ie: PowerMetric.configure ). ( I5cf20 , b/227229375 )
Added TotalPowerMetric and TotalEnergyMetric for measuring total power and energy in each system category in macrobenchmarks. ( I3b26b , b/224557371 )

Исправления ошибок

Fixed an issue where compiled methods were not correctly being reset between each macrobenchmark on unrooted builds. This unfortunately requires reinstalling the apk each iteration, which will clear application data for each macrobenchmark. ( I31c74 , b/230665435 )
Fix trace recording crash on API 21/22 ( If7fd6 , b/227509388 , b/227510293 , b/227512788 )
Overhaul activity launch completion detection to fix 'Unable to read any metrics' exception in startup macrobenchmarks. ( Ia517c )

Version 1.1.1

9 ноября 2022 г.

androidx.benchmark:benchmark-*:1.1.1 is released. Version 1.1.1 contains these commits.

Исправления ошибок

Fixes android.system.ErrnoException: open failed: EACCES which would occur on some Android11 (API 30)+ devices. This is a cherry-pick of a fix from 1.2.0-alpha01 . ( aosp/2072249 )

Версия 1.1.0

June 15, 2022

androidx.benchmark:benchmark-*:1.1.0 is released. Version 1.1.0 contains these commits.

This version is identical to androidx.benchmark:benchmark-*:1.1.0-rc03 .

Important changes since 1.0.0

Support for Jetpack Macrobenchmarks, which allows you to measure whole-app interactions like startup and scrolling , provides the ability to capture traces & measure trace sections .
Support for Baseline Profiles
- CompilationMode.Partial to measure the effectiveness of Baseline Profiles.
- @BaselineProfileRule to automatically generate Baseline profiles for a given critical user journey.
Support for Allocation metrics & profiling during Microbenchmark runs.

Version 1.1.0-rc03

June 1, 2022

androidx.benchmark:benchmark-*:1.1.0-rc03 is released. Version 1.1.0-rc03 contains these commits.

Исправления ошибок

Avoid reinstalling the target package on every benchmark iteration. ( aosp/2093027 , b/231976084 )
Remove the 300ms delay from pressHome() . ( aosp/2086030 , b/231322975 )
Improve Macrobenchmark iteration speed by optimizing Shell commands used under the hood. ( aosp/2086023 , b/231323582 )
Support for Managed Gradle Devices when generating Baseline Profiles with Macrobenchmarks. ( aosp/2062228 , b/228926421 )

Version 1.1.0-rc02

May 11, 2022

androidx.benchmark:benchmark-*:1.1.0-rc02 is released. Version 1.1.0-rc02 contains these commits.

Note that this release includes a behavior change, as apps are now fully reinstalled in between each benchmark to ensure accurate measurements.

Bug Fixes/Behavior Changes

Fixed an issue where app compilation was not correctly reset between macrobenchmarks, and not reset at all on unrooted builds. This fixes many cases where running multiple tests would result in CompilationMode having little to no effect on measurements. To workaround this problem, the target app is now fully reinstalling each test method, which will clear application data between each macrobenchmark. ( I31c74 , b/230665435 )
As this prevents apps from setting up state before tests, it is now possible to skip compilation / reinstallation to enable working around this. You can for example fully compile the target with a shell command cmd package compile -f -m speed <package> , and then bypass macrobenchmark's compilation step.
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.compilation.enabled, 'false'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.compilation.enabled=false
```
Made it possible to share a module between macrobenchmarks and baseline profile generating tests by adding androidx.benchmark.enabledRules instrumentation argument. Pass in 'Macrobenchmark', or 'BaselineProfile' to just run one type of test, eg when generating BaselineProfiles on an emulator. ( I756b7 , b/230371561 )
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.enabledRules', 'BaselineProfile'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
```

Version 1.1.0-rc01

April 20, 2022

androidx.benchmark:benchmark-*:1.1.0-rc01 is released. Version 1.1.0-rc01 contains these commits.

Исправления ошибок

Baseline profile output links in Android Studio now use a unique file name. This way the output always reflects the latest results of using a BaselineProfileRule . ( aosp/2057008 , b/228203086 )

Version 1.1.0-beta06

April 6, 2022

androidx.benchmark:benchmark-*:1.1.0-beta06 is released. Version 1.1.0-beta06 contains these commits.

Исправления ошибок

Fix trace recording crash on API 21/22 ( If7fd6 , b/227509388 )
Overhaul activity launch completion detection to fix 'Unable to read any metrics' exception in startup macrobenchmarks. ( Ia517c )
Fix startup metrics for Macrobenchmarks when CompilationMode.None() is used. Before this change, CompilationMode.Partial() would appear to be slower than Compilation.None() . ( 611ac9 ).

Version 1.1.0-beta05

March 23, 2022

androidx.benchmark:benchmark-*:1.1.0-beta05 is released. Version 1.1.0-beta05 contains these commits.

Исправления ошибок

Kill package after skipping profile installation when using CompilationMode.None . ( aosp/1991373 )
Fixed an issue where Macrobenchmarks is unable to collect startup metrics when using StartupMode.COLD . ( aosp/2012227 b/218668335 )

Version 1.1.0-beta04

February 23, 2022

androidx.benchmark:benchmark-*:1.1.0-beta04 is released. Version 1.1.0-beta04 contains these commits.

Исправления ошибок

Fix missing metrics on Android 10, and NoSuchElementException caused by process names not being captured correctly in traces. ( Ib4c17 , b/218668335 )
Use PowerManager for thermal throttling detection on Q (API 29) and higher. This significantly reduces frequency of false positives in thermal throttling detection (benchmark retry after 90 second cooldown), and speeds up benchmarks significantly on user builds. It also provides throttle detection even when clocks are locked (if they're locked too high for the device's physical environment). ( I9c027 , b/217497678 , b/131755853 )
Filter simpleperf sampled profiling to measureRepeated thread only to simplify inspection ( Ic3e12 , b/217501939 )
Support metrics from named UI subprocesses in multi-process apps ( Ice6c0 , b/215988434 )
Filter Baseline Profile rules to target Android 9 (SDK 28). aosp/1980331 b/216508418
Skip Profile Installation when using Compilation.None() . Additionally, report warnings when the app is using an older version of androidx.profileinstaller and Android Gradle Plugin. aosp/1977029

Version 1.1.0-beta03

February 9, 2022

androidx.benchmark:benchmark-*:1.1.0-beta03 is released. Version 1.1.0-beta03 contains these commits.

Изменения API

Added AudioUnderrunMetric into macrobenchmark library under experimental flag to allow detection of audio underruns ( Ib5972 )

BaselineProfileRule no longer accepts a setup block as this functioned the same as the profileBlock . ( Ic7dfe , b/215536447 )

For eg

@Test
fun collectBaselineProfile() {
    baselineRule.collectBaselineProfile(
        packageName = PACKAGE_NAME,
        setupBlock = {
            startActivityAndWait()
        },
        profileBlock = {
            // ...
        }
    )
}

@Test
fun collectBaselineProfile() {
    baselineRule.collectBaselineProfile(
        packageName = PACKAGE_NAME,
        profileBlock = {
            startActivityAndWait()
            // ...
        }
    )
}

Исправления ошибок

Fixed issue where microbench profiler traces would fail to be updated in subsequent runs when linked in Studio output ( I5ae4d , b/214917025 )
Prevent compilation shell commands on API 23 ( Ice380 )
Renamed FrameCpuTime -> FrameDurationCpu , FrameUiTime -> FrameDurationUi to clarify these are durations, not timestamps, and to match prefixes. ( I0eba3 , b/216337830 )

Version 1.1.0-beta02

January 26, 2022

androidx.benchmark:benchmark-*:1.1.0-beta02 is released. Version 1.1.0-beta02 contains these commits.

Исправления ошибок

Microbenchmark Stack Sampling / Method Tracing Profile results are now linked in Studio output, similar to other profiling outputs, and do not suppress the allocation metric. ( Idcb65 , b/214440748 , b/214253245 )
BaselineProfileRule now prints the adb pull command in logcat and Studio output for pulling generated BaselineProfile text file. ( f08811 )

Version 1.1.0-beta01

January 12, 2022

androidx.benchmark:benchmark-*:1.1.0-beta01 is released. Version 1.1.0-beta01 contains these commits.

Исправления ошибок

Fixes profiler argument enable being ignored. ( I37373 , b/210619998 )
Removed deprecated CompliationModes ( I98186 , b/213467659 )
Switched baseline profile arg of CompilationMode.Partial to enum for clarity. ( Id67ea )

Version 1.1.0-alpha13

December 15, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha13 is released. Version 1.1.0-alpha13 contains these commits.

Изменения API

Add low-overhead System Tracing to microbench output on Android Q (API 29+). Note that this does not currently capture custom tracing (via android.os.Trace or androidx.tracing Jetpack APIs) to avoid affecting results. This tracing should be useful in diagnosing instability, especially from sources outside the benchmark. ( I298be , b/205636583 , b/145598917 )
Clarify CompilationModes into three classes - Full, None, Partial. Previously they were inconsistently named after compilation arguments (which we now treat as implementation details) and features. This makes the tradeoffs, potential combinations, and behavior across platform versions more clear. ( I3d7bf , b/207132597 )
Setup and measure are now always in pairs, in order. You can now query the package name and iteration (though the iteration may be null in certain warmup scenarios). ( Id3b68 , b/208357448 , b/208369635 )

Исправления ошибок

Fixed CompilationMode.Speed incorrectly treated as None ( I01137 )

Version 1.1.0-alpha12

November 17, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha12 is released. Version 1.1.0-alpha12 contains these commits.

Новые возможности

Add experimental TraceSectionMetric for custom trace-based timing measurements. ( I99db1 , b/204572664 )

Исправления ошибок

Wake device each iteration, to ensure UI can be tested - requires lockscreen is disabled. ( Ibfa28 , b/180963442 )
Fixes multiple crashes in StackSampling profiling mode on emulators and non-rooted devices ( Icdbda , b/202719335 )
Removed 0.5 second sleep at the end of each iteration - if you see missing metrics with this change, please file a bug. ( Iff6aa )
Reduce chances of dropped data, and lower memory overhead from tracing ( Id2544 , b/199324831 , b/204448861 )
Reduce trace size by ~40% by switching to compact sched storage format. ( Id5fb6 , b/199324831 )
Updated implementations of startup metrics to always end at end of renderthread. This will be more contistent across platform versions, and more closely map to in-app measurements. ( Ic6b55 )

Version 1.1.0-alpha11

November 3, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha11 is released. Version 1.1.0-alpha11 contains these commits.

Изменения API

Macrobenchmark now has a minSdkVersion of 23 . ( If2655 )
Adds a new experimental BaselineProfileRule which is capable of generating baseline profiles for app's critical user journey. Detailed documentation to follow. ( Ibbefa , b/203692160 )
Removes measureRepeated interface variant, which was added for java callers, as it caused ambiguity in completing/resolving the method. Java callers will again need to return Unit.Instance from measureRepeated. If this is an inconvenience, please file a bug, we can revisit this in a future version. ( Ifb23e , b/204331495 )

Version 1.1.0-alpha10

October 27, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha10 is released. Version 1.1.0-alpha10 contains these commits.

Изменения API

Backport StartupTimingMetric to work back to API 23. This new implementation also better handles reportFullyDrawn() to wait until corresponding content has been rendered. ( If3ac9 , b/183129298 )
Added JvmOverloads to multiple MacrobenchmarkScope methods for Java callers. ( I644fe , b/184546459 )
Provide alternative MacrobenchmarkRule.measureRepeated function that uses a Consumer<MacrobenchmarkScope> for idiomatic usage in Java language. ( If74ab , b/184546459 )

Исправления ошибок

Fix for traces not starting early enough, and missing metric data. This is expected to fix "Unable to read any metrics during benchmark" exceptions that were caused by the library itself. ( I6dfcb , b/193827052 , b/200302931 )
FrameNegativeSlack has been renamed to FrameOverrun to clarify its meaning - how much the frame went over its time budget. ( I6c2aa , b/203008701 )

Version 1.1.0-alpha09

October 13, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha09 is released. Version 1.1.0-alpha09 contains these commits.

Исправления ошибок

Support dropping Kernel page cache without root on API 31/S+, which will increase accuracy of StartupMode.COLD launches. ( Iecfdb , b/200160030 )

Version 1.1.0-alpha08

September 29, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha08 is released. Version 1.1.0-alpha08 contains these commits.

Изменения API

Enable scrolling macrobenchmarks to run back to API 23 ( If39c2 , b/183129298 )
Add new type of sampled metric to UI and JSON output, focused on percentiles of multiple samples per iteration. ( I56247 , b/199940612 )
Switch to floating point metrics throughout the benchmark libraries (truncated in the Studio UI). ( I69249 , b/197008210 )

Версия 1.1.0-альфа07

1 сентября 2021 г.

androidx.benchmark:benchmark-*:1.1.0-alpha07 is released. Version 1.1.0-alpha07 contains these commits.

Изменения API

Raised min API to 21 to reflect the intended lowest API level to be supported in the future. Current min API supported continues to be conveyed via RequiredApi(), and is currently 29 ( I440d6 , b/183129298 )

Исправления ошибок

Fixes ProfileInstaller to make it easier for apps using baseline profiles to run MacroBenchmarks using CompilationMode.BaselineProfile . ( I42657 , b/196074999 ) NOTE: requires also updating to androidx.profileinstaller:profileinstaller:1.1.0-alpha04 or greater.
StartupMode.COLD + CompilationMode.None benchmarks are now more stable. ( I770cd , b/196074999 )

Версия 1.1.0-альфа06

August 18, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha06 is released. Версия 1.1.0-alpha06 содержит эти коммиты.

Изменения API

Added androidx.benchmark.iterations instrumentation argument to allow manual overriding of iteration count when testing/profiling locally. ( 6188be , b/194137879 )

Исправления ошибок

Switched to Simpleperf as default sampling profiler on API 29+. ( Ic4b34 , b/158303822 )

Известные проблемы

CompilationMode.BaselineProfile is a work in progress. Avoid using it to determine how good a profile is for now.

Версия 1.1.0-альфа05

August 4, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha05 is released. Версия 1.1.0-alpha05 содержит эти коммиты.

1.1.0-alpha04 was cancelled before release due to a sporatic crash. b/193827052

Изменения API

Switched startActivityAndWait to invoke launch via am start , which reduces the time of each measurement iteration by approximately 5 seconds, at the cost of no longer supporting intent parcelables. ( I5a6f5 , b/192009149

Исправления ошибок

Reduce aggressiveness of thermal throttle detection, and recompute baseline if throttles are detected frequently. ( I7327b )
Fixes FrameTimingMetric to work on Android S beta ( Ib60cc , b/193260119 )
Use an EmptyActivity to bring the target app out of a force-stopped state to better support CompilationMode.BaselineProfile . ( Id7cac , b/192084204 )
Changed trace file extension to .perfetto-trace to match platform standard. ( I4c236 , b/174663039 )
StartupTimingMetric now outputs the "fullyDrawnMs" metric to measure time until your application has completed rendering. To define this metric for your app, call Activity.reportFullyDrawn when your initial content is ready, such as when your initial list items are loaded from DB or network. (reportFullyDrawn method available without build version checks on ComponentActivity). Note that your test must run long enough to capture the metric (startActivityAndWait doesn't wait for reportFullyDrawn). ( If1141 , b/179176560 )
Reduce cost of appending Ui metadata to traces by 50+ ms ( Ic8390 , b/193923003 )
Drastically increased polling frequency when stopping tracing, which can reduce eg startup benchmark runtime by 30+% ( Idfbc1 , b/193723768 )

Версия 1.1.0-альфа03

June 16, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha03 is released. Версия 1.1.0-alpha03 содержит эти коммиты.

Новые возможности

Added a new CompilationMode.BaselineProfile to support profiles installed using the Jetpack ProfileInstaller library . ( aosp/1720930 )

Исправления ошибок

The sample Gradle code for suppressing benchmark errors has been updated to use a non-deprecated API with a syntax that also supports .gradle.kts users.
Например,
```
testInstrumentationRunnerArguments["androidx.benchmark.suppressErrors"] = "EMULATOR,LOW-BATTERY"
```

Версия 1.1.0-альфа02

18 мая 2021 г.

androidx.benchmark:benchmark-*:1.1.0-alpha02 is released. Версия 1.1.0-alpha02 содержит эти коммиты.

Новые возможности

Macrobenchmark artifacts added ( androidx.benchmark:benchmark-macro-junit4 and androidx.benchmark:benchmark-macro )

Capture startup, scrolling/animation performance metrics from your app, locally or in CI
Capture and inspect traces from within Android Studio

Исправления ошибок

Workaround shell permissions issue with output directory on Android 12 (Note - may require updating Android Gradle Plugin to 7.0.0 canary and Android Studio to Arctic Fox (2020.3.1), to continue capturing output files on affected devices). ( Icb039 )
Support configuration caching in BenchmarkPlugin ( 6be1c1 , b/159804788 )
Simplified file output - on by default, in a directory that doesn't require requestLegacyExternalStorage=true ( 8b5a4d , b/172376362 )
Fixes library printing logcat warnings about not finding JIT thread on platform versions where it is not present. ( I9cc63 , b/161847393 )
Fix for reading device max frequency. ( I55c7a )

Версия 1.1.0-альфа01

10 июня 2020 г.

New Features of 1.1

Allocation Metric - Benchmarks now run an additional phase after warmup and timing, capturing allocation counts. Allocations can cause performance problems on older versions of the platform (140ns in O became 8ns in M - measured on Nexus5X, with locked clocks). This metric is displayed in Android Studio console output, as well as in the
Profiling support - You can now capture profiling data for a benchmark run, to inspect why your code may be running slowly. Benchmark supports capturing either method tracing, or method sampling from ART. These files can be inspected with the Profiler inside Android Studio using File > Open .
The Benchmark Gradle plugin now provides defaults for simpler setup:
- testBuildType is set to release by default, to avoid using dependencies with code coverage built-in. The release buildType is also configured as the default buildType, which allows Android Studio to automatically select the correct build variant when opening a project for the first time. ( b/138808399 )
- signingConfig.debug is used as the default signing config ( b/153583269 )

** Bug Fixes **

Significantly reduced the warmup transition overhead, where the first measurement for each benchmark was artificially higher than others. This issue was more pronounced in very small benchmarks (1 microsecond or less). ( b/142058671 )
Fixed InstrumentationResultParser error printed for each benchmark when running from command line. ( I64988 , b/154248456 )

Известные проблемы

Command line, gradle invocations of Benchmark do not print out results directly. You can work around this by either running through Studio, or parsing the JSON output file for results.
Benchmark reporting fails to pull the report from devices that have an app installed with an applicationId ending with either “android” or “download” (case insensitive). Users hitting this issue should upgrade the Android Gradle Plugin to 4.2-alpha01 or later.

Версия 1.0.0

Benchmark Version 1.0.0

November 20, 2019

Основные особенности версии 1.0.0

The Benchmark library allows you to write performance benchmarks of app code and get results quickly.

Major features include:

Clock stabilization
Automatic thread prioritization
Support for UI performance testing, such as in the RecyclerView Sample
JIT-aware warmup and looping
JSON benchmark output for post-processing

Версия 1.0.0-rc01

October 23, 2019

Новые возможности

Added systrace tracing to benchmarks

Исправления ошибок

Fixed metric instability issue where JIT wouldn't finish before warm up due to deprioritization ( b/140773023 )
Unified JSON output directory across Android Gradle Plugin 3.5 and 3.6

Версия 1.0.0-бета01

October 9, 2019

Новые возможности

Run garbage collection before each warmup to reduce memory pressure from one benchmark to leak to the next ( b/140895105 )

Исправления ошибок

Added androidx.annotation:android-experimental-lint dependency, so that Java code will correctly produce lint errors when experimental API is not used, similar to what is provided by the Kotlin experimental annotation for Kotlin callers.
Now correctly detects usage of additionalTestOutputDir instrumentation argument for output in Android Gradle Plugin 3.6, to know when AGP will handle data copy.
Fix undetected clock frequency in JSON to correctly print -1 ( b/141945670 ).

Version 1.0.0-alpha06

September 18, 2019

Новые возможности

Added a check for incorrectly using the old package for the test runner, which now provides a more-helpful error message

API меняется

The experimental annotation ExperimentalAnnotationReport is now correctly public. Usage of the experimental BenchmarkState#report API now requires this annotation

Версия 1.0.0-альфа05

September 5, 2019

API меняется

BenchmarkState.reportData API is now marked experimental

Исправления ошибок

Fix for the clock-locking script, which would fail on devices that were either missing the cut or expr shell utilities.
Fixed an issue with ./gradlew lockClocks task that would hang on devices that were rooted with an older version of the su utility, which did not support the -c flag.

Версия 1.0.0-альфа04

August 7, 2019

Новые возможности

Gradle plugin
- Now automatically disables test coverage, and sets the AndroidBenchmarkRunner by default ( b/138374050 )
- Added support for new AGP-based data copy, when running benchmarks and when using AGP 3.6+
JSON format additions
- Output total benchmark test run time ( b/133147694 )
- @Parameterized benchmarks that use a name string (for example @Parameters(name = "size={0},depth={1}") ) now output parameter names and values per benchmark in the JSON output ( b/132578772 )
Dry Run mode ( b/138785848 )
- Added a "dry run" mode for running each benchmark loop only once, to check for errors/crashes without capturing measurements. This can be useful eg for, for example, quickly running benchmarks in presubmit to check that they're not broken.

API меняется

Module structure has changed, splitting the library ( b/138451391 )
- benchmark:benchmark-junit4 contains classes with JUnit dependency: AndroidBenchmarkRunner , and BenchmarkRule , both of which have moved into the androidx.benchmark.junit4 package
- benchmark:benchmark-common contains the rest of the logic, including the BenchmarkState API
- This split will allow the library to support benchmarking without JUnit4 APIs in the future
Configuration warnings are now treated as errors, and will crash the test ( b/137653596 )
- This is done to further encourage accurate measurements, especially in CI
- These errors can be reduced back to warnings with an instrumentation argument. For example: -e androidx.benchmark.suppressErrors "DEBUGGABLE,LOW_BATTERY"

Исправления ошибок

Errors when writing to external storage on Q devices provide more-descriptive messages, with suggestions of how to resolve the issue
Screens are automatically turned on during benchmark runs, instead of failing when the screen is off

External contributions

Thanks to Sergey Zakharov for contributing JSON output improvements and the fix for screen off issues!

Версия 1.0.0-альфа03

July 2, 2019

Новые возможности

Expose sleep duration due to thermal throttling per benchmark in the full JSON report

Исправления ошибок

The Gradle plugin should no longer be required to be applied after Android plugins and the Android block
Adds support for benchmark reports on Android 10 devices using scoped storage

Версия 1.0.0-альфа02

June 6, 2019

API меняется

Overhauled JSON schema. Further changes to the JSON schema are likely to be limited to additions:
- Reorganized the result object structure to support additional metric groups in the future ( b/132713021 )
- Added test run context information, such as device and build info and whether clocks are locked, to the top-level object ( b/132711920 )
- Time metric names now have 'ns' in their name ( b/132714527 )
- Additional stats added per reported metric (maximum, median, minimum), and removed simplified 'nanos' summary stat ( b/132713851 )
Removed XML output ( b/132714414 )
Thermal throttle detection removed from BenchmarkState.reportData API ( b/132887006 )

Исправления ошибок

Fixed ./gradlew lockClocks not sticking on some recent OS devices ( b/133424037 )
Throttling detection disabled for emulator ( b/132880807 )

Версия 1.0.0-альфа01

7 мая 2019 г.

androidx.benchmark:benchmark:1.0.0-alpha01 is released. The commits included in this version are available here .

Эталон

User Guide Code Sample Codelab

Справочник по API
androidx.benchmark
androidx.benchmark.junit4
androidx.benchmark.macro
androidx.benchmark.macro.junit4

Accurately measure your code's performance within Android Studio.

Latest Update	Стабильный выпуск	Кандидат на выпуск	Бета-версия	Альфа-релиз
9 апреля 2025 г.	1.3.4	-	-	1.4.0-alpha11

Объявление зависимостей

Macrobenchmark

To use Macrobenchmark in your project, add the following dependencies to your build.gradle file for your macrobenchmark module :

классный

dependencies {
  androidTestImplementation "androidx.benchmark:benchmark-macro-junit4:1.3.4"
}

Котлин

dependencies {
  androidTestImplementation("androidx.benchmark:benchmark-macro-junit4:1.3.4")
}

Microbenchmark

To use Microbenchmark in your project, add the following dependencies to your build.gradle file for your microbenchmark module :

классный

dependencies {
    androidTestImplementation "androidx.benchmark:benchmark-junit4:1.3.4"
}

android {
    ...
    defaultConfig {
        ...
        testInstrumentationRunner "androidx.benchmark.junit4.AndroidBenchmarkRunner"
    }
}

Котлин

dependencies {
    androidTestImplementation("androidx.benchmark:benchmark-junit4:1.3.4")
}

android {
    ...
    defaultConfig {
        ...
        testInstrumentationRunner = "androidx.benchmark.junit4.AndroidBenchmarkRunner"
    }
}

Чтобы использовать плагин, включите следующую строку в блок `plugins` вашего файла build.gradle верхнего уровня:

классный

plugins {
  id 'androidx.benchmark' version '1.3.4' apply false
}

Котлин

plugins {
  id("androidx.benchmark") version "1.3.4" apply false
}

Затем примените плагин к файлу build.gradle вашего тестового модуля.

классный

plugins {
  id 'androidx.benchmark'
}

Котлин

plugins {
    id("androidx.benchmark")
}

Обратная связь

Создать новую задачу

Дополнительную информацию см. в документации по системе отслеживания проблем .

Version 1.4

Version 1.4.0-alpha11

9 апреля 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha11 is released. Version 1.4.0-alpha11 contains these commits .

Изменения API

Changed TraceProcessor 's argument timeout: Duration to timeoutMs long for Java caller usability. ( I9fbb5 )
Mark TraceProcessor constructor as internal. Callers should use TraceProcessor.startServer or TraceProcessor.runServer . ( Ia8c5b )

Исправления ошибок

When killing the application with MacrobenchmarkScope.killProcess , validate the kill command results to prevent silent failure, and timeout error. ( I84555 )

Dependency Updates

This library now targets Kotlin 2.0 language level and requires KGP 2.0.0 or newer. ( Idb6b5 )

Version 1.4.0-alpha10

26 марта 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha10 is released. Version 1.4.0-alpha10 contains these commits .

Изменения API

Increased default timeout for TraceProcessor server load and querying to 120 seconds (from 60/30 previously), and made both configurable with one timeout parameter. ( Ifec87 )

Исправления ошибок

Fixed a few issues that would occur when benchmarking or capturing profiles of an app without profileinstaller , and it's included BroadcastReciever . This only affects runs on rooted devices. ( Ied308 )

Version 1.4.0-alpha09

March 12, 2025

androidx.benchmark:benchmark-*:1.4.0-alpha09 is released. Version 1.4.0-alpha09 contains these commits .

Изменения API

Added TraceProcessor and Session API with closable Handles, for easier usage with custom lifecycles. This is also a step towards enabling easier Coroutine and Java API usage. The extension functions to TraceProcessor.runServer {} are now marked experimental, as they're likely to move, and be made non-experimental in the future. ( I358b4 )

Исправления ошибок

Fixed Benchmark and Baseline Profile capture not working with API 36 due to a change in pgrep toybox that now requires -a to print the full command line. ( Idc991 )
Filter default tracing config to reduce risk of data loss in traces on newer API levels. ( I54e8a )
Added experimental androidx.benchmark.killExistingPerfettoRecordings instrumentation argument, which can be set to false to allow pre existing perfetto trace capture to continue. By default pre existing perfetto trace captures on device are killed to prevent interference. ( I02a3c )
The JSON field context.osCodenameAbbreviated will now be REL for released OS versions at and above API 35, as non-numeric code names are no longer supported by the underlying platform. ( Ib17fd )
Fixes crash in FrameTimingMetric when resynced frames occur. ( I7c6f4 , b/394610806 )
No longer assume Choreographer#doFrame is the top of the stack frame on the main thread for FrameTimingQuery . ( Iee0e0 , b/340206285 )

Version 1.4.0-alpha08

12 февраля 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha08 is released. Version 1.4.0-alpha08 contains these commits .

Изменения API

Moved TraceProcessor.runSession() extension APIs to be experimental, as they are likely to move to be concrete constructors eventually on Android. ( Ib0528 , b/393640753 )
Most of the implementation of Startup Insights are now public/experimental, and move to the TraceProcessor artifact See StartupInsights . ( I0aa00 )
Deprecate BenchmarkRule.runWithTimingDisabled {} in favor of BenchmarkRule.runWithMeasurementDisabled {} , which more clearly describes the behavior - all metrics are paused. Additionally, expose the MicrobenchmarkScope superclass since redeclaring the runWithMeasurementDisabled function to open access isn't possible, since it's inline. ( I9e23b , b/389149423 , b/149979716 )
Benchmark libraries have moved to Kotlin 2.0. ( I9d1e0 )
Removed androidx.benchmark.startupProfiles.enable instrumentation argument. It is no longer useful, as it can be controlled by includeInStartupProfile arg in BaselineProfileRule.collect() . ( I39eb4 )

Исправления ошибок

Reduced the amount of Microbenchmark internal functions called during profiling to make eg Method traces more clear ( Ifaed8 )
Speculative fix for crashes: 'Failed to stop [ ProcessPid(processName=perfetto, pid=...) ]'. Now Benchmark will log a message instead of crashing when a background Perfetto process isn't able to be stopped before running the benchmark. ( I37d3e , b/323601788 )
Fix IllegalStateExceptions with 'Expected pm dump-profiles stdout' label that were caused by overly strict output format check. ( I358dc )

Version 1.4.0-alpha07

29 января 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha07 is released. Version 1.4.0-alpha07 contains these commits .

Новые возможности

BaselineProfileRule now introduces a collectWithResults(...) API which includes a list of paths to the computed profiles. ( I056f8 )
Added androidx.benchmark.measureRepeatedOnMainThread.throwOnDeadline instrumentation argument, which can be set to false to disable the throw on deadline behavior of measureRepeatedOnMainThread for local testing. Not otherwise recommended, as this increases the likelihood of ANRs during tests. ( Idbeec , b/353226476 )

Изменения API

Added @JvmOverloads to MicrobenchmarkConfig constructor. ( I13fd3 )
Refactored BenchmarkRule to be built on top of coroutines, and support better yield() behavior. This refactor removed several experimental BenchmarkState APIs, but will be followed by replacements as needed. Additionally, added runWithMeasurementDisabled to clarify behavior (all measurement is paused). In the future, runWithTimingDisabled will be deprecated. ( I19837 , b/389149423 , b/311242861 )
Move PerfettoTraceProcessor to TraceProcessor in a new androidx.benchmark:benchmark-traceprocessor artifact, and make most of its API non-experimental. Any custom TraceMetric or anything reading from traces will need to update to the new TraceProcessor import. The new TraceProcessor API works exactly like the old one, but is a standalone interface library (somewhat analogous to the androidx.sqlite layer from Room) with an Android-specific implementation built into macrobenchmark. The new artifact can be used on JVM as well, but currently you'll need to start your own copy of the TraceProcessor binary and offer a port to connect to it on. ( I3a767 , I62563 , b/381134564 )

Исправления ошибок

Throw a clearer error message when MacrobenchmarkScope.startActivityAndWait fails to launch the target process (potentially due to a crash in the target process), instead of the more ambiguous 'Unable to confirm activity launch completion' message ( I3539b )
Fixed several syntax errors in Kotlin samples, and syntax highlighting in several Java / build.gradle samples. ( Ib3808 )
Clarified ArtMetric and CaptureInfo parameter docs. ( I96e60 )

Version 1.4.0-alpha06

December 11, 2024

androidx.benchmark:benchmark-*:1.4.0-alpha06 is released. Version 1.4.0-alpha06 contains these commits .

Изменения API

Removed usage of @Language("sql") in PerfettoTraceProcessor.Session.query() , as Studio highlighting/parsing is broken. ( Idc2fa , b/377733398 )

Исправления ошибок

This library now uses JSpecify nullness annotations , which are type-use. Kotlin developers should use the following compiler arguments to enforce correct usage: -Xjspecify-annotations=strict , -Xtype-enhancement-improvements-strict-mode ( I46810 , b/326456246 )
Corrected ArtMetric to report class load (not init), and improved docs to clarify runtime behavior. ( I9915c )
On Android Multiuser, execute commands as root only on rooted devices. ( I88b44 )

Version 1.4.0-alpha05

November 13, 2024

androidx.benchmark:benchmark-*:1.4.0-alpha05 is released. Version 1.4.0-alpha05 contains these commits .

Исправления ошибок

Fixed an issue on API 34+ where CompilationMode.None() would have inconsistent performance not representative of initial, worst case performance. This works around a platform change which allows ART's compilation state verify to partially compile apps (only affecting class loading) shortly after first launch. ( Ie48d0 )
Fixed issue where (especially short) traces could be captured that wouldn't report measurement from built-in Macrobenchmark Metrics, due to the process name being truncated within the Perfetto trace. Now macrobenchmark works around this issue by looking for the truncated package name in all built-in queries, in addition to the expected package name. Note that custom TraceMetric implementations or other direct callers of PerfettoSession.query can implement this same behavior by changing process.name LIKE "$packageName" in a Perfetto query to instead be (process.name LIKE "$packageName" OR process.name LIKE "$(packageName.takeLast(15))") . ( I5bf01 , b/377565760 )

Version 1.4.0-alpha04

30 октября 2024 г.

androidx.benchmark:benchmark-*:1.4.0-alpha04 is released. Version 1.4.0-alpha04 contains these commits .

Новые возможности

(Experimental) Enable Baseline Profile generation, and benchmarking on apps installed to a secondary user, for example any app on headless Android Auto devices. This support has been tested in some scenarios, but let us know with a bug if it doesn't work for you. ( I9fcbe , b/356684617 , b/373641155 )

Исправления ошибок

isProfileable is now always overridden in benchmark builds, and isDebuggable is also now always overridden in both benchmark and nonMinified (baseline profile capture) builds. ( I487fa , b/369213505 )
Fixes compilation detection on some physical devices prior to API 28 - affects json context.compilationMode , as well as behavior of androidx.benchmark.requireAot=true (which no longer incorrectly throws) ( Ic3e08 , b/374362482 )
In CpuEventCounter metrics, throw if invalid measurements are observed (eg instructions/cpucycles==0) ( I8c503 )

Version 1.4.0-alpha03

16 октября 2024 года

androidx.benchmark:benchmark-*:1.4.0-alpha03 is released. Version 1.4.0-alpha03 contains these commits .

Изменения API

Macrobenchmark : Adds ArtMetric , which can be used to inspect profile coverage or general Android RunTime performance. Captures number and total duration of JIT, class init (where available), and class verification. Additionally, changes CaptureInfo to include optional ART mainline version with default. ( I930f7 )
Add coefficientOfVariation to Benchmark JSON output to show stability within a given benchmark run. ( Ib14ea )

Исправления ошибок

Fixed CollectBaselineProfileTask when AVD device has spaces in it. ( Ia0225 , b/371642809 )
Speculative fix for errors from StartupMode.COLD exceptions: Package <packagename> must not be running prior to cold start! . Now, MacrobenchmarkScope.killProcess() (including the one run before each iteration, used to implement StartupMode.COLD behavior) will wait to verify that the app's processes have all stopped running. ( I60aa6 , b/351582215 )
Fixed issue where UNLOCKED_ error would show up on some rooted emulators. ( Ic5117 )
This library now uses JSpecify nullness annotations , which are type-use. Kotlin developers should use the following compiler arguments to enforce correct usage: -Xjspecify-annotations=strict , -Xtype-enhancement-improvements-strict-mode ( I7104f , b/326456246 )

Version 1.4.0-alpha02

2 октября 2024 г.

androidx.benchmark:benchmark-*:1.4.0-alpha02 is released. Version 1.4.0-alpha02 contains these commits .

Изменения API

Moved Gradle tasks lockClocks and unlockClocks to be on benchmark projects, instead of available at the top level. This change was necessary as there is unfortunately no way to register these as top level actions without breaking project isolation. ( I02b8f , b/363325823 )

Исправления ошибок

BaselineProfileRule now collects profiles for multi-process apps by signaling each running process at the end of the block to dump profiles. If a profile based compilation never successfully finds a process to broadcast to, the compilation will fail, as it's unexpected to have profile data within. Additionally, added an instrumentation argument to control dump wait duration: androidx.benchmark.saveProfileWaitMillis ( I0f519 , b/366231469 )
From Benchmark 1.3.2 : Fixed Firebase Test Lab (FTL) being unable to pull Baseline Profile or Macrobenchmark result files from the Baseline Profile Gradle Plugin. ( I2f678 , b/285187547 )

To use FTL apply the plugin to the baseline profile module in the plugin block, with:

  plugins {
      ...
      id("com.google.firebase.testlab")
  }

and then configure firebase test lab with:

  firebaseTestLab {

      // Credentials for FTL service
      serviceAccountCredentials.set(file("credentials.json"))

      // Creates one or more managed devices to run the tests on.
      managedDevices {
          "ftlDeviceShiba34" {
              device = "shiba"
              apiLevel = 34
          }
      }

      // Ensures the baseline profile is pulled from the device.
      // Note that this will be automated as well later with aosp/3272935.
      testOptions {
          results {
              directoriesToPull.addAll("/storage/emulated/0/Android/media/${android.namespace}")
          }
      }
  }

Also the created FTL device needs to be added to the baseline profile extension:

  baselineProfile {
      managedDevices += "ftlDeviceShiba34"
      useConnectedDevices = false
  }

Version 1.4.0-alpha01

18 сентября 2024 года

androidx.benchmark:benchmark-*:1.4.0-alpha01 is released. Version 1.4.0-alpha01 contains these commits .

New Feature - App Startup Insights

Initial version of app startup insights can be enabled in Macrobenchmark. ( 09fae38 )

To enable in a startup benchmark:

  @Test
  fun startup {
      macrobenchmarkRule.measureRepeated(
          …
          packageName = "com.example.my.application.id"
          metrics = listOf(StartupTimingMetric()),
          iterations = 5,
          startupMode = StartupMode.COLD,
          compilationMode = CompilationMode.None(),
          experimentalConfig = ExperimentalConfig(startupInsightsConfig = StartupInsightsConfig(isEnabled = true))
          ) {
          scope.startActivityAndWait(...)
      }
  }

Then running your startup benchmark will analyze the trace to look for common problems, and print them after metrics to Studio test output in the benchmark tab, eg:

StartupBenchmark_startup[startup=COLD,compilationMode=None]
├── Metrics
│   ├──   timeToFullDisplayMs                min  1,147.2,   median  1,208.8,   max  1,307.4
│   └──   timeToInitialDisplayMs             min  1,147.2,   median  1,208.8,   max  1,307.4
├── App Startup Insights
│   ├── App in debuggable mode (expected: false)
│   │   └── seen in iterations: 0(true) 1(true) 2(true) 3(true) 4(true) 5(true) 6(true) 7(true) 8(true) 9(true)
│   ├── Potential CPU contention with another process (expected: < 100000000ns)
│   │   └── seen in iterations: 4(105022546ns)
│   └── Main Thread - Binder transactions blocked (expected: false)
│       └── seen in iterations: 7(true)
└── Traces
    └── Iteration 0 1 2 3 4 5 6 7 8 9

This feature is still a work-in-progress, with improvements to documentation and extensibility to follow, but feedback is welcome.

Новые возможности

Added gradle property androidx.baselineprofile.suppressWarnings to suppress all baseline profile warnings. ( 314153a )
Microbench metrics are now displayed in Perfetto traces as counters. ( 3214854 )
Add experimental scripts for disabling jit (requires root / runtime restart), and resetting device perf/test state. These are not currently published as gradle tasks. ( 7c3732b )
Added benchmark argument to skip tests when running on emulator. When automaticGenerationDuring build is enabled, benchmarks will also trigger baseline profile generation. This will fail, if emulators are used. With the new argument skipBenchmarksOnEmulator we can instead skip the test. ( 0c2ddcd )
Change perf event enable logic to run on API 23+ ( 2550048 )

Изменения API

Existing experimental PerfettoConfig argument to MacrobenchmarkRule.measureRepeated() moved to the new ExperimentalConfig object.

Исправления ошибок

Increase lockClocks.sh retry count ( 99e9dac )
Don't create nonMinified and benchmark build types if existing. Due to a bug, even if nonMinified and benchmark build types existed, they were going to be recreated. ( e75f0a5 )
Ignore non-terminating slices from TraceSectionMetric results. ( a927d20 )
Improved emulator check to consider sdk_ prefix. ( 1587de8 )
Treat non-running packages as cleared in FrameTimingGfxInfoMetric . ( 35cc79c )
Fix androidx.benchmark.cpuEventCounter producing corrupt values for non-Instruction events. ( 06edd59 )
Fix resumeTiming/runWithTimingDisabled to respect metric priority order, and significantly reduce impact of lower priority metric pause/resume on higher priority metric results. For example, if using cpu perf counters via cpuEventCounter.enable instrumentation argument, timeNs is no longer significantly reduced when pause/resume occur. ( 5de0968 )

Версия 1.3

Version 1.3.4

26 марта 2025 г.

androidx.benchmark:benchmark-*:1.3.4 is released. Version 1.3.4 contains these commits .

Исправления ошибок

Fixed Gradle Project Isolation incompatibilities in the Benchmark Baseline Gradle Plugin. ( b/404523257 )

Version 1.3.3

16 октября 2024 года

androidx.benchmark:benchmark-*:1.3.3 is released. Version 1.3.3 contains these commits .

Исправления ошибок

Fixed CollectBaselineProfileTask when AVD device has spaces in it ( Ia0225 , b/371642809 )

Version 1.3.2

2 октября 2024 г.

androidx.benchmark:benchmark-*:1.3.2 is released. Version 1.3.2 contains these commits .

Исправления ошибок

Fixed Firebase Test Lab (FTL) being unable to pull Baseline Profile or Macrobenchmark result files from the Baseline Profile Gradle Plugin. ( I2f678 , b/285187547 )

To use FTL apply the plugin to the baseline profile module in the plugin block, with:

  plugins {
      ...
      id("com.google.firebase.testlab")
  }

and then configure firebase test lab with:

  firebaseTestLab {

      // Credentials for FTL service
      serviceAccountCredentials.set(file("credentials.json"))

      // Creates one or more managed devices to run the tests on.
      managedDevices {
          "ftlDeviceShiba34" {
              device = "shiba"
              apiLevel = 34
          }
      }

      // Ensures the baseline profile is pulled from the device.
      // Note that this will be automated as well later with aosp/3272935.
      testOptions {
          results {
              directoriesToPull.addAll("/storage/emulated/0/Android/media/${android.namespace}")
          }
      }
  }

Also the created FTL device needs to be added to the baseline profile extension:

  baselineProfile {
      managedDevices += "ftlDeviceShiba34"
      useConnectedDevices = false
  }

Version 1.3.1

18 сентября 2024 года

androidx.benchmark:benchmark-*:1.3.1 is released. Version 1.3.1 contains these commits .

Исправления ошибок

Added gradle property androidx.baselineprofile.suppressWarnings to suppress all baseline profile warnings ( I7c36e , b/349646646 )
Fixed Baseline Profile Gradle Plugin to use pre-existing nonMinified… and benchmark… if created by the app instead of creating wrappers. ( Ia8934 , b/361370179 )
Fixed java.lang.AssertionError: ERRORS (not suppressed): EMULATOR when automaticGenerationDuringBuild is enabled on emulators. New argument is used to instead skip the test. ( If3f51 , b/355515798 )
Microbenchmark minification - keep subclasses of org.junit.runner.notification.RunListener in benchmark library proguard ( Ic8ed5 , b/354264743 )
Fix TraceSectionMetric to Ignore non-terminating slices. Previously these were considered to have -1 duration, eg during summation or finding minimum duration. ( If74b7 )
Fixed an issue in FrameTimingGfxInfoMetric where starting the metric would crash if the process wasn't already running. ( I6e412 )

Version 1.3.0

21 августа 2024 года

androidx.benchmark:benchmark-*:1.3.0 is released. Version 1.3.0 contains these commits .

Microbenchmark changes since 1.2.0

Method tracing is on by default in microbenchmarks when running on most devices
- Method tracing runs as a separate phase, after measurements - this enables accurate measurements and method traces to both be output from a single benchmark run
- Method tracing on some Android OS and ART versions will affect later measurement phases - on these versions, method tracing is off by default and a warning is printed to Studio output
Main thread benchmarks and ANRs
- Added measureRepeatedOnMainThread for UI thread benchmarks (eg those that interact with Compose/View UIs) to avoid ANRs when running for many seconds.
- Method traces are skipped if expected to overrun the ANR avoidance deadline. Set androidx.benchmark.profiling.skipWhenDurationRisksAnr to false to disable this behavior (not recommended for CI runs, as ANRs can cause problem in long CI runs).
Minification
- Embedded proguard rules to improve microbenchmarking with minification enabled
- Minification/R8 in a library module requires AGP 8.3, and can be enabled via android.buildTypes.release.androidTest.enableMinification in your build.gradle
- Experimental BlackHole.consume() API added to prevent dead code elimination ( If6812 , b/286091643 )
Метрики
- Experimental cpu event counter feature (metrics from perf_event_open , which requires root on most versions of the platform), access via InstrumentationArgument androidx.benchmark.cpuEventCounter.enable (can be set to true ), and androidx.benchmark.cpuEventCounter.events can be set eg to ( Instructions,CpuCycles ). This should be supported on some userdebug emulators, but support has not been tested across all available emulators

MACRObenchmark changes since 1.2.0

Method tracing overhaul for macrobenchmarks.
- Now method traces are scoped to the duration of the measureBlock , and can capture multiple sessions if the process starts multiple times.
- Previously, method tracing would only work for StartupMode.COLD benchmarks, and capture nothing for measureBlocks that didn't restart the target process
- Fixed method traces flush in macrobenchmark, so that method traces should be fully captured and valid, even on slower devices. ( I6349a , b/329904950 )
Correctly dump ART profile during individual warmUp iterations when process is killed so CompilationMode.Partial(warmup=N) measurements are more accurate. ( I17923 )
Drop Shader broadcast failure message
- Added debugging suggestions to drop shader broadcast failure message
- Add two instrumentation arguments for overriding shader dropping behavior to workaround crashes when benchmarking apps without ProfileInstaller 1.3:
  - androidx.benchmark.dropShaders.enable=true/false : can be used to skip all shader dropping (including that done in StartupMode.Cold launches), esp when benchmarking apps that don't yet use profileinstaller 1.3
  - androidx.benchmark.dropShaders.throwOnFailure=true/false : can be used to tolerate failures when trying to drop shaders, for example when benchmarking apps without profileinstaller 1.3 ( I4f573 )
Added experimental MacrobenchmarkRule#measureRepeated variant which takes a custom PerfettoConfig for fully customized Perfetto trace recording. Note that incorrectly configured configs may cause built in Metric classes to fail. ( Idfd3d , b/309841164 , b/304038384 )
Cancel background dexopt jobs before running a Macrobenchmark to reduce interference. ( I989ed )
Macrobenchmark now waits for 1 second for the target application to flush an ART profile (previously it waited for 500 ms). ( I85a50 , b/316082056 )
TraceSectionMetric overhaul
- Note : TraceSectionMetric changes below can affect outputs in CI usage, and may create discontinuities, or break parsing
- Sum is now the default, as most usage of this metric is for repeated events, and first would discard data in these cases
- Changed to be more customizable, with more available modes
- Mode names are now embedded in metric output name (in Studio and JSON)
- Now supports slices created using Trace.{begin|end}AsyncSection .
Метрики
- Power - Added PowerMetric.deviceSupportsHighPrecisionTracking , PowerMetric.deviceBatteryHasMinimumCharge() and PowerMetric.deviceSupportsPowerEnergy()
- Renamed Metric.getResult to getMeasurements to match return type
- Added log.w / exception labels to all startup detection failures. This does not change current behavior (so some errors throw, and others silently fail to detect the startup), just makes it more understandable. Generally the ones that Log.w() and fail to report startup metrics are those where non-frame events are missing, exceptions are thrown when startup is detected except for frame timing information (from UI/RT slices). ( Id240f , b/329145809 )
- Added frameCount measurement to FrameTimingMetric to aid in discovery of scenarios where measurements change because the number of frames produced changed (new animations added, invalidation issues fixed). ( I1e5aa )
- Clarified that frameOverrunMs is the preferred metric for tracking when available in docs, and why. ( I18749 , b/329478323 )
- Fixes issue where unterminated frames at the beginning and end of the trace could be paired together, which would incorrectly report as a single extremely long frame. ( I39353 , b/322232828 )
- Improve FrameTimingMetric error when frames aren't produced, and always output link to trace when failing metric parsing to assist in diagnosing problem. ( I956b9 )
- Fixed crash in FrameTimingMetric failing to parse frame id, especially on certain OEM devices. ( Ia24bc , b/303823815 , b/306235276 )
- Relaxed strictness of checks in FrameMetrics , and added more detail to error messages. ( Iadede )

Baseline Profile capture / Gradle plugin changes since 1.2.0

Increased max recommended version of AGP to 9.0.0-alpha01.
Ensure mergeArtProfile and mergeStartupProfile tasks always wait for baseline profile generation. ( I623d6 , b/343086054 )
Generating a baseline profile successfully will output a summary of what changed ( I824c8 , b/269484510 )
Added DSL to disable warnings ( Ic4deb , b/331237001 )
Fix to ensure benchmarks use generated baseline profiles when automaticGenerationDuringBuild is off ( Ic144f , b/333024280 )
Fix BaselineProfile gradle plugin property overrides to enable baseline profile generation and benchmarking when customizing a nonMinified or benchmark build type. ( Ib8f05 , b/324837887 )
Fix for including library baseline profiles in AAR prior to AGP 8.3.0-alpha15. ( I1d2af , b/313992099 )
Fixed baseline and startup profile output url at the end of generation task. ( I802e5 , b/313976958 )

Other significant changes since 1.2.0

Trace capture
- Reduced EXITCODE 2 error when starting perfetto from an error to logged warning
- Enable AIDL tracing by default in benchmarks(requires API 28) ( Ia0af2 , b/341852305 )
- Enable porter tag tracing by default in benchmarks. This captures, for example, wakelock tracepoints. ( Icfe44 , b/286551983 )
- Increased trace capture start timeout to avoid crashes when starting tracing on slower devices ( I98841 , b/329145808 )
- Added public API PerfettoTraceProcessor.Session.queryMetrics APIs with JSON, textproto, and proto binary (undecoded) variants. These allow you to query metrics built into TraceProcessor ( I54d7f , b/304038382 )
- Enable blocking start on Perfetto trace record to reduce risk of missing data at beginning of trace. Only supported on API 33+. ( Ie6e41 , b/310760059 )
JSON output
- Added additional information in benchmark context in JSON output:
  - context.artMainlineVersion - integer version of Art mainline module (if present on device, -1 otherwise)
  - context.build.id - Equals android.os.Build.ID
  - context.build.version.codename - Equals android.os.Build.VERSION.CODENAME
  - context.build.version.abbreviatedCodename - corresponds to first letter of pre-release codename (including on release builds) ( Ie5020 )
- Added profilerOutput list to JSON output for easier tooling around profiling traces (eg Perfetto, Method traces) ( I05ddd , b/332604449 )
- Added a warning when Android Test Orchestrator is used in benchmark modules, as this will cause per-module output JSON files to be repeatedly overwritten. ( Ia1af6 , b/286899049 )
- Throw when filenames are longer than 200 chars to avoid unclear crashes when writing or post-processing files. ( I4a5ab )

Version 1.3.0-rc01

7 августа 2024 г.

androidx.benchmark:benchmark-*:1.3.0-rc01 is released. Version 1.3.0-rc01 contains these commits .

Исправления ошибок

Fix androidx.benchmark.cpuEventCounter producing corrupt values for non-Instruction events ( I7386a , b/286306579 )
Fix resumeTiming / runWithTimingDisabled to respect metric priority order, and significantly reduce impact of lower priority metric pause/resume on higher priority metric results. For example, if using cpu perf counters via cpuEventCounter.enable instrumentation argument, timeNs is no longer significantly reduced when pause/resume occur. ( I39c2e , b/286306579 , b/307445225 )
Reduced chance of stack sampling causing measureRepeatedOnMainThread from hitting main thread hard timeout by moving stack sampling conversion off main thread. ( I487a8 , b/342237318 )
Removed manual outlining of access to new platform APIs since this happens automatically via API modeling when using R8 with AGP 7.3 or later (eg R8 version 3.3) and for all builds when using AGP 8.1 or later (eg D8 version 8.1). Clients who are not using AGP are advised to update to D8 version 8.1 or later. See this article for more details. ( I9496c , b/345472586 )
Added agp version check to send package name as instr arg. Previous to AGP 8.4.0 the target app package name cannot be send to the instrumentation app via instrumentation arguments. ( 0c72a3f )

Version 1.3.0-beta02

10 июля 2024 г.

androidx.benchmark:benchmark-*:1.3.0-beta02 is released. Version 1.3.0-beta02 contains these commits .

Исправления ошибок

Gracefully handle EXITCODE 2 when starting Perfetto to log a warning, but proceed.

Version 1.3.0-beta01

12 июня 2024 г.

androidx.benchmark:benchmark-*:1.3.0-beta01 is released. Version 1.3.0-beta01 contains these commits .

Изменения API

Renamed MethodTracing.affectsMeasurementOnThisDevice to AFFECTS_MEASUREMENT_ON_THIS_DEVICE for consistency. ( I1bdfa )
Added experimental BlackHole.consume() api to prevent dead code elimination in microbenchmarks. ( If6812 , b/286091643 )
Microbenchmark will now correctly throw to prevent method tracing from interfering with measurements. This occurs on certain devices when method tracing is forced on (via instrumentation args or MicrobenchmarkConfig ), and if a measurement is attempted after a method trace. Affected devices are running API 26-30 or certain ART mainline module versions affected by this interference, and can be detected at runtime via ProfilerConfig.MethodTracing.affectsMeasurementOnThisDevice . ( Iafb92 , b/303660864 )

Исправления ошибок

Bumped max agp version recommended to 9.0.0-alpha01. ( I5bbb0 )
Added compilation mode to benchmark context ( If5612 , b/325512900 )
Enable AIDL tracing by default (requires API 28) ( Ia0af2 , b/341852305 )
Added additional information in benchmark context in JSON output:
- context.artMainlineVersion - integer version of Art mainline module (if present on device, -1 otherwise)
- context.build.id - Equals android.os.Build.ID
- context.build.version.codename - Equals android.os.Build.VERSION.CODENAME
- context.build.version.abbreviatedCodename - corresponds to first letter of pre-release codename (even on release builds) ( Ie5020 )
Fixes StackSampling to respect androidx.benchmark.profiling.sampleDurationSeconds ( Ib1d53 )
Change macro->common dependency to be api() , so it's easier to use eg PerfettoTrace and PerfettoConfig . ( Icdae3 , b/341851833 )
Ensure mergeArtProfile and mergeStartupProfile tasks always wait for baseline profile generation. ( I623d6 , b/343086054 )
Consider variant enable state when deciding whether variant should be enabled. ( I5d19e , b/343249144 )
Increased default start timeout for perfetto trace processor. ( I87e8c , b/329145808 )

Version 1.3.0-alpha05

May 14, 2024

androidx.benchmark:benchmark-*:1.3.0-alpha05 is released. Version 1.3.0-alpha05 contains these commits .

Исправления ошибок

Throw clearer exception when macrobench metric returns zero values for all iterations ( Iab58f , b/314931695 )
Additional workaround rules added to microbench proguard rules, including support for listener rules and other observed warnings / errors. ( I14d8f , b/329126308 , b/339085669 )
Method tracing runs as a separate phase during a Macrobenchmark, and it no longer affects measurements. ( If9a50 , b/285912360 , b/336588271 )
Added extra debugging suggestions to drop shader broadcast failure message. ( I5efa6 , b/325502725 )

Version 1.3.0-alpha04

1 мая 2024 г.

androidx.benchmark:benchmark-*:1.3.0-alpha04 is released. Version 1.3.0-alpha04 contains these commits .

Изменения API

Added experimental MacrobenchmarkRule#measureRepeated variant which takes a custom PerfettoConfig for fully customized Perfetto trace recording. Note that incorrectly configured configs may cause built in Metric classes to fail. ( Idfd3d , b/309841164 , b/304038384 )
Rename PowerMetric.deviceSupportsPowerEnergy to PowerMetric.deviceSupportsHighPrecisionTracking for clarity ( I5b82f )
Added PowerMetric.deviceBatteryHasMinimumCharge() and PowerMetric.deviceSupportsPowerEnergy() to enable changing or skipping benchmarks based on device power measurement capability. ( I6a591 , b/322121218 )

Исправления ошибок

Added comparison with previous baseline profile ( I824c8 , b/269484510 )
Added DSL to disable warnings ( Ic4deb , b/331237001 )
Changed exception to info log when benchmark variants are disabled ( I8a517 , b/332772491 )
Make it simpler to capture method traces for a Macrobenchmark is scoped to the duration of the actual measureBlock() . Previously, it started at target process launch and only supported cold starts ( Iee85a , b/300651094 )
Avoid crashing when perfetto trace processor is slow to start ( I98841 , b/329145808 )

Version 1.3.0-alpha03

April 17, 2024

androidx.benchmark:benchmark-*:1.3.0-alpha03 is released. Version 1.3.0-alpha03 contains these commits .

Новые возможности

Adds public API PerfettoTraceProcessor.Session.queryMetrics APIs with JSON, textproto, and proto binary (undecoded) variants. These allow you to query metrics built into TraceProcessor ( I54d7f , b/304038382 )
Added profilerOutput to JSON output for easier tooling around profiling traces (eg perfetto, method traces). ( I05ddd , b/332604449 )
Added power tag to benchmark Perfetto Config. This captures, for example, wakelock tracepoints. ( Icfe44 , b/286551983 )
Added inst argument androidx.benchmark.profiling.skipWhenDurationRisksAnr , can be set to false to avoid skipping method traces when expected duration may cause an ANR - strongly recommended to avoid in CI runs.
Added experimental inst argument androidx.benchmark.profiling.perfCompare.enable , set this to true to run comparison timing between measurement and profiling phases. Useful in eg evaluating overhead of method tracing. ( I61fb4 , b/329146942 )

Изменения API

Changed TraceSectionMetric.Mode to sealed class to enable future expansion without breaking exhaustive when statements ( I71f7b )
Added TraceSectionMetric.Mode.Average and .Count , and reordered args so the more common argument (mode) was earlier in the arg list, reducing need for specifying parameter names. ( Ibf0b0 , b/315830077 , b/322167531 )
Renamed Metric.getResult to getMeasurements to match return type ( I42595 )

Исправления ошибок

Fix to ensure benchmarks use generated baseline profiles when automaticGenerationDuringBuild is off ( Ic144f , b/333024280 )
Fix BaselineProfile gradle plugin property overrides to enable baseline profile generation and benchmarking when customizing a nonMinified or benchmark build type. ( Ib8f05 , b/324837887 )
Fixed method traces flush in macrobenchmark, so that method traces should be fully captured and valid, even on slower devices. ( I6349a , b/329904950 )
Enable blocking start on Perfetto trace record to reduce risk of missing data at beginning of trace. Only supported on API 33+. ( Ie6e41 , b/310760059 )
Added a warning when Android Test Orchestrator is used in benchmark modules, as this will cause per-module output JSON files to be repeatedly overwritten. ( Ia1af6 , b/286899049 )
Force ',' (comma) thousands separators for consistency in Studio output, ignoring device locale ( I3e921 , b/313496656 )
TraceSectionMetric now supports slices created using Trace.{begin|end}AsyncSection . ( I91b32 , b/300434906 )
Added log.w / exception labels to all startup detection failures. This does not change current behavior (so some errors throw, and others silently fail to detect the startup), just makes it more understandable. Generally the ones that Log.w() and fail to report startup metrics are those where non-frame events are missing, exceptions are thrown when startup is detected except for frame timing information (from UI/RT slices). ( Id240f , b/329145809 )
Cancel background dexopt jobs before running a Macrobenchmark to reduce interference. ( I989ed )
Added frameCount measurement to FrameTimingMetric to aid in discovery of scenarios where measurements change because the number of frames produced changed (new animations added, invalidation issues fixed). ( I1e5aa )
Clarified that frameOverrunMs is the preferred metric for tracking when available in docs, and why. ( I18749 , b/329478323 )

Version 1.3.0-alpha02

20 марта 2024 г.

androidx.benchmark:benchmark-*:1.3.0-alpha02 is released. Version 1.3.0-alpha02 contains these commits .

Новые возможности

Experimental R8 support in microbench via embedded proguard rules. Note that this support is experimental, and requires AGP 8.3 for minification of library module tests. Use the following to enable R8 minification/optimization in your benchmark module's build.gradle , which should lead to a significant performance increase, depending on workload. ( I738a3 , b/184378053 )
```
android {
    buildTypes.release.androidTest.enableMinification = true
}
```

Исправления ошибок

Fixes method tracing warning to be on separate line from microbench output. ( I0455c , b/328308833 )

Version 1.3.0-alpha01

February 21, 2024

androidx.benchmark:benchmark-*:1.3.0-alpha01 is released. Version 1.3.0-alpha01 contains these commits.

Изменения API

Renamed MicrobenchmarkConfig boolean parameters to avoid unnecessary word 'should' ( Ia8f00 , b/303387299 )
Added BenchmarkRule.measureRepeatedOnMainThread so main thread benchmarks (eg ones touching Views or Compose UIs) can avoid triggering ANRs, especially during large suites in CI. ( I5c86d )
Added FrameTimingGfxInfoMetric , an experimental alternate implementation of FrameTimingMetric with measurements coming directly from the platform, rather than extracted from the Perfetto trace. ( I457cb , b/322232828 )
Add the ability to dump an ART profile during individual warmUp iterations. ( I17923 )
Several changes to TraceSectionMetric API:
- Add Mode.Min , Mode.Max
- Add label argument to override section name as metric label
- Added mode name to output to clarify metric meaning
- Changed default to sum, as most usage of this metric is for repeated events Be aware of this changes in CI usage, as it may create discontinuities or break parsing. ( Ic1e82 , b/301892382 , b/301955938 )

Исправления ошибок

Improved error message in baseline profile gradle plugin when specified managed device does not exist ( Idea2b , b/313803289 )
Fix for including library baseline profiles in AAR prior to AGP 8.3.0-alpha15 ( I1d2af , b/313992099 )
Fixed baseline and startup profile output url at the end of generation task ( I802e5 , b/313976958 )
Adjusted data source timeouts to attempt to fix java.lang.IllegalStateException: Failed to stop [ProcessPid(processName=perfetto, pid=...)] ( I8dc7d , b/323601788 )
Add two instrumentation arguments for overriding shader dropping behavior to workaround crashes when benchmarking apps without ProfileInstaller 1.3:
- androidx.benchmark.dropShaders.enable=true/false : can be used to skip all shader dropping (including that done in StartupMode.Cold launches), esp when benchmarking apps that don't yet use profileinstaller 1.3
- androidx.benchmark.dropShaders.throwOnFailure=true/false : can be used to tolerate failures when trying to drop shaders, for example when benchmarking apps without profileinstaller 1.3 ( I4f573 )
Skip method tracing on UI thread when expected to take longer than a few seconds, and cleanup method traces when throwing. ( I6e768 )
Throw when filenames are longer than 200 chars to avoid unclear crashes when writing or post-processing files. ( I4a5ab )
Fixes issue where unterminated frames at the beginning and end of the trace could be paired together, which would incorrectly report as a single extremely long frame. ( I39353 , b/322232828 )
Use --skip verification on API 30+ when reinstalling a package on API 30-33 to clear ART profiles on user builds. This helps bypass Play Protect warnings that cause failures on some class of devices. ( Ic9e36 )
Use am force-stop to kill apps when not a system app like System UI or Launcher. ( I5e028 )
Macrobenchmark now waits for 1 second for the target application to flush an ART profile (previously it waited for 500 ms ). ( I85a50 , b/316082056 )
Improve FrameTimingMetric error when frames aren't produced, and always output link to trace when failing metric parsing to assist in diagnosing problem. ( I956b9 )
Fixed crash in FrameTimingMetric failing to parse frame id, especially on certain OEM devices. ( Ia24bc , b/303823815 , b/306235276 )
Relaxed strictness of checks in FrameMetrics , and added more detail to error messages. ( Iadede )

Version 1.2

Version 1.2.4

April 17, 2024

androidx.benchmark:benchmark-*:1.2.4 is released. Version 1.2.4 contains these commits .

Исправления ошибок

Fixes baseline profile srcset not being set up in benchmark variants. Also fixes automaticGenerationDuringBuild in libraries causing a circular dependency. ( I28ab7 , b/333024280 )
Use am force-stop to kill apps when not a system app like System UI or Launcher. This fixes StartupMode.COLD benchmarks crashing from "Package $package must not be running prior to cold start!" due to process kill not fully succeeding. ( I5e028 )

Version 1.2.3

January 24, 2024

androidx.benchmark:benchmark-*:1.2.3 is released. Version 1.2.3 contains these commits.

Исправления ошибок

Removed exception from Baseline Profile Gradle Plugin when AGP version is 8.3.0 or higher.
Fix for including library baseline profiles in AAR prior to AGP 8.3.0-alpha15.

Version 1.2.2

1 декабря 2023 года

androidx.benchmark:benchmark-*:1.2.2 is released. Version 1.2.2 contains these commits.

Baseline Profiles

Execution logs will show the baseline profile output file path as a local file URI ( aosp/2843918 , aosp/2853665 , b/313976958 )

Версия 1.2.1

November 15, 2023

androidx.benchmark:benchmark-*:1.2.1 is released. Version 1.2.1 contains these commits.

Новые возможности

Improved error message when user disables test variants ( b/307478189 )
Added properties to support AS test run integration ( b/309805233 ), ( b/309116324 )

Version 1.2.0

October 18, 2023

androidx.benchmark:benchmark-*:1.2.0 is released. Version 1.2.0 contains these commits.

Important changes since 1.1.0

Baseline Profiles

New Baseline Profile Gradle Plugin automates capturing and including baseline profiles in your test and build workflow.
BaselineProfileRule.collect now stable, a streamlined and simplified version of the previous experimental BaselineProfileRule.collectBaselineProfile API
- Just specify packageName , and drive your app
For libraries generating baseline profiles, you can now filter the rules generated either in code ( BaselineProfileRule.collect argument), or even more simply in the gradle plugin
Исправления
- Fixed baseline profile collection on Android U+ ( Id1392 , b/277645214 )

Macrobenchmark

Сборник
- Macrobenchmark now correctly fully resets compilation state for each compile - this requires reinstalling the APK prior to Android 14, so benchmarking on Android 14+ is strongly recommended if you want to persist state (like user login) in what's being measured.
- You can also work around this by controlling app compilation separately, and skipping compilation with CompilationMode.Ignore() or instrumentation argument
Instrumentation Arguments
- Support for androidx.benchmark.dryRunMode.enable instrumentation argument, (already available in microbenchmark) for quicker validation runs (eg when creating the benchmark, or in presubmit)
- Support for androidx.benchmark.profiling.mode=StackSampling and MethodTracing .
- Added androidx.benchmark.enabledRules to allow runtime filtering baseline profile vs macrobenchmark rule tests
- Added androidx.benchmark.perfettoSdkTracing.enable argument to enable tracing with tracing-perfetto, eg Compose recomposition tracing. Note that when used with StartupMode.COLD , timing will be significantly affected as the tracing library is loaded and enabled during app startup.
Требования
- Macrobenchmark now requires ProfileInstaller 1.3.0 or greater in the target app, to enable profile capture / reset, and shader cache clearing.
New Experimental Metric APIs
- Added experimental TraceSectionMetric , which allows for extracting simple timing from trace("") {} blocks in your app, or TraceMetric for leveraging the full query capability of Perfetto TraceProcessor .
- Added experimental PowerMetric to capture power usage information
- Added experimental MemoryCountersMetric to count page faults
- Added experimental PerfettoTraceProcessor API, which is used internally to extract metrics from System traces (aka Perfetto traces)
Исправления
- Fixed crashes when installing or extracting profiles from an app installed from multiple APKs (eg from app bundle).
- Fixed FrameTimingMetric ignoring frames with inconsistent frame IDs (generally, frames during ripples on API 31+) ( I747d2 , b/279088460 )
- Fixed parsing errors on traces > 64MB ( Ief831 , b/269949822 )
- Clarified errors when device (especially emulator) OS image not correctly configured for tracing, or compilation
- Skip battery level check for devices without battery (micro and macro)
- Improved file output, with more clear errors for invalid output directories, and safer defaults
- Improved stability of StartupMode.COLD by consistently dropping the shader cache (also exposed via MacrobenchmarkScope.dropShaderCache )
- Fixed leanback fallback for startActivityAndWait .

Microbenchmark

Функции
- Profiling was moved to a separate phase, after other metrics, so one test run can display both accurate timing and profiling results.
Experimental APIs
- Added experimental MicrobenchmarkConfig API for defining custom metrics and configuring tracing and profiling. Can be used to capture method traces, or capture tracepoints (but be aware of tracing overhead).
- Added experimental APIs for controlling BenchmarkState separately from BenchmarkRule , without JUnit
- Added experimental PerfettoTrace record to enable capturing Perfetto traces, with custom configuration, separate from benchmark APIs.
Исправления
- Workaround missing leading whitespaces in Android Studio benchmark output.
- Fix issue where warnings could fail to print in Android Studio benchmark output.
- Fixed SampledProfiling crash on Android 13 (API 33) and higher.
- Massively improved performance of dryRunMode by skipping IsolationActivity and Perfetto tracing (Up to 10x faster dry run mode on older OS versions).

Version 1.2.0-rc02

October 6, 2023

androidx.benchmark:benchmark-*:1.2.0-rc02 is released. Version 1.2.0-rc02 contains these commits.

Исправления ошибок

Fix Benchmark file output to no longer break BaselineProfile Plugin file copying. Files were generated and copied off device, but had been renamed such that the gradle plugin wouldn't see them. ( I8dbcc , b/303034735 , b/296453339 )
Clarified tracing-perfetto loading error messages when injecting from macrobenchmark module into target application.

Version 1.2.0-rc01

20 сентября 2023 года

androidx.benchmark:benchmark-*:1.2.0-rc01 is released. Version 1.2.0-rc01 contains these commits.

Исправления ошибок

An exception (with remedy instructions) is now thrown when Perfetto SDK tracing fails to initialize in a Benchmark. ( I6c878 , b/286228781 )
Fix OOM crash when converting ART method trace -> perfetto format. ( I106bd , b/296905344 )
(Macrobenchmark) Clarified method tracing label when linked in Studio test output, and fixed method tracing filenames to be unique on device/host, so they won't be overwritten when more than one benchmark is run. ( I08e65 , b/285912360 )
Ensures that the device is awake when capturing a baseline profile. ( I503fc )

Version 1.2.0-beta05

August 30, 2023

androidx.benchmark:benchmark-*:1.2.0-beta05 is released. Version 1.2.0-beta05 contains these commits.

Новые возможности

The Baseline Profile Gradle Plugin now supports Android Gradle Plugin 8.3. ( aosp/2715214 )

Version 1.2.0-beta04

August 23, 2023

androidx.benchmark:benchmark-*:1.2.0-beta04 is released. Version 1.2.0-beta04 contains these commits.

Новые возможности

The Baseline Profiles Gradle plugin now supports Android Gradle Plugin 8.3. ( aosp/2715214 )

Исправления ошибок

Fix failures in writing / moving and pulling files (especially those from parameterized tests) by sanitizing output file names further, avoiding '=' and ':' in output file names. ( I759d8 )

Version 1.2.0-beta03

August 9, 2023

androidx.benchmark:benchmark-*:1.2.0-beta03 is released. Version 1.2.0-beta03 contains these commits.

Изменения API

Added argument to filter TraceSectionMetric to only the target package, on by default ( Ia219b , b/292208786 )

Исправления ошибок

Renamed fullTracing.enable instrumentation argument to perfettoSdkTracing.enable for consistency with artifact name, and other references. fullTracing.enable will continue to work as a fallback. ( I7cc00 )
Benchmark library internal tracepoints (including microbenchmark loop/phase tracing) will now show up in Studio system trace viewer, and nest under the correct process in Perfetto. ( I6b2e7 , b/293510459 )
Removed macrobenchmark NOT-PROFILEABLE error on API 31+, and skip profileable check on eng/userdebug rooted devices. ( I2abac , b/291722507 )
When using Dex Layout Optimizations, startup profile rules are also now considered as baseline profile rules. ( aosp/2684246 , b/293889189 )

Version 1.2.0-beta02

July 26, 2023

androidx.benchmark:benchmark-*:1.2.0-beta02 is released. Version 1.2.0-beta02 contains these commits.

Изменения API

Added experimental APIs for microbench custom metrics and configuration (eg profiler, and tracing). ( I86101 , b/291820856 )

Исправления ошибок

Report error in macrobench when OS is misconfigured for tracing, as was recently fixed in API 26/28 ARM64 emulators. ( I0a328 , b/282191686 )
Added detail to compilation reset failure to suggest updating emulator, as some emulators have recently fixed this issue. ( I8c815 , b/282191686 )
Make androidx.test.uiautomator:uiautomator:2.2.0 an api instead of an implementation dependency. ( I1981e )

Version 1.2.0-beta01

July 18, 2023

androidx.benchmark:benchmark-*:1.2.0-beta01 is released. Version 1.2.0-beta01 contains these commits.

Исправления ошибок

Fix warnings being sometimes suppressed in Benchmark output in Studio, and workaround leading whitespaces from Benchmark output not showing up in Studio ( Ia61d0 , b/227205461 , b/286306579 , b/285912360 )
Fixed comment for FrameTimingMetric . The submetric is named frameDurationCpuMs . ( Ib097f , b/288830934 ).

Version 1.2.0-alpha16

June 21, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha16 is released. Version 1.2.0-alpha16 contains these commits.

Изменения API

BaselineProfileRule.collectBaselineProfile() API has been renamed to BaselineProfileRule.collect() . ( I4b665 )

Исправления ошибок

Macrobenchmark support for androidx.benchmark.profiling.mode = MethodTracing . ( I7ad37 , b/285912360 )
Microbenchmark profiling moved to a separate phase, so it occurs in sequence after measurement, instead of replacing it. MethodTracing trace sections are also now included in the captured Perfetto trace, if present. ( I9f657 , b/285014599 )
Add count measurement to TraceSectionMetric with Mode.Sum . ( Ic121a , b/264398606 )

Version 1.2.0-alpha15

7 июня 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha15 is released. Version 1.2.0-alpha15 contains these commits.

Новые возможности

Added experimental MemoryUsageMetric for tracking memory usage of a target application. ( I56453 , b/133147125 , b/281749311 )
Add support for fully custom Perfetto configs with PerfettoTrace.record ( If9d75 , b/280460183 )
Added property to skip baseline profile generation. Usage: ./gradlew assemble -Pandroidx.baselineprofile.skipgeneration . ( I37fda , b/283447020 )

Изменения API

The collectBaselineProfile API always generates stable baseline profiles. The collectStableBaselineProfile API has been removed and collectBaselineProfile should be used instead. ( I17262 , b/281078707 )
Changed BaselineProfileRule 's filterPredicate arg to non-null, with a equivalent default value so that the default filter behavior is more clear in docs. ( I3816e )

Исправления ошибок

Disable IsolationActivity and Perfetto tracing in dryRunMode to significantly improve performance, as these were majority of runtime. ( Ie4f7d )
Support for call stack sampling in Macrobenchmarks using instrumentation test arguments androidx.benchmark.profiling.mode=StackSampling and androidx.benchmark.profiling.sampleFrequency . ( I1d13b , b/282188489 )
Fixes crash when dropping shaders on Android U (API 34), as well as on emulators. ( I031ca , b/274314544 )

Version 1.2.0-alpha14

May 3, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha14 is released. Version 1.2.0-alpha14 contains these commits.

Исправления ошибок

Fix FrameTimingMetric ignoring frames with inconsistent frame IDs. This would cause some animations on recent platform versions (API 31+) to ignore many frames while RenderThread was animating (eg during a ripple). ( I747d2 , b/279088460 )
Fixed trace processor parsing for traces larger than 64Mb. ( Ief831 , b/269949822 )
Fixed baseline profile generation on Android U failing because of the different output of pm dump-profiles command. ( Id1392 , b/277645214 )
Fix GPU clock locking script to compare strings correctly ( I53e54 , b/213935715 )

Version 1.2.0-alpha13

April 5, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha13 is released. Version 1.2.0-alpha13 contains these commits.

Изменения API

Added profile type parameter when generating baseline profiles to support upcoming startup profile feature ( Ie20d7 , b/275093123 )
Added new experimental TraceMetric API for defining fully custom metrics based on content of a Perfetto trace. ( I4ce31 , b/219851406 )
Add an experimental metric to determine the number of page faults during a benchmark. ( I48db0 )

Version 1.2.0-alpha12

March 22, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha12 is released. Version 1.2.0-alpha12 contains these commits.

Новые возможности

The new baseline profile gradle plugin is released in alpha version, making it easier to generate a baseline profile and simplifying the developer workflow.

Изменения API

Removed Perfetto tracing support on API 21 and 22, which includes both Microbenchmarks and the experimental PerfettoTrace APIs. Prior to this version, UiAutomation connections were unreliable on some devices. ( I78e8c )
Added public experimental API for PerfettoTraceProcessor to enable parsing trace content. This is a step toward fully custom metrics based on Perfetto trace data. ( I2659e , b/219851406 )

Version 1.2.0-alpha11

March 8, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha11 is released. Version 1.2.0-alpha11 contains these commits.

Исправления ошибок

Fixed crashes in MacrobenchmarkRule and BaselineProfileRule when reinstalling or extracting profiles from an app bundle with multiple APKs. ( I0d8c8 , b/270587281 )

Version 1.2.0-alpha10

22 февраля 2023 г.

androidx.benchmark:benchmark-*:1.2.0-alpha10 is released. Version 1.2.0-alpha10 contains these commits.

Новые возможности

On Android 14+, Macrobenchmark no longer reinstalls target applications to reset compilation state, thanks to a new platform feature. Previously it was necessary to have a rooted device, or to deal with all application state (eg user login) being removed before each benchmark runs. ( I9b08c , b/249143766 )

Исправления ошибок

Fix DryRunMode to no longer crash with empty profile, due to compilation skipping. Instead, it runs a single iteration and extracts the profile to ensure something is captured. ( I2f05d , b/266403227 )
Fix PowerMetric crash when checking for powerstats presence on old API levels. ( 5faaf9 , b/268253898 )

Version 1.2.0-alpha09

January 11, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha09 is released. Version 1.2.0-alpha09 contains these commits.

Исправления ошибок

Enabled passing None to androidx.benchmark.enabledRules instrumentation arg to disable all benchmarks / baseline profile generation. ( I3d7fd , b/258671856 )
Fix PerfettoTrace capture in app modules (ie non-self-instrumenting test APKs) ( I12cfc )
Fixed baseline profile adb pull argument order in Studio output ( I958d1 , b/261781624 )
Arm emulator api 33 is now correctly recognized as such when trying to run a macrobenchmark and will correctly print the warning. ( 69133b , b/262209591 )
Skip battery level check on devices without battery in Macrobenchmark ( fe4114 , b/232448937 )

Version 1.2.0-alpha08

December 7, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha08 is released. Version 1.2.0-alpha08 contains these commits.

Изменения API

Added experimental new APIs PerfettoTrace.record {} and PerfettoTraceRule to capture Perfetto traces (also known as System Traces) as part of a test, to inspect test behavior and performance. ( I3ba16 )
BaselineProfileRule now accepts a filter predicate instead of a list of package prefixes. This gives the test full control on filtering. ( I93240 )
Add an experimental API BaselineProfileRule.collectStableBaselineProfile which waits until a baseline profile is stable for N iterations. ( I923f3 )
Add the ability to specify an output file name prefix when generating baseline profiles using BaselineProfileRule . ( I7b59f , b/260318655 )

Исправления ошибок

Improve safety of file output writing, which should prevent output files from silently not being written / appended, especially on API 21/22. ( If8c44 , b/227510293 )
Fix simpleperf trace output to create and place the file correctly. This should also more generally fix issues where a file is unsuccessfully pulled by gradle. ( I12a1c , b/259424099 )
Improve profileinstaller error message printed when profileinstaller is too old. This now tells you to update profileinstaller version (1.2.1) for measuring baseline profiles on API 31 through 33, instead of saying it's not supported. ( Ia517f , b/253519888 )
Fix several shell command failures onerror message Print needed API <=23, including failed perfetto capture binary setup and trace capture failures ( Ib6b87 , b/258863685 )
Automatically sort generated profile rules to minimize the number of changes as they change over time (when checking-in profile rules into source control). ( Ie2509 )
Fixed crash on unrooted builds below Android 13 (API 33) with message Expected no stderr from echo 3 > /proc/sys/vm/drop_caches ( I6c245 , b/259508183 )

  <!-- workaround bug in profileinstaller 1.3.0-alpha02, remove when updating to alpha03+ -->
  <receiver
    android:name="androidx.profileinstaller.ProfileInstallReceiver"
    android:permission="android.permission.DUMP"
    android:exported="true">
    <intent-filter>
        <action android:name="androidx.profileinstaller.action.BENCHMARK_OPERATION" />
    </intent-filter>
  </receiver>

Version 1.2.0-alpha07

9 ноября 2022 г.

androidx.benchmark:benchmark-*:1.2.0-alpha07 is released. Version 1.2.0-alpha07 contains these commits.

Изменения API

Adds PowerMetric API for measuring energy and power in Macrobenchmarks. ( Ife601 , b/220183779 )
Fixed MacrobenchmarkScope.dropShaderCache() to actually drop the shader cache. This removes roughly 20ms of noise from StartupMode.COLD benchmarks, as shaders are now consistently cleared each iteration. Previously, Partial compilation using warmup iterations would report incorrectly fast numbers, as shader caching was more likely to happen during warmup. This fix requires either a rooted device, or using profileinstaller:1.3.0-alpha02 in the target app. For ProfileInstaller library's API changes, please refer to ProfileInstaller 1.30-alpha02 page. ( Ia5171 , b/231455742 )
Added TraceSectionMode("label", Mode.Sum) , allowing measurement of total time spent on multiple trace sections with the same label. For instance, TraceSectionMetric("inflate", Mode.Sum) will report a metric inflateMs for the total time in a macrobenchmark spent on inflation. Also removed API 29 requirement, as TraceSectionMetric works together with androidx.tracing.Trace back to lower API levels, with the use of forceEnableAppTracing within the target app. ( Id7b68 , b/231455742 )

Исправления ошибок

Improved safety of all internal shell commands by validating all output/errors. ( I5984d , b/255402908 , b/253094958 )
Specify device in baseline profile adb pull command, so the pull command can be simply copied if multiple devices are connected (up to one emulator) ( I6ac6c , b/223359380 )
Add error if macrobenchmark test apk isn't set up as self-instrumenting. This error prevents macrobenchmarking from within the target app's process. In process, macrobench wouldn't be able to compile/kill/cold start the app, or control its own permissions ( I4279b )
Fixed an issue in measureRepeated() where StartupMode.COLD wouldn't kill the target process after setupBlock . Now setupBlock interacting with the app will not leave the app process running, and an invalid cold start measurement. ( I8ebb7 )

Version 1.2.0-alpha06

October 24, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha06 is released. Version 1.2.0-alpha06 contains these commits.

Изменения API

BaselineProfileRule no longer requires root on Android 13 (API 33), and is no longer experimental. ( Ie0a7d , b/250083467 , b/253094958 )
- This change also fixes how profiles from an app are flushed to disk on unrooted devices, but requires updating the target app's profileinstaller dependency.
- To use BaselineProfileRule or CompilationMode.Partial(warmupIterations) on an unrooted device, you must also update your target app to use androidx.profileinstaller.profileinstaller:1.3.0-alpha01 . This enables flushing the profile to disk correctly, so that it can be compiled/extracted.

Исправления ошибок

Fixes SampledProfiling crash on API 33+. ( I40743 , b/236109374 )

Version 1.2.0-alpha05

October 5, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha05 is released. Version 1.2.0-alpha05 contains these commits.

Исправления ошибок

Fix frame breakdown in Studio system trace viewer for benchmark captured traces ( I3f3ae , b/239677443 )
Correct FrameTimingMetric to list FrameOverrun as requiring API 31 instead of 29 ( I716dd , b/220702554 )
Set iteration in BaselineProfileRule , and throw clearly if target package not installed (was already done for MacrobenchmarkRule). ( Ic09a3 , b/227991471 )

Version 1.2.0-alpha04

September 21, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha04 is released. Version 1.2.0-alpha04 contains these commits.

Новые возможности

Add support for dryRunMode.enable instrumentation argument to macrobenchmark (already available in micro) for faster local development, and validating app automation (eg in presubmit). This overrides iterations to 1, skips compilation, suppresses all configuration errors , and disables measurement .json file output. ( Ib51b4 , b/175149857 )
On Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.dryRunMode.enable=true
```
In build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.dryRunMode.enable', 'true'
    }
}
```

Исправления ошибок

Fixed StartupTimingMetric to no longer require measured Activities to be launched through MacrobenchmarkScope.startActivityAndWait() . This means the metric can pick up launches from eg notifications, Context.startActivity() , in-app Activity based navigation, or shell commands. ( Ia2de6 , b/245414235 )
Fix bug where startActivityAndWait would timeout trying to wait for launch completion on emulators by reducing strictness of frame detection. ( Ibe2c6 , b/244594339 , b/228946895 )

Version 1.2.0-alpha03

September 7, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha03 is released. Version 1.2.0-alpha03 contains these commits.

Новые возможности

Added experimental APIs for using BenchmarkState independently, separate from BenchmarkRule / JUnit4 . ( Id478f , b/228489614 )

Исправления ошибок

Added Leanback fallback for startActivityAndWait . ( 01ed77 , b/242899915 )

Version 1.2.0-alpha02

August 24, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha02 is released. Version 1.2.0-alpha02 contains these commits.

Изменения API

Default to am force stop for MacrobenchmarkScope.killProcess() , even when rooted, except during Baseline Profile generation. This can be overridden with an optional boolean argument. ( 02cce9 , b/241214097 )

Исправления ошибок

Support baseline profile generation for System apps. ( I900b8 , b/241214097 )
Support checking for ODPM power metrics on unrooted devices. ( a38c78 , b/229623230 )

Version 1.2.0-alpha01

July 27, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha01 is released. Version 1.2.0-alpha01 contains these commits.

Новые возможности

New tracing-perfetto-common component allowing tooling to enable Perfetto SDK tracing in an app that exposes it ( I2cc7f )
Added androidx.benchmark.enabledRules instrumentation argument to enable filtering macrobenchmark runs to just benchmarks, or just baseline profile generation. Pass in 'Macrobenchmark', or 'BaselineProfile' to just run one type of test, eg when just generating BaselineProfiles on an emulator. Comma-separated list also Supported. ( I756b7 , b/230371561 )
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.enabledRules', 'BaselineProfile'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
```

Изменения API

Added new PowerMetric for measuring energy and power tasks in benchmarks. ( I9f39b , b/220183779 )
Added a new compilation mode CompilationMode.Ignore to skip profile reset and compilation. ( Ibbcf8 , b/230453509 )
Added a new parameter to BaselineProfileRule#collectBaselineProfile to filter output file by package names ( If7338 , b/220146561 )
Enables developer to discharge device to measure power drain. ( I6a6cb )
Added the ability to clear shader cache in MacrobenchmarkScope . ( I32122 )
Enables developer to configure display of metric type and detail desired subsystem categories. ( I810c9 )
Previously an UnsupportedOperationException was thrown in the benchmark if run on an unsupported device. Now UOE only occurs if the metric is used on the unsupported device (ie: PowerMetric.configure ). ( I5cf20 , b/227229375 )
Added TotalPowerMetric and TotalEnergyMetric for measuring total power and energy in each system category in macrobenchmarks. ( I3b26b , b/224557371 )

Исправления ошибок

Fixed an issue where compiled methods were not correctly being reset between each macrobenchmark on unrooted builds. This unfortunately requires reinstalling the apk each iteration, which will clear application data for each macrobenchmark. ( I31c74 , b/230665435 )
Fix trace recording crash on API 21/22 ( If7fd6 , b/227509388 , b/227510293 , b/227512788 )
Overhaul activity launch completion detection to fix 'Unable to read any metrics' exception in startup macrobenchmarks. ( Ia517c )

Version 1.1.1

9 ноября 2022 г.

androidx.benchmark:benchmark-*:1.1.1 is released. Version 1.1.1 contains these commits.

Исправления ошибок

Fixes android.system.ErrnoException: open failed: EACCES which would occur on some Android11 (API 30)+ devices. This is a cherry-pick of a fix from 1.2.0-alpha01 . ( aosp/2072249 )

Версия 1.1.0

June 15, 2022

androidx.benchmark:benchmark-*:1.1.0 is released. Version 1.1.0 contains these commits.

This version is identical to androidx.benchmark:benchmark-*:1.1.0-rc03 .

Important changes since 1.0.0

Support for Jetpack Macrobenchmarks, which allows you to measure whole-app interactions like startup and scrolling , provides the ability to capture traces & measure trace sections .
Support for Baseline Profiles
- CompilationMode.Partial to measure the effectiveness of Baseline Profiles.
- @BaselineProfileRule to automatically generate Baseline profiles for a given critical user journey.
Support for Allocation metrics & profiling during Microbenchmark runs.

Version 1.1.0-rc03

June 1, 2022

androidx.benchmark:benchmark-*:1.1.0-rc03 is released. Version 1.1.0-rc03 contains these commits.

Исправления ошибок

Avoid reinstalling the target package on every benchmark iteration. ( aosp/2093027 , b/231976084 )
Remove the 300ms delay from pressHome() . ( aosp/2086030 , b/231322975 )
Improve Macrobenchmark iteration speed by optimizing Shell commands used under the hood. ( aosp/2086023 , b/231323582 )
Support for Managed Gradle Devices when generating Baseline Profiles with Macrobenchmarks. ( aosp/2062228 , b/228926421 )

Version 1.1.0-rc02

May 11, 2022

androidx.benchmark:benchmark-*:1.1.0-rc02 is released. Version 1.1.0-rc02 contains these commits.

Note that this release includes a behavior change, as apps are now fully reinstalled in between each benchmark to ensure accurate measurements.

Bug Fixes/Behavior Changes

Fixed an issue where app compilation was not correctly reset between macrobenchmarks, and not reset at all on unrooted builds. This fixes many cases where running multiple tests would result in CompilationMode having little to no effect on measurements. To workaround this problem, the target app is now fully reinstalling each test method, which will clear application data between each macrobenchmark. ( I31c74 , b/230665435 )
As this prevents apps from setting up state before tests, it is now possible to skip compilation / reinstallation to enable working around this. You can for example fully compile the target with a shell command cmd package compile -f -m speed <package> , and then bypass macrobenchmark's compilation step.
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.compilation.enabled, 'false'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.compilation.enabled=false
```
Made it possible to share a module between macrobenchmarks and baseline profile generating tests by adding androidx.benchmark.enabledRules instrumentation argument. Pass in 'Macrobenchmark', or 'BaselineProfile' to just run one type of test, eg when generating BaselineProfiles on an emulator. ( I756b7 , b/230371561 )
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.enabledRules', 'BaselineProfile'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
```

Version 1.1.0-rc01

April 20, 2022

androidx.benchmark:benchmark-*:1.1.0-rc01 is released. Version 1.1.0-rc01 contains these commits.

Исправления ошибок

Baseline profile output links in Android Studio now use a unique file name. This way the output always reflects the latest results of using a BaselineProfileRule . ( aosp/2057008 , b/228203086 )

Version 1.1.0-beta06

April 6, 2022

androidx.benchmark:benchmark-*:1.1.0-beta06 is released. Version 1.1.0-beta06 contains these commits.

Исправления ошибок

Fix trace recording crash on API 21/22 ( If7fd6 , b/227509388 )
Overhaul activity launch completion detection to fix 'Unable to read any metrics' exception in startup macrobenchmarks. ( Ia517c )
Fix startup metrics for Macrobenchmarks when CompilationMode.None() is used. Before this change, CompilationMode.Partial() would appear to be slower than Compilation.None() . ( 611ac9 ).

Version 1.1.0-beta05

March 23, 2022

androidx.benchmark:benchmark-*:1.1.0-beta05 is released. Version 1.1.0-beta05 contains these commits.

Исправления ошибок

Kill package after skipping profile installation when using CompilationMode.None . ( aosp/1991373 )
Fixed an issue where Macrobenchmarks is unable to collect startup metrics when using StartupMode.COLD . ( aosp/2012227 b/218668335 )

Version 1.1.0-beta04

February 23, 2022

androidx.benchmark:benchmark-*:1.1.0-beta04 is released. Version 1.1.0-beta04 contains these commits.

Исправления ошибок

Fix missing metrics on Android 10, and NoSuchElementException caused by process names not being captured correctly in traces. ( Ib4c17 , b/218668335 )
Use PowerManager for thermal throttling detection on Q (API 29) and higher. This significantly reduces frequency of false positives in thermal throttling detection (benchmark retry after 90 second cooldown), and speeds up benchmarks significantly on user builds. It also provides throttle detection even when clocks are locked (if they're locked too high for the device's physical environment). ( I9c027 , b/217497678 , b/131755853 )
Filter simpleperf sampled profiling to measureRepeated thread only to simplify inspection ( Ic3e12 , b/217501939 )
Support metrics from named UI subprocesses in multi-process apps ( Ice6c0 , b/215988434 )
Filter Baseline Profile rules to target Android 9 (SDK 28). aosp/1980331 b/216508418
Skip Profile Installation when using Compilation.None() . Additionally, report warnings when the app is using an older version of androidx.profileinstaller and Android Gradle Plugin. aosp/1977029

Version 1.1.0-beta03

February 9, 2022

androidx.benchmark:benchmark-*:1.1.0-beta03 is released. Version 1.1.0-beta03 contains these commits.

Изменения API

Added AudioUnderrunMetric into macrobenchmark library under experimental flag to allow detection of audio underruns ( Ib5972 )

BaselineProfileRule no longer accepts a setup block as this functioned the same as the profileBlock . ( Ic7dfe , b/215536447 )

For eg

@Test
fun collectBaselineProfile() {
    baselineRule.collectBaselineProfile(
        packageName = PACKAGE_NAME,
        setupBlock = {
            startActivityAndWait()
        },
        profileBlock = {
            // ...
        }
    )
}

@Test
fun collectBaselineProfile() {
    baselineRule.collectBaselineProfile(
        packageName = PACKAGE_NAME,
        profileBlock = {
            startActivityAndWait()
            // ...
        }
    )
}

Исправления ошибок

Fixed issue where microbench profiler traces would fail to be updated in subsequent runs when linked in Studio output ( I5ae4d , b/214917025 )
Prevent compilation shell commands on API 23 ( Ice380 )
Renamed FrameCpuTime -> FrameDurationCpu , FrameUiTime -> FrameDurationUi to clarify these are durations, not timestamps, and to match prefixes. ( I0eba3 , b/216337830 )

Version 1.1.0-beta02

January 26, 2022

androidx.benchmark:benchmark-*:1.1.0-beta02 is released. Version 1.1.0-beta02 contains these commits.

Исправления ошибок

Microbenchmark Stack Sampling / Method Tracing Profile results are now linked in Studio output, similar to other profiling outputs, and do not suppress the allocation metric. ( Idcb65 , b/214440748 , b/214253245 )
BaselineProfileRule now prints the adb pull command in logcat and Studio output for pulling generated BaselineProfile text file. ( f08811 )

Version 1.1.0-beta01

January 12, 2022

androidx.benchmark:benchmark-*:1.1.0-beta01 is released. Version 1.1.0-beta01 contains these commits.

Исправления ошибок

Fixes profiler argument enable being ignored. ( I37373 , b/210619998 )
Removed deprecated CompliationModes ( I98186 , b/213467659 )
Switched baseline profile arg of CompilationMode.Partial to enum for clarity. ( Id67ea )

Version 1.1.0-alpha13

December 15, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha13 is released. Version 1.1.0-alpha13 contains these commits.

Изменения API

Add low-overhead System Tracing to microbench output on Android Q (API 29+). Note that this does not currently capture custom tracing (via android.os.Trace or androidx.tracing Jetpack APIs) to avoid affecting results. This tracing should be useful in diagnosing instability, especially from sources outside the benchmark. ( I298be , b/205636583 , b/145598917 )
Clarify CompilationModes into three classes - Full, None, Partial. Previously they were inconsistently named after compilation arguments (which we now treat as implementation details) and features. This makes the tradeoffs, potential combinations, and behavior across platform versions more clear. ( I3d7bf , b/207132597 )
Setup and measure are now always in pairs, in order. You can now query the package name and iteration (though the iteration may be null in certain warmup scenarios). ( Id3b68 , b/208357448 , b/208369635 )

Исправления ошибок

Fixed CompilationMode.Speed incorrectly treated as None ( I01137 )

Version 1.1.0-alpha12

November 17, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha12 is released. Version 1.1.0-alpha12 contains these commits.

Новые возможности

Add experimental TraceSectionMetric for custom trace-based timing measurements. ( I99db1 , b/204572664 )

Исправления ошибок

Wake device each iteration, to ensure UI can be tested - requires lockscreen is disabled. ( Ibfa28 , b/180963442 )
Fixes multiple crashes in StackSampling profiling mode on emulators and non-rooted devices ( Icdbda , b/202719335 )
Removed 0.5 second sleep at the end of each iteration - if you see missing metrics with this change, please file a bug. ( Iff6aa )
Reduce chances of dropped data, and lower memory overhead from tracing ( Id2544 , b/199324831 , b/204448861 )
Reduce trace size by ~40% by switching to compact sched storage format. ( Id5fb6 , b/199324831 )
Updated implementations of startup metrics to always end at end of renderthread. This will be more contistent across platform versions, and more closely map to in-app measurements. ( Ic6b55 )

Version 1.1.0-alpha11

November 3, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha11 is released. Version 1.1.0-alpha11 contains these commits.

Изменения API

Macrobenchmark now has a minSdkVersion of 23 . ( If2655 )
Adds a new experimental BaselineProfileRule which is capable of generating baseline profiles for app's critical user journey. Detailed documentation to follow. ( Ibbefa , b/203692160 )
Removes measureRepeated interface variant, which was added for java callers, as it caused ambiguity in completing/resolving the method. Java callers will again need to return Unit.Instance from measureRepeated. If this is an inconvenience, please file a bug, we can revisit this in a future version. ( Ifb23e , b/204331495 )

Version 1.1.0-alpha10

October 27, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha10 is released. Version 1.1.0-alpha10 contains these commits.

Изменения API

Backport StartupTimingMetric to work back to API 23. This new implementation also better handles reportFullyDrawn() to wait until corresponding content has been rendered. ( If3ac9 , b/183129298 )
Added JvmOverloads to multiple MacrobenchmarkScope methods for Java callers. ( I644fe , b/184546459 )
Provide alternative MacrobenchmarkRule.measureRepeated function that uses a Consumer<MacrobenchmarkScope> for idiomatic usage in Java language. ( If74ab , b/184546459 )

Исправления ошибок

Fix for traces not starting early enough, and missing metric data. This is expected to fix "Unable to read any metrics during benchmark" exceptions that were caused by the library itself. ( I6dfcb , b/193827052 , b/200302931 )
FrameNegativeSlack has been renamed to FrameOverrun to clarify its meaning - how much the frame went over its time budget. ( I6c2aa , b/203008701 )

Version 1.1.0-alpha09

October 13, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha09 is released. Version 1.1.0-alpha09 contains these commits.

Исправления ошибок

Support dropping Kernel page cache without root on API 31/S+, which will increase accuracy of StartupMode.COLD launches. ( Iecfdb , b/200160030 )

Version 1.1.0-alpha08

September 29, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha08 is released. Version 1.1.0-alpha08 contains these commits.

Изменения API

Enable scrolling macrobenchmarks to run back to API 23 ( If39c2 , b/183129298 )
Add new type of sampled metric to UI and JSON output, focused on percentiles of multiple samples per iteration. ( I56247 , b/199940612 )
Switch to floating point metrics throughout the benchmark libraries (truncated in the Studio UI). ( I69249 , b/197008210 )

Версия 1.1.0-альфа07

1 сентября 2021 г.

androidx.benchmark:benchmark-*:1.1.0-alpha07 is released. Version 1.1.0-alpha07 contains these commits.

Изменения API

Raised min API to 21 to reflect the intended lowest API level to be supported in the future. Current min API supported continues to be conveyed via RequiredApi(), and is currently 29 ( I440d6 , b/183129298 )

Исправления ошибок

Fixes ProfileInstaller to make it easier for apps using baseline profiles to run MacroBenchmarks using CompilationMode.BaselineProfile . ( I42657 , b/196074999 ) NOTE: requires also updating to androidx.profileinstaller:profileinstaller:1.1.0-alpha04 or greater.
StartupMode.COLD + CompilationMode.None benchmarks are now more stable. ( I770cd , b/196074999 )

Версия 1.1.0-альфа06

August 18, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha06 is released. Версия 1.1.0-alpha06 содержит эти коммиты.

Изменения API

Added androidx.benchmark.iterations instrumentation argument to allow manual overriding of iteration count when testing/profiling locally. ( 6188be , b/194137879 )

Исправления ошибок

Switched to Simpleperf as default sampling profiler on API 29+. ( Ic4b34 , b/158303822 )

Известные проблемы

CompilationMode.BaselineProfile is a work in progress. Avoid using it to determine how good a profile is for now.

Версия 1.1.0-альфа05

August 4, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha05 is released. Версия 1.1.0-alpha05 содержит эти коммиты.

1.1.0-alpha04 was cancelled before release due to a sporatic crash. b/193827052

Изменения API

Switched startActivityAndWait to invoke launch via am start , which reduces the time of each measurement iteration by approximately 5 seconds, at the cost of no longer supporting intent parcelables. ( I5a6f5 , b/192009149

Исправления ошибок

Reduce aggressiveness of thermal throttle detection, and recompute baseline if throttles are detected frequently. ( I7327b )
Fixes FrameTimingMetric to work on Android S beta ( Ib60cc , b/193260119 )
Use an EmptyActivity to bring the target app out of a force-stopped state to better support CompilationMode.BaselineProfile . ( Id7cac , b/192084204 )
Changed trace file extension to .perfetto-trace to match platform standard. ( I4c236 , b/174663039 )
StartupTimingMetric now outputs the "fullyDrawnMs" metric to measure time until your application has completed rendering. To define this metric for your app, call Activity.reportFullyDrawn when your initial content is ready, such as when your initial list items are loaded from DB or network. (reportFullyDrawn method available without build version checks on ComponentActivity). Note that your test must run long enough to capture the metric (startActivityAndWait doesn't wait for reportFullyDrawn). ( If1141 , b/179176560 )
Reduce cost of appending Ui metadata to traces by 50+ ms ( Ic8390 , b/193923003 )
Drastically increased polling frequency when stopping tracing, which can reduce eg startup benchmark runtime by 30+% ( Idfbc1 , b/193723768 )

Версия 1.1.0-альфа03

June 16, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha03 is released. Версия 1.1.0-alpha03 содержит эти коммиты.

Новые возможности

Added a new CompilationMode.BaselineProfile to support profiles installed using the Jetpack ProfileInstaller library . ( aosp/1720930 )

Исправления ошибок

The sample Gradle code for suppressing benchmark errors has been updated to use a non-deprecated API with a syntax that also supports .gradle.kts users.
Например,
```
testInstrumentationRunnerArguments["androidx.benchmark.suppressErrors"] = "EMULATOR,LOW-BATTERY"
```

Версия 1.1.0-альфа02

18 мая 2021 г.

androidx.benchmark:benchmark-*:1.1.0-alpha02 is released. Версия 1.1.0-alpha02 содержит эти коммиты.

Новые возможности

Macrobenchmark artifacts added ( androidx.benchmark:benchmark-macro-junit4 and androidx.benchmark:benchmark-macro )

Capture startup, scrolling/animation performance metrics from your app, locally or in CI
Capture and inspect traces from within Android Studio

Исправления ошибок

Workaround shell permissions issue with output directory on Android 12 (Note - may require updating Android Gradle Plugin to 7.0.0 canary and Android Studio to Arctic Fox (2020.3.1), to continue capturing output files on affected devices). ( Icb039 )
Support configuration caching in BenchmarkPlugin ( 6be1c1 , b/159804788 )
Simplified file output - on by default, in a directory that doesn't require requestLegacyExternalStorage=true ( 8b5a4d , b/172376362 )
Fixes library printing logcat warnings about not finding JIT thread on platform versions where it is not present. ( I9cc63 , b/161847393 )
Fix for reading device max frequency. ( I55c7a )

Версия 1.1.0-альфа01

10 июня 2020 г.

New Features of 1.1

Allocation Metric - Benchmarks now run an additional phase after warmup and timing, capturing allocation counts. Allocations can cause performance problems on older versions of the platform (140ns in O became 8ns in M - measured on Nexus5X, with locked clocks). This metric is displayed in Android Studio console output, as well as in the
Profiling support - You can now capture profiling data for a benchmark run, to inspect why your code may be running slowly. Benchmark supports capturing either method tracing, or method sampling from ART. These files can be inspected with the Profiler inside Android Studio using File > Open .
The Benchmark Gradle plugin now provides defaults for simpler setup:
- testBuildType is set to release by default, to avoid using dependencies with code coverage built-in. The release buildType is also configured as the default buildType, which allows Android Studio to automatically select the correct build variant when opening a project for the first time. ( b/138808399 )
- signingConfig.debug is used as the default signing config ( b/153583269 )

** Bug Fixes **

Significantly reduced the warmup transition overhead, where the first measurement for each benchmark was artificially higher than others. This issue was more pronounced in very small benchmarks (1 microsecond or less). ( b/142058671 )
Fixed InstrumentationResultParser error printed for each benchmark when running from command line. ( I64988 , b/154248456 )

Известные проблемы

Command line, gradle invocations of Benchmark do not print out results directly. You can work around this by either running through Studio, or parsing the JSON output file for results.
Benchmark reporting fails to pull the report from devices that have an app installed with an applicationId ending with either “android” or “download” (case insensitive). Users hitting this issue should upgrade the Android Gradle Plugin to 4.2-alpha01 or later.

Версия 1.0.0

Benchmark Version 1.0.0

November 20, 2019

Основные особенности версии 1.0.0

The Benchmark library allows you to write performance benchmarks of app code and get results quickly.

Major features include:

Clock stabilization
Automatic thread prioritization
Support for UI performance testing, such as in the RecyclerView Sample
JIT-aware warmup and looping
JSON benchmark output for post-processing

Версия 1.0.0-rc01

October 23, 2019

Новые возможности

Added systrace tracing to benchmarks

Исправления ошибок

Fixed metric instability issue where JIT wouldn't finish before warm up due to deprioritization ( b/140773023 )
Unified JSON output directory across Android Gradle Plugin 3.5 and 3.6

Версия 1.0.0-бета01

October 9, 2019

Новые возможности

Run garbage collection before each warmup to reduce memory pressure from one benchmark to leak to the next ( b/140895105 )

Исправления ошибок

Added androidx.annotation:android-experimental-lint dependency, so that Java code will correctly produce lint errors when experimental API is not used, similar to what is provided by the Kotlin experimental annotation for Kotlin callers.
Now correctly detects usage of additionalTestOutputDir instrumentation argument for output in Android Gradle Plugin 3.6, to know when AGP will handle data copy.
Fix undetected clock frequency in JSON to correctly print -1 ( b/141945670 ).

Version 1.0.0-alpha06

September 18, 2019

Новые возможности

Added a check for incorrectly using the old package for the test runner, which now provides a more-helpful error message

API меняется

The experimental annotation ExperimentalAnnotationReport is now correctly public. Usage of the experimental BenchmarkState#report API now requires this annotation

Версия 1.0.0-альфа05

September 5, 2019

API меняется

BenchmarkState.reportData API is now marked experimental

Исправления ошибок

Fix for the clock-locking script, which would fail on devices that were either missing the cut or expr shell utilities.
Fixed an issue with ./gradlew lockClocks task that would hang on devices that were rooted with an older version of the su utility, which did not support the -c flag.

Версия 1.0.0-альфа04

August 7, 2019

Новые возможности

Gradle plugin
- Now automatically disables test coverage, and sets the AndroidBenchmarkRunner by default ( b/138374050 )
- Added support for new AGP-based data copy, when running benchmarks and when using AGP 3.6+
JSON format additions
- Output total benchmark test run time ( b/133147694 )
- @Parameterized benchmarks that use a name string (for example @Parameters(name = "size={0},depth={1}") ) now output parameter names and values per benchmark in the JSON output ( b/132578772 )
Dry Run mode ( b/138785848 )
- Added a "dry run" mode for running each benchmark loop only once, to check for errors/crashes without capturing measurements. This can be useful eg for, for example, quickly running benchmarks in presubmit to check that they're not broken.

API меняется

Module structure has changed, splitting the library ( b/138451391 )
- benchmark:benchmark-junit4 contains classes with JUnit dependency: AndroidBenchmarkRunner , and BenchmarkRule , both of which have moved into the androidx.benchmark.junit4 package
- benchmark:benchmark-common contains the rest of the logic, including the BenchmarkState API
- This split will allow the library to support benchmarking without JUnit4 APIs in the future
Configuration warnings are now treated as errors, and will crash the test ( b/137653596 )
- This is done to further encourage accurate measurements, especially in CI
- These errors can be reduced back to warnings with an instrumentation argument. For example: -e androidx.benchmark.suppressErrors "DEBUGGABLE,LOW_BATTERY"

Исправления ошибок

Errors when writing to external storage on Q devices provide more-descriptive messages, with suggestions of how to resolve the issue
Screens are automatically turned on during benchmark runs, instead of failing when the screen is off

External contributions

Thanks to Sergey Zakharov for contributing JSON output improvements and the fix for screen off issues!

Версия 1.0.0-альфа03

July 2, 2019

Новые возможности

Expose sleep duration due to thermal throttling per benchmark in the full JSON report

Исправления ошибок

The Gradle plugin should no longer be required to be applied after Android plugins and the Android block
Adds support for benchmark reports on Android 10 devices using scoped storage

Версия 1.0.0-альфа02

June 6, 2019

API меняется

Overhauled JSON schema. Further changes to the JSON schema are likely to be limited to additions:
- Reorganized the result object structure to support additional metric groups in the future ( b/132713021 )
- Added test run context information, such as device and build info and whether clocks are locked, to the top-level object ( b/132711920 )
- Time metric names now have 'ns' in their name ( b/132714527 )
- Additional stats added per reported metric (maximum, median, minimum), and removed simplified 'nanos' summary stat ( b/132713851 )
Removed XML output ( b/132714414 )
Thermal throttle detection removed from BenchmarkState.reportData API ( b/132887006 )

Исправления ошибок

Fixed ./gradlew lockClocks not sticking on some recent OS devices ( b/133424037 )
Throttling detection disabled for emulator ( b/132880807 )

Версия 1.0.0-альфа01

7 мая 2019 г.

androidx.benchmark:benchmark:1.0.0-alpha01 is released. The commits included in this version are available here .

Эталон

User Guide Code Sample Codelab

Справочник по API
androidx.benchmark
androidx.benchmark.junit4
androidx.benchmark.macro
androidx.benchmark.macro.junit4

Accurately measure your code's performance within Android Studio.

Latest Update	Стабильный выпуск	Кандидат на выпуск	Бета-версия	Альфа-релиз
9 апреля 2025 г.	1.3.4	-	-	1.4.0-alpha11

Объявление зависимостей

Macrobenchmark

To use Macrobenchmark in your project, add the following dependencies to your build.gradle file for your macrobenchmark module :

классный

dependencies {
  androidTestImplementation "androidx.benchmark:benchmark-macro-junit4:1.3.4"
}

Котлин

dependencies {
  androidTestImplementation("androidx.benchmark:benchmark-macro-junit4:1.3.4")
}

Microbenchmark

To use Microbenchmark in your project, add the following dependencies to your build.gradle file for your microbenchmark module :

классный

dependencies {
    androidTestImplementation "androidx.benchmark:benchmark-junit4:1.3.4"
}

android {
    ...
    defaultConfig {
        ...
        testInstrumentationRunner "androidx.benchmark.junit4.AndroidBenchmarkRunner"
    }
}

Котлин

dependencies {
    androidTestImplementation("androidx.benchmark:benchmark-junit4:1.3.4")
}

android {
    ...
    defaultConfig {
        ...
        testInstrumentationRunner = "androidx.benchmark.junit4.AndroidBenchmarkRunner"
    }
}

Чтобы использовать плагин, включите следующую строку в блок `plugins` вашего файла build.gradle верхнего уровня:

классный

plugins {
  id 'androidx.benchmark' version '1.3.4' apply false
}

Котлин

plugins {
  id("androidx.benchmark") version "1.3.4" apply false
}

Затем примените плагин к файлу build.gradle вашего тестового модуля.

классный

plugins {
  id 'androidx.benchmark'
}

Котлин

plugins {
    id("androidx.benchmark")
}

Обратная связь

Создать новую задачу

Дополнительную информацию см. в документации по системе отслеживания проблем .

Version 1.4

Version 1.4.0-alpha11

9 апреля 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha11 is released. Version 1.4.0-alpha11 contains these commits .

Изменения API

Changed TraceProcessor 's argument timeout: Duration to timeoutMs long for Java caller usability. ( I9fbb5 )
Mark TraceProcessor constructor as internal. Callers should use TraceProcessor.startServer or TraceProcessor.runServer . ( Ia8c5b )

Исправления ошибок

When killing the application with MacrobenchmarkScope.killProcess , validate the kill command results to prevent silent failure, and timeout error. ( I84555 )

Dependency Updates

This library now targets Kotlin 2.0 language level and requires KGP 2.0.0 or newer. ( Idb6b5 )

Version 1.4.0-alpha10

26 марта 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha10 is released. Version 1.4.0-alpha10 contains these commits .

Изменения API

Increased default timeout for TraceProcessor server load and querying to 120 seconds (from 60/30 previously), and made both configurable with one timeout parameter. ( Ifec87 )

Исправления ошибок

Fixed a few issues that would occur when benchmarking or capturing profiles of an app without profileinstaller , and it's included BroadcastReciever . This only affects runs on rooted devices. ( Ied308 )

Version 1.4.0-alpha09

March 12, 2025

androidx.benchmark:benchmark-*:1.4.0-alpha09 is released. Version 1.4.0-alpha09 contains these commits .

Изменения API

Added TraceProcessor and Session API with closable Handles, for easier usage with custom lifecycles. This is also a step towards enabling easier Coroutine and Java API usage. The extension functions to TraceProcessor.runServer {} are now marked experimental, as they're likely to move, and be made non-experimental in the future. ( I358b4 )

Исправления ошибок

Fixed Benchmark and Baseline Profile capture not working with API 36 due to a change in pgrep toybox that now requires -a to print the full command line. ( Idc991 )
Filter default tracing config to reduce risk of data loss in traces on newer API levels. ( I54e8a )
Added experimental androidx.benchmark.killExistingPerfettoRecordings instrumentation argument, which can be set to false to allow pre existing perfetto trace capture to continue. By default pre existing perfetto trace captures on device are killed to prevent interference. ( I02a3c )
The JSON field context.osCodenameAbbreviated will now be REL for released OS versions at and above API 35, as non-numeric code names are no longer supported by the underlying platform. ( Ib17fd )
Fixes crash in FrameTimingMetric when resynced frames occur. ( I7c6f4 , b/394610806 )
No longer assume Choreographer#doFrame is the top of the stack frame on the main thread for FrameTimingQuery . ( Iee0e0 , b/340206285 )

Version 1.4.0-alpha08

12 февраля 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha08 is released. Version 1.4.0-alpha08 contains these commits .

Изменения API

Moved TraceProcessor.runSession() extension APIs to be experimental, as they are likely to move to be concrete constructors eventually on Android. ( Ib0528 , b/393640753 )
Most of the implementation of Startup Insights are now public/experimental, and move to the TraceProcessor artifact See StartupInsights . ( I0aa00 )
Deprecate BenchmarkRule.runWithTimingDisabled {} in favor of BenchmarkRule.runWithMeasurementDisabled {} , which more clearly describes the behavior - all metrics are paused. Additionally, expose the MicrobenchmarkScope superclass since redeclaring the runWithMeasurementDisabled function to open access isn't possible, since it's inline. ( I9e23b , b/389149423 , b/149979716 )
Benchmark libraries have moved to Kotlin 2.0. ( I9d1e0 )
Removed androidx.benchmark.startupProfiles.enable instrumentation argument. It is no longer useful, as it can be controlled by includeInStartupProfile arg in BaselineProfileRule.collect() . ( I39eb4 )

Исправления ошибок

Reduced the amount of Microbenchmark internal functions called during profiling to make eg Method traces more clear ( Ifaed8 )
Speculative fix for crashes: 'Failed to stop [ ProcessPid(processName=perfetto, pid=...) ]'. Now Benchmark will log a message instead of crashing when a background Perfetto process isn't able to be stopped before running the benchmark. ( I37d3e , b/323601788 )
Fix IllegalStateExceptions with 'Expected pm dump-profiles stdout' label that were caused by overly strict output format check. ( I358dc )

Version 1.4.0-alpha07

29 января 2025 г.

androidx.benchmark:benchmark-*:1.4.0-alpha07 is released. Version 1.4.0-alpha07 contains these commits .

Новые возможности

BaselineProfileRule now introduces a collectWithResults(...) API which includes a list of paths to the computed profiles. ( I056f8 )
Added androidx.benchmark.measureRepeatedOnMainThread.throwOnDeadline instrumentation argument, which can be set to false to disable the throw on deadline behavior of measureRepeatedOnMainThread for local testing. Not otherwise recommended, as this increases the likelihood of ANRs during tests. ( Idbeec , b/353226476 )

Изменения API

Added @JvmOverloads to MicrobenchmarkConfig constructor. ( I13fd3 )
Refactored BenchmarkRule to be built on top of coroutines, and support better yield() behavior. This refactor removed several experimental BenchmarkState APIs, but will be followed by replacements as needed. Additionally, added runWithMeasurementDisabled to clarify behavior (all measurement is paused). In the future, runWithTimingDisabled will be deprecated. ( I19837 , b/389149423 , b/311242861 )
Move PerfettoTraceProcessor to TraceProcessor in a new androidx.benchmark:benchmark-traceprocessor artifact, and make most of its API non-experimental. Any custom TraceMetric or anything reading from traces will need to update to the new TraceProcessor import. The new TraceProcessor API works exactly like the old one, but is a standalone interface library (somewhat analogous to the androidx.sqlite layer from Room) with an Android-specific implementation built into macrobenchmark. The new artifact can be used on JVM as well, but currently you'll need to start your own copy of the TraceProcessor binary and offer a port to connect to it on. ( I3a767 , I62563 , b/381134564 )

Исправления ошибок

Throw a clearer error message when MacrobenchmarkScope.startActivityAndWait fails to launch the target process (potentially due to a crash in the target process), instead of the more ambiguous 'Unable to confirm activity launch completion' message ( I3539b )
Fixed several syntax errors in Kotlin samples, and syntax highlighting in several Java / build.gradle samples. ( Ib3808 )
Clarified ArtMetric and CaptureInfo parameter docs. ( I96e60 )

Version 1.4.0-alpha06

December 11, 2024

androidx.benchmark:benchmark-*:1.4.0-alpha06 is released. Version 1.4.0-alpha06 contains these commits .

Изменения API

Removed usage of @Language("sql") in PerfettoTraceProcessor.Session.query() , as Studio highlighting/parsing is broken. ( Idc2fa , b/377733398 )

Исправления ошибок

This library now uses JSpecify nullness annotations , which are type-use. Kotlin developers should use the following compiler arguments to enforce correct usage: -Xjspecify-annotations=strict , -Xtype-enhancement-improvements-strict-mode ( I46810 , b/326456246 )
Corrected ArtMetric to report class load (not init), and improved docs to clarify runtime behavior. ( I9915c )
On Android Multiuser, execute commands as root only on rooted devices. ( I88b44 )

Version 1.4.0-alpha05

November 13, 2024

androidx.benchmark:benchmark-*:1.4.0-alpha05 is released. Version 1.4.0-alpha05 contains these commits .

Исправления ошибок

Fixed an issue on API 34+ where CompilationMode.None() would have inconsistent performance not representative of initial, worst case performance. This works around a platform change which allows ART's compilation state verify to partially compile apps (only affecting class loading) shortly after first launch. ( Ie48d0 )
Fixed issue where (especially short) traces could be captured that wouldn't report measurement from built-in Macrobenchmark Metrics, due to the process name being truncated within the Perfetto trace. Now macrobenchmark works around this issue by looking for the truncated package name in all built-in queries, in addition to the expected package name. Note that custom TraceMetric implementations or other direct callers of PerfettoSession.query can implement this same behavior by changing process.name LIKE "$packageName" in a Perfetto query to instead be (process.name LIKE "$packageName" OR process.name LIKE "$(packageName.takeLast(15))") . ( I5bf01 , b/377565760 )

Version 1.4.0-alpha04

30 октября 2024 г.

androidx.benchmark:benchmark-*:1.4.0-alpha04 is released. Version 1.4.0-alpha04 contains these commits .

Новые возможности

(Experimental) Enable Baseline Profile generation, and benchmarking on apps installed to a secondary user, for example any app on headless Android Auto devices. This support has been tested in some scenarios, but let us know with a bug if it doesn't work for you. ( I9fcbe , b/356684617 , b/373641155 )

Исправления ошибок

isProfileable is now always overridden in benchmark builds, and isDebuggable is also now always overridden in both benchmark and nonMinified (baseline profile capture) builds. ( I487fa , b/369213505 )
Fixes compilation detection on some physical devices prior to API 28 - affects json context.compilationMode , as well as behavior of androidx.benchmark.requireAot=true (which no longer incorrectly throws) ( Ic3e08 , b/374362482 )
In CpuEventCounter metrics, throw if invalid measurements are observed (eg instructions/cpucycles==0) ( I8c503 )

Version 1.4.0-alpha03

16 октября 2024 года

androidx.benchmark:benchmark-*:1.4.0-alpha03 is released. Version 1.4.0-alpha03 contains these commits .

Изменения API

Macrobenchmark : Adds ArtMetric , which can be used to inspect profile coverage or general Android RunTime performance. Captures number and total duration of JIT, class init (where available), and class verification. Additionally, changes CaptureInfo to include optional ART mainline version with default. ( I930f7 )
Add coefficientOfVariation to Benchmark JSON output to show stability within a given benchmark run. ( Ib14ea )

Исправления ошибок

Fixed CollectBaselineProfileTask when AVD device has spaces in it. ( Ia0225 , b/371642809 )
Speculative fix for errors from StartupMode.COLD exceptions: Package <packagename> must not be running prior to cold start! . Now, MacrobenchmarkScope.killProcess() (including the one run before each iteration, used to implement StartupMode.COLD behavior) will wait to verify that the app's processes have all stopped running. ( I60aa6 , b/351582215 )
Fixed issue where UNLOCKED_ error would show up on some rooted emulators. ( Ic5117 )
This library now uses JSpecify nullness annotations , which are type-use. Kotlin developers should use the following compiler arguments to enforce correct usage: -Xjspecify-annotations=strict , -Xtype-enhancement-improvements-strict-mode ( I7104f , b/326456246 )

Version 1.4.0-alpha02

2 октября 2024 г.

androidx.benchmark:benchmark-*:1.4.0-alpha02 is released. Version 1.4.0-alpha02 contains these commits .

Изменения API

Moved Gradle tasks lockClocks and unlockClocks to be on benchmark projects, instead of available at the top level. This change was necessary as there is unfortunately no way to register these as top level actions without breaking project isolation. ( I02b8f , b/363325823 )

Исправления ошибок

BaselineProfileRule now collects profiles for multi-process apps by signaling each running process at the end of the block to dump profiles. If a profile based compilation never successfully finds a process to broadcast to, the compilation will fail, as it's unexpected to have profile data within. Additionally, added an instrumentation argument to control dump wait duration: androidx.benchmark.saveProfileWaitMillis ( I0f519 , b/366231469 )
From Benchmark 1.3.2 : Fixed Firebase Test Lab (FTL) being unable to pull Baseline Profile or Macrobenchmark result files from the Baseline Profile Gradle Plugin. ( I2f678 , b/285187547 )

To use FTL apply the plugin to the baseline profile module in the plugin block, with:

  plugins {
      ...
      id("com.google.firebase.testlab")
  }

and then configure firebase test lab with:

  firebaseTestLab {

      // Credentials for FTL service
      serviceAccountCredentials.set(file("credentials.json"))

      // Creates one or more managed devices to run the tests on.
      managedDevices {
          "ftlDeviceShiba34" {
              device = "shiba"
              apiLevel = 34
          }
      }

      // Ensures the baseline profile is pulled from the device.
      // Note that this will be automated as well later with aosp/3272935.
      testOptions {
          results {
              directoriesToPull.addAll("/storage/emulated/0/Android/media/${android.namespace}")
          }
      }
  }

Also the created FTL device needs to be added to the baseline profile extension:

  baselineProfile {
      managedDevices += "ftlDeviceShiba34"
      useConnectedDevices = false
  }

Version 1.4.0-alpha01

18 сентября 2024 года

androidx.benchmark:benchmark-*:1.4.0-alpha01 is released. Version 1.4.0-alpha01 contains these commits .

New Feature - App Startup Insights

Initial version of app startup insights can be enabled in Macrobenchmark. ( 09fae38 )

To enable in a startup benchmark:

  @Test
  fun startup {
      macrobenchmarkRule.measureRepeated(
          …
          packageName = "com.example.my.application.id"
          metrics = listOf(StartupTimingMetric()),
          iterations = 5,
          startupMode = StartupMode.COLD,
          compilationMode = CompilationMode.None(),
          experimentalConfig = ExperimentalConfig(startupInsightsConfig = StartupInsightsConfig(isEnabled = true))
          ) {
          scope.startActivityAndWait(...)
      }
  }

Then running your startup benchmark will analyze the trace to look for common problems, and print them after metrics to Studio test output in the benchmark tab, eg:

StartupBenchmark_startup[startup=COLD,compilationMode=None]
├── Metrics
│   ├──   timeToFullDisplayMs                min  1,147.2,   median  1,208.8,   max  1,307.4
│   └──   timeToInitialDisplayMs             min  1,147.2,   median  1,208.8,   max  1,307.4
├── App Startup Insights
│   ├── App in debuggable mode (expected: false)
│   │   └── seen in iterations: 0(true) 1(true) 2(true) 3(true) 4(true) 5(true) 6(true) 7(true) 8(true) 9(true)
│   ├── Potential CPU contention with another process (expected: < 100000000ns)
│   │   └── seen in iterations: 4(105022546ns)
│   └── Main Thread - Binder transactions blocked (expected: false)
│       └── seen in iterations: 7(true)
└── Traces
    └── Iteration 0 1 2 3 4 5 6 7 8 9

This feature is still a work-in-progress, with improvements to documentation and extensibility to follow, but feedback is welcome.

Новые возможности

Added gradle property androidx.baselineprofile.suppressWarnings to suppress all baseline profile warnings. ( 314153a )
Microbench metrics are now displayed in Perfetto traces as counters. ( 3214854 )
Add experimental scripts for disabling jit (requires root / runtime restart), and resetting device perf/test state. These are not currently published as gradle tasks. ( 7c3732b )
Added benchmark argument to skip tests when running on emulator. When automaticGenerationDuring build is enabled, benchmarks will also trigger baseline profile generation. This will fail, if emulators are used. With the new argument skipBenchmarksOnEmulator we can instead skip the test. ( 0c2ddcd )
Change perf event enable logic to run on API 23+ ( 2550048 )

Изменения API

Existing experimental PerfettoConfig argument to MacrobenchmarkRule.measureRepeated() moved to the new ExperimentalConfig object.

Исправления ошибок

Increase lockClocks.sh retry count ( 99e9dac )
Don't create nonMinified and benchmark build types if existing. Due to a bug, even if nonMinified and benchmark build types existed, they were going to be recreated. ( e75f0a5 )
Ignore non-terminating slices from TraceSectionMetric results. ( a927d20 )
Improved emulator check to consider sdk_ prefix. ( 1587de8 )
Treat non-running packages as cleared in FrameTimingGfxInfoMetric . ( 35cc79c )
Fix androidx.benchmark.cpuEventCounter producing corrupt values for non-Instruction events. ( 06edd59 )
Fix resumeTiming/runWithTimingDisabled to respect metric priority order, and significantly reduce impact of lower priority metric pause/resume on higher priority metric results. For example, if using cpu perf counters via cpuEventCounter.enable instrumentation argument, timeNs is no longer significantly reduced when pause/resume occur. ( 5de0968 )

Версия 1.3

Version 1.3.4

26 марта 2025 г.

androidx.benchmark:benchmark-*:1.3.4 is released. Version 1.3.4 contains these commits .

Исправления ошибок

Fixed Gradle Project Isolation incompatibilities in the Benchmark Baseline Gradle Plugin. ( b/404523257 )

Version 1.3.3

16 октября 2024 года

androidx.benchmark:benchmark-*:1.3.3 is released. Version 1.3.3 contains these commits .

Исправления ошибок

Fixed CollectBaselineProfileTask when AVD device has spaces in it ( Ia0225 , b/371642809 )

Version 1.3.2

2 октября 2024 г.

androidx.benchmark:benchmark-*:1.3.2 is released. Version 1.3.2 contains these commits .

Исправления ошибок

Fixed Firebase Test Lab (FTL) being unable to pull Baseline Profile or Macrobenchmark result files from the Baseline Profile Gradle Plugin. ( I2f678 , b/285187547 )

To use FTL apply the plugin to the baseline profile module in the plugin block, with:

  plugins {
      ...
      id("com.google.firebase.testlab")
  }

and then configure firebase test lab with:

  firebaseTestLab {

      // Credentials for FTL service
      serviceAccountCredentials.set(file("credentials.json"))

      // Creates one or more managed devices to run the tests on.
      managedDevices {
          "ftlDeviceShiba34" {
              device = "shiba"
              apiLevel = 34
          }
      }

      // Ensures the baseline profile is pulled from the device.
      // Note that this will be automated as well later with aosp/3272935.
      testOptions {
          results {
              directoriesToPull.addAll("/storage/emulated/0/Android/media/${android.namespace}")
          }
      }
  }

Also the created FTL device needs to be added to the baseline profile extension:

  baselineProfile {
      managedDevices += "ftlDeviceShiba34"
      useConnectedDevices = false
  }

Version 1.3.1

18 сентября 2024 года

androidx.benchmark:benchmark-*:1.3.1 is released. Version 1.3.1 contains these commits .

Исправления ошибок

Added gradle property androidx.baselineprofile.suppressWarnings to suppress all baseline profile warnings ( I7c36e , b/349646646 )
Fixed Baseline Profile Gradle Plugin to use pre-existing nonMinified… and benchmark… if created by the app instead of creating wrappers. ( Ia8934 , b/361370179 )
Fixed java.lang.AssertionError: ERRORS (not suppressed): EMULATOR when automaticGenerationDuringBuild is enabled on emulators. New argument is used to instead skip the test. ( If3f51 , b/355515798 )
Microbenchmark minification - keep subclasses of org.junit.runner.notification.RunListener in benchmark library proguard ( Ic8ed5 , b/354264743 )
Fix TraceSectionMetric to Ignore non-terminating slices. Previously these were considered to have -1 duration, eg during summation or finding minimum duration. ( If74b7 )
Fixed an issue in FrameTimingGfxInfoMetric where starting the metric would crash if the process wasn't already running. ( I6e412 )

Version 1.3.0

21 августа 2024 года

androidx.benchmark:benchmark-*:1.3.0 is released. Version 1.3.0 contains these commits .

Microbenchmark changes since 1.2.0

Method tracing is on by default in microbenchmarks when running on most devices
- Method tracing runs as a separate phase, after measurements - this enables accurate measurements and method traces to both be output from a single benchmark run
- Method tracing on some Android OS and ART versions will affect later measurement phases - on these versions, method tracing is off by default and a warning is printed to Studio output
Main thread benchmarks and ANRs
- Added measureRepeatedOnMainThread for UI thread benchmarks (eg those that interact with Compose/View UIs) to avoid ANRs when running for many seconds.
- Method traces are skipped if expected to overrun the ANR avoidance deadline. Set androidx.benchmark.profiling.skipWhenDurationRisksAnr to false to disable this behavior (not recommended for CI runs, as ANRs can cause problem in long CI runs).
Minification
- Embedded proguard rules to improve microbenchmarking with minification enabled
- Minification/R8 in a library module requires AGP 8.3, and can be enabled via android.buildTypes.release.androidTest.enableMinification in your build.gradle
- Experimental BlackHole.consume() API added to prevent dead code elimination ( If6812 , b/286091643 )
Метрики
- Experimental cpu event counter feature (metrics from perf_event_open , which requires root on most versions of the platform), access via InstrumentationArgument androidx.benchmark.cpuEventCounter.enable (can be set to true ), and androidx.benchmark.cpuEventCounter.events can be set eg to ( Instructions,CpuCycles ). This should be supported on some userdebug emulators, but support has not been tested across all available emulators

MACRObenchmark changes since 1.2.0

Method tracing overhaul for macrobenchmarks.
- Now method traces are scoped to the duration of the measureBlock , and can capture multiple sessions if the process starts multiple times.
- Previously, method tracing would only work for StartupMode.COLD benchmarks, and capture nothing for measureBlocks that didn't restart the target process
- Fixed method traces flush in macrobenchmark, so that method traces should be fully captured and valid, even on slower devices. ( I6349a , b/329904950 )
Correctly dump ART profile during individual warmUp iterations when process is killed so CompilationMode.Partial(warmup=N) measurements are more accurate. ( I17923 )
Drop Shader broadcast failure message
- Added debugging suggestions to drop shader broadcast failure message
- Add two instrumentation arguments for overriding shader dropping behavior to workaround crashes when benchmarking apps without ProfileInstaller 1.3:
  - androidx.benchmark.dropShaders.enable=true/false : can be used to skip all shader dropping (including that done in StartupMode.Cold launches), esp when benchmarking apps that don't yet use profileinstaller 1.3
  - androidx.benchmark.dropShaders.throwOnFailure=true/false : can be used to tolerate failures when trying to drop shaders, for example when benchmarking apps without profileinstaller 1.3 ( I4f573 )
Added experimental MacrobenchmarkRule#measureRepeated variant which takes a custom PerfettoConfig for fully customized Perfetto trace recording. Note that incorrectly configured configs may cause built in Metric classes to fail. ( Idfd3d , b/309841164 , b/304038384 )
Cancel background dexopt jobs before running a Macrobenchmark to reduce interference. ( I989ed )
Macrobenchmark now waits for 1 second for the target application to flush an ART profile (previously it waited for 500 ms). ( I85a50 , b/316082056 )
TraceSectionMetric overhaul
- Note : TraceSectionMetric changes below can affect outputs in CI usage, and may create discontinuities, or break parsing
- Sum is now the default, as most usage of this metric is for repeated events, and first would discard data in these cases
- Changed to be more customizable, with more available modes
- Mode names are now embedded in metric output name (in Studio and JSON)
- Now supports slices created using Trace.{begin|end}AsyncSection .
Метрики
- Power - Added PowerMetric.deviceSupportsHighPrecisionTracking , PowerMetric.deviceBatteryHasMinimumCharge() and PowerMetric.deviceSupportsPowerEnergy()
- Renamed Metric.getResult to getMeasurements to match return type
- Added log.w / exception labels to all startup detection failures. This does not change current behavior (so some errors throw, and others silently fail to detect the startup), just makes it more understandable. Generally the ones that Log.w() and fail to report startup metrics are those where non-frame events are missing, exceptions are thrown when startup is detected except for frame timing information (from UI/RT slices). ( Id240f , b/329145809 )
- Added frameCount measurement to FrameTimingMetric to aid in discovery of scenarios where measurements change because the number of frames produced changed (new animations added, invalidation issues fixed). ( I1e5aa )
- Clarified that frameOverrunMs is the preferred metric for tracking when available in docs, and why. ( I18749 , b/329478323 )
- Fixes issue where unterminated frames at the beginning and end of the trace could be paired together, which would incorrectly report as a single extremely long frame. ( I39353 , b/322232828 )
- Improve FrameTimingMetric error when frames aren't produced, and always output link to trace when failing metric parsing to assist in diagnosing problem. ( I956b9 )
- Fixed crash in FrameTimingMetric failing to parse frame id, especially on certain OEM devices. ( Ia24bc , b/303823815 , b/306235276 )
- Relaxed strictness of checks in FrameMetrics , and added more detail to error messages. ( Iadede )

Baseline Profile capture / Gradle plugin changes since 1.2.0

Increased max recommended version of AGP to 9.0.0-alpha01.
Ensure mergeArtProfile and mergeStartupProfile tasks always wait for baseline profile generation. ( I623d6 , b/343086054 )
Generating a baseline profile successfully will output a summary of what changed ( I824c8 , b/269484510 )
Added DSL to disable warnings ( Ic4deb , b/331237001 )
Fix to ensure benchmarks use generated baseline profiles when automaticGenerationDuringBuild is off ( Ic144f , b/333024280 )
Fix BaselineProfile gradle plugin property overrides to enable baseline profile generation and benchmarking when customizing a nonMinified or benchmark build type. ( Ib8f05 , b/324837887 )
Fix for including library baseline profiles in AAR prior to AGP 8.3.0-alpha15. ( I1d2af , b/313992099 )
Fixed baseline and startup profile output url at the end of generation task. ( I802e5 , b/313976958 )

Other significant changes since 1.2.0

Trace capture
- Reduced EXITCODE 2 error when starting perfetto from an error to logged warning
- Enable AIDL tracing by default in benchmarks(requires API 28) ( Ia0af2 , b/341852305 )
- Enable porter tag tracing by default in benchmarks. This captures, for example, wakelock tracepoints. ( Icfe44 , b/286551983 )
- Increased trace capture start timeout to avoid crashes when starting tracing on slower devices ( I98841 , b/329145808 )
- Added public API PerfettoTraceProcessor.Session.queryMetrics APIs with JSON, textproto, and proto binary (undecoded) variants. These allow you to query metrics built into TraceProcessor ( I54d7f , b/304038382 )
- Enable blocking start on Perfetto trace record to reduce risk of missing data at beginning of trace. Only supported on API 33+. ( Ie6e41 , b/310760059 )
JSON output
- Added additional information in benchmark context in JSON output:
  - context.artMainlineVersion - integer version of Art mainline module (if present on device, -1 otherwise)
  - context.build.id - Equals android.os.Build.ID
  - context.build.version.codename - Equals android.os.Build.VERSION.CODENAME
  - context.build.version.abbreviatedCodename - corresponds to first letter of pre-release codename (including on release builds) ( Ie5020 )
- Added profilerOutput list to JSON output for easier tooling around profiling traces (eg Perfetto, Method traces) ( I05ddd , b/332604449 )
- Added a warning when Android Test Orchestrator is used in benchmark modules, as this will cause per-module output JSON files to be repeatedly overwritten. ( Ia1af6 , b/286899049 )
- Throw when filenames are longer than 200 chars to avoid unclear crashes when writing or post-processing files. ( I4a5ab )

Version 1.3.0-rc01

7 августа 2024 г.

androidx.benchmark:benchmark-*:1.3.0-rc01 is released. Version 1.3.0-rc01 contains these commits .

Исправления ошибок

Fix androidx.benchmark.cpuEventCounter producing corrupt values for non-Instruction events ( I7386a , b/286306579 )
Fix resumeTiming / runWithTimingDisabled to respect metric priority order, and significantly reduce impact of lower priority metric pause/resume on higher priority metric results. For example, if using cpu perf counters via cpuEventCounter.enable instrumentation argument, timeNs is no longer significantly reduced when pause/resume occur. ( I39c2e , b/286306579 , b/307445225 )
Reduced chance of stack sampling causing measureRepeatedOnMainThread from hitting main thread hard timeout by moving stack sampling conversion off main thread. ( I487a8 , b/342237318 )
Removed manual outlining of access to new platform APIs since this happens automatically via API modeling when using R8 with AGP 7.3 or later (eg R8 version 3.3) and for all builds when using AGP 8.1 or later (eg D8 version 8.1). Clients who are not using AGP are advised to update to D8 version 8.1 or later. See this article for more details. ( I9496c , b/345472586 )
Added agp version check to send package name as instr arg. Previous to AGP 8.4.0 the target app package name cannot be send to the instrumentation app via instrumentation arguments. ( 0c72a3f )

Version 1.3.0-beta02

10 июля 2024 г.

androidx.benchmark:benchmark-*:1.3.0-beta02 is released. Version 1.3.0-beta02 contains these commits .

Исправления ошибок

Gracefully handle EXITCODE 2 when starting Perfetto to log a warning, but proceed.

Version 1.3.0-beta01

12 июня 2024 г.

androidx.benchmark:benchmark-*:1.3.0-beta01 is released. Version 1.3.0-beta01 contains these commits .

Изменения API

Renamed MethodTracing.affectsMeasurementOnThisDevice to AFFECTS_MEASUREMENT_ON_THIS_DEVICE for consistency. ( I1bdfa )
Added experimental BlackHole.consume() api to prevent dead code elimination in microbenchmarks. ( If6812 , b/286091643 )
Microbenchmark will now correctly throw to prevent method tracing from interfering with measurements. This occurs on certain devices when method tracing is forced on (via instrumentation args or MicrobenchmarkConfig ), and if a measurement is attempted after a method trace. Affected devices are running API 26-30 or certain ART mainline module versions affected by this interference, and can be detected at runtime via ProfilerConfig.MethodTracing.affectsMeasurementOnThisDevice . ( Iafb92 , b/303660864 )

Исправления ошибок

Bumped max agp version recommended to 9.0.0-alpha01. ( I5bbb0 )
Added compilation mode to benchmark context ( If5612 , b/325512900 )
Enable AIDL tracing by default (requires API 28) ( Ia0af2 , b/341852305 )
Added additional information in benchmark context in JSON output:
- context.artMainlineVersion - integer version of Art mainline module (if present on device, -1 otherwise)
- context.build.id - Equals android.os.Build.ID
- context.build.version.codename - Equals android.os.Build.VERSION.CODENAME
- context.build.version.abbreviatedCodename - corresponds to first letter of pre-release codename (even on release builds) ( Ie5020 )
Fixes StackSampling to respect androidx.benchmark.profiling.sampleDurationSeconds ( Ib1d53 )
Change macro->common dependency to be api() , so it's easier to use eg PerfettoTrace and PerfettoConfig . ( Icdae3 , b/341851833 )
Ensure mergeArtProfile and mergeStartupProfile tasks always wait for baseline profile generation. ( I623d6 , b/343086054 )
Consider variant enable state when deciding whether variant should be enabled. ( I5d19e , b/343249144 )
Increased default start timeout for perfetto trace processor. ( I87e8c , b/329145808 )

Version 1.3.0-alpha05

May 14, 2024

androidx.benchmark:benchmark-*:1.3.0-alpha05 is released. Version 1.3.0-alpha05 contains these commits .

Исправления ошибок

Throw clearer exception when macrobench metric returns zero values for all iterations ( Iab58f , b/314931695 )
Additional workaround rules added to microbench proguard rules, including support for listener rules and other observed warnings / errors. ( I14d8f , b/329126308 , b/339085669 )
Method tracing runs as a separate phase during a Macrobenchmark, and it no longer affects measurements. ( If9a50 , b/285912360 , b/336588271 )
Added extra debugging suggestions to drop shader broadcast failure message. ( I5efa6 , b/325502725 )

Version 1.3.0-alpha04

1 мая 2024 г.

androidx.benchmark:benchmark-*:1.3.0-alpha04 is released. Version 1.3.0-alpha04 contains these commits .

Изменения API

Added experimental MacrobenchmarkRule#measureRepeated variant which takes a custom PerfettoConfig for fully customized Perfetto trace recording. Note that incorrectly configured configs may cause built in Metric classes to fail. ( Idfd3d , b/309841164 , b/304038384 )
Rename PowerMetric.deviceSupportsPowerEnergy to PowerMetric.deviceSupportsHighPrecisionTracking for clarity ( I5b82f )
Added PowerMetric.deviceBatteryHasMinimumCharge() and PowerMetric.deviceSupportsPowerEnergy() to enable changing or skipping benchmarks based on device power measurement capability. ( I6a591 , b/322121218 )

Исправления ошибок

Added comparison with previous baseline profile ( I824c8 , b/269484510 )
Added DSL to disable warnings ( Ic4deb , b/331237001 )
Changed exception to info log when benchmark variants are disabled ( I8a517 , b/332772491 )
Make it simpler to capture method traces for a Macrobenchmark is scoped to the duration of the actual measureBlock() . Previously, it started at target process launch and only supported cold starts ( Iee85a , b/300651094 )
Avoid crashing when perfetto trace processor is slow to start ( I98841 , b/329145808 )

Version 1.3.0-alpha03

April 17, 2024

androidx.benchmark:benchmark-*:1.3.0-alpha03 is released. Version 1.3.0-alpha03 contains these commits .

Новые возможности

Adds public API PerfettoTraceProcessor.Session.queryMetrics APIs with JSON, textproto, and proto binary (undecoded) variants. These allow you to query metrics built into TraceProcessor ( I54d7f , b/304038382 )
Added profilerOutput to JSON output for easier tooling around profiling traces (eg perfetto, method traces). ( I05ddd , b/332604449 )
Added power tag to benchmark Perfetto Config. This captures, for example, wakelock tracepoints. ( Icfe44 , b/286551983 )
Added inst argument androidx.benchmark.profiling.skipWhenDurationRisksAnr , can be set to false to avoid skipping method traces when expected duration may cause an ANR - strongly recommended to avoid in CI runs.
Added experimental inst argument androidx.benchmark.profiling.perfCompare.enable , set this to true to run comparison timing between measurement and profiling phases. Useful in eg evaluating overhead of method tracing. ( I61fb4 , b/329146942 )

Изменения API

Changed TraceSectionMetric.Mode to sealed class to enable future expansion without breaking exhaustive when statements ( I71f7b )
Added TraceSectionMetric.Mode.Average and .Count , and reordered args so the more common argument (mode) was earlier in the arg list, reducing need for specifying parameter names. ( Ibf0b0 , b/315830077 , b/322167531 )
Renamed Metric.getResult to getMeasurements to match return type ( I42595 )

Исправления ошибок

Fix to ensure benchmarks use generated baseline profiles when automaticGenerationDuringBuild is off ( Ic144f , b/333024280 )
Fix BaselineProfile gradle plugin property overrides to enable baseline profile generation and benchmarking when customizing a nonMinified or benchmark build type. ( Ib8f05 , b/324837887 )
Fixed method traces flush in macrobenchmark, so that method traces should be fully captured and valid, even on slower devices. ( I6349a , b/329904950 )
Enable blocking start on Perfetto trace record to reduce risk of missing data at beginning of trace. Only supported on API 33+. ( Ie6e41 , b/310760059 )
Added a warning when Android Test Orchestrator is used in benchmark modules, as this will cause per-module output JSON files to be repeatedly overwritten. ( Ia1af6 , b/286899049 )
Force ',' (comma) thousands separators for consistency in Studio output, ignoring device locale ( I3e921 , b/313496656 )
TraceSectionMetric now supports slices created using Trace.{begin|end}AsyncSection . ( I91b32 , b/300434906 )
Added log.w / exception labels to all startup detection failures. This does not change current behavior (so some errors throw, and others silently fail to detect the startup), just makes it more understandable. Generally the ones that Log.w() and fail to report startup metrics are those where non-frame events are missing, exceptions are thrown when startup is detected except for frame timing information (from UI/RT slices). ( Id240f , b/329145809 )
Cancel background dexopt jobs before running a Macrobenchmark to reduce interference. ( I989ed )
Added frameCount measurement to FrameTimingMetric to aid in discovery of scenarios where measurements change because the number of frames produced changed (new animations added, invalidation issues fixed). ( I1e5aa )
Clarified that frameOverrunMs is the preferred metric for tracking when available in docs, and why. ( I18749 , b/329478323 )

Version 1.3.0-alpha02

20 марта 2024 г.

androidx.benchmark:benchmark-*:1.3.0-alpha02 is released. Version 1.3.0-alpha02 contains these commits .

Новые возможности

Experimental R8 support in microbench via embedded proguard rules. Note that this support is experimental, and requires AGP 8.3 for minification of library module tests. Use the following to enable R8 minification/optimization in your benchmark module's build.gradle , which should lead to a significant performance increase, depending on workload. ( I738a3 , b/184378053 )
```
android {
    buildTypes.release.androidTest.enableMinification = true
}
```

Исправления ошибок

Fixes method tracing warning to be on separate line from microbench output. ( I0455c , b/328308833 )

Version 1.3.0-alpha01

February 21, 2024

androidx.benchmark:benchmark-*:1.3.0-alpha01 is released. Version 1.3.0-alpha01 contains these commits.

Изменения API

Renamed MicrobenchmarkConfig boolean parameters to avoid unnecessary word 'should' ( Ia8f00 , b/303387299 )
Added BenchmarkRule.measureRepeatedOnMainThread so main thread benchmarks (eg ones touching Views or Compose UIs) can avoid triggering ANRs, especially during large suites in CI. ( I5c86d )
Added FrameTimingGfxInfoMetric , an experimental alternate implementation of FrameTimingMetric with measurements coming directly from the platform, rather than extracted from the Perfetto trace. ( I457cb , b/322232828 )
Add the ability to dump an ART profile during individual warmUp iterations. ( I17923 )
Several changes to TraceSectionMetric API:
- Add Mode.Min , Mode.Max
- Add label argument to override section name as metric label
- Added mode name to output to clarify metric meaning
- Changed default to sum, as most usage of this metric is for repeated events Be aware of this changes in CI usage, as it may create discontinuities or break parsing. ( Ic1e82 , b/301892382 , b/301955938 )

Исправления ошибок

Improved error message in baseline profile gradle plugin when specified managed device does not exist ( Idea2b , b/313803289 )
Fix for including library baseline profiles in AAR prior to AGP 8.3.0-alpha15 ( I1d2af , b/313992099 )
Fixed baseline and startup profile output url at the end of generation task ( I802e5 , b/313976958 )
Adjusted data source timeouts to attempt to fix java.lang.IllegalStateException: Failed to stop [ProcessPid(processName=perfetto, pid=...)] ( I8dc7d , b/323601788 )
Add two instrumentation arguments for overriding shader dropping behavior to workaround crashes when benchmarking apps without ProfileInstaller 1.3:
- androidx.benchmark.dropShaders.enable=true/false : can be used to skip all shader dropping (including that done in StartupMode.Cold launches), esp when benchmarking apps that don't yet use profileinstaller 1.3
- androidx.benchmark.dropShaders.throwOnFailure=true/false : can be used to tolerate failures when trying to drop shaders, for example when benchmarking apps without profileinstaller 1.3 ( I4f573 )
Skip method tracing on UI thread when expected to take longer than a few seconds, and cleanup method traces when throwing. ( I6e768 )
Throw when filenames are longer than 200 chars to avoid unclear crashes when writing or post-processing files. ( I4a5ab )
Fixes issue where unterminated frames at the beginning and end of the trace could be paired together, which would incorrectly report as a single extremely long frame. ( I39353 , b/322232828 )
Use --skip verification on API 30+ when reinstalling a package on API 30-33 to clear ART profiles on user builds. This helps bypass Play Protect warnings that cause failures on some class of devices. ( Ic9e36 )
Use am force-stop to kill apps when not a system app like System UI or Launcher. ( I5e028 )
Macrobenchmark now waits for 1 second for the target application to flush an ART profile (previously it waited for 500 ms ). ( I85a50 , b/316082056 )
Improve FrameTimingMetric error when frames aren't produced, and always output link to trace when failing metric parsing to assist in diagnosing problem. ( I956b9 )
Fixed crash in FrameTimingMetric failing to parse frame id, especially on certain OEM devices. ( Ia24bc , b/303823815 , b/306235276 )
Relaxed strictness of checks in FrameMetrics , and added more detail to error messages. ( Iadede )

Version 1.2

Version 1.2.4

April 17, 2024

androidx.benchmark:benchmark-*:1.2.4 is released. Version 1.2.4 contains these commits .

Исправления ошибок

Fixes baseline profile srcset not being set up in benchmark variants. Also fixes automaticGenerationDuringBuild in libraries causing a circular dependency. ( I28ab7 , b/333024280 )
Use am force-stop to kill apps when not a system app like System UI or Launcher. This fixes StartupMode.COLD benchmarks crashing from "Package $package must not be running prior to cold start!" due to process kill not fully succeeding. ( I5e028 )

Version 1.2.3

January 24, 2024

androidx.benchmark:benchmark-*:1.2.3 is released. Version 1.2.3 contains these commits.

Исправления ошибок

Removed exception from Baseline Profile Gradle Plugin when AGP version is 8.3.0 or higher.
Fix for including library baseline profiles in AAR prior to AGP 8.3.0-alpha15.

Version 1.2.2

1 декабря 2023 года

androidx.benchmark:benchmark-*:1.2.2 is released. Version 1.2.2 contains these commits.

Baseline Profiles

Execution logs will show the baseline profile output file path as a local file URI ( aosp/2843918 , aosp/2853665 , b/313976958 )

Версия 1.2.1

November 15, 2023

androidx.benchmark:benchmark-*:1.2.1 is released. Version 1.2.1 contains these commits.

Новые возможности

Improved error message when user disables test variants ( b/307478189 )
Added properties to support AS test run integration ( b/309805233 ), ( b/309116324 )

Version 1.2.0

October 18, 2023

androidx.benchmark:benchmark-*:1.2.0 is released. Version 1.2.0 contains these commits.

Important changes since 1.1.0

Baseline Profiles

New Baseline Profile Gradle Plugin automates capturing and including baseline profiles in your test and build workflow.
BaselineProfileRule.collect now stable, a streamlined and simplified version of the previous experimental BaselineProfileRule.collectBaselineProfile API
- Just specify packageName , and drive your app
For libraries generating baseline profiles, you can now filter the rules generated either in code ( BaselineProfileRule.collect argument), or even more simply in the gradle plugin
Исправления
- Fixed baseline profile collection on Android U+ ( Id1392 , b/277645214 )

Macrobenchmark

Сборник
- Macrobenchmark now correctly fully resets compilation state for each compile - this requires reinstalling the APK prior to Android 14, so benchmarking on Android 14+ is strongly recommended if you want to persist state (like user login) in what's being measured.
- You can also work around this by controlling app compilation separately, and skipping compilation with CompilationMode.Ignore() or instrumentation argument
Instrumentation Arguments
- Support for androidx.benchmark.dryRunMode.enable instrumentation argument, (already available in microbenchmark) for quicker validation runs (eg when creating the benchmark, or in presubmit)
- Support for androidx.benchmark.profiling.mode=StackSampling and MethodTracing .
- Added androidx.benchmark.enabledRules to allow runtime filtering baseline profile vs macrobenchmark rule tests
- Added androidx.benchmark.perfettoSdkTracing.enable argument to enable tracing with tracing-perfetto, eg Compose recomposition tracing. Note that when used with StartupMode.COLD , timing will be significantly affected as the tracing library is loaded and enabled during app startup.
Требования
- Macrobenchmark now requires ProfileInstaller 1.3.0 or greater in the target app, to enable profile capture / reset, and shader cache clearing.
New Experimental Metric APIs
- Added experimental TraceSectionMetric , which allows for extracting simple timing from trace("") {} blocks in your app, or TraceMetric for leveraging the full query capability of Perfetto TraceProcessor .
- Added experimental PowerMetric to capture power usage information
- Added experimental MemoryCountersMetric to count page faults
- Added experimental PerfettoTraceProcessor API, which is used internally to extract metrics from System traces (aka Perfetto traces)
Исправления
- Fixed crashes when installing or extracting profiles from an app installed from multiple APKs (eg from app bundle).
- Fixed FrameTimingMetric ignoring frames with inconsistent frame IDs (generally, frames during ripples on API 31+) ( I747d2 , b/279088460 )
- Fixed parsing errors on traces > 64MB ( Ief831 , b/269949822 )
- Clarified errors when device (especially emulator) OS image not correctly configured for tracing, or compilation
- Skip battery level check for devices without battery (micro and macro)
- Improved file output, with more clear errors for invalid output directories, and safer defaults
- Improved stability of StartupMode.COLD by consistently dropping the shader cache (also exposed via MacrobenchmarkScope.dropShaderCache )
- Fixed leanback fallback for startActivityAndWait .

Microbenchmark

Функции
- Profiling was moved to a separate phase, after other metrics, so one test run can display both accurate timing and profiling results.
Experimental APIs
- Added experimental MicrobenchmarkConfig API for defining custom metrics and configuring tracing and profiling. Can be used to capture method traces, or capture tracepoints (but be aware of tracing overhead).
- Added experimental APIs for controlling BenchmarkState separately from BenchmarkRule , without JUnit
- Added experimental PerfettoTrace record to enable capturing Perfetto traces, with custom configuration, separate from benchmark APIs.
Исправления
- Workaround missing leading whitespaces in Android Studio benchmark output.
- Fix issue where warnings could fail to print in Android Studio benchmark output.
- Fixed SampledProfiling crash on Android 13 (API 33) and higher.
- Massively improved performance of dryRunMode by skipping IsolationActivity and Perfetto tracing (Up to 10x faster dry run mode on older OS versions).

Version 1.2.0-rc02

October 6, 2023

androidx.benchmark:benchmark-*:1.2.0-rc02 is released. Version 1.2.0-rc02 contains these commits.

Исправления ошибок

Fix Benchmark file output to no longer break BaselineProfile Plugin file copying. Files were generated and copied off device, but had been renamed such that the gradle plugin wouldn't see them. ( I8dbcc , b/303034735 , b/296453339 )
Clarified tracing-perfetto loading error messages when injecting from macrobenchmark module into target application.

Version 1.2.0-rc01

20 сентября 2023 года

androidx.benchmark:benchmark-*:1.2.0-rc01 is released. Version 1.2.0-rc01 contains these commits.

Исправления ошибок

An exception (with remedy instructions) is now thrown when Perfetto SDK tracing fails to initialize in a Benchmark. ( I6c878 , b/286228781 )
Fix OOM crash when converting ART method trace -> perfetto format. ( I106bd , b/296905344 )
(Macrobenchmark) Clarified method tracing label when linked in Studio test output, and fixed method tracing filenames to be unique on device/host, so they won't be overwritten when more than one benchmark is run. ( I08e65 , b/285912360 )
Ensures that the device is awake when capturing a baseline profile. ( I503fc )

Version 1.2.0-beta05

August 30, 2023

androidx.benchmark:benchmark-*:1.2.0-beta05 is released. Version 1.2.0-beta05 contains these commits.

Новые возможности

The Baseline Profile Gradle Plugin now supports Android Gradle Plugin 8.3. ( aosp/2715214 )

Version 1.2.0-beta04

August 23, 2023

androidx.benchmark:benchmark-*:1.2.0-beta04 is released. Version 1.2.0-beta04 contains these commits.

Новые возможности

The Baseline Profiles Gradle plugin now supports Android Gradle Plugin 8.3. ( aosp/2715214 )

Исправления ошибок

Fix failures in writing / moving and pulling files (especially those from parameterized tests) by sanitizing output file names further, avoiding '=' and ':' in output file names. ( I759d8 )

Version 1.2.0-beta03

August 9, 2023

androidx.benchmark:benchmark-*:1.2.0-beta03 is released. Version 1.2.0-beta03 contains these commits.

Изменения API

Added argument to filter TraceSectionMetric to only the target package, on by default ( Ia219b , b/292208786 )

Исправления ошибок

Renamed fullTracing.enable instrumentation argument to perfettoSdkTracing.enable for consistency with artifact name, and other references. fullTracing.enable will continue to work as a fallback. ( I7cc00 )
Benchmark library internal tracepoints (including microbenchmark loop/phase tracing) will now show up in Studio system trace viewer, and nest under the correct process in Perfetto. ( I6b2e7 , b/293510459 )
Removed macrobenchmark NOT-PROFILEABLE error on API 31+, and skip profileable check on eng/userdebug rooted devices. ( I2abac , b/291722507 )
When using Dex Layout Optimizations, startup profile rules are also now considered as baseline profile rules. ( aosp/2684246 , b/293889189 )

Version 1.2.0-beta02

July 26, 2023

androidx.benchmark:benchmark-*:1.2.0-beta02 is released. Version 1.2.0-beta02 contains these commits.

Изменения API

Added experimental APIs for microbench custom metrics and configuration (eg profiler, and tracing). ( I86101 , b/291820856 )

Исправления ошибок

Report error in macrobench when OS is misconfigured for tracing, as was recently fixed in API 26/28 ARM64 emulators. ( I0a328 , b/282191686 )
Added detail to compilation reset failure to suggest updating emulator, as some emulators have recently fixed this issue. ( I8c815 , b/282191686 )
Make androidx.test.uiautomator:uiautomator:2.2.0 an api instead of an implementation dependency. ( I1981e )

Version 1.2.0-beta01

July 18, 2023

androidx.benchmark:benchmark-*:1.2.0-beta01 is released. Version 1.2.0-beta01 contains these commits.

Исправления ошибок

Fix warnings being sometimes suppressed in Benchmark output in Studio, and workaround leading whitespaces from Benchmark output not showing up in Studio ( Ia61d0 , b/227205461 , b/286306579 , b/285912360 )
Fixed comment for FrameTimingMetric . The submetric is named frameDurationCpuMs . ( Ib097f , b/288830934 ).

Version 1.2.0-alpha16

June 21, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha16 is released. Version 1.2.0-alpha16 contains these commits.

Изменения API

BaselineProfileRule.collectBaselineProfile() API has been renamed to BaselineProfileRule.collect() . ( I4b665 )

Исправления ошибок

Macrobenchmark support for androidx.benchmark.profiling.mode = MethodTracing . ( I7ad37 , b/285912360 )
Microbenchmark profiling moved to a separate phase, so it occurs in sequence after measurement, instead of replacing it. MethodTracing trace sections are also now included in the captured Perfetto trace, if present. ( I9f657 , b/285014599 )
Add count measurement to TraceSectionMetric with Mode.Sum . ( Ic121a , b/264398606 )

Version 1.2.0-alpha15

7 июня 2023 года

androidx.benchmark:benchmark-*:1.2.0-alpha15 is released. Version 1.2.0-alpha15 contains these commits.

Новые возможности

Added experimental MemoryUsageMetric for tracking memory usage of a target application. ( I56453 , b/133147125 , b/281749311 )
Add support for fully custom Perfetto configs with PerfettoTrace.record ( If9d75 , b/280460183 )
Added property to skip baseline profile generation. Usage: ./gradlew assemble -Pandroidx.baselineprofile.skipgeneration . ( I37fda , b/283447020 )

Изменения API

The collectBaselineProfile API always generates stable baseline profiles. The collectStableBaselineProfile API has been removed and collectBaselineProfile should be used instead. ( I17262 , b/281078707 )
Changed BaselineProfileRule 's filterPredicate arg to non-null, with a equivalent default value so that the default filter behavior is more clear in docs. ( I3816e )

Исправления ошибок

Disable IsolationActivity and Perfetto tracing in dryRunMode to significantly improve performance, as these were majority of runtime. ( Ie4f7d )
Support for call stack sampling in Macrobenchmarks using instrumentation test arguments androidx.benchmark.profiling.mode=StackSampling and androidx.benchmark.profiling.sampleFrequency . ( I1d13b , b/282188489 )
Fixes crash when dropping shaders on Android U (API 34), as well as on emulators. ( I031ca , b/274314544 )

Version 1.2.0-alpha14

May 3, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha14 is released. Version 1.2.0-alpha14 contains these commits.

Исправления ошибок

Fix FrameTimingMetric ignoring frames with inconsistent frame IDs. This would cause some animations on recent platform versions (API 31+) to ignore many frames while RenderThread was animating (eg during a ripple). ( I747d2 , b/279088460 )
Fixed trace processor parsing for traces larger than 64Mb. ( Ief831 , b/269949822 )
Fixed baseline profile generation on Android U failing because of the different output of pm dump-profiles command. ( Id1392 , b/277645214 )
Fix GPU clock locking script to compare strings correctly ( I53e54 , b/213935715 )

Version 1.2.0-alpha13

April 5, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha13 is released. Version 1.2.0-alpha13 contains these commits.

Изменения API

Added profile type parameter when generating baseline profiles to support upcoming startup profile feature ( Ie20d7 , b/275093123 )
Added new experimental TraceMetric API for defining fully custom metrics based on content of a Perfetto trace. ( I4ce31 , b/219851406 )
Add an experimental metric to determine the number of page faults during a benchmark. ( I48db0 )

Version 1.2.0-alpha12

March 22, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha12 is released. Version 1.2.0-alpha12 contains these commits.

Новые возможности

The new baseline profile gradle plugin is released in alpha version, making it easier to generate a baseline profile and simplifying the developer workflow.

Изменения API

Removed Perfetto tracing support on API 21 and 22, which includes both Microbenchmarks and the experimental PerfettoTrace APIs. Prior to this version, UiAutomation connections were unreliable on some devices. ( I78e8c )
Added public experimental API for PerfettoTraceProcessor to enable parsing trace content. This is a step toward fully custom metrics based on Perfetto trace data. ( I2659e , b/219851406 )

Version 1.2.0-alpha11

March 8, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha11 is released. Version 1.2.0-alpha11 contains these commits.

Исправления ошибок

Fixed crashes in MacrobenchmarkRule and BaselineProfileRule when reinstalling or extracting profiles from an app bundle with multiple APKs. ( I0d8c8 , b/270587281 )

Version 1.2.0-alpha10

22 февраля 2023 г.

androidx.benchmark:benchmark-*:1.2.0-alpha10 is released. Version 1.2.0-alpha10 contains these commits.

Новые возможности

On Android 14+, Macrobenchmark no longer reinstalls target applications to reset compilation state, thanks to a new platform feature. Previously it was necessary to have a rooted device, or to deal with all application state (eg user login) being removed before each benchmark runs. ( I9b08c , b/249143766 )

Исправления ошибок

Fix DryRunMode to no longer crash with empty profile, due to compilation skipping. Instead, it runs a single iteration and extracts the profile to ensure something is captured. ( I2f05d , b/266403227 )
Fix PowerMetric crash when checking for powerstats presence on old API levels. ( 5faaf9 , b/268253898 )

Version 1.2.0-alpha09

January 11, 2023

androidx.benchmark:benchmark-*:1.2.0-alpha09 is released. Version 1.2.0-alpha09 contains these commits.

Исправления ошибок

Enabled passing None to androidx.benchmark.enabledRules instrumentation arg to disable all benchmarks / baseline profile generation. ( I3d7fd , b/258671856 )
Fix PerfettoTrace capture in app modules (ie non-self-instrumenting test APKs) ( I12cfc )
Fixed baseline profile adb pull argument order in Studio output ( I958d1 , b/261781624 )
Arm emulator api 33 is now correctly recognized as such when trying to run a macrobenchmark and will correctly print the warning. ( 69133b , b/262209591 )
Skip battery level check on devices without battery in Macrobenchmark ( fe4114 , b/232448937 )

Version 1.2.0-alpha08

December 7, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha08 is released. Version 1.2.0-alpha08 contains these commits.

Изменения API

Added experimental new APIs PerfettoTrace.record {} and PerfettoTraceRule to capture Perfetto traces (also known as System Traces) as part of a test, to inspect test behavior and performance. ( I3ba16 )
BaselineProfileRule now accepts a filter predicate instead of a list of package prefixes. This gives the test full control on filtering. ( I93240 )
Add an experimental API BaselineProfileRule.collectStableBaselineProfile which waits until a baseline profile is stable for N iterations. ( I923f3 )
Add the ability to specify an output file name prefix when generating baseline profiles using BaselineProfileRule . ( I7b59f , b/260318655 )

Исправления ошибок

Improve safety of file output writing, which should prevent output files from silently not being written / appended, especially on API 21/22. ( If8c44 , b/227510293 )
Fix simpleperf trace output to create and place the file correctly. This should also more generally fix issues where a file is unsuccessfully pulled by gradle. ( I12a1c , b/259424099 )
Improve profileinstaller error message printed when profileinstaller is too old. This now tells you to update profileinstaller version (1.2.1) for measuring baseline profiles on API 31 through 33, instead of saying it's not supported. ( Ia517f , b/253519888 )
Fix several shell command failures onerror message Print needed API <=23, including failed perfetto capture binary setup and trace capture failures ( Ib6b87 , b/258863685 )
Automatically sort generated profile rules to minimize the number of changes as they change over time (when checking-in profile rules into source control). ( Ie2509 )
Fixed crash on unrooted builds below Android 13 (API 33) with message Expected no stderr from echo 3 > /proc/sys/vm/drop_caches ( I6c245 , b/259508183 )

  <!-- workaround bug in profileinstaller 1.3.0-alpha02, remove when updating to alpha03+ -->
  <receiver
    android:name="androidx.profileinstaller.ProfileInstallReceiver"
    android:permission="android.permission.DUMP"
    android:exported="true">
    <intent-filter>
        <action android:name="androidx.profileinstaller.action.BENCHMARK_OPERATION" />
    </intent-filter>
  </receiver>

Version 1.2.0-alpha07

9 ноября 2022 г.

androidx.benchmark:benchmark-*:1.2.0-alpha07 is released. Version 1.2.0-alpha07 contains these commits.

Изменения API

Adds PowerMetric API for measuring energy and power in Macrobenchmarks. ( Ife601 , b/220183779 )
Fixed MacrobenchmarkScope.dropShaderCache() to actually drop the shader cache. This removes roughly 20ms of noise from StartupMode.COLD benchmarks, as shaders are now consistently cleared each iteration. Previously, Partial compilation using warmup iterations would report incorrectly fast numbers, as shader caching was more likely to happen during warmup. This fix requires either a rooted device, or using profileinstaller:1.3.0-alpha02 in the target app. For ProfileInstaller library's API changes, please refer to ProfileInstaller 1.30-alpha02 page. ( Ia5171 , b/231455742 )
Added TraceSectionMode("label", Mode.Sum) , allowing measurement of total time spent on multiple trace sections with the same label. For instance, TraceSectionMetric("inflate", Mode.Sum) will report a metric inflateMs for the total time in a macrobenchmark spent on inflation. Also removed API 29 requirement, as TraceSectionMetric works together with androidx.tracing.Trace back to lower API levels, with the use of forceEnableAppTracing within the target app. ( Id7b68 , b/231455742 )

Исправления ошибок

Improved safety of all internal shell commands by validating all output/errors. ( I5984d , b/255402908 , b/253094958 )
Specify device in baseline profile adb pull command, so the pull command can be simply copied if multiple devices are connected (up to one emulator) ( I6ac6c , b/223359380 )
Add error if macrobenchmark test apk isn't set up as self-instrumenting. This error prevents macrobenchmarking from within the target app's process. In process, macrobench wouldn't be able to compile/kill/cold start the app, or control its own permissions ( I4279b )
Fixed an issue in measureRepeated() where StartupMode.COLD wouldn't kill the target process after setupBlock . Now setupBlock interacting with the app will not leave the app process running, and an invalid cold start measurement. ( I8ebb7 )

Version 1.2.0-alpha06

October 24, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha06 is released. Version 1.2.0-alpha06 contains these commits.

Изменения API

BaselineProfileRule no longer requires root on Android 13 (API 33), and is no longer experimental. ( Ie0a7d , b/250083467 , b/253094958 )
- This change also fixes how profiles from an app are flushed to disk on unrooted devices, but requires updating the target app's profileinstaller dependency.
- To use BaselineProfileRule or CompilationMode.Partial(warmupIterations) on an unrooted device, you must also update your target app to use androidx.profileinstaller.profileinstaller:1.3.0-alpha01 . This enables flushing the profile to disk correctly, so that it can be compiled/extracted.

Исправления ошибок

Fixes SampledProfiling crash on API 33+. ( I40743 , b/236109374 )

Version 1.2.0-alpha05

October 5, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha05 is released. Version 1.2.0-alpha05 contains these commits.

Исправления ошибок

Fix frame breakdown in Studio system trace viewer for benchmark captured traces ( I3f3ae , b/239677443 )
Correct FrameTimingMetric to list FrameOverrun as requiring API 31 instead of 29 ( I716dd , b/220702554 )
Set iteration in BaselineProfileRule , and throw clearly if target package not installed (was already done for MacrobenchmarkRule). ( Ic09a3 , b/227991471 )

Version 1.2.0-alpha04

September 21, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha04 is released. Version 1.2.0-alpha04 contains these commits.

Новые возможности

Add support for dryRunMode.enable instrumentation argument to macrobenchmark (already available in micro) for faster local development, and validating app automation (eg in presubmit). This overrides iterations to 1, skips compilation, suppresses all configuration errors , and disables measurement .json file output. ( Ib51b4 , b/175149857 )
On Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.dryRunMode.enable=true
```
In build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.dryRunMode.enable', 'true'
    }
}
```

Исправления ошибок

Fixed StartupTimingMetric to no longer require measured Activities to be launched through MacrobenchmarkScope.startActivityAndWait() . This means the metric can pick up launches from eg notifications, Context.startActivity() , in-app Activity based navigation, or shell commands. ( Ia2de6 , b/245414235 )
Fix bug where startActivityAndWait would timeout trying to wait for launch completion on emulators by reducing strictness of frame detection. ( Ibe2c6 , b/244594339 , b/228946895 )

Version 1.2.0-alpha03

September 7, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha03 is released. Version 1.2.0-alpha03 contains these commits.

Новые возможности

Added experimental APIs for using BenchmarkState independently, separate from BenchmarkRule / JUnit4 . ( Id478f , b/228489614 )

Исправления ошибок

Added Leanback fallback for startActivityAndWait . ( 01ed77 , b/242899915 )

Version 1.2.0-alpha02

August 24, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha02 is released. Version 1.2.0-alpha02 contains these commits.

Изменения API

Default to am force stop for MacrobenchmarkScope.killProcess() , even when rooted, except during Baseline Profile generation. This can be overridden with an optional boolean argument. ( 02cce9 , b/241214097 )

Исправления ошибок

Support baseline profile generation for System apps. ( I900b8 , b/241214097 )
Support checking for ODPM power metrics on unrooted devices. ( a38c78 , b/229623230 )

Version 1.2.0-alpha01

July 27, 2022

androidx.benchmark:benchmark-*:1.2.0-alpha01 is released. Version 1.2.0-alpha01 contains these commits.

Новые возможности

New tracing-perfetto-common component allowing tooling to enable Perfetto SDK tracing in an app that exposes it ( I2cc7f )
Added androidx.benchmark.enabledRules instrumentation argument to enable filtering macrobenchmark runs to just benchmarks, or just baseline profile generation. Pass in 'Macrobenchmark', or 'BaselineProfile' to just run one type of test, eg when just generating BaselineProfiles on an emulator. Comma-separated list also Supported. ( I756b7 , b/230371561 )
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.enabledRules', 'BaselineProfile'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
```

Изменения API

Added new PowerMetric for measuring energy and power tasks in benchmarks. ( I9f39b , b/220183779 )
Added a new compilation mode CompilationMode.Ignore to skip profile reset and compilation. ( Ibbcf8 , b/230453509 )
Added a new parameter to BaselineProfileRule#collectBaselineProfile to filter output file by package names ( If7338 , b/220146561 )
Enables developer to discharge device to measure power drain. ( I6a6cb )
Added the ability to clear shader cache in MacrobenchmarkScope . ( I32122 )
Enables developer to configure display of metric type and detail desired subsystem categories. ( I810c9 )
Previously an UnsupportedOperationException was thrown in the benchmark if run on an unsupported device. Now UOE only occurs if the metric is used on the unsupported device (ie: PowerMetric.configure ). ( I5cf20 , b/227229375 )
Added TotalPowerMetric and TotalEnergyMetric for measuring total power and energy in each system category in macrobenchmarks. ( I3b26b , b/224557371 )

Исправления ошибок

Fixed an issue where compiled methods were not correctly being reset between each macrobenchmark on unrooted builds. This unfortunately requires reinstalling the apk each iteration, which will clear application data for each macrobenchmark. ( I31c74 , b/230665435 )
Fix trace recording crash on API 21/22 ( If7fd6 , b/227509388 , b/227510293 , b/227512788 )
Overhaul activity launch completion detection to fix 'Unable to read any metrics' exception in startup macrobenchmarks. ( Ia517c )

Version 1.1.1

9 ноября 2022 г.

androidx.benchmark:benchmark-*:1.1.1 is released. Version 1.1.1 contains these commits.

Исправления ошибок

Fixes android.system.ErrnoException: open failed: EACCES which would occur on some Android11 (API 30)+ devices. This is a cherry-pick of a fix from 1.2.0-alpha01 . ( aosp/2072249 )

Версия 1.1.0

June 15, 2022

androidx.benchmark:benchmark-*:1.1.0 is released. Version 1.1.0 contains these commits.

This version is identical to androidx.benchmark:benchmark-*:1.1.0-rc03 .

Important changes since 1.0.0

Support for Jetpack Macrobenchmarks, which allows you to measure whole-app interactions like startup and scrolling , provides the ability to capture traces & measure trace sections .
Support for Baseline Profiles
- CompilationMode.Partial to measure the effectiveness of Baseline Profiles.
- @BaselineProfileRule to automatically generate Baseline profiles for a given critical user journey.
Support for Allocation metrics & profiling during Microbenchmark runs.

Version 1.1.0-rc03

June 1, 2022

androidx.benchmark:benchmark-*:1.1.0-rc03 is released. Version 1.1.0-rc03 contains these commits.

Исправления ошибок

Avoid reinstalling the target package on every benchmark iteration. ( aosp/2093027 , b/231976084 )
Remove the 300ms delay from pressHome() . ( aosp/2086030 , b/231322975 )
Improve Macrobenchmark iteration speed by optimizing Shell commands used under the hood. ( aosp/2086023 , b/231323582 )
Support for Managed Gradle Devices when generating Baseline Profiles with Macrobenchmarks. ( aosp/2062228 , b/228926421 )

Version 1.1.0-rc02

May 11, 2022

androidx.benchmark:benchmark-*:1.1.0-rc02 is released. Version 1.1.0-rc02 contains these commits.

Note that this release includes a behavior change, as apps are now fully reinstalled in between each benchmark to ensure accurate measurements.

Bug Fixes/Behavior Changes

Fixed an issue where app compilation was not correctly reset between macrobenchmarks, and not reset at all on unrooted builds. This fixes many cases where running multiple tests would result in CompilationMode having little to no effect on measurements. To workaround this problem, the target app is now fully reinstalling each test method, which will clear application data between each macrobenchmark. ( I31c74 , b/230665435 )
As this prevents apps from setting up state before tests, it is now possible to skip compilation / reinstallation to enable working around this. You can for example fully compile the target with a shell command cmd package compile -f -m speed <package> , and then bypass macrobenchmark's compilation step.
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.compilation.enabled, 'false'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.compilation.enabled=false
```
Made it possible to share a module between macrobenchmarks and baseline profile generating tests by adding androidx.benchmark.enabledRules instrumentation argument. Pass in 'Macrobenchmark', or 'BaselineProfile' to just run one type of test, eg when generating BaselineProfiles on an emulator. ( I756b7 , b/230371561 )
Eg in Your macrobenchmark's build.gradle:
```
android {
    defaultConfig {
        testInstrumentationRunnerArgument 'androidx.benchmark.enabledRules', 'BaselineProfile'
    }
}
```
Or from the Gradle command line:
```
./gradlew macrobenchmark:cC -P android.testInstrumentationRunnerArguments.androidx.benchmark.enabledRules=BaselineProfile
```

Version 1.1.0-rc01

April 20, 2022

androidx.benchmark:benchmark-*:1.1.0-rc01 is released. Version 1.1.0-rc01 contains these commits.

Исправления ошибок

Baseline profile output links in Android Studio now use a unique file name. This way the output always reflects the latest results of using a BaselineProfileRule . ( aosp/2057008 , b/228203086 )

Version 1.1.0-beta06

April 6, 2022

androidx.benchmark:benchmark-*:1.1.0-beta06 is released. Version 1.1.0-beta06 contains these commits.

Исправления ошибок

Fix trace recording crash on API 21/22 ( If7fd6 , b/227509388 )
Overhaul activity launch completion detection to fix 'Unable to read any metrics' exception in startup macrobenchmarks. ( Ia517c )
Fix startup metrics for Macrobenchmarks when CompilationMode.None() is used. Before this change, CompilationMode.Partial() would appear to be slower than Compilation.None() . ( 611ac9 ).

Version 1.1.0-beta05

March 23, 2022

androidx.benchmark:benchmark-*:1.1.0-beta05 is released. Version 1.1.0-beta05 contains these commits.

Исправления ошибок

Kill package after skipping profile installation when using CompilationMode.None . ( aosp/1991373 )
Fixed an issue where Macrobenchmarks is unable to collect startup metrics when using StartupMode.COLD . ( aosp/2012227 b/218668335 )

Version 1.1.0-beta04

February 23, 2022

androidx.benchmark:benchmark-*:1.1.0-beta04 is released. Version 1.1.0-beta04 contains these commits.

Исправления ошибок

Fix missing metrics on Android 10, and NoSuchElementException caused by process names not being captured correctly in traces. ( Ib4c17 , b/218668335 )
Use PowerManager for thermal throttling detection on Q (API 29) and higher. This significantly reduces frequency of false positives in thermal throttling detection (benchmark retry after 90 second cooldown), and speeds up benchmarks significantly on user builds. It also provides throttle detection even when clocks are locked (if they're locked too high for the device's physical environment). ( I9c027 , b/217497678 , b/131755853 )
Filter simpleperf sampled profiling to measureRepeated thread only to simplify inspection ( Ic3e12 , b/217501939 )
Support metrics from named UI subprocesses in multi-process apps ( Ice6c0 , b/215988434 )
Filter Baseline Profile rules to target Android 9 (SDK 28). aosp/1980331 b/216508418
Skip Profile Installation when using Compilation.None() . Additionally, report warnings when the app is using an older version of androidx.profileinstaller and Android Gradle Plugin. aosp/1977029

Version 1.1.0-beta03

February 9, 2022

androidx.benchmark:benchmark-*:1.1.0-beta03 is released. Version 1.1.0-beta03 contains these commits.

Изменения API

Added AudioUnderrunMetric into macrobenchmark library under experimental flag to allow detection of audio underruns ( Ib5972 )

BaselineProfileRule no longer accepts a setup block as this functioned the same as the profileBlock . ( Ic7dfe , b/215536447 )

For eg

@Test
fun collectBaselineProfile() {
    baselineRule.collectBaselineProfile(
        packageName = PACKAGE_NAME,
        setupBlock = {
            startActivityAndWait()
        },
        profileBlock = {
            // ...
        }
    )
}

@Test
fun collectBaselineProfile() {
    baselineRule.collectBaselineProfile(
        packageName = PACKAGE_NAME,
        profileBlock = {
            startActivityAndWait()
            // ...
        }
    )
}

Исправления ошибок

Fixed issue where microbench profiler traces would fail to be updated in subsequent runs when linked in Studio output ( I5ae4d , b/214917025 )
Prevent compilation shell commands on API 23 ( Ice380 )
Renamed FrameCpuTime -> FrameDurationCpu , FrameUiTime -> FrameDurationUi to clarify these are durations, not timestamps, and to match prefixes. ( I0eba3 , b/216337830 )

Version 1.1.0-beta02

January 26, 2022

androidx.benchmark:benchmark-*:1.1.0-beta02 is released. Version 1.1.0-beta02 contains these commits.

Исправления ошибок

Microbenchmark Stack Sampling / Method Tracing Profile results are now linked in Studio output, similar to other profiling outputs, and do not suppress the allocation metric. ( Idcb65 , b/214440748 , b/214253245 )
BaselineProfileRule now prints the adb pull command in logcat and Studio output for pulling generated BaselineProfile text file. ( f08811 )

Version 1.1.0-beta01

January 12, 2022

androidx.benchmark:benchmark-*:1.1.0-beta01 is released. Version 1.1.0-beta01 contains these commits.

Исправления ошибок

Fixes profiler argument enable being ignored. ( I37373 , b/210619998 )
Removed deprecated CompliationModes ( I98186 , b/213467659 )
Switched baseline profile arg of CompilationMode.Partial to enum for clarity. ( Id67ea )

Version 1.1.0-alpha13

December 15, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha13 is released. Version 1.1.0-alpha13 contains these commits.

Изменения API

Add low-overhead System Tracing to microbench output on Android Q (API 29+). Note that this does not currently capture custom tracing (via android.os.Trace or androidx.tracing Jetpack APIs) to avoid affecting results. This tracing should be useful in diagnosing instability, especially from sources outside the benchmark. ( I298be , b/205636583 , b/145598917 )
Clarify CompilationModes into three classes - Full, None, Partial. Previously they were inconsistently named after compilation arguments (which we now treat as implementation details) and features. This makes the tradeoffs, potential combinations, and behavior across platform versions more clear. ( I3d7bf , b/207132597 )
Setup and measure are now always in pairs, in order. You can now query the package name and iteration (though the iteration may be null in certain warmup scenarios). ( Id3b68 , b/208357448 , b/208369635 )

Исправления ошибок

Fixed CompilationMode.Speed incorrectly treated as None ( I01137 )

Version 1.1.0-alpha12

November 17, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha12 is released. Version 1.1.0-alpha12 contains these commits.

Новые возможности

Add experimental TraceSectionMetric for custom trace-based timing measurements. ( I99db1 , b/204572664 )

Исправления ошибок

Wake device each iteration, to ensure UI can be tested - requires lockscreen is disabled. ( Ibfa28 , b/180963442 )
Fixes multiple crashes in StackSampling profiling mode on emulators and non-rooted devices ( Icdbda , b/202719335 )
Removed 0.5 second sleep at the end of each iteration - if you see missing metrics with this change, please file a bug. ( Iff6aa )
Reduce chances of dropped data, and lower memory overhead from tracing ( Id2544 , b/199324831 , b/204448861 )
Reduce trace size by ~40% by switching to compact sched storage format. ( Id5fb6 , b/199324831 )
Updated implementations of startup metrics to always end at end of renderthread. This will be more contistent across platform versions, and more closely map to in-app measurements. ( Ic6b55 )

Version 1.1.0-alpha11

November 3, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha11 is released. Version 1.1.0-alpha11 contains these commits.

Изменения API

Macrobenchmark now has a minSdkVersion of 23 . ( If2655 )
Adds a new experimental BaselineProfileRule which is capable of generating baseline profiles for app's critical user journey. Detailed documentation to follow. ( Ibbefa , b/203692160 )
Removes measureRepeated interface variant, which was added for java callers, as it caused ambiguity in completing/resolving the method. Java callers will again need to return Unit.Instance from measureRepeated. If this is an inconvenience, please file a bug, we can revisit this in a future version. ( Ifb23e , b/204331495 )

Version 1.1.0-alpha10

October 27, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha10 is released. Version 1.1.0-alpha10 contains these commits.

Изменения API

Backport StartupTimingMetric to work back to API 23. This new implementation also better handles reportFullyDrawn() to wait until corresponding content has been rendered. ( If3ac9 , b/183129298 )
Added JvmOverloads to multiple MacrobenchmarkScope methods for Java callers. ( I644fe , b/184546459 )
Provide alternative MacrobenchmarkRule.measureRepeated function that uses a Consumer<MacrobenchmarkScope> for idiomatic usage in Java language. ( If74ab , b/184546459 )

Исправления ошибок

Fix for traces not starting early enough, and missing metric data. This is expected to fix "Unable to read any metrics during benchmark" exceptions that were caused by the library itself. ( I6dfcb , b/193827052 , b/200302931 )
FrameNegativeSlack has been renamed to FrameOverrun to clarify its meaning - how much the frame went over its time budget. ( I6c2aa , b/203008701 )

Version 1.1.0-alpha09

October 13, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha09 is released. Version 1.1.0-alpha09 contains these commits.

Исправления ошибок

Support dropping Kernel page cache without root on API 31/S+, which will increase accuracy of StartupMode.COLD launches. ( Iecfdb , b/200160030 )

Version 1.1.0-alpha08

September 29, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha08 is released. Version 1.1.0-alpha08 contains these commits.

Изменения API

Enable scrolling macrobenchmarks to run back to API 23 ( If39c2 , b/183129298 )
Add new type of sampled metric to UI and JSON output, focused on percentiles of multiple samples per iteration. ( I56247 , b/199940612 )
Switch to floating point metrics throughout the benchmark libraries (truncated in the Studio UI). ( I69249 , b/197008210 )

Версия 1.1.0-альфа07

1 сентября 2021 г.

androidx.benchmark:benchmark-*:1.1.0-alpha07 is released. Version 1.1.0-alpha07 contains these commits.

Изменения API

Raised min API to 21 to reflect the intended lowest API level to be supported in the future. Current min API supported continues to be conveyed via RequiredApi(), and is currently 29 ( I440d6 , b/183129298 )

Исправления ошибок

Fixes ProfileInstaller to make it easier for apps using baseline profiles to run MacroBenchmarks using CompilationMode.BaselineProfile . ( I42657 , b/196074999 ) NOTE: requires also updating to androidx.profileinstaller:profileinstaller:1.1.0-alpha04 or greater.
StartupMode.COLD + CompilationMode.None benchmarks are now more stable. ( I770cd , b/196074999 )

Версия 1.1.0-альфа06

August 18, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha06 is released. Версия 1.1.0-alpha06 содержит эти коммиты.

Изменения API

Added androidx.benchmark.iterations instrumentation argument to allow manual overriding of iteration count when testing/profiling locally. ( 6188be , b/194137879 )

Исправления ошибок

Switched to Simpleperf as default sampling profiler on API 29+. ( Ic4b34 , b/158303822 )

Известные проблемы

CompilationMode.BaselineProfile is a work in progress. Avoid using it to determine how good a profile is for now.

Версия 1.1.0-альфа05

August 4, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha05 is released. Версия 1.1.0-alpha05 содержит эти коммиты.

1.1.0-alpha04 was cancelled before release due to a sporatic crash. b/193827052

Изменения API

Switched startActivityAndWait to invoke launch via am start , which reduces the time of each measurement iteration by approximately 5 seconds, at the cost of no longer supporting intent parcelables. ( I5a6f5 , b/192009149

Исправления ошибок

Reduce aggressiveness of thermal throttle detection, and recompute baseline if throttles are detected frequently. ( I7327b )
Fixes FrameTimingMetric to work on Android S beta ( Ib60cc , b/193260119 )
Use an EmptyActivity to bring the target app out of a force-stopped state to better support CompilationMode.BaselineProfile . ( Id7cac , b/192084204 )
Changed trace file extension to .perfetto-trace to match platform standard. ( I4c236 , b/174663039 )
StartupTimingMetric now outputs the "fullyDrawnMs" metric to measure time until your application has completed rendering. To define this metric for your app, call Activity.reportFullyDrawn when your initial content is ready, such as when your initial list items are loaded from DB or network. (reportFullyDrawn method available without build version checks on ComponentActivity). Note that your test must run long enough to capture the metric (startActivityAndWait doesn't wait for reportFullyDrawn). ( If1141 , b/179176560 )
Reduce cost of appending Ui metadata to traces by 50+ ms ( Ic8390 , b/193923003 )
Drastically increased polling frequency when stopping tracing, which can reduce eg startup benchmark runtime by 30+% ( Idfbc1 , b/193723768 )

Версия 1.1.0-альфа03

June 16, 2021

androidx.benchmark:benchmark-*:1.1.0-alpha03 is released. Версия 1.1.0-alpha03 содержит эти коммиты.

Новые возможности

Added a new CompilationMode.BaselineProfile to support profiles installed using the Jetpack ProfileInstaller library . ( aosp/1720930 )

Исправления ошибок

The sample Gradle code for suppressing benchmark errors has been updated to use a non-deprecated API with a syntax that also supports .gradle.kts users.
Например,
```
testInstrumentationRunnerArguments["androidx.benchmark.suppressErrors"] = "EMULATOR,LOW-BATTERY"
```

Версия 1.1.0-альфа02

18 мая 2021 г.

androidx.benchmark:benchmark-*:1.1.0-alpha02 is released. Версия 1.1.0-alpha02 содержит эти коммиты.

Новые возможности

Macrobenchmark artifacts added ( androidx.benchmark:benchmark-macro-junit4 and androidx.benchmark:benchmark-macro )

Capture startup, scrolling/animation performance metrics from your app, locally or in CI
Capture and inspect traces from within Android Studio

Исправления ошибок

Workaround shell permissions issue with output directory on Android 12 (Note - may require updating Android Gradle Plugin to 7.0.0 canary and Android Studio to Arctic Fox (2020.3.1), to continue capturing output files on affected devices). ( Icb039 )
Support configuration caching in BenchmarkPlugin ( 6be1c1 , b/159804788 )
Simplified file output - on by default, in a directory that doesn't require requestLegacyExternalStorage=true ( 8b5a4d , b/172376362 )
Fixes library printing logcat warnings about not finding JIT thread on platform versions where it is not present. ( I9cc63 , b/161847393 )
Fix for reading device max frequency. ( I55c7a )

Версия 1.1.0-альфа01

10 июня 2020 г.

New Features of 1.1

Allocation Metric - Benchmarks now run an additional phase after warmup and timing, capturing allocation counts. Allocations can cause performance problems on older versions of the platform (140ns in O became 8ns in M - measured on Nexus5X, with locked clocks). This metric is displayed in Android Studio console output, as well as in the
Profiling support - You can now capture profiling data for a benchmark run, to inspect why your code may be running slowly. Benchmark supports capturing either method tracing, or method sampling from ART. These files can be inspected with the Profiler inside Android Studio using File > Open .
The Benchmark Gradle plugin now provides defaults for simpler setup:
- testBuildType is set to release by default, to avoid using dependencies with code coverage built-in. The release buildType is also configured as the default buildType, which allows Android Studio to automatically select the correct build variant when opening a project for the first time. ( b/138808399 )
- signingConfig.debug is used as the default signing config ( b/153583269 )

** Bug Fixes **

Significantly reduced the warmup transition overhead, where the first measurement for each benchmark was artificially higher than others. This issue was more pronounced in very small benchmarks (1 microsecond or less). ( b/142058671 )
Fixed InstrumentationResultParser error printed for each benchmark when running from command line. ( I64988 , b/154248456 )

Известные проблемы

Command line, gradle invocations of Benchmark do not print out results directly. You can work around this by either running through Studio, or parsing the JSON output file for results.
Benchmark reporting fails to pull the report from devices that have an app installed with an applicationId ending with either “android” or “download” (case insensitive). Users hitting this issue should upgrade the Android Gradle Plugin to 4.2-alpha01 or later.

Версия 1.0.0

Benchmark Version 1.0.0

November 20, 2019

Основные особенности версии 1.0.0

The Benchmark library allows you to write performance benchmarks of app code and get results quickly.

Major features include:

Clock stabilization
Automatic thread prioritization
Support for UI performance testing, such as in the RecyclerView Sample
JIT-aware warmup and looping
JSON benchmark output for post-processing

Версия 1.0.0-rc01

October 23, 2019

Новые возможности

Added systrace tracing to benchmarks

Исправления ошибок

Fixed metric instability issue where JIT wouldn't finish before warm up due to deprioritization ( b/140773023 )
Unified JSON output directory across Android Gradle Plugin 3.5 and 3.6

Версия 1.0.0-бета01

October 9, 2019

Новые возможности

Run garbage collection before each warmup to reduce memory pressure from one benchmark to leak to the next ( b/140895105 )

Исправления ошибок

Added androidx.annotation:android-experimental-lint dependency, so that Java code will correctly produce lint errors when experimental API is not used, similar to what is provided by the Kotlin experimental annotation for Kotlin callers.
Now correctly detects usage of additionalTestOutputDir instrumentation argument for output in Android Gradle Plugin 3.6, to know when AGP will handle data copy.
Fix undetected clock frequency in JSON to correctly print -1 ( b/141945670 ).

Version 1.0.0-alpha06

September 18, 2019

Новые возможности

Added a check for incorrectly using the old package for the test runner, which now provides a more-helpful error message

API меняется

The experimental annotation ExperimentalAnnotationReport is now correctly public. Usage of the experimental BenchmarkState#report API now requires this annotation

Версия 1.0.0-альфа05

September 5, 2019

API меняется

BenchmarkState.reportData API is now marked experimental

Исправления ошибок

Fix for the clock-locking script, which would fail on devices that were either missing the cut or expr shell utilities.
Fixed an issue with ./gradlew lockClocks task that would hang on devices that were rooted with an older version of the su utility, which did not support the -c flag.

Версия 1.0.0-альфа04

August 7, 2019

Новые возможности

Gradle plugin
- Now automatically disables test coverage, and sets the AndroidBenchmarkRunner by default ( b/138374050 )
- Added support for new AGP-based data copy, when running benchmarks and when using AGP 3.6+
JSON format additions
- Output total benchmark test run time ( b/133147694 )
- @Parameterized benchmarks that use a name string (for example @Parameters(name = "size={0},depth={1}") ) now output parameter names and values per benchmark in the JSON output ( b/132578772 )
Dry Run mode ( b/138785848 )
- Added a "dry run" mode for running each benchmark loop only once, to check for errors/crashes without capturing measurements. This can be useful eg for, for example, quickly running benchmarks in presubmit to check that they're not broken.

API меняется

Module structure has changed, splitting the library ( b/138451391 )
- benchmark:benchmark-junit4 contains classes with JUnit dependency: AndroidBenchmarkRunner , and BenchmarkRule , both of which have moved into the androidx.benchmark.junit4 package
- benchmark:benchmark-common contains the rest of the logic, including the BenchmarkState API
- This split will allow the library to support benchmarking without JUnit4 APIs in the future
Configuration warnings are now treated as errors, and will crash the test ( b/137653596 )
- This is done to further encourage accurate measurements, especially in CI
- These errors can be reduced back to warnings with an instrumentation argument. For example: -e androidx.benchmark.suppressErrors "DEBUGGABLE,LOW_BATTERY"

Исправления ошибок

Errors when writing to external storage on Q devices provide more-descriptive messages, with suggestions of how to resolve the issue
Screens are automatically turned on during benchmark runs, instead of failing when the screen is off

External contributions

Thanks to Sergey Zakharov for contributing JSON output improvements and the fix for screen off issues!

Версия 1.0.0-альфа03

July 2, 2019

Новые возможности

Expose sleep duration due to thermal throttling per benchmark in the full JSON report

Исправления ошибок

The Gradle plugin should no longer be required to be applied after Android plugins and the Android block
Adds support for benchmark reports on Android 10 devices using scoped storage

Версия 1.0.0-альфа02

June 6, 2019

API меняется

Overhauled JSON schema. Further changes to the JSON schema are likely to be limited to additions:
- Reorganized the result object structure to support additional metric groups in the future ( b/132713021 )
- Added test run context information, such as device and build info and whether clocks are locked, to the top-level object ( b/132711920 )
- Time metric names now have 'ns' in their name ( b/132714527 )
- Additional stats added per reported metric (maximum, median, minimum), and removed simplified 'nanos' summary stat ( b/132713851 )
Removed XML output ( b/132714414 )
Thermal throttle detection removed from BenchmarkState.reportData API ( b/132887006 )

Исправления ошибок

Fixed ./gradlew lockClocks not sticking on some recent OS devices ( b/133424037 )
Throttling detection disabled for emulator ( b/132880807 )

Версия 1.0.0-альфа01

7 мая 2019 г.

androidx.benchmark:benchmark:1.0.0-alpha01 is released. The commits included in this version are available here .