As app developers, we strive to deliver exceptional user experiences that are fast, smooth, and engaging. To achieve this, it's crucial to measure and optimize our apps' performance throughout the development cycle. In this article, we'll explore how to leverage Google's Jetpack benchmarking libraries, specifically the Macrobenchmark library, to ensure your Android app provides an outstanding user experience.

Prerequisites

Before diving into the world of Macrobenchmarking, you should have a solid understanding of:

  • Android development and testing
  • Gradle
  • CircleCI account
  • Firebase and Google Cloud platform accounts
  • Android Studio Narwhal Feature Drop or higher

About the Project

The sample project we'll be working with is an expansion of an earlier testing sample used in a blog post on testing Android apps in CI/CD pipelines. We've added a benchmark job to the CI/CD build, which runs app startup benchmarks using the Jetpack Macrobenchmark library.

Jetpack Benchmarking

Android Jetpack offers two types of benchmarking: Microbenchmark and Macrobenchmark. While Microbenchmark is ideal for measuring application code performance (e.g., caching or similar processes), Macrobenchmark focuses on measuring your app's overall performance in areas like app startup and scrolling. Our sample application uses macrobenchmark tests to measure app startup.

Both libraries work seamlessly with the familiar Android Instrumentation framework, which runs on connected devices and emulators.

Setting Up the Library

The official Jetpack site provides detailed documentation on setting up the Macrobenchmark library - [Macrobenchmark Setup](https://developer.android.com/jetpack/benchmarking/macrobenchmark). Since this tutorial won't cover every step in fine detail (as they might change in future preview releases), we'll provide an overview of the procedure:

  1. Create the Macrobenchmark module
  2. Configure the target app
  3. Set up signing

Module Creation

Create a new test module named macrobenchmark. In Android Studio, create an Android library module and change the build.gradle.kts file to use the libs.plugins.test plugin. The new module needs a minimum SDK level of API 29 (Android 10).

Target App Configuration

  1. Add the tag to your app's AndroidManifest.xml to enable detailed trace information capture.
  2. Create a benchmark build type that mimics your release configuration but uses debug signing for local testing.
  3. Ensure your app includes ProfilerInstaller 1.3 or higher for profile capture functionality.

Signing Setup

Specify the local release signing config, which can use the existing debug config for the benchmark build type.

Please refer to the official Macrobenchmark library documentation for complete step-by-step instructions.

Writing and Executing Macrobenchmark Tests

The Macrobenchmark library introduces a few new JUnit rules and metrics. Here's a simple startup benchmark test taken from the performance-samples project on GitHub:

This startup benchmark measures how long it takes for your app to launch and become interactive using the StartupTimingMetric(). The test runs multiple iterations (defined by DEFAULT_ITERATIONS) to get consistent measurements, providing concrete performance data you can use to track regressions over time.

To generate Baseline Profiles alongside your benchmarks, include a profile generator:

This baseline profile generator creates Baseline Profiles - a powerful Android optimization technique that improves app performance by about 30% from the first launch. It guides Android Runtime (ART) to perform Ahead-of-Time (AOT) compilation on critical code paths, and the includeInStartupProfile = true parameter also generates Startup Profiles for DEX file layout optimization.

When you ship an app with Baseline Profiles, Google Play processes them and delivers optimized code to users immediately after installation.