Profile a Protected Audience auction

Protected Audience auctions can be analyzed (either visually or by SQL query with Perfetto. Ad techs can use profiling with Perfetto to measure the performance of Protected Audience auctions, including:

  • Bidding and scoring script CPU time
  • Latency of HTTP requests such as key/value service
  • Performance impact of cold cache vs. hot cache
  • More vs. fewer custom audiences
  • Larger vs. smaller sets of signals
  • Different bidding logic scripts per custom audience as compared to using the same script for all bidding

Setup

  1. Clone the Perfetto and Privacy Sandbox repositories.

    git clone https://android.googlesource.com/platform/external/perfetto
    git clone https://github.com/android/privacy-sandbox-samples
    
  2. In Android Studio, open the Protected Audience sample app from the privacy-sandbox-samples/Fledge/FledgeKotlin directory.

  3. Build and install the sample app on your test device or emulator.

Run an auction and take a Perfetto trace

  1. Set up and deploy test HTTPS endpoints. Take a note of the URL of the hosted endpoints as they're required for the Protected Audience demo app to function.
  2. Launch the demo app with a test endpoint URL specified. Replace <test-endpoint-url> with a hosted endpoint URL that you recorded in the previous step.

    adb shell am start -n com.example.adservices.samples.fledge.sampleapp/.MainActivity \
      -e baseUrl "<test-endpoint-url>"
    
  3. Toggle the "Shoes CA" to ensure there's at least 1 custom audience active.

    Screenshot of demo app. The Shoes CA is toggled.
    Figure 1. Protected Audience Demo App.
  4. Record a trace using the trace_config.textproto file from the Privacy Sandbox DevTools GitHub repository:

    ./perfetto/tools/record_android_trace \
      -c path/to/trace_config.textproto
    
  5. Tap the "Run Ad Selection" button and wait for the auction results. When the auction completes, the output displays a message such as "Would display ad from http://example.com/bidding/render_shoes".

  6. In your terminal, terminate (CTRL+C) the record_android_trace program to finish the trace. The Perfetto UI opens in your browser with the trace data loaded.

Visually explore traces in Perfetto

  1. Search for "RunOnDeviceAdSelection" using the address bar at the top of the UI. Click Enter to complete the search and show the results:

    Perfetto UI looking at the Protected Audience auction. Tracks such as RunOnDeviceAdSelection and RunBidding are visible.
    Figure 2. Single-buyer Protected Audience auction in Perfetto.
  2. Click a trace to inspect it. Details such as execution latency are available here.

    Perfetto UI inspecting a trace segment. Latency details are visible.
    Figure 3. Inspecting a trace segment.

Protected Audience-specific trace segments

The Protected Audience auction is a complex process and there are many different segments captured by the Perfetto trace. This table documents what each trace segment represents.

Time Segment Description Frequency
Pre auction RunOnDeviceAdSelection Auction end-to-end Per auction
Bidding (buy-side) FilterContextualAds Perform app install and frequency cap filtering on contextual ads Per auction
GetBuyersCustomAudience Load buyer's custom audience from database Per buyer
FilterCustomAudiences Perform app install and frequency cap filtering on custom audiences Per auction
GetTrustedBiddingSignals Load buyer's bidding signals Per buyer
RunBiddingPerCustomAudience Ad bidding for a single custom audience Per custom audience
GetBuyerDecisionLogic Load buyer's JavaScript from the network, or database, if cached Per custom audience
RunBidding JavaScript execution for a buyer Per buyer
GenerateBids JavaScript execution for a custom audience Per custom audience
Scoring (sell-side) GetTrustedScoringSignals Load seller's scoring signals Per seller
RunAdScoring JavaScript execution for scoring Per auction
ScoreAd JavaScript execution for an ad Per ad
GetAdSelectionLogic Load seller's ad selection logic Per auction
RunAdOutcomeSelection Final filtering Per auction
Post auction PersistOnDeviceAdSelection Write auction result to database Per auction

Query for average execution latency

Perfetto can use SQL queries to get precise measurement of what's happening inside a particular trace.

This section documents how to measure the average execution latency for JavaScript execution.

  1. In Perfetto, navigate to "Query (SQL)" in the left navigation pane.
  2. Enter the following query:

    SELECT AVG(dur)
    FROM slice
    WHERE slice.name GLOB 'FetchPayload';
    
  3. Run the query and inspect the results.

    SQL query results. Average latency is 17693688 nanoseconds.
    Figure 4. SQL query results in nanoseconds.