GFX test notes
GPU Rendering Benchmarks
Chromium has some GPU Rendering Benchmarks which use the Telemetry framework.
The smoothness test loads a set of 25 very common web pages, uses Telemetry to
scroll them, and then measures some performance metrics.
To run the test, you first need to check out the full Chromium source, and a Chrome OS Test Image.
cd chromium/src
tools/perf/run_measurement --browser=cros-chrome-guest --remote=${remote_ip} smoothness tools/perf/page_sets/top_25.json --allow-live-sites
For the Google and Facebook site tests, you can to provide credentials in a ~/.telemetry-credentials file:
cp tools/telemetry/examples/credentials_example.json ~/.telemetry-credentials
vi ~/.telemetry-credentials
Add Google and Facebook credentials, but the example file format has a bug: there should be a ',' between the "google" and "facebook" blocks.
The previous example tested using live sites. There are times where it is preferable to test using pre-cached site data. Telemetry lets you do that using a "web page replay" tool which captures a snapshot of a set of webpages (.wpr).
Several such snapshots have already been captured, such as the "top_25" test set which we are running here. Since the page contents are copyright of the respective web page owners, however, they are not available as part of the publicly available chromium source tree.
However, if you have access to the Chrome (src-internal) tree, you can check out the repository containing test data and run the test withough "--allow-live-sites":
# Prerequisite: To access the pre-recorded page sets, first set up a chrome src-internal password if not already set.
# See: <https://www.googlesource.com/new-password>
cd chromium/src/tools/perf
git clone https://chrome-internal.googlesource.com/chrome/tools/perf/data
cd chromium/src
tools/perf/run_measurement --browser=cros-chrome-guest --remote=${remote_ip} smoothness tools/perf/page_sets/top_25.json
The run_measurement, is useful for running arbitrary measurements on different
page sets. However, the performance bots run their benchmarks, using the
run_benchmark script. The equivalent benchmark to the above rrun_measurement
example is:
cd src/tools/perf
./run_benchmark run --browser=cros-chrome-guest --remote=${remote_ip} smoothness_top25
The scripts run_benchmark/run_measurement both launch the python script test_runner.py, which in turn starts up a local webpagereplay.py server.
Beware that the webpagereplay.py will fail to start if /etc/resolv.conf
contains the line "nameserver 127.0.0.1":
webpagereplay.ReplayNotStartedError: Web Page Replay failed to start. Log output:
dnsproxy.DnsProxyException: Invalid nameserver: 127.0.0.1 (causes an infinte loop)
This can happen when running on a laptop running Ubuntu 12.04+, since by default
Ubuntu 12.04 uses the resolvconf which uses a local dnsmasq server to do
actual dns resolution. Thus, the nameserver in /etc/resov.conf will be
127.0.0.1. To temporarily work around this, first fetch the real dns nameserver
using nm-tool, and use this value to manually update the nameserver entry in
/etc/resolv.conf. Note that any manual changes to /etc/resolv.conf will get
overwritten by the system at boot - or whenever resolvconf gets run.
Testing without Vsync
In /sbin/session_manager_setup.sh, add the following to the chrome parameter
list: --disable-gpu-vsync
Experimental WebGL autotests
WebGLManyPlanetsDeep
./run_remote_tests.sh --board=${board} --remote=${remote_ip} graphics_WebGLManyPlanetsDeep
Example results on mali 2013-wk20 w/ kernel/chromeos-3.4:
graphics_WebGLManyPlanetsDeep/graphics_WebGLManyPlanetsDeep avg_fps 42.5563759729
graphics_WebGLManyPlanetsDeep/graphics_WebGLManyPlanetsDeep avg_render_time 0.00651348182884
Example results on mali 2012-wk45 w/ kernel/chromeos-3.4:
graphics_WebGLManyPlanetsDeep/graphics_WebGLManyPlanetsDeep avg_fps 59.1172358698
graphics_WebGLManyPlanetsDeep/graphics_WebGLManyPlanetsDeep avg_render_time 0.00622212837838
WebGLAquarium
./run_remote_tests.sh --board=${board} --remote=${remote_ip} graphics_WebGLAquarium
mali-2012_wk45 on 3.4 on daisy @ CPU=1.4 MHz GPU=533MHz
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0001_fishes 34.7671616056
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0010_fishes 38.0083871349
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0050_fishes 36.1185803129
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0100_fishes 33.5180697395
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0250_fishes 29.4186162358
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0500_fishes 21.4609046677
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_1000_fishes 14.0662410091
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0001_fishes 0.0115492735466
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0010_fishes 0.0120682393055
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0050_fishes 0.0137373693937
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0100_fishes 0.0159920819915
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0250_fishes 0.0222824848304
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0500_fishes 0.0312746594998
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_1000_fishes 0.0478747122113
mali-2013_wk20 on 3.4 on daisy @ CPU=1.4 MHz GPU=533MHz
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0001_fishes 26.237068422
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0010_fishes 28.0702920668
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0050_fishes 27.0979309871
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0100_fishes 26.2101456206
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0250_fishes 23.3153728582
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_0500_fishes 21.0018305341
graphics_WebGLAquarium/graphics_WebGLAquarium avg_fps_1000_fishes 15.375419297
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0001_fishes 0.0117292154996
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0010_fishes 0.0115656813221
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0050_fishes 0.0134754876296
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0100_fishes 0.0157825582097
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0250_fishes 0.0217831682377
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_0500_fishes 0.0322741695138
graphics_WebGLAquarium/graphics_WebGLAquarium avg_render_time_1000_fishes 0.0508704025854
Restoring OOBE to make Telemetry happy
Telemetry gets confused if it cannot figure how to log in to the machine. This can happen, for instance, if you've logged in as another user and telemetry can't find the OOBE (Out Of Box Experience). To fix this, try restoring the oobe:
sudo pkill -9 chrome && rm -rf /home/chronos/Local\ State /home/chronos/.oobe_completed && restart ui
Enable tracing in mali-ddk
- Add --no-sandbox to chrome start in /sbin/sesssion_manager_startup.sh
- Change –enable-gpu-sandbox to –disable-gpu-sandbox
- Disable self.lCCFlags.append('-pedantic') in bldsys/toolchain_abstr.py for trace defines to build.
- Change the #if 0 to #if 1 in cros/cros_tracing.h
- Change the cros_trace_enabled to 1 in cros/cros_tracing.c
- Each time you boot the board, run this to enable the Mali tracing:
chmod a+wx /sys
chmod a+wx /sys/kernel
chmod a+wx /sys/kernel/debug
chmod a+wx /sys/kernel/debug/tracing/
chmod a+wx /sys/kernel/debug/tracing/trace_marker
Then include cros/cros_tracing.h in any files with functions you want to trace and add:
CROS_TRACE_ENABLE();
CROS_TRACE_BLOCK("trace block title");
Using valgrind on the gpu-process
Chromium OS includes an ebuild for valgrind. So you can just gmerge it to your target:
gmerge valgrind
Then you just do valgrind myprog.
For the gpu process, you need to pass a flag to chrome to give it the gpu launcher.
Add the following in in session_manager_setup.sh
--gpu-launcher="usr/bin/valgrind --trace-children=yes --error-limit=no"