| Deflaking tests |
| kevers@ addressed some test flakes in animations and scroll-snap. These flakes were largely triggered due to common gotchas like antialiasing, floating point precision, or test timing. |
| Animation timing checks have been updated to accommodate 1 microsecond accuracy. This fix was not a pure test change as there was a lurking issue with numerical precision and setting the finished state of an animation. It is not just strict equality checks that are problematic when working with floating point numbers. |
| Several layout tests in animations make use of an internal pause for testing API, which predates and is largely deprecated by the web animations API (WAAPI). One test flake was addressed simply by switching over to WAAPI. |
| Several timing flakes were fixed in scroll-snap tests, by injecting a listener for scroll events. Unlike web animations, scroll animations do not fire an event to indicate completion of the scroll operation, and the end is inferred based on an absence of position updates over time. Unfortunately, this can lead to flakes when testing machines get bogged down and tests start running slower than expected. Simply waiting on an initial scroll event before starting the countdown, largely addresses this type of flake in scroll-snap tests. Once scrolling starts, the number of animation frames (rAFs) required to complete the scroll has a strict upper bound. |
| xidachen@ also fixed quite a few paint worklet flaky tests by using the standard animation API. |
| Before: we are taking a screenshot at rAF, which doesn’t guarantee the start of the animation. |
| Now: with the animation ready API, we take a screenshot when the animation has started which prevents the flakiness. |
| |
| Before |
| |
| After |
| WPT test coverage |
| |
| kevers@ and girard@ measured test coverage in WPT for input/scroll/animation. The data can help us understand which category of tests needs more attention. |
| TestDriver Action API |
| |
| lanwei@ kept improving the TestDriver action API and made more WPT tests automatic. |
| The following two pictures show the wpt dashboard for the TestDriver action API. |
| |
|