Audio/Video

Everything you need to know about audio/video inside Chromium and Chromium OS!

Whom To Contact

It's best to have discussions on chromium-dev@chromium.org or media-dev@chromium.org for media specific matters.

We are component Internals>Media on the Chromium bug tracker.

Documentation

See media/README.md. For historical reference, here's the original design doc for HTML5 audio/video.

Codec and Container Support

Container formats

• MP4 (QuickTime/ MOV / ISO-BMFF / CMAF)

• Ogg

• WebM

• WAV

• HLS [Only on Android and only single-origin manifests]

Codec formats (Decode Only)

==Audio==

• FLAC

• MP3

• Opus

• PCM 8-bit unsigned integer

• PCM 16-bit signed integer little endian

• PCM 32-bit float little endian

• PCM μ-law

• Vorbis

• AAC [Main, LC, HE profiles only, xHE-AAC on Android P+, macOS, Windows 11] [Google Chrome only]

==Video==

• AV1

• VP8

• VP9

• H.264 [Google Chrome only]

• H.265 [Google Chrome only and only where supported by the underlying OS]

Code Location

Chromium

media/ - Home to all things media! media/audio - OS audio input/output abstractions media/video/capture - OS camera input abstraction media/video - software/hardware video decoder interfaces + implementations third_party/ffmpeg - Chromium's copy of FFmpeg third_party/libvpx - Chromium's copy of libvpx

Blink

third_party/blink/renderer/core/html/media/html_media_element.{cpp,h,idl} - media element base class

third_party/blink/renderer/core/html/media/html_audio_element.{cpp,h,idl} - audio element implementation

third_party/blink/renderer/core/html/media/html_video_element.{cpp,h,idl} - video element implementation

Particularly Interesting Bits

media/base/mime_util.cc - defines canPlayType() behaviour and file extension mapping

media/blink/buffered_data_source.{cc,h} - Chromium's main implementation of DataSource for the media pipeline

media/blink/buffered_resource_loader.{cc,h} - Implements the sliding window buffering strategy (see below)

third_party/blink/public/platform/web_media_player.h - Blink's media player interface for providing HTML5 audio/video functionality

media/blink/webmediaplayer_impl.{cc,h} - Chromium's main implementation of WebMediaPlayer

How does everything get instantiated?

WebFrameClient::createMediaPlayer() is the Blink embedder API for creating a WebMediaPlayer and passing it back to Blink. Every HTML5 audio/video element will ask the embedder to create a WebMediaPlayer.

For Chromium this is handled in RenderFrameImpl.

GN Flags

There are a few GN flags which can alter the behaviour of Chromium's HTML5 audio/video implementation.

ffmpeg_branding

Overrides which version of FFmpeg to use

Default: $(branding)

Values:

Chrome - includes additional proprietary codecs (MP3, etc..) for use with Google Chrome

Chromium - builds default set of codecs

proprietary_codecs

Alters the list of codecs Chromium claims to support, which affects <source> and canPlayType() behaviour

Default: 0(gyp)/false(gn)

Values:

0/false - <source> and canPlayType() assume the default set of codecs

1/true - <source> and canPlayType() assume they support additional proprietary codecs

How the %#$& does buffering work?

Chromium uses a combination of range requests and an in-memory sliding window to buffer media. We have a low and high watermark that is used to determine when to purposely stall the HTTP request and when to resume the HTTP request.

It's complicated, so here's a picture: