WebCodecs API
The WebCodecs API provides low-level access to the individual frames of a video stream and chunks of audio. It gives web developers the ability to encode and decode video and audio using the browser's built-in codecs (like H.264, VP8, VP9, AV1, AAC, Opus, etc.).
1. Why WebCodecs?
Before WebCodecs, web apps had to rely on high-level APIs like <video> or WebRTC, or use slow, WASM-compiled software codecs if they needed frame-level manipulation. WebCodecs exposes the efficient, hardware-accelerated media components already present in the browser.
Use cases include:
- Video editing in the browser.
- Low-latency game streaming.
- Real-time video processing (AI/ML).
2. Video Decoding
To decode video, you use VideoDecoder.
const decoder = new VideoDecoder({
output: (frame) => {
// frame is a VideoFrame object
console.log('Decoded frame:', frame);
// Draw to canvas, process, etc.
// ctx.drawImage(frame, 0, 0);
frame.close(); // Important: Release memory!
},
error: (e) => console.error(e),
});
// Configure the decoder
decoder.configure({
codec: 'vp8',
codedWidth: 640,
codedHeight: 480,
});
// Feed chunks to the decoder (e.g., from a file or network stream)
// chunk is an EncodedVideoChunk
// decoder.decode(chunk);3. Video Encoding
To encode video frames into chunks, you use VideoEncoder.
const encoder = new VideoEncoder({
output: (chunk, metadata) => {
// chunk is an EncodedVideoChunk
console.log('Encoded chunk size:', chunk.byteLength);
// Send chunk to server or mux into a file
},
error: (e) => console.error(e),
});
encoder.configure({
codec: 'vp8',
width: 640,
height: 480,
bitrate: 2_000_000, // 2 Mbps
framerate: 30,
});
// Create a VideoFrame (e.g., from a Canvas or ImageBitmap)
// const frame = new VideoFrame(canvas, { timestamp: 0 });
// Encode the frame
// encoder.encode(frame, { keyFrame: true });
// frame.close();4. Audio
Similar interfaces exist for audio: AudioDecoder and AudioEncoder. They work with AudioData (raw audio) and EncodedAudioChunk.
5. Integration with Streams
WebCodecs works well with the Streams API (ReadableStream, WritableStream) to process media pipelines efficiently.
programming/javascript/vanilla/javascript programming/javascript/vanilla/streams-api programming/javascript/vanilla/canvas-api