Streams API
The Streams API allows you to programmatically access streams of data received over the network or created by other means and process them chunk by chunk. This is efficient for handling large datasets without holding everything in memory.
1. The Concept
Traditionally, if you wanted to process a resource (like a video, a text file, or a JSON response), you had to wait for the entire resource to download before you could do anything with it.
With Streams, you can start processing the data as soon as the first chunk arrives.
2. Readable Streams
A ReadableStream represents a source of data from which you can read.
Consuming a Stream (Fetch API)
The Fetch API Response.body is a ReadableStream.
fetch('https://example.com/large-text.txt')
.then(response => {
const reader = response.body.getReader();
const decoder = new TextDecoder();
return new ReadableStream({
start(controller) {
function push() {
reader.read().then(({ done, value }) => {
if (done) {
controller.close();
return;
}
// Process the chunk (value is a Uint8Array)
const chunk = decoder.decode(value, { stream: true });
console.log('Received chunk:', chunk);
controller.enqueue(value);
push();
});
}
push();
}
});
})
.then(stream => new Response(stream))
.then(response => response.text())
.then(text => console.log('Final text length:', text.length));Creating a Readable Stream
You can create your own stream to push data to a consumer.
const stream = new ReadableStream({
start(controller) {
controller.enqueue("Hello ");
controller.enqueue("World");
controller.close();
}
});3. Writable Streams
A WritableStream represents a destination for data.
const writableStream = new WritableStream({
write(chunk) {
console.log('Writing chunk:', chunk);
},
close() {
console.log('Stream closed');
},
abort(err) {
console.error('Stream error:', err);
}
});
const writer = writableStream.getWriter();
writer.write('Hello');
writer.write('Streams');
writer.close();4. Transform Streams
A TransformStream consists of a pair of streams: a writable stream (input) and a readable stream (output). It allows you to take data, transform it, and pass it along.
const transformStream = new TransformStream({
transform(chunk, controller) {
controller.enqueue(chunk.toUpperCase());
}
});
const writableStream = transformStream.writable;
const readableStream = transformStream.readable;5. Piping
The real power comes from connecting streams together using pipeTo() (Readable -> Writable) or pipeThrough() (Readable -> Transform -> Readable).
// Example: Fetch -> Decode -> Log
fetch('data.txt').then(async (response) => {
const stream = response.body
.pipeThrough(new TextDecoderStream()) // Transform bytes to text
.pipeTo(new WritableStream({
write(chunk) {
console.log(chunk);
}
}));
});programming/javascript/vanilla/javascript programming/javascript/vanilla/ajax-fetch programming/javascript/vanilla/promises-async-await