Intl.Segmenter
The Intl.Segmenter object enables locale-sensitive text segmentation. It allows you to split a string into meaningful segments, such as graphemes (user-perceived characters), words, or sentences.
1. The Problem with split()
Traditionally, developers used string.split(' ') to count words or split sentences. However, this fails for languages that don't use spaces (like Japanese or Chinese) or for complex emojis that are made of multiple code points.
const str = "π¨βπ©βπ§βπ¦";
console.log(str.length); // 11 (It's actually 4 code points joined by ZWJ)
console.log([...str].length); // 7 (Still not 1 user-perceived character)2. Basic Usage
The constructor takes locales and options. The segment() method returns an iterable of segments.
const segmenter = new Intl.Segmenter('en', { granularity: 'grapheme' });
const segments = segmenter.segment("π¨βπ©βπ§βπ¦");
for (const { segment, index, input } of segments) {
console.log(segment); // "π¨βπ©βπ§βπ¦" (Correctly identified as 1 grapheme)
}3. Granularity
The granularity option determines how the string is split.
grapheme: Split by user-perceived characters (default).word: Split by words.sentence: Split by sentences.
Word Segmentation
const text = "Hello, world! How are you?";
const wordSegmenter = new Intl.Segmenter('en', { granularity: 'word' });
for (const seg of wordSegmenter.segment(text)) {
console.log(`'${seg.segment}' (isWordLike: ${seg.isWordLike})`);
}
/* Output:
'Hello' (isWordLike: true)
',' (isWordLike: false)
' ' (isWordLike: false)
'world' (isWordLike: true)
...
*/Sentence Segmentation
const paragraph = "Hello world. This is a test! Is it working?";
const sentenceSegmenter = new Intl.Segmenter('en', { granularity: 'sentence' });
for (const seg of sentenceSegmenter.segment(paragraph)) {
console.log(seg.segment);
}4. Handling Languages without Spaces
Intl.Segmenter is essential for languages like Japanese.
const japaneseText = "εΎθΌ©γ―η«γ§γγ";
const segmenter = new Intl.Segmenter('ja', { granularity: 'word' });
const words = [...segmenter.segment(japaneseText)].filter(s => s.isWordLike).map(s => s.segment);
console.log(words); // ["εΎθΌ©", "γ―", "η«", "γ§", "γγ"]programming/javascript/vanilla/javascript programming/javascript/vanilla/intl-collator