graphemelyWhat is a lone surrogate and how do I remove it?
A lone surrogate is half of an emoji (or any character outside the Basic Multilingual Plane) left behind when a string is cut by code unit. Detect it with isWellFormed(str) and replace it with U+FFFD using toWellFormed(str).
The fix, copy and paste
import { isWellFormed, toWellFormed } from '@kwy404/graphemely';
const broken = '\u{1F600}'.slice(0, 1); // '\uD83D'
isWellFormed(broken); // false
toWellFormed('ok' + broken); // 'ok\u{FFFD}'
encodeURIComponent(toWellFormed(broken)); // '%EF%BF%BD', no URIErrorEmoji in the code samples are written as JavaScript escapes, for example \u{1F468} for a man. In your own code you can type the emoji directly.
Why lone surrogates are a problem
encodeURIComponentthrowsURIError: URI malformed.TextEncoderand most UTF-8 encoders silently replace them with U+FFFD, so data changes on its way to a server.- Some APIs and databases reject the request as invalid UTF-8.
- They render as a replacement box or a question mark in a diamond.
Where they come from
Almost always from cutting text with code unit indices: str.slice(0, n), str.substring, str[i], str.split('') or a maximum length enforced by an input field. The real fix is to cut by grapheme cluster with truncate or slice, so they never appear. toWellFormed is the safety net for data that is already broken.
Native methods
ES2024 added String.prototype.isWellFormed and toWellFormed. They are missing on Node 18 and older browsers. The graphemely versions behave the same and work everywhere.
Install
npm install @kwy404/graphemelypnpm add @kwy404/graphemelyyarn add @kwy404/graphemelybun add @kwy404/graphemelyWorks in Node 16 and newer, all modern browsers, Deno and Bun. ESM and CommonJS, TypeScript types included, zero dependencies, about 2 kB gzipped. Try your own text in the playground.
FAQ
A UTF-16 code unit in the range U+D800 to U+DFFF that is not part of a valid high and low surrogate pair. It is half of a character.
Call isWellFormed(str) from graphemely, or str.isWellFormed() on runtimes that support ES2024.
Call toWellFormed(str), which replaces each one with U+FFFD, the replacement character. Then fix the code that cut the string by code unit.
Related guides
Truncate a string with emoji
Cut text to a length with an ellipsis, never half an emoji.
String length with emoji
Why an emoji has length 2, and how to count what users see.
Reverse a string with Unicode
Reverse text without scrambling flags, accents or emoji.
Split a string into characters
Get the visible characters, not code units or code points.
Substring without breaking emoji
A Unicode safe substring and slice with negative indices.