Lesson 14·Unit 5 · Strings, names, and input·Strings, bytes, and what users see·Lv 301·~8 min
Unicode pitfalls: strings that lie about what they are
'ID'.toLowerCase() returns 'ıd' in Turkish locale, and that's how Spring CVE-2024-38820 happened. 'café' === 'café' returns false if one's NFC and one's NFD. 'family emoji'.length is 11, not 1. Strings are not what your code thinks they are.
By the end
- ·Give a string's three different lengths, and say which one a character limit should be counting.
- ·Normalize at your system's edges, so two spellings that look identical compare equal.
- ·Explain why uppercasing needs a locale pinned to it, and what breaks in Turkish when you forget.
The problem
JavaScript strings are sequences of UTF-16 code units, not characters. "café" can be three or four units depending on whether the é is precomposed (NFC) or decomposed (NFD). Lowercase conversion is locale-dependent: Turkish has a dotless i. Equality comparison without normalization fails for usernames typed on different platforms. Security flaws compound: a username uniqueness check that doesn't normalize lets two users have visually identical accounts.
How it works
Three different answers to “how long is this string?”
A string has three lengths, and JavaScript hands you the least useful one. There are code points (what Unicode assigns), UTF-16 code units (what .length counts, with surrogate pairs counting double), and grapheme clusters (what a human calls one character). The family emoji is one grapheme built from seven code points occupying eleven code units. Truncate by .length and your "character limit" can slice between a mother and her children. Intl.Segmenter with grapheme granularity is the only count that matches what users see.
Equality is layered the same way. Composed café (é as one code point) and decomposed café (e plus a combining accent) render identically and compare unequal until you normalize, usually to NFC, at your system's boundaries. Skip it and "identical" usernames coexist and lookups miss.
Even lowercasing needs a locale. Turkish distinguishes dotted and dotless i, so 'ID'.toLocaleLowerCase('tr') is ıd, not id. That divergence caused a real Spring CVE, where a blocklist compared post-lowercase on the host's locale. The rules: normalize at the boundary, compare with the right tool (a pinned locale for security checks, Intl.Collator for humans), and never slice UI text by UTF-16 index.
See it yourself
Try these, in order
Each step triggers a specific failure you should recognize on sight.
- 1On the Turkish I tab, type ID (plain ASCII) into the input. en lowercases to “id”, while tr gives “ıd”, dotless. The equality badge flips false. That divergence is the CVE mechanism: a blocklist that compares post-lowercase misses on Turkish hosts.
- 2Open the NFC vs NFD tab and read the code-point dumps. Two visually identical cafés: 4 code points vs 5. The === badge says false until .normalize('NFC') makes them comparable. Filenames from macOS arrive NFD, and typed input arrives NFC.
- 3Still on that tab, scroll to “Normalize your own string”. Read the three rows for the seeded text, then replace it with an accented word of your own or a filename pasted from Finder. The seed is café spelled the decomposed way. “As typed” shows .length = 5 and five code points ending in U+0301. The NFC row shows the same word at 4, and the input === input.normalize('NFC') badge reads false. Whatever you paste, the badge tells you whether that byte sequence would survive an NFC boundary unchanged.
- 4On the Emoji length tab, pick the family emoji 👨👩👧👦. .length says 11, code points say 7, graphemes say 1. Any character-limit, truncation, or cursor logic keyed to .length cuts through the middle of a person.
- 5Pick the Scotland flag 🏴. A black flag followed by invisible “tag characters” spelling gbsct: even longer in code units. One visible glyph, fourteen code units.
Type a string. Watch the same string lowercased in English and Turkish produce different output. Try İD, STRAİT, or the literal word ID.
toLocaleLowerCase("en-US")
i̇d
toLocaleLowerCase("tr-TR")
id
en === tr ?
false
Security implication
A blocked-list filter rejects fields named "id" or "userid", written with bare toLocaleLowerCase(), no locale argument. That call follows the host locale: on a Turkish user's device it lowercases ID to ıd (dotless ı), which does not match the filter, and the field gets through. Plain toLowerCase() is locale-independent and always produces id. (The default sample İD shows the reverse: on a Turkish device it lowercases to plain id and collides with a name it should not match.)
Locale-independent toLowerCase() blocks "İD": no · Bare toLocaleLowerCase() on a Turkish device blocks it: yes
This is the shape of CVE-2024-38820 in Spring, and similar bugs in GitHub auth and Active Directory.
toLocaleLowerCase(): it follows whatever locale the host runtime uses. Pin the locale explicitly with toLocaleLowerCase("en-US"), or use plain toLowerCase(), which is locale-independent (root-locale case mapping) by spec.If you remember one thing
A string is not what it looks like. Normalize before comparing, segment by graphemes before counting or slicing, and pin the locale for case operations.
What to do about it
- Normalize every user-supplied string on input with
.normalize("NFC")before storing or comparing. This is one line of code and prevents an entire class of bugs. - Never use
.toLowerCase()for case-insensitive comparison without specifying a locale. Use.toLocaleLowerCase("und")(the unknown locale) for machine-only comparison, and only use real locales for display. - For string length that matches user perception, use
[...str].length(counts code points) orIntl.Segmenterwithgranularity: "grapheme"(counts user-visible characters, including emoji ZWJ sequences). - Run Unicode Skeleton checks on usernames / handles to prevent IDN homograph spoofing (e.g., Cyrillic
аvs Latina).
Use this with
Stakeholders
Moments
- ·Designing username / handle uniqueness rules
- ·Pre-merge code review for any string comparison
- ·Security audit
Field note
Spring Framework's CVE-2024-38820 is the Turkish-i bug with a CVSS score. Case-insensitive field matching used a default-locale lowercase, so on certain locales a disallowed field name stopped matching its blocklist entry. That is the toLowerCase('ID') → 'ıd' divergence this demo runs live. A one-argument fix (Locale.ROOT / a pinned locale) prevents the whole class.
Quick check
3 questions · pass at 2+
Question 1/3
What does 'ID'.toLocaleLowerCase('tr') return?
Question 2/3
For the family emoji 👨👩👧👦, what do .length, [...spread].length, and Intl.Segmenter graphemes report?
Question 3/3
Two strings render identically as “café” but === returns false. Most likely cause?