Text Encoding Explained: Part 2 — UTF-8, UTF-16 and Emojis

In Part 1, we saw how ASCII assigns numbers to characters, and how Unicode gave every character in the world a unique code point — a U+ identity.

But a code point is still just an identity. It is not something a computer can store, send, or write to a file.

That is where encoding comes in.

😀 is U+1F600 — that's its identity. But computers don't store Unicode code points directly. Computers store bytes.

So a new question appears. How does a Unicode code point become bytes in memory?

That's exactly what encodings solve.

What Is Encoding?

Encoding is a set of rules for turning a code point into actual bytes.

A code point like U+0041 is just a number. Encoding decides how that number gets written down — how many bytes it takes, and in what order.

In simple terms: Code Point -> Bytes

Without encoding, a code point is just an abstract identity. It cannot be stored in memory, saved to a file, or sent over a network.

Why Isn't Unicode Alone Enough?

Unicode solves the identity problem. Every character gets one unique number, no matter the language.

But Unicode never says how to store that number.

😀 is U+1F600. Should a computer store that as 2 bytes? 3 bytes? 4 bytes? Unicode does not answer this.

A computer still doesn't know:

• How many bytes should this occupy?
• How should it be written into memory?
• How should it be sent over the internet?
• How should another computer read it?

Unicode intentionally leaves those questions unanswered.

If every system stored numbers differently with no agreed format, text would break every time it moved between systems. That is exactly the kind of encoding mismatch you have probably seen — a file saved on one system turning into strange symbols on another.

So the industry needed standard, agreed-upon rules for turning code points into bytes. These rules are called UTF.

What Is UTF?

UTF stands for Unicode Transformation Format.

Unicode answers "what character is this?" UTF answers "how should this character be stored?"

Character -> Unicode Code Point -> UTF Encoding -> Bytes in Memory

So UTF is the specific set of rules that turns a Unicode identity into actual bytes.

The Three Common UTF Encodings

Today you'll mostly encounter three Unicode encodings.

• UTF-8
• UTF-16
• UTF-32

All three can represent every Unicode character. They just differ in how many bytes they use to do it.

UTF-8

UTF-8 is the most widely used text encoding today and represents every Unicode character. Almost every website, REST API, JSON document, database, and programming language uses it by default.

UTF-8 stores characters using 8-bit (1 byte) units. But it is variable-width — a character can take 1, 2, 3, or 4 bytes, depending on the code point. 4 bytes is the maximum allowed, and that's enough to hold every Unicode character.

• ASCII characters (U+0000 to U+007F) → 1 byte
• Most non-Latin scripts → 2–3 bytes
• Emoji and rarer symbols → 4 bytes

For example:

Notice something interesting — English characters occupy only one byte. That's because ASCII values only go up to 127, which fits inside 7 bits — and one UTF-8 unit is 8 bits. So every ASCII character fits in a single UTF-8 byte, with room to spare (1 extra bit left unused).

That is why an English-only text file looks identical whether it is saved as ASCII or UTF-8.

This backward compatibility is a big reason UTF-8 became the default for the web, JSON, most APIs, and Linux systems.

UTF-16

UTF-16 also represents every Unicode character. But it stores characters using 16-bit (2-byte) units, called code units. So: 1 code unit = 16 bits = 2 bytes.

Unicode's Basic Multilingual Plane — code points from U+0000 to U+FFFF (0 to 65,535 in decimal) — covers most everyday characters: English, and most other widely used scripts. All of these fit inside a single 16-bit code unit.

But 16 bits can only hold up to 65,535 values. Some code points go beyond that — like most emoji. 😀 is U+1F600, which is 128,512 in decimal — far past 65,535. It simply cannot fit inside one 16-bit code unit.

So UTF-16 stores it using two 16-bit code units together. This is called a surrogate pair.

Notice something — 2 code units means 2 x 16 bits = 32 bits = 4 bytes. That's the same maximum size UTF-8 uses for its biggest characters too.

So UTF-16 doesn't go bigger than UTF-8 to fit emoji. It just handles that size differently — UTF-8 flexibly grows one byte at a time up to 4, while UTF-16 jumps straight from 1 code unit to 2 (a full 4 bytes) with nothing in between.

UTF-16 is used as the internal in-memory string representation inside certain languages or platforms — like Java, JavaScript engines, and Windows.

UTF-8 vs UTF-16 — Same Character, Different Storage

Character -> Unicode Code Point -> Encoding -> Code Units -> Bytes in Memory

For 😀:

Same character. Same code point. Same total bytes. But a different number of code units — because UTF-8's code unit is 1 byte (8 bits), while UTF-16's code unit is 2 bytes (16 bits).

So Why Does .length() Give Different Answers for the Same Emoji?

Java's String and JavaScript's String are both built on UTF-16 internally. .length doesn't count "characters" — it counts UTF-16 code units.

Since 😀 needs a surrogate pair (2 code units), .length() returns 2.

Does Python also use UTF-16 internally?

No — Python does not store strings as UTF-16. Python counts Unicode code points directly (from the programmer's perspective), not UTF-16 code units like Java.

len("😀") returns 1, because 😀 is one Unicode code point (U+1F600).

Whereas in Java, "😀".length() returns 2, because Java counts UTF-16 code units, and this emoji needs two of them.

Neither is wrong — they're just counting two different things:

• Java counts UTF-16 code units.
• Python counts Unicode code points.

Bonus Example — The Flag Emoji

Here's a fun one that pushes what we just learned a step further: 🇮🇳

You'd assume it's one emoji, one code point, like 😀 was. It isn't.

A flag emoji is actually made of two code points — one for each letter of the country code. India's code is "IN," so 🇮🇳 is built from two special "Regional Indicator" characters: one representing I, one representing N. Your screen sees these two sitting together and renders them as one flag.

🇮🇳 = U+1F1EE (Regional Indicator I) + U+1F1F3 (Regional Indicator N)

Both of these code points are outside the BMP — same situation as 😀 — so each one needs a surrogate pair in UTF-16.

In Java / JavaScript: "🇮🇳".length() returns 4.

Two code points, each needing 2 UTF-16 code units → 2 + 2 = 4. .length() is just counting code units, same rule as before — it's just adding it up across two code points instead of one.

In Python: len("🇮🇳") returns 2.

Python counts code points, not code units — same rule as before too. But this time there genuinely are 2 code points (I + N), not one. So Python correctly reports 2.

UTF-32

We've seen two encodings so far — UTF-8, which flexes between 1 and 4 bytes, and UTF-16, which mostly uses 2 bytes but jumps to a surrogate pair when needed. There's a third one worth knowing: UTF-32.

UTF-32 stores every character using a fixed 32-bit (4-byte) unit — always, no exceptions.

There is no variable width and no surrogate pair. Every single code point, from A to 😀, takes exactly 4 bytes.

This makes UTF-32 simple to work with — the nth character always starts at a fixed, predictable byte position.

But it is expensive. Even plain English text, which needs just 1 byte per character in UTF-8, takes 4 bytes per character in UTF-32. This is why UTF-32 is rarely used for storage or transmission — mostly just internally, in some programming environments, for simpler processing.

Comparison Table

One Visible Character Doesn't Always Mean One Stored Character

This is the biggest takeaway from this article.

The character you see on your screen is not necessarily equal to the storage used internally.

The same Unicode character may occupy different amounts of memory depending on the encoding.

The character never changes. Only its representation changes.

Putting It All Together

After both parts, the complete picture looks like this.

Visible Character -> Unicode Code Point -> UTF Encoding -> Bytes Stored in Memory

For 😀:

😀 -> U+1F600 -> UTF-8 -> 4 bytes

😀 -> U+1F600 -> UTF-16 -> 2 code units (4 bytes)

Unicode tells us what the character is. UTF tells us how it is stored.

Conclusion

At first, text handling looks deceptively simple. But underneath, several layers work together:

ASCII -> Unicode -> Code Points -> UTF Encodings -> Bytes in Memory