Why QR Codes Mangle Accents, Emoji and the Euro Sign

There is a thread on the Adobe support forums, and others almost identical to it on the ODK forum and CreativePro, that all describe the same thing. Someone generates a QR code containing a name, a menu item or an address. It scans. The text comes back as café instead of café, or as Pras?s Bakery instead of Pras’s Bakery. The advice that follows is always some mixture of "try a different generator", "change the vCard version" and "modern scanners handle UTF-8 now".

That last one is the received wisdom, and it is the part that is wrong. Not because scanners are bad at UTF-8 — most are fine at it — but because nothing inside an ordinary QR code says the text is UTF-8. The reader is guessing, and there is a documented, checkable reason why.

This article explains what actually happens to a non-ASCII character between your keyboard and someone else's phone, why the euro sign fails when the pound sign does not, and how to tell in about five seconds whether a broken code is the generator's fault or the reader's — because the fix is completely different depending on the answer.

The Short Answer

• QR codes store bytes, not characters. Turning bytes back into text needs a character set, and the code usually does not name one.
• ISO/IEC 18004 makes the default for byte mode ISO/IEC 8859-1 — as implemented, one byte per code point for U+0000 to U+00FF and nothing above.
• Characters above U+00FF are destroyed when the code is generated, typically replaced with a question mark. No scanner can recover them.
• Characters at or below U+00FF survive, but can still be misread, which is where é comes from.
• The marker that would remove the ambiguity is called ECI, and assignment 26 means UTF-8. Most generators emit it only when the calling code explicitly asks for a character set.
• So the practical test is not "is this Unicode" but "is every character at or below U+00FF".
• The characters that break most often are the ones your word processor inserts for you: curly quotes, en and em dashes, the ellipsis, and the euro sign.

Two failures that look the same and are not

Almost every "special characters break my QR code" report is one of two distinct faults. They produce different symptoms, they happen at opposite ends of the process, and only one of them is fixable by changing scanners.

This is the single most useful distinction in the whole subject, and it is almost never drawn. A question mark means the data is gone and the print run is wasted. Gibberish means the data is intact and you have a display problem. Telling them apart takes one scan — you just have to look at the raw decoded text rather than at whatever your phone decides to do with it.

Why the default is a character set from 1987

QR codes have four ways of packing data: numeric, alphanumeric, byte and kanji. Anything that is not purely digits or purely capitals-and-a-few-symbols ends up in byte mode, which stores raw 8-bit values with no opinion about what they mean.

ISO/IEC 18004 has to say something about how those bytes become text, and what it says is that the default interpretation is ISO/IEC 8859-1 — Latin-1. That set was published in 1987. In the form every library implements, it maps each byte value straight onto the matching code point, so it covers U+0000 through U+00FF and stops.

You can see this in the source of the reference implementation. ZXing, which sits underneath a very large share of Android scanning and an enormous amount of server-side generation, declares it in one line in its QR encoder:

static final Charset DEFAULT_BYTE_MODE_ENCODING = StandardCharsets.ISO_8859_1;

Text goes through that charset unless the caller says otherwise. And the standard behaviour when a Java charset encoder meets a character it cannot map is to substitute a replacement character, which for ISO-8859-1 is the question mark. That is where the question marks come from. They are written into the symbol at generation time, permanently, and the resulting code scans perfectly — it just contains the wrong text.

The euro test

Here is the line, and it is sharper than most people expect. The pound sign is U+00A3. The euro sign is U+20AC. Latin-1 was finalised more than a decade before the euro existed, so there is no byte for it. A price list in sterling survives a Latin-1 generator untouched; the same list in euros comes back as ?18,50.

The same logic sorts every other character cleanly, and the result is counter-intuitive. The dusty-looking symbols from the typewriter era are all safe. The modern typographic ones are all fatal.

Two consequences worth stating plainly. First, French is not quite safe: every accent it uses is inside Latin-1, but œ is not, so cœur and sœur break while crème brûlée does not. Second, Polish, Czech, Hungarian, Romanian, Turkish, Greek and every non-Latin script are entirely outside the set. There is no partial success available there; a Latin-1 generator will hollow out the whole string.

You can check any specific string against this in the QR code character encoding checker, which lists every non-ASCII character you have used, its code point, and whether it survives.

Why "just use UTF-8" does not finish the job

Suppose your generator does support UTF-8. It writes é as the two bytes C3 A9 instead of the single byte E9. Nothing is lost. So why does anything still go wrong?

Because the symbol still does not say so. Byte mode is a sequence of 8-bit values with a default meaning of Latin-1, and writing UTF-8 bytes into it does not change the default. The mechanism that would change it is ECI — Extended Channel Interpretation — an optional marker placed in front of the data declaring which character set follows. Assignment number 26 is UTF-8.

ECI has existed for as long as most people have been making QR codes. The problem is when it gets emitted. In ZXing's encoder, an ECI segment is appended only when three things are true at once: the mode is byte, the caller supplied a character-set hint, and that character set differs from the ISO-8859-1 default. A caller who simply asks for a QR code of some text — which is what the overwhelming majority of integrations do — satisfies none of that. The result is a symbol full of UTF-8 bytes with nothing anywhere inside it saying they are UTF-8.

This is the honest version of "modern scanners handle UTF-8". They do handle it. They just have not been told that is what they are looking at.

What a reader actually does with an unlabelled code

You might expect a reader meeting byte-mode data with no ECI to do what the standard says and decode it as Latin-1. In practice, essentially none of them do, because that would mangle the very large number of real codes in the world that carry unlabelled UTF-8. Instead they guess.

ZXing's guessing routine, in its StringUtils class, works down a list. It checks for a UTF-16 byte-order mark. It checks whether the bytes form valid UTF-8 multi-byte sequences. It checks for Shift_JIS, including a heuristic about runs of katakana. It falls to ISO-8859-1 by elimination when that is the only remaining candidate. And if every test is inconclusive, it returns Charset.defaultCharset() — the character set of the device doing the reading.

That final fallback deserves more attention than it gets. It means the text that comes out of a QR code can depend on the locale settings of the phone in someone's hand. Two people can scan the same printed code, in the same room, and see different words. It also means "it works on my phone" is not evidence that a code is correct — it is evidence that your phone guessed the way you hoped.

Practice varies between implementations here, and other stacks make other choices: iOS, Android's ML Kit and desktop reader software all differ in their heuristics and in whether they honour ECI at all. That variation is precisely the problem. An unlabelled code is not decoded, it is interpreted, and different products interpret differently.

Where the bad characters actually come from

Almost nobody types a curly apostrophe. They type a straight one, and something replaces it.

Microsoft Word, Google Docs, Apple Pages and Adobe InDesign all apply smart-quote substitution by default. The straight apostrophe ' (U+0027, safely inside ASCII) becomes ’ (U+2019, outside Latin-1). Two hyphens become an em dash. Three dots become an ellipsis. Straight quotes become curly ones. Every one of those substitutions moves a character from the safe group to the fatal group, and none of them is visible unless you are looking for it.

This explains a pattern that otherwise looks inexplicable: text that worked when the developer tested it stops working after the copywriter has been through it. Nothing about the QR pipeline changed. The apostrophe did.

The practical rule is to check the string after it has been through every document it is going to pass through, not before, and to paste rather than retype it into the generator.

What the characters cost you in space

Even when a character survives, it is not free. Byte mode charges 8 bits per byte, not per character, and UTF-8 spends more than one byte on everything above ASCII.

An emoji costs as much room as four Latin letters. Two Chinese characters cost as much as six. On a short payload that can be enough to push the symbol up a version, which means more modules in the same printed area, which means each module is smaller and the code is harder to scan — the subject of the QR code size guide. The capacity calculator shows where the version boundaries fall.

The same arithmetic works in your favour in one specific case. Alphanumeric mode covers 45 characters — the ten digits, capital A to Z, space, and $ % * + - . / : — and packs two of them into 11 bits, or 5.5 bits each. HELLO WORLD needs 61 data bits; hello world needs 88. It is all or nothing, though: one lower-case letter, comma or apostrophe anywhere in the string drops the whole thing back to byte mode.

Diagnosing a broken code in four steps

1. Read the raw decoded text, not the action. A scanner that opens the link immediately hides the evidence. You need to see the characters that came out.
2. Look for question marks. If the text contains ? where a character should be, the code is dead. The character was replaced at generation and no reader can bring it back. Regenerate.
3. Look for gibberish letters instead. Sequences like é, ü or ’ mean the bytes are intact and the reader chose the wrong character set. The code will work correctly elsewhere; whether that is good enough depends on who is scanning it.
4. If it is a URL, sidestep the whole problem. Percent-encode the non-ASCII parts of the path and query, and use punycode for an internationalised domain. A percent-encoded URL is pure ASCII, and pure ASCII is identical in every character set involved. The URL encoder will do the conversion.

Step one is the one people skip, and it is the one that decides everything else. A phone that silently opens a link, or silently joins a network, never shows you what it read. Scan & Generate displays the decoded text first, which is what makes the question-mark-versus-gibberish test possible at all — and it is the same habit that catches a malicious link before it opens.

Practical rules by field

The right approach depends on what the code carries, because some payloads have an ASCII escape hatch and some do not.

Frequently Asked Questions

Why does my QR code show é instead of é?

Your generator wrote the character as its two UTF-8 bytes, C3 and A9, and the reader interpreted those bytes one at a time as ISO/IEC 8859-1, where they mean à and ©. The data in the code is completely correct — only the interpretation is wrong. The same code may well read correctly on another phone, because readers guess the character set independently.

What does a question mark in the scanned text mean?

It means the character was destroyed when the code was generated rather than when it was read. An encoder falling back to ISO/IEC 8859-1 substitutes a question mark for anything it cannot map, and that substitution is permanent: the original character is not stored anywhere in the symbol. No scanner, setting or app can recover it. The code has to be regenerated.

Why does the euro sign break when the pound sign works?

ISO/IEC 8859-1 dates from 1987 and covers code points U+0000 to U+00FF. The pound sign is U+00A3 and sits inside that range. The euro sign is U+20AC, added to Unicode long afterwards, and has no byte in Latin-1 at all. It is the clearest illustration that the dividing line is a numeric one about code points, not a judgement about how exotic a character looks.

Can a QR code contain emoji?

Physically yes, reliably no. Byte mode will carry any byte value, and an emoji is four UTF-8 bytes. But emoji sit far outside ISO/IEC 8859-1, so a generator without explicit UTF-8 support replaces each one with a question mark, and a reader that guesses wrongly shows several stray symbols instead. They also cost as much space as four Latin letters. For anything printed, put emoji on the poster around the code rather than in the data inside it.

What is ECI and should I care?

ECI stands for Extended Channel Interpretation. It is an optional marker a generator can place in front of the data to declare which character set the bytes use, and assignment number 26 means UTF-8. You should care because it is the only thing that removes the guesswork. If your content is not pure ASCII, the question to ask a generator is not whether it supports Unicode but whether it emits an ECI marker — many only do so when the calling code explicitly names a character set.

Why does the same code read differently on two phones?

Because with no ECI marker the reader has to infer the character set, and different readers infer differently. ZXing checks for a byte-order mark, then valid UTF-8 sequences, then Shift_JIS, then settles on ISO-8859-1 by elimination — and where all of those are inconclusive it falls back to the default character set of the device itself. A code whose interpretation depends on the reader's locale is not a broken code so much as an underspecified one.

Why did my QR code stop working after the copy was edited?

Almost certainly smart quotes. Word, Google Docs, Pages and InDesign automatically replace the straight apostrophe with a curly one, double hyphens with an em dash and three dots with an ellipsis. The originals are ASCII and perfectly safe; the replacements are all above U+00FF and all break. Nothing in the QR pipeline changed — the text did, invisibly, on its way through a document.

Does using only capital letters make a QR code smaller?

Yes, when every character qualifies. Alphanumeric mode covers 45 characters — the digits, capital A to Z, space and $ % * + - . / : — and encodes them at 5.5 bits each instead of byte mode's 8. HELLO WORLD needs 61 data bits against 88 for hello world. The catch is that it is all or nothing: a single lower-case letter, comma or apostrophe puts the whole string back into byte mode.

The Short Version

A QR code stores bytes and only weakly implies what they mean. The default meaning is a character set finalised in 1987 that stops at U+00FF, so the real question about any string is not whether it is Unicode but whether every character sits at or below that line. Above it, characters are destroyed at generation and come back as question marks. At or below it, they survive but may be misinterpreted and come back as gibberish. An ECI marker is what closes the gap, and most generators only emit one when they are explicitly asked to.

The characters most likely to bite you are not the obviously foreign ones. They are the curly apostrophe, the em dash and the euro sign — three characters that your own software inserts on your behalf, in text you believed was plain.