What Is Character Encoding in Spatial Data?

Character encoding is the rule used to convert text characters into bytes and back again. In spatial data, an encoding problem can corrupt names such as São Tomé, Yaoundé or non-Latin scripts while the coordinates and geometry remain completely valid.

That is why an encoding failure can look strangely selective: the map draws in the correct place, but labels, categories or geographic identifiers are unreadable.

Why mojibake appears

Text becomes garbled when bytes written under one encoding are decoded under another. A sequence intended to represent é in UTF-8, for example, can be interpreted as multiple characters under a legacy single-byte encoding, producing familiar strings such as é; replacement characters such as � can appear when the decoder encounters a byte sequence it cannot interpret.

The important distinction is between the stored bytes and the interpretation applied to them. If the original bytes are still intact, choosing the correct encoding may recover the text losslessly. If a previous export already replaced unknown characters with ?, some information may be permanently lost.

UTF-8 is a common modern default

UTF-8 is a byte-oriented Unicode encoding that can represent the full Unicode character repertoire. It has become the dominant interchange encoding for web and many modern data formats.

That does not mean every legacy GIS file is UTF-8, since older shapefiles, CSV exports and database extracts may use Windows-1252, ISO-8859 variants or regional encodings. The file format may also provide weak or external encoding metadata.

Modern formats that explicitly use Unicode or carry encoding information reduce ambiguity, but conversion tools can still mis-handle text if the source encoding was guessed incorrectly.

Why the problem matters geographically

Encoding errors are not merely cosmetic when text fields carry geographic meaning.

They can break:

• place-name labels;

• joins on text identifiers;

• multilingual names;

• category filters;

• address matching;

• gazetteer searches;

• metadata and source citations.

A country name that renders incorrectly may fail to join to a statistical table even though the visible corruption looks minor, while a damaged local-language name can erase information that cannot be reconstructed from an English label.

Diagnose before re-saving

If you see garbled text, preserve the original file first. Re-saving a wrongly decoded dataset can overwrite recoverable bytes with corrupted Unicode text.

Then inspect:

1. the file format and any encoding metadata;

2. documentation from the data producer;

3. the same text in a known reference source;

4. whether the corruption pattern is consistent with a particular encoding mismatch;

5. whether raw bytes are still available.

If one program displays the text correctly and another does not, compare their detected or configured encodings.

Format-specific considerations

CSV files do not inherently carry a universal, reliable encoding declaration in the way a database schema might, so tool defaults matter.

Shapefile attribute text is stored in DBF files, and encoding may be described using a .cpg file or legacy code-page information. Missing or ignored metadata can produce inconsistent results across software.

GeoJSON is defined in terms of Unicode and uses UTF-8 for interchange, which removes much of the ambiguity at the format level. Converting a damaged legacy source to GeoJSON, however, does not magically restore characters already decoded incorrectly.

Normalisation is different from encoding

Two Unicode strings can look identical while using different underlying sequences—for example, a precomposed accented character versus a base character followed by a combining mark.

That is a Unicode normalisation issue rather than a character-encoding issue, although it can still affect exact string comparisons and joins.

If labels look correct but matching fails unexpectedly, inspect whitespace, case and Unicode normalisation rather than assuming the encoding is wrong.

Recovery should preserve geography and raw text

A careful repair workflow is:

1. keep an untouched copy of the source;

2. identify the likely source encoding from documentation or known text;

3. decode from that encoding once;

4. convert the result to UTF-8 or another chosen Unicode encoding;

5. verify representative names across languages and symbols;

6. rerun joins or filters that depend on the repaired fields;

7. record the conversion in provenance metadata.

Do not repeatedly try encodings and save over the same file. Each incorrect round trip can make recovery harder.

Coordinates can survive while names break

Geometry coordinates are numeric, so an encoding mismatch can leave spatial placement untouched while corrupting the human meaning attached to features. That selective failure is exactly why encoding belongs in spatial-data quality checks.

If the map is in the right place but the names are wrong, investigate the text representation before changing anything geographic.

References

1. IETF RFC 3629 — UTF-8. Defines UTF-8 for Internet interchange.

2. Unicode FAQ — UTF-8, UTF-16, UTF-32 & BOM. Explains Unicode encoding forms and UTF-8 behaviour.

Related content

• Why Are Leading Zeros Missing from Geographic Codes? — identifier formatting

• What Is a Geographic Identifier? — identifiers versus labels

• Why Place Names Are Ambiguous — names as geographic evidence