Fix CSV Encoding

“Müller” instead of “Müller”, “’” instead of an apostrophe, boxes or question marks where Cyrillic or Chinese should be: garbled characters are the most common CSV problem after wrong separators. They come from text encodings, the rules that turn letters into bytes. This tool reads the file with the right encoding, repairs text that was already damaged, and saves a clean UTF-8 file.

Two different problems

Sometimes the file is fine but was read with the wrong encoding: a Windows-1252 export opened as UTF-8, or the other way round. Choosing the right encoding fixes it completely. In other cases the damage is inside the file: UTF-8 text was once read as Windows-1252 and saved again, so the garbled characters are now stored as real text. That needs a repair. This tool handles both.

Automatic detection

The tool looks for a byte order mark first, which identifies UTF-8 and UTF-16 with certainty. Without one, it recognizes UTF-16 from its pattern of zero bytes, accepts the file as UTF-8 if every byte sequence is valid, and otherwise reads it as Windows-1252, the most common legacy encoding in Western Europe and the Americas. The detected encoding is shown under the file name and in the summary.

Choosing another encoding

Detection cannot tell legacy encodings apart reliably, because they all use the same byte range. For Central European, Cyrillic, Greek, Turkish, Arabic, Japanese, Chinese or Korean files, open the encoding list under the file name and pick the right one while watching the preview. The list of special characters found in the file helps: when it shows the letters you expect, the choice is right.

Repairing double-encoded text

“Repair mixed-up characters” looks for the typical patterns left when UTF-8 was misread as Windows-1252, such as ü for ü, é for é and – for a dash, turns them back into bytes and decodes them correctly. Each run is checked: if it does not form valid UTF-8, it is left alone, so genuine text like “Ångström” is never changed. Repaired cells are highlighted in the preview and counted.

Saving clean UTF-8

The download is UTF-8 with the original separator, so the file still imports wherever it did before. Add a byte order mark for Excel, which otherwise may misread UTF-8 when the file is opened by double-click; leave it out for databases and scripts. Choose Windows or Unix line endings to suit the destination.

Prevent it next time

When exporting from a system, choose “UTF-8” or “Unicode” if offered. In Excel, use “CSV UTF-8 (comma delimited)” in the Save As dialog instead of plain “CSV”.

UTF-16 files from Excel

Excel’s “Unicode Text” export writes UTF-16 with tabs between values. Many programs reject it. Drop such a file here: the encoding is recognized from its byte order mark and the tab separator is detected, and the download is a regular UTF-8 file that keeps tabs as the separator.

Related tools

If the problem is columns rather than characters, change the separator with Change CSV Delimiter. To open the repaired file as a workbook, use CSV to Excel. For cleaning stray spaces and invisible characters in plain text, try the Text Cleaner.