Text encoding convert
Convert text between character encodings without silent data loss.
Working…
Questions people ask
- Will this tool silently mangle my characters?
- That is precisely what it refuses to do. Under the default strict policy, a character the target cannot represent stops the run with the offset and codepoint named (U+20AC and so on) and NOTHING is written. The numeric-refs option is the only substitute, and it is explicit and reversible: every gap becomes a &#xHHHH; reference in the output.
- How does auto decide the source encoding?
- Proofs first: a byte-order mark or a strictly valid UTF-8 input is used directly. Otherwise the detector is consulted, and its top candidate is trusted only with clean quality signals (zero erroneous-character ratio, high language coherence). An uncertain reading is refused with the detector's proposals quoted, never silently trusted. Declare the encoding explicitly when auto declines.
- What happens to byte-order marks?
- Explicitly per choice: a UTF-8 BOM in front of an explicit utf-8 source is stripped before decoding (a BOM is a container mark, not text); utf-16/utf-32 sources consume their own; the utf-8-sig target writes one and plain utf-8 never does. BOM-aware sources (utf-16) still require the mark; a missing BOM is a loud refusal, not a guess.
- Are line endings preserved?
- Yes. The conversion works on bytes and codecs, and neither decoding nor encoding rewrites line termination: CRLF, LF and CR all pass through byte-for-byte.