encoding.js: shifting between Shift_JIS, EUC-JP and UTF-8 in JavaScript

The problem: a JavaScript string cannot hold Shift_JIS

encoding.js is a JavaScript library for converting and detecting character encodings, supporting Japanese character encodings, Shift_JIS, EUC-JP and ISO-2022-JP, and Unicode formats, UTF-8 and UTF-16.

The reason it exists is a language limitation, and the README states it precisely. Since JavaScript string values are internally encoded as UTF-16 code units, data in other encodings cannot be handled directly as JavaScript strings. The library works around this by representing encoded data as arrays of numeric code values instead of strings, which enables conversion between different character encodings.

The example given makes it concrete: the array [130, 160] represents the character in Shift_JIS. Those are the two bytes of a two-byte Shift_JIS sequence, and they are ordinary JavaScript numbers.

That choice has a second benefit that is easy to miss. Because the data is an array of numbers, it is a normal JavaScript value, so it can be passed around, sliced, compared and transferred without any string involved. And when you pass these arrays to encoding.js methods you can use regular arrays as well as typed arrays such as Uint8Array or Node.js Buffer objects, which means the same code handles a Buffer from a file read and a plain array from JSON.

One more detail shapes how the library is built. The source code of encoding.js itself is written using only ASCII characters, except for comments, which the README says makes it easy to work with when embedded in other code or used in environments with limited character support. That is a small constraint with a real payoff, since a library that has to be embedded in a page or a constrained toolchain cannot afford non-ASCII bytes in its own source.

Getting bytes back out of the numeric representation

The numeric array representation is the mechanism, but you often need a string, and the README is careful about the limits of that conversion.

Numeric arrays of character codes can be converted to strings using methods such as codeToString, and stringToCode goes the other way. But because of the JavaScript specification constraints, some character encodings may not be handled properly when converted directly to strings. That caveat is the important one, and it is why the library also offers two lossy-but-textual representations.

The first is percent-encoding. urlEncode and urlDecode convert numeric arrays to and from percent-encoded strings such as '%82%A0', which is the form a URL query string carries. The second is Base64, with base64Encode and base64Decode doing the same for Base64 strings.

So there are three ways to move data in and out of the library, and the choice depends on what the data has to survive. codeToString for text you will actually display, where the encoding has to be one JavaScript can represent. urlEncode for query parameters. base64Encode for binary payloads moving through a JSON API.

The API list in the table of contents makes the shape of the library clear, and it is worth noticing how much of it is about representation rather than about encoding. There is detect, convert, urlEncode, urlDecode, base64Encode, base64Decode, codeToString and stringToCode, plus a separate section for Japanese Zenkaku and Hankaku conversion, which is the half-width to full-width katakana transformation that turns up when text passes between legacy encodings and modern ones.

Installing the package and the two import styles

encoding.js is published on npm under the package name encoding-japanese, which does not match the repository name, and the installation is:

npm install --save encoding-japanese

The import style is up to you, and both are documented:

import Encoding from 'encoding-japanese';

const Encoding = require('encoding-japanese');

The TypeScript situation changed recently and the README is explicit about the migration. As of version 2.4.0, encoding.js includes TypeScript type definitions in index.d.ts, so no additional package is required, and after upgrading to v2.4.0 or later you can remove the separate types package if you had it installed:

npm uninstall @types/encoding-japanese

The definitions were originally maintained on DefinitelyTyped, and the README credits the two people who did that work. This is the pattern most of the JavaScript ecosystem has been converging on, shipping types in the package, and for a library like this one the type definitions matter more than usual, because the calling convention is unusual. A function that takes a string will not accept your byte array, and a function that takes a Buffer may also accept a Uint8Array or a plain array, and the type definitions are how a TypeScript user discovers which is which.

The package manifest backs this up. The main entry is src/index.js and the types field points at index.d.ts, and the files array ships the bundled encoding.js, the minified encoding.min.js, the type definitions and the src directory, so the sources travel with the package for anyone who wants to read them.

In a browser, from a CDN, and how the build works

For a browser without a bundler, the package includes both a full and a minified build, and the script tag is all you need:

<!-- To include the full version -->
<script src="encoding.js"></script>

<!-- Or, to include the minified version for production -->
<script src="encoding.min.js"></script>

When the script loads, the object Encoding is defined in the global scope, so window.Encoding. There is a warning about the git clone route: cloning the repository might give you the master branch, which could still be in a development state, so the release list is the safer source for a browser build.

The CDN route is the same file from a package CDN, and the README names unpkg, cdnjs and jsDelivr as the options:

<script src="https://unpkg.com/[email protected]/encoding.min.js"></script>

The build script explains what the bundle actually is. The compile step runs browserify over src/index.js with a standalone flag and a banner file, and the minify step runs uglifyjs with ascii_only set to true, which is the setting that keeps the output ASCII-safe in line with the source constraint. A watch script exists for development, and the test script builds first, then type-checks with tsc, then runs mocha.

The engine requirement is Node 18 or newer, and the repository is not archived with the last push on 2026-10-19. The release history shows steady work: 2.4.0 on 2026-09-14, 2.3.0 on 2026-08-30 and 2.2.0 from 2024-06-08. The licence is MIT, and the project keeps a Japanese README alongside the English one.

The options that decide what happens to characters you cannot represent

The convert method is the one with a real decision surface, and the table of contents lists four separate topics under it because there are four different answers to the same question.

First, there is the return type. You can ask for a specific type rather than the default. Then there is the handling for unrepresentable characters, and three documented strategies for it.

One is replacing characters with HTML entities when they cannot be represented. Another is ignoring them. The third is throwing an Error when they cannot be represented, which is the strict option and the one you want if a silent data loss is worse than a failed request.

The choice of default matters more than it might appear, because an encoding conversion is exactly the operation where characters go missing. Shift_JIS and EUC-JP do not cover the same characters as Unicode, and a document that mixes a modern symbol into legacy text will lose it somewhere. A library that drops such characters quietly produces output that looks right and is subtly wrong, which is harder to detect than a failure. That is why the explicit error mode is the one to prefer in a data pipeline, and why the other two exist for display contexts where a placeholder is better than a stack trace.

There is also a specific option for the byte order mark in UTF-16, which matters because a UTF-16 file with a BOM and one without it are read differently by different consumers, and this is one of the classic ways a text pipeline breaks across systems.

Where encoding.js is the wrong choice

The first limitation is the coverage. The supported set is the Japanese legacy encodings and the Unicode formats, and the topics list makes that explicit with shift-jis, cp932, euc-jp and jis alongside utf-8, utf-16, unicode and base64. If your data is Windows-1252, ISO-8859-1 or GBK, this library is not the tool, and a general iconv-based implementation or a native library is.

The second limitation is detection, and it is inherent to the problem. Detecting an encoding from bytes is a guess when the signal is weak, and the library's own README does not make strong claims about the accuracy of detect, which tells you something. Short strings are the hard case, because a handful of bytes may be valid in several encodings. If your input has a reliable marker, a BOM, a known header, a fixed length, detection is straightforward. If your input is a three character fragment of arbitrary user text, no library can be reliable there.

The third limitation is the ecosystem. Node has TextDecoder with the full ICU set of encodings, and modern browsers do too, and both handle UTF-16 and UTF-8 natively with no library at all. encoding.js earns its place for the legacy Japanese encodings that TextDecoder does not cover, and for the array-based representation that lets you move raw encoded bytes through code that would otherwise mangle them.

That is also the honest comparison with a native iconv binding. A native module gives you the whole encoding set and is usually faster, at the cost of a build step, a platform-specific binary, and a dependency your browser build has to shim. encoding.js is pure JavaScript with no native part, it runs the same in Node and in a page, and it is the only one of the two that gives you numeric code arrays as a first class value. If you need only UTF-8 and UTF-16, use the platform. If you need Shift_JIS in a browser without a bundler, this is the reason it exists.

Editorial conclusion

Adopt encoding.js if you handle Japanese legacy encodings in a JavaScript or TypeScript codebase, especially Shift_JIS and EUC-JP data arriving from older systems, and if you want to run the same code in Node and in a browser without a native dependency. Do not adopt it as a general text encoding library, since its coverage is Japanese encodings plus UTF-8 and UTF-16 rather than the full range of legacy code pages, and do not expect it to guess reliably for short or ambiguous input. Before you rely on detection, test it on the actual byte sequences your system produces, and read the sections on unrepresentable characters, because a lossy conversion is the failure mode you will hit first.

Frequently asked questions

Official sources

1. Issues
2. polygonplanet/encoding.js on GitHub
3. README
4. Releases