Pinyin sort order

The standard for alphabetically sorting Hanyu Pinyin is given in the ABC dictionary series edited by John DeFrancis and issued by the University of Hawaii Press.

Here’s the basic idea:

The ordering is primarily simply alphabetical. Diacritical marks, punctuation, juncture and capitalization are only taken into account when the strings being compared are otherwise identical. For example, píng’?n sorts before p?ny?n, because pingan sorts before pinyin, because g precedes y alphabetically.

Only when two strings are alphabetically identical is non-alphabetical information taken into account.

The series’ Reader’s Guide presents the specifics of the sort order. Since I don’t have to worry about how much space this takes up on my site, I have reformatted the information slightly to give the examples as numbered lists.

Head entry transcriptions with the same sequence of letters are ordered first strictly by letter sequence regardless of tones, then by initial syllable tone in the sequence 0 1 2 3 4. For entries with the same initial tone, arrangement is by the tone of the second syllable, again in the order 0 1 2 3 4. For example:

  1. sh?shi
  2. sh?sh?
  3. sh?shí
  4. sh?sh?
  5. sh?shì
  6. shísh?
  7. shíshì
  8. sh?sh?
  9. shìsh?

Irrespective of tones, entries with the vowel u precede those with ü.
For example:

  1. l?
  2. l?
  3. l?
  4. l?
  1. n?

Entries without apostrophe precede those with apostrophe. For example:

  1. biànargue
  2. b?’ànthe other shore

Lower-case entries precede upper-case entries. For example:

  1. hòujìnaftereffect
  2. Hòu JìnLater Jin dynasty

For entries with identical spelling, including tones, arrangement is by order of frequency….

For most users, the most important thing to note is that the neutral tone is regarded as 0, not as 5. Thus, the order is not? á ? à a,” but “a ? á ? à.” And, because lowercase comes before uppercase, notA a ? ? Á á ? ? À à” but “a A ? ? á Á ? ? à À.

One can see this in action in the A entries for the ABC English-Chinese, Chinese-English Dictionary. And here are some sample pages from an earlier ABC dictionary.

The ABC series follows the example of the Hanyu Pinyin Cihui (?????? / Hàny? P?ny?n Cíhuì) (example), with only one minor difference, as noted by Tom Bishop:

HPC [Hanyu Pinyin Cihui] gave hyphens and spaces the same priority as apostrophes, so that lìg?ng sorted before l?-g?ng, in spite of the tones. Usage of hyphens and spaces in pinyin is still far from being fully standardized. (The same is true in English orthography.) Consequently, for collation it makes sense to give less weight to hyphens and spaces, and more weight to tones, thus sorting l?-g?ng before lìg?ng. In ABC, hyphens and spaces don’t affect the sort order unless they change the pronunciation in the same way that apostrophe would; for example, 1míng-àn ?? and 2míng’àn ?? are treated as homophones, and they sort after m?ng?n ??.

Pinyin font: the Brill

Some of the Pinyin-friendly font families I provide examples of on this blog are fun but not exactly the sort of thing you’d want to use in a book or other serious project. Others, though, are solid examples of the subtle and exacting art of type design. Today’s entry belongs in the latter group.

Brill — a Leiden-based publisher of work in the humanities, social sciences, law, and science — has released “the Brill,” a new font family designed to support the Latin and Greek scripts “to the fullest extent possible.” IPA and the Slavic parts of the Cyrillic range are also covered. This can handle the needs of just about any romanized script, including Hanyu Pinyin.

As someone with Brill explained to me:

Instead of limiting the fonts’ character set to known characters and character-plus-diacritic combinations, we chose a dynamic model in which, using OpenType GPOS features, any base character can carry any diacritic above or below it, and in which diacritics can be stacked as well—not forgetting all the precomposed characters that are already present in the Unicode Standard, of course. Finally, a huge assortment of punctuation marks, editorial marks, and other symbols known to occur in Brill publications were added to the spec.

In total, the Brill contains more than 5,100 characters. And that already immense range can be extended through combining diacritics, as noted above.

Even better, the Brill is free for non-commercial use. You can download it after agreeing to the End User License Agreement license. (See the bottom of that page and then the bottom of the page that follows.)

The Brill is available now in roman and italic styles. Bold and bold italic versions will be released later this year, probably before July.

The Brill is considerably different than Brill Online, which has been available for some time and was aimed at helping users of Brill’s online reference works. Brill Online is based on v. 1.00 of the Gentium family of fonts. The glyph set was extended to support some very rare characters, such as Aegean numbers. “In essence it became a hybrid Latin-Greek-Cyrillic-IPA and ‘pi’ font family.”

Thanks to Lin Ai of Zhongweb.net for the heads up that this had been released, and to Dominique de Roo and Pim Rietbroek of Brill for patiently helping me with my questions.

Pinyin font: Linux Biolinum

The highly useful and Pinyin-friendly Linux Libertine has a companion font family: Linux Biolinum.

Biolinum is designed for emphasis, e.g. of titles. You can also use it for short passages of text. For longer texts a serif font such as the Libertine should be used for readability. The Biolinum has the same vertical metrics and visual weight as the Libertine, so that it fits perfectly to the Libertine and can be also used for emphasizing within the body text.

Linux Biolinum Capitals and Linux Biolinum Keyboard don’t presently work with Pinyin. But the other styles do, as this sample of Linux Biolinum with Pinyin text shows.

Pinyin Dongwuyuan: an illustrated Pinyin alphabet

Here’s a new book I made for fun: P?ny?n Dòngwùyuán (4.3 MB PDF).

It goes through the letters of the alphabet: A is for ?nchun, B is for b?nm?, C is for chángj?nglù, etc., all the way through Z, which is for zh?ngyú.

But X is not for xióngm?o. I’m sick of pandas. Let’s let some other animals have some time in the spotlight.

Although technically speaking the Pinyin alphabet is the same as that for English, I prefer to go with A–Z, minus V but plus Ü.

O and R were the tricky ones to find animals for.

Perhaps some teachers will print this out and hang it up in their classrooms. Or kids could use it as a coloring book. You have my permission to do just about anything you like with this — other than add Chinese characters. (The world already has plenty of material in Hanzi, but not nearly enough in Pinyin.)

I made sure to include multiples of some common morphemes (e.g., b?nm?, h?im?, and m?; è and zh?ng; h?im? and h?i’?u; niú, w?niú, and x?niú), which I hope will be useful.

For fonts, I used the Linux Libertine family.

This took me far longer to make than I thought it would, so I hope some people enjoy it or at least find it interesting.

Pinyin font: Linux Libertine

Linux Libertine in Wikipedia logoLinux Libertine is perhaps most familiar as the font used in the Wikipedia logo. This surprisingly large font family also works well with Hanyu Pinyin, though a few adjustments need to be made before all of the fonts in this family work as they should with Pinyin texts.

Here’s how those working on Linux Libertine describe it:

We work on a versatile font family. It is designed to give you an alternative for fonts like T*mes New Roman. We’re creating free software and publish our fonts under terms of the GPL and OFL. Please have a look at the paragraph concerning the license.

It is our aim to support the many western languages and provide many special characters. Our fonts cover the codepages of Western Latin, Greek, Cyrillic (with their specific enhancements), Hebrew, IPA and many more. Furthermore, typographical features such as ligatures, small capitals, different number styles, scientific symbols, etc. are implemented in this font. Linux Libertine thus contains more than 2000 characters.

Here’s what it looks like with Pinyin. (Click to view a PDF, which is much clearer.)
screenshot of Linux Libertine in action on Pinyin text

image of a rhinocerous (xiniu) and the word 'xiniu' in Linux Libertine

All in all: Not bad.

Some fonts *not* to use for Pinyin

One of the traditions in advance of Chinese New Year is housecleaning — something not among my favorite activities. But I thought I’d do a bit of housecleaning of half-finished posts and get at least one up before the new year (tomorrow). So here it is.

Although I occasionally bemoan the fact that relatively few font families are made such that they can handle Hanyu Pinyin with tone marks (at least not right out of the box), it’s worth noting that some of the commonly found fonts that do cover all of the letters and diacritics really suck at it and should be avoided when writing in Pinyin.

Typically, such fonts were designed mainly with Hanzi in mind.

Here’s one example:
screenshot of a Pinyin text set in Adobe Ming -- and, boy oh boy, is it ever hideous

Hideous.

That was Adobe Ming. Yes, Adobe.

I’ll go ahead and point out the obvious problems:

And I’m not so sure about the consistency of the x-height either. Those stubby little descenders are puzzling, too, but are not necessarily wrong.

Perhaps the designers intended these letters for use in vertically aligned text — though I don’t think these forms would work well even then. Perhaps there’s some context in which these might make sense, though I’m inclined to doubt this. Perhaps the designers have an irrational hatred of romanization and wanted to make Pinyin look as ugly as possible. Whatever the reason, even though this and the other Unicode-compliant fonts below have all of the letters with diacritics that Pinyin requires, using them for Pinyin texts would be a very bad idea.

Since there is apparently still some confusion about why the “?” form (in contrast to the normal “a” form) is incorrect, see the chart below.

table showing that the fonts discussed in this post that use the rounded style for the letter 'a' do so only with diacritics, not elsewhere. This is wrong. The rounded a's should not be used at all.

Note how the odd form of the letter a does not appear in regular text or even in double-width forms; instead, it’s seen only when accompanied by a tone mark. In other words, even within individual fonts the ? form is treated not as a normal “a” that happens to look that way but as something specifically for Pinyin, which is flat-out wrong. Other than the addition of diacritics themselves, there is no reason to alter letter shapes in any way for Pinyin.

Let’s get back to the broader issue. Here are some more examples of fonts that render Pinyin in ugly ways. (Click image to view PDF.)

click to view PDF with much larger and clearer text

To aid Web searches, here’s a text list of the fonts above, none of which should be used for Hanyu Pinyin:

  • Adobe Fangsong Std
  • Adobe Heiti Std
  • Adobe Kaiti Std
  • Adobe Ming Std
  • Adobe Song Std
  • MS Gothic
  • MS Mincho
  • MS PGothic
  • MS PMincho
  • MS UI Gothic
  • NSimSun
  • SimHei
  • SimSun

SimSun is probably the least awful of the bunch. But even so, there’s no good reason to use it instead of something else that would do the job much better, such as Gentium:
screenshot of the same Pinyin text, but this one is set in Gentium -- and it looks great

Generally speaking, if you wouldn’t want to use a font for English, French, Italian, etc., then don’t use it for Hanyu Pinyin.

Say no to making Pinyin ugly!

I wish you all a happy and P?ny?n-rich year of the dragon.