ChatGPT writes poetry in Pinyin, sort of

I asked ChatGPT to “write a poem in Mandarin using Pinyin.”

Its first response took the form of a sonnet:

qīngchūn yǒu tài duō de mèngxiǎng,
xīwàng yǒu yīgè xiǎng zhù de dìfāng.
yīgè rén zài huáshān shàng,
tā de mèng yǒu tèbié de chàng.

zài fēngláng zhī xiàn,
tā yòng bàozhǐ zhànwàng huāliàn.
yǒngqì bǎ dōngfēng zài shǒu,
zài yùndòng zhōng fàng sīwáng zhī hòu.

mèngzhōng yǒu tài duō de jìhuà,
yǒnggǎn bǎ tāmen dōu chéngshí huà.
yǒngyuan bùyào bèi shìjiè suǒ bǎozhòng,
zhǐyào huílái bùwàng jiùshì yīzhǒng.

qīngchūn yǒu tài duō de mèngxiǎng,
wǒmen yīqǐ xiǎng zhù zhè yīqiè de dìfāng.

That one won’t win any prizes, but the result was still much better than the one produced when I asked ChatGPT to make another:

Sì jiāo qíng yuán, xìng fú suì suì,
Chūn fēng yè sè, ruò dàng huā kāi.
Dōu shì shuāng shǒu, shuāng shǒu yī,
Xīn lián xīn shì, shuāng xīn xī.

Qīng sè yǔ tiān, yǔ zhuǎn yǔ duō,
Cǎo yuǎn shān jǐn, dōu xīng huā tuō.
Yuè liàng guāng míng, míng yǔ ruò,
Qíng qíng yǔ hǎo, yǔ zhōng yǔ.

Shēn tián yǔ sǎng, qíng hǎo yǔ,
Liàng yǔ fēn fēn, shǎn chéng yǔ.
Qīng chuān huáng hé, yǔ yíng cháo,
Chūn sè xīn róu, xìng fú mǎo.

Wēn huā yǔ chūn, yǔ xǐ xīn,
Sān yuán sì jì, yǔ zhōng shuāng.
Chūn tiān xìng fú, zhuǎn yǔ shuāng,
Qíng huā yǔ jīng, yǔ yīng huáng.

That opaque mo no syl la bic mess is infected by the spirit of Literary Sinitic, which Pinyin isn’t designed to handle.

I’m hopeful that ChatGPT will get trained properly on the rules of Hanyu Pinyin — both the short-form rules of Pinyin and the longer (and superior) work of Yin Binyong — and so will be able to produce proper Pinyin. But maybe wǒ yǒu tài duō de mèngxiǎng.

As one of my heroes, Hu Shih, wrote one hundred years ago,

Nǐ bùnéng zuò wǒ de shī,
zhèngrú wǒ bùnéng zuò nǐ de mèng

(你不能做我的詩,
正如我不能做你的夢)

NB: I should probably remind everyone, should you wish to include Chinese characters or Pinyin with tone marks in a comment, be sure to encode them first or they’ll end up scrambled here. (Not my fault. Sorry.)

Microsoft Translator and Pinyin

screenshot of the text described in the post, as treated by Microsoft Translator

If supplied with the following,

談中國的“語”和“文”的問題,我覺得最好能先了解一下在中國通用的語言。中國的主要語言有哪些?為甚麼我說這個,而不說那個?因為環境?因為被強迫?因為我愛這個語言?因為有必要?因為這個語言很重要?也想想什麼是中國人的共同語言。用一個共同語言有必要嗎?為什麼?別的漢語的去向會怎麼樣?如果你使用中國的共同語言普通話,你了解這個語言的語法(比如“的, 得, 地“ 和“了” 的不同用法)嗎? 知道這個語言的基本音節(不包括聲調)只有408個嗎?

Microsoft Translator produces the following Hanyu Pinyin:

tán zhōngguóde “yǔ” hé “wén”dewèntí, wǒjuéde zuìhǎo néng xiānliǎojiě yì xiàzài zhōngguó tōngyòng de yǔyán。 zhōngguóde zhǔyào yǔyán yǒu nǎxiē? wèishénme wǒshuō zhège ,érbùshuōnàgè? yīnwéi huánjìng? yīnwéi bèi qiǎngpò? yīnwéi wǒài zhège yǔyán? yīnwéi yǒubìyào? yīnwéi zhège yǔyán hěnzhòngyào? yě xiǎngxiǎng shénmeshì zhōngguórén de gòngtóngyǔyán。 yòng yígè gòngtóngyǔyán yǒubìyào ma? wèishénme? biéde hànyǔ de qùxiàng huì zěnmeyàng? rúguǒnǐ shǐyòng zhōngguóde gòngtóngyǔyán pǔtōnghuà , nǐ liǎojiě zhège yǔyán de yǔfǎ ( bǐrú “de,dé, de ”hé“le” de bùtóng yòngfǎ )ma? zhīdào zhège yǔyán de jīběn yīnjié (bùbāokuòshēngtiáo) zhǐyǒu 408gèma?

This has a number of obvious problems:

  • failure to capitalize the first letter in a sentence
  • failure to capitalize proper nouns (e.g., “zhongguo” should be “Zhongguo”) (Here is how to handle proper nouns in Pinyin.)
  • frequent appending of “de” to the word before it (Here is how to handle de in Pinyin.)
  • incorrect punctuation, e.g., commas, periods, parentheses, and question marks were not converted from their double-width (i.e., Chinese character) forms to regular roman forms (“,。?()” should appear instead as “,.?()”)
  • incorrect word parsing (sometimes)

In short: Thumbs-down for now. But it might not take too much work for Microsoft to make this significantly better.

How to find Chinese characters in an MS Word document

Recently someone wrote me with a problem. She had a book-length manuscript, most of which was in English. It also had some Chinese characters interspersed throughout the text. She needed to make some alterations to just the parts in Chinese characters and was hoping to avoid going through the entire Microsoft Word document line by line and changing the Chinese characters phrase by phrase. That could have taken hours or even days.

Fortunately, there’s a much easier and much faster way. So here’s how to search for Chinese characters inside a Microsoft Word document.

First, the simplest and easiest way. Copy the following line:
[⺀-■]{1,}

In Microsoft Word, use Ctrl+H to bring up the Find and Replace box.

  1. Paste the text you just copied in the Find what box.
  2. Click on the More >> button to reveal additional options.
  3. Select Use wildcards.

find_chinese_characters_word

find_chinese_characters_word_wildcard

Then Find away. That’s all there is to it. You can alter all the Chinese characters you find at the same time if you so desire.

Pro tip: If you want to change something about the Chinese characters, you might be better off in the long run making a new Word style and changing all the relevant characters to that style and then adjusting the style to meet your needs. Use ReplaceFormatStyle....

———–

Now comes a longer explanation, which you can safely ignore if the above worked fine for you.

But in case the special code above didn’t work for you or if you’d like to understand this a little better, here’s some more information on how to enter [⺀-■]{1,} yourself and why it works.

Basically, what you’re searching for is a range of characters, such as everything from A to Z. But in this case you’re going to be looking for everything from the start to the finish of Unicode’s set of graphs related to Chinese characters. Word calls this a wildcard search. Others refer to the use of wildcards as “regular expressions,” or “regex” for short.

Searches for ranges go in square brackets, with a hyphen between the first character and the last one, e.g. [A-Z].

The part at the end, {1,}, just tells Microsoft Word to look for one or more of the previous expression, so it will locate entire sections in Chinese characters, not just one character at a time. That will save you a lot of time and trouble.

OK, to get those special characters in a Word document, use

  1. Insert
  2. Symbol
  3. More Symbols

insert_symbol_more_symbols

Next,

  1. Under Font, select (Asian text).
  2. Under Subset, scroll down until you can select the CJK Radicals Supplement.
  3. Word should have already selected ⺀ (CJK Radical Repeat) for you. If not, you can click on it.
  4. Click the Insert button.

symbols_asian_text_cjk_insert

If needed, repeat Insert → Symbol → More Symbols.

sdsd

This time, with Font, still set at (Asian text):

  1. Under Subset, scroll all the way down until you can select the Halfwidth and Fullwidth Forms.
  2. Scroll all the way down the selection of glyphs and select the very last one.
  3. Click the Insert button.

half_fullwidth_forms

On my system at the moment that final character is a “halfwidth black square.” But as Unicode — and fonts — expand, the final character may be something else. Just use whatever is last and you should be fine. Just be sure to type in the square brackets, the hyphen, and the {1,} to complete the expression:
[⺀-■]{1,}

In case anyone’s wondering, no, you can’t just enter Unicode code numbers, because searches for those (u^ +number) won’t work when “Use wildcards” is on. So you have to enter the characters themselves.

This method can be easily adapted to suit searches for Greek letters, Cyrillic, etc.

I hope you find it useful.

Google Web fonts and Pinyin — December 2011 update

When I put up my first post on Google Web fonts (Google Web fonts and Hanyu Pinyin), that site offered 252 font families, 29 of which cover at least parts of Latin Extended. Now, some three months later, the total has grown to 342 font families, with 70 of those covering at least parts of Latin Extended.

Only two of the new families, however, support Hanyu Pinyin with tone marks: Ubuntu Condensed and Ubuntu Mono. That brings the total to eight Google Web fonts that support Hanyu Pinyin: four serifs and four sans serifs.

Serif

  • EB Garamond
  • Gentium Basic
  • Gentium Book Basic
  • Neuton

Sans Serif

  • Andika
  • Ubuntu
  • Ubuntu Condensed
  • Ubuntu Mono

Here’s what the two new families, Ubuntu Condensed and Ubuntu Mono, look like next to the earlier Ubuntu.

example of Ubuntu, Ubuntu Condensed, and Ubuntu Mono in action on Hanyu Pinyin

For reference, here’s the total list of Latin Extended, with Pinyin-compliant fonts in bold.

Serif Faces

  1. Bitter
  2. Cardo
  3. Caudex
  4. EB Garamond
  5. Enriqueta
  6. Gentium Basic
  7. Gentium Book Basic
  8. Neuton
  9. Playfair Display
  10. Radley
  11. Sorts Mill Goudy

Sans Serif Faces

  1. Andika
  2. Anonymous Pro
  3. Anton
  4. Chango
  5. Didact Gothic
  6. Francois One
  7. Fresca
  8. Istok Web
  9. Jockey One
  10. Jura
  11. Marmelad
  12. Open Sans Condensed
  13. Open Sans
  14. Play
  15. Signika Negative
  16. Signika
  17. Tenor Sans
  18. Ubuntu
  19. Ubuntu Condensed
  20. Ubuntu Mono
  21. Varela
  22. Viga

Display Faces (all fail)

  1. Abril Fatface
  2. Arbutus
  3. Bubblegum Sans
  4. Butcherman Caps
  5. Chicle
  6. Eater Caps
  7. Forum
  8. Kelly Slab
  9. Knewave
  10. Lobster
  11. MedievalSharp
  12. Modern Antiqua
  13. Nosifer Caps
  14. Piedra
  15. Passion One
  16. Plaster
  17. Rammetto One
  18. Ribeye Marrow
  19. Ribeye
  20. Righteous
  21. Ruslan Display
  22. Stint Ultra Condensed

Handwriting Faces (all fail)

  1. Aguafina Script
  2. Aladin
  3. Devonshire
  4. Dr Sugiyama
  5. Fondamento
  6. Herr Von Muellerhoff
  7. Marck Script
  8. Miss Fajardos
  9. Miss Saint Delafield
  10. Monsieur La Doulaise
  11. Mr Bedford
  12. Mr Dafoe
  13. Mr De Haviland
  14. Mrs Sheppards
  15. Niconne
  16. Patrick Hand

Google improves its maps of Taiwan

Two years ago when Google switched to Hanyu Pinyin in its maps of Taiwan, it did a poor job … despite the welcome use of tone marks.

Here are some of the problems I noted at the time:

  • The Hanyu Pinyin is given as Bro Ken Syl La Bles. (Terrible! Also, this is a new style for Google Maps. Street names in Tongyong were styled properly: e.g., Minsheng, not Min Sheng.)
  • The names of MRT stations remain incorrectly presented. For example, what is referred to in all MRT stations and on all MRT maps as “NTU Hospital” is instead referred to in broken Pinyin as “Tái Dà Yī Yuàn” (in proper Pinyin this would be Tái-Dà Yīyuàn); and “Xindian City Hall” (or “Office” — bleah) is marked as Xīn Diàn Shì Gōng Suǒ (in proper Pinyin: “Xīndiàn Shìgōngsuǒ” or perhaps “Xīndiàn Shì Gōngsuǒ“). Most but not all MRT stations were already this incorrect way (in Hanyu Pinyin rather than Tongyong) in Google Maps.
  • Errors in romanization point to sloppy conversions. For example, an MRT station in Banqiao is labeled Xīn Bù rather than as Xīnpǔ. (埔 is one of those many Chinese characters with multiple Mandarin pronunciations.)
  • Tongyong Pinyin is still used in the names of most cities and townships (e.g., Banciao, not Banqiao).

I’m pleased to report that Google Maps has recently made substantial improvements.

First, and of fundamental importance, word parsing has finally been implemented for the most part. No more Bro Ken Syl La Bles. Hallelujah!

Here’s what this section of a map of Tainan looked like two years ago:

And here’s how it is now:

Oddly, “Jiànxīng Jr High School” has been changed to “Tainan Municipal Chien-Shing Jr High School Library” — which is wordy, misleading (library?), and in bastardized Wade-Giles (misspelled bastardized Wade-Giles, at that). And “Girl High School” still hasn’t been corrected to “Girls’ High School”. (We’ll also see that problem in the maps for Taipei.)

But for the most part things are much better, including — at last! — a correct apostrophe: Yǒu’ài St.

As these examples from Taipei show, the apostrophe isn’t just a one-off. Someone finally got this right.

Rén’ài, not Renai.
screenshot from Google Maps, showing how the correct Rén'ài (rather than the incorrect Renai) is used

Cháng’ān, not Changan.
screenshot from Google Maps, showing how the correct Cháng'ān is used

Well, for the most part right. Here we have the correct Dà’ān (and correct Ruì’ān) but also the incorrect Daan and Ta-An. But at least the street names are correct.
click for larger screenshot from Google Maps, showing how the correct Dà'ān (and correct Ruì'ān) is used but also the incorrect Daan and Ta-An

Second, MRT station names have been fixed … mostly. Most all MRT station names are now in the mixture of romanization and English that Taipei uses, with Google Maps also unfortunately following even the incorrect ones. A lot of this was fixed long ago. The stops along the relatively new Luzhou line, however, are all written wrong, as one long string of Pinyin.

To match the style used for other stations, this should be MRT Songjiang Nanjing, not Jieyunsongjiangnanjing.
screenshot from Google Maps, showing how the Songjiang-Nanjing MRT station is labeled 'Jieyunsongjiangnanjing Station' (with tone marks)

Third, misreadings of poyinzi (pòyīnzì/破音字) have largely been corrected.

Chéngdū, not Chéng Dōu.
screenshot from Google Maps, showing how the correct 'Chéngdū Rd' is used

Like I said: have largely been corrected. Here we have the correct Chéngdū and Chóngqìng (rather than the previous maps’ Chéng Dōu and Zhòng Qìng) but also the incorrect Houbu instead of the correct Houpu.
screenshot from Google Maps, showing how the correct Chóngqìng Rd and Chéngdū St are used but also how the incorrect Houbu (instead of Houpu) is shown

But at least the major ones are correct.

Unfortunately, the fourth point I raised two years ago (Tongyong Pinyin instead of Hanyu Pinyin at the district and city levels) has still not been addressed. So Google is still providing Tongyong Pinyin rather than the official Hanyu Pinyin at some levels. Most of the names in this map, for example, are distinctly in Tongyong Pinyin (e.g., Lujhou, Sinjhuang, and Banciao, rather than Luzhou, Xinzhuang, and Banqiao).

Google did go in and change the labels on some places from city to district when Taiwan revised their names; but, oddly enough, the company didn’t fix the romanization at the same time. But with any luck we won’t have to wait so long before Google finally takes care of that too.

Or perhaps we’ll have a new president who will revive Tongyong Pinyin and Google will throw out all its good work.

Google Translate’s Pinyin converter: now with apostrophes

Google has taken another major step toward making Google Translate‘s Pinyin converter decent. Finally, apostrophes.

Not long ago “阿爾巴尼亞然而仁愛蓮藕普洱茶” would have yielded “Āěrbāníyǎ ránér rénài liánǒu pǔěr chá.” But now Google produces the correct “Ā’ěrbāníyǎ rán’ér rén’ài lián’ǒu pǔ’ěr chá.” (Well, one could debate whether that last one should be pǔ’ěr chá, pǔ’ěrchá, Pǔ’ěr chá, Pǔ’ěr Chá, or Pǔ’ěrchá. But the apostrophe is undoubtedly correct regardless.)

Also, the -men suffix is now solid with words (e.g., 朋友們 –> péngyoumen and 孩子們 –> háizimen). This is a small thing but nonetheless welcome.

The most significant remaining fundamental problem is the capitalization and parsing of proper nouns.

And numbers are still wrong, with everything being written separately. For example, “七千九百四十三萬五千六百五十八” should be rendered as “qīqiān jiǔbǎi sìshísān wàn wǔqiān liùbǎi wǔshíbā.” But Google is still giving this as “qī qiān jiǔ bǎi sì shí sān wàn wǔ qiān liù bǎi wǔ shí bā.”

On the other hand, Google is starting to deal with “le”, with it being appended to verbs. This is a relatively tricky thing to get right, so I’m not surprised Google doesn’t have the details down yet.

So there’s still a lot of work to be done. But at least progress is being made in areas of fundamental importance. I’m heartened by the progress.

Related posts:

The current state:
screen shot of what Google Translate's Pinyin converter produces as of late September 2011

Script font for Pinyin

Unfortunately, relatively few fonts support Hanyu Pinyin (with tone marks, that is). So I was surprised to come across Pecita, by Philippe Cochy. This is the first script typeface I recall seeing that covers Pinyin … and a lot more.

It might be too individualistic for much Pinyin use. But I’m very glad to know it exists and hope to see many more creations like it.

GIF of Pecita in action: A-Z, a-z, plus the diacritics used in Pinyin and a pinyin pangram

Pecita is licensed under the SIL Open Font License, Version 1.1.

Additional links:

Google Web fonts and Hanyu Pinyin

Back in the last century, getting Web browsers to correctly display Pinyin was such a troublesome task that I remember once even employing GIFs of first- and third-tone letters to get those to look right. So there were a whole lotta IMG tags in my text. Sure, I put the necessary info in ALT tags (e.g., “alt=’a3′”), just in case. But, still, I shudder to recall having to resort to that particular hack.

Things are better now, though still far from ideal. Something that promises to considerably improve the situation of website viewers not all having the same font you may wish to use is CSS3’s @font-face, which allows those creating Web pages to employ fonts that are provided online. Google is helping with this through its Google Web Fonts. (Current count: 252 font families.)

But is anything in Google’s collection capable of dealing with Hanyu Pinyin? Armed with a handy-dandy Pinyin pangram, I had a look at what Google has made available.

Not surprisingly, most of the 29 font families marked as offering the “Latin Extended” character set failed to handle the entire Hanyu Pinyin set. The ??? group is the most likely to be unsupported at present, with third-tone vowels also frequently missing.

Here are the Google Web fonts that do support Hanyu Pinyin with tone marks:
Serifs

  • EB Garamond (227 KB)
  • Gentium Basic (263 KB — and about the same for each of the three accompanying styles: italic, bold, bold italic)
  • Gentium Book Basic (267 KB — and about the same for each of the three accompanying styles: italic, bold, bold italic)
  • Neuton (56 KB — and about the same for each of the five accompanying styles: italic, bold, light, extra light, extra bold)

screenshot of the Pinyin fonts above

Note:

  • Neuton has relatively weak tone marks, so I wouldn’t recommend it for Web pages aimed at beginning students of Mandarin.

Sans Serifs

  • Andika (1.4 MB)
  • Ubuntu (350 KB) — available in eight styles

screenshot of the Pinyin fonts above

Some Ubuntu sample PDFs: Ubuntu regular, Ubuntu italic, Ubuntu bold, Ubuntu bold italic, Ubuntu light, Ubuntu light italic, Ubuntu medium, Ubuntu medium italic.

Andika sample PDF.

Note:

  • Andika’s relatively large size (1.4 MB) makes it unsuitable for @font-face use because of download time. (Its license, however, would permit someone with the time and energy to crack it open and remove lots of the glyphs not needed for Pinyin, thus reducing the size.) More fundamentally, though, I don’t much like the look of it; but YMMV.

Since Google is likely to expand the number of fonts it offers, I’m including the list of all 29 faces I tried for this experiment, which should make it easier for those wanting to test only new fonts. (It is possible, however, that Pinyin support will be added later to some fonts that fail in this area now. If anyone hears of any such changes, please let me know.) Use of bold indicates Pinyin support; everything else failed.

Display Faces with Latin Extended (all fail)

  • Abril Fatface
  • Forum
  • Kelly Slab
  • Lobster
  • MedievalSharp
  • Modern Antiqua
  • Ruslan Display
  • Tenor Sans

Handwriting Faces with Latin Extended (all fail)

  • Patrick Hand

Serif Faces with Latin Extended

  • Cardo
  • Caudex
  • EB Garamond
  • Gentium Basic
  • Gentium Book Basic
  • Neuton
  • Playfair Display
  • Sorts Mill Goudy

Sans Serif Faces with Latin Extended

  • Andika
  • Anonymous Pro
  • Anton
  • Didact Gothic
  • Francois One
  • Istok Web
  • Jura
  • Open Sans
  • Open Sans Condensed
  • Play
  • Ubuntu
  • Varela

Additional resource: SIL Fonts for downloading (including the full versions of Andika and Gentium).