Hi,

For those that don't know me I'm currently trying to organize a Fedora
special interest fonts group. Fedora is used as base by the OLPC
project. In the course of discussing the OLPC font situation we've
identified a problem in the current *nix desktops. Jim Gettys, of
multiple fames, currently at OLPC, asked me to post about it there.

The problem

Modern text rendering is extremely language dependent. Since the
unicode.org consortium, in its infinite wisdom, lumped regional
variants at the same code points (chinese han vs japanese han, arabic
arabic vs farsi arabic, russian cyrillic vs balcanic cyrillic), text
layouting libs like pango absolutely need to know what language is
being rendered to make the right glyph choices.

It's not sufficient to pass them unicode strings — the same unicode
point is used for different regional variants of the same glyph, so
language context is needed to select the right one¹.

This is not a problem for pre-existing text as most international text
formats (HTML, ODF) allow tagging text with language info.

It is a huge problem for text the user is typing. The applications
have absolutely no way to know what language the user is currently
inputing. Therefore, they're not giving the right info to their text
layouting lib, and it makes mistakes.

False solutions

1. Use the user locale. Well I'm typing in English right now, and I
can assure you my locale is not an English-speaking location

2. Query the current keyboard layout or input method. X allows apps to
query the current keyboard layout via xkb. Unfortunately there is no
1←→1 relationship between layout/IM and languages. I'm currently
typing English with a latin layout not designed for english. Some
layouts (like the latest canadian one) are explicitely multilingual.
Some countries use US layouts because they're cheaper than defining a
different local standard. As a result inferring language from layout
or IM is extremely unreliable.

3. Add language toggles to every app the user types in. I hope no one
here thinks that's a smart solution. It's the one OpenOffice.org
chose, though².

Good solution

Windows solved this problem years ago. Instead of having a keyboard
layout/IM switcher, you add a language switcher to the desktop.
Switching languages may require a layout/IM change, or not. When it
requires a layout/IM change the switcher behaves like the current
GNOME or KDE keyboard applet. The smart thing is that even when there
is no layout/IM change needed the user can still press his switcher
hotkey, and apps are informed the language being typed changed.

(In other words the switcher manages a list of language,layout/IM
tupples, instead of a raw layout/IM list. The user can define the same
layout/IM for different list elements. When he cycles through the list
only language is displayed. And apps can query this info)

The problem, as pointed by Sergey Udaltsov (svu), is the X protocol
made no provision to relay language state, just layout state.

But according to Jim Gettys « This is just about trivial to add by
using X properties.  It just requires agreement on a convention. »

So I'm posting there to ask the xdg people to agree on a X property
convention. Or find another solution than the one pointed by Jim. I
don't care, I just want my fonts rendered properly ;) .

As an aside, correct text rendering is not the only use requiring live
language change info. Every app that does spellchecking needs to know
what dictionary to load when the user is typing.

I hope my long message was clear to everyone.

¹ The right variant can be found in the same font if it's a modern
opentype font with locl, or in another font if you rely on
language-specific fontconfig priorities

² The argument being getting *nix desktops agree to a desktop-wide
language notification system is a lost cause
http://blogs.sun.com/GullFOSS/entry/language_selection_statusbar_control
http://qa.openoffice.org/issues/show_bug.cgi?id=1034

Regards,

-- 
Nicolas Mailhot