International Web

While working on web development, I got stuck for a few hours trying to uncover the magic behind Internationalization, and how I am supposed to deal with it from the client (the web browser) down to the database backend engine, through the code that handles the application logic.

Some basic definitions worth having clear:

• Unicode is not an encoding. It is a system to represent any character in any writing system in use today: go here and ask your browser what encoding this page is using.
• Localized web applications might use any of the many character encoding systems: for instance, this Spanish website uses ISO8859-1.
• UTF-8 is a Unicode encoding system and is the preferred way of sending data back and forth from web browsers for its widespread use.

The safest and easiest approach then, seems to rely on the following rules:

• In your application, decode the data to Unicode as soon as it gets to your hands, and work with it all throughout your application processing (including Database backends).
  
• Once your output data is ready, encode it into UTF-8 and send it back to the web browser client as such.
  

Read this great tutorial from Python's Pylons website.