Easy multi-language models in Django

2012-07-27 by Senko Rašić

If you’ve written an application that needs to be localized into multiple languages, you’ve probably faced the problem of translating the application interface and data to the target languages.

In Django, the first part of the problem, interface translation, is more or less a solved one. Django has good i18n support and uses standard gettext framework for handling the translated strings. But what about the data in the database? Sadly, there isn’t a good general solution to this. There are a number of Django apps that can help you out, but only if your use case closely matches the authors’.

Here’s a simple solution that may fit your use case and is easy to implement and understand. The problem addressed here is:

  • some of the fields of your Django model need to be translated to multiple languages,
  • you want to be able to select the correct “output” language at runtime (no hardcoding or special template tags),
  • if a translation is not available for a field, it should fall-back to the default language,
  • it should be easy to add or remove languages.

Basic idea

The basic idea is to create one model field for each of the language needed for the attribute, with the language suffix added. So, for example, if the original field is a CharField called message, and English and Croatian languages need to be supported, we’ll create message_en and message_hr character fields. To mapmessage to correct variant at runtime, falling back to the default when needed, we’ll use a little bit of __getattribute__ magic. We’ll also need a way to select the language we want, and a way to propagate the selection throughout related models.

class MultilingualModel(models.Model):
    # fallback/default language code
    default_language = None

    # currently selected language
    selected_language = None

    class Meta:
        abstract = True

    def select_language(self, lang):
        """Select a language"""
        self.selected_language = lang
        return self

    def __getattribute__(self, name):
        def get(x):
            return super(MultilingualModel, self).__getattribute__(x)

        try:
            # Try to get the original field, if exists
            value = get(name)
            # If we can select language on the field as well, do it
            if isinstance(value, MultilingualModel):
                value.select_language(get('selected_language'))
            return value
        except AttributeError, e:
            # Try the translated variant, falling back to default if no
            # language has been explicitly selected
            lang = self.selected_language
            if not lang:
                lang = self.default_language
            if not lang:
                raise

            value = get(name + '_' + lang)

            # If the translated variant is empty, fallback to default
            if isinstance(value, basestring) and value == u'':
                value = get(name + '_' + self.default_language)

        return value

We can now use this model as a base class in our models. For example, here’s a model containing a single message string that should be translated:

class Message(MultilingualModel):
    default_language = 'en'

    msg_en = models.CharField(max_length=255, default='', blank=True)
    msg_hr = models.CharField(max_length=255, default='', blank=True)

    def __unicode__(self):
        return self.msg

And use it in the rest of the code like this:

>>> m = Message.objects.create(msg_en='Hello', msg_hr='Zdravo')
>>> m.select_language('hr').msg
'Zdravo'
>>> m.select_language('en').msg
'Hello'

So far, so good. But the situation complicates if we have related models:

class MessagePart(MultilingualModel):

    default_language = 'en'

    message = models.ForeignKey(Message, related_name='parts')
    part_en = models.CharField(max_length=255, default='', blank=True)
    part_hr = models.CharField(max_length=255, default='', blank=True)

    def __unicode__(self):
        return self.part

If we only select language on the first model, it won’t be propagated on the related fields:

>>> p = MessagePart.objects.create(part_en='World',
...     part_hr='Svijete', message=m)
>>> m.select_language('hr').parts.get()
'World'

To achieve this, we can extend ModelManager and QuerySet so they propagate the language selection:

class MultilingualQuerySet(models.query.QuerySet):
    selected_language = None

    def __init__(self, *args, **kwargs):
        super(MultilingualQuerySet, self).__init__(*args, **kwargs)

    def select_language(self, lang):
        self.selected_language = lang
        return self

    def iterator(self):
        result_iter = super(MultilingualQuerySet, self).iterator()
        for result in result_iter:
            if hasattr(result, 'select_language'):
                result.select_language(self.selected_language)
            yield result

    def _clone(self, *args, **kwargs):
        qs = super(MultilingualQuerySet, self)._clone(*args, **kwargs)
        if hasattr(qs, 'select_language'):
            qs.select_language(self.selected_language)
        return qs

class MultilingualManager(models.Manager):
    use_for_related_fields = True
    selected_language = None

    def select_language(self, lang):
        self.selected_language = lang
        return self

    def get_query_set(self):
        qs = MultilingualQuerySet(self.model, using=self._db)
        return qs.select_language(self.selected_language)

We also have to remember to modify MultilingualModel base class to use MultilingualManager, and to check for MultilingualManager and MultilingualQuerySet when propagating the selection in __getattribute__ method.

The end result

Let’s try the previous example again:

>>> m.select_language('hr').parts.get()
'Svijete'

Also, we can do really useful stuff like set language on all the queried objects, so we don’t have to worry later:

>>> Message.objects.select_language('hr').all()

What about adding new languages? If you’re using a database migration tool like South, it’s really easy. Just create new fields for the language you’re adding, and set the default strings to ” (empty). The code will just fall back to default language until you fill in all the fields for the new language, just like gettext.

As we can see, this is an easy way to add quite powerful multilingual capabilities to our Django models.

Author
Senko Rašić
We’re small, experienced and passionate team of web developers, doing custom app development and web consulting.