Step by step

We now have a way to convert from .wml to .po and back to .wml again, but what exactly is the process from start to finish?

Here's how it will work, step by step, from a new .wml, to a translated .po, to .wml again, to a part of the website.

Step 1:
I have written foo.wml, a new .wml file. I commit this to the repository.

Step 2:
I need to convert foo.wml to foo.po. I run wml2po.

Step 3:
I now have foo.po in the repository as well, but pootle doesn't know about it. Pootle needs to be poked to learn about new files. Restart pootle.

Step 4:
Pootle now knows about foo.po and people can begin to translate.

Step 5:
It is soon time to build the website and I want to include translated pages of foo.wml. People forget to click on the 'commit'-button in the pootle interface when they are done translating foo.po. I log on to the pootle admin interface and force pootle to commit foo.po for all languages.

Step 6:
All work in pootle has been committed and I can update the translated documents, for example website/de/foo.wml. Run po2wml to convert translation/projects/website/de/foo.po to website/de/foo.wml and commit.

Step 7:
I build the website and deploy it.

Old code

I have uploaded the code that I have previously written to show that I have been hacking away on stuff, and also what I have been hacking away on. Most of the scripts there should work, but I can't promise anything.

Anyways, they are there for everyone to look at and maybe also learn from (kids, don't use PATH as a variable name). Also, some people might like to see how the ideas and the code have evolved over time.

The code is in the repository, translation/trunk/tools/gsoc09/other.

wml2po and po2wml

Four days ago I asked, on the tor-gsoc mailing list, if we could split the contents of website/ into po/ and wml/. Roger Dingledine said that it would be possible, but it would be even better to keep the wml files in the website module and the po files in the translation module.

He also said that "we can give pootle or other services read-write access to the "translation" module, but they can treat the "website" module as read-only. The process of building the website can then treat the "translation" module as read-only".

After a little bit of testing I came up with a good solution. So, in my case, I can have the wml files in

"/home/runa/tor/website"

and the resulting po files in

"/home/runa/tor/translation/projects/website"

I also wrote the second script, po2wml, that will take the translated po files and convert them back to wml files.

The updated version of wml2po has been added to the repository, together with po2wml.

In SVN: wml2po

Some say it's better late than never, I say it's something I should have done a little bit earlier. It took some time before I managed to get on the right track and find a solution that everyone seemed happy with. Hopefully, this is it. The script, wml2po is now in the tor svn repository.

Almost ready: wml2po

The script, wml2po, is almost ready. It assumes that the contents of website/ is split into wml/ and po/. The resulting po files are now being put where they belong, i.e.:

• website/wml/en/index.wml to website/po/$language/index.po


• website/wml/press/en/index.wml to website/po/press/$language/index.po

Also, the hash of each po file is generated before and after the file has been updated. The only thing left is to make sure that script has a proper lockfile.

My progress so far

I just sent an email to the tor-gsoc list asking if it is possible to split the contents of website/ into po/ and wml/. Doing so will make things a whole lot easier when it comes to figuring out where to put the resulting po files.

I also sent an email to my mentor telling him about my progress so far. What I wrote is pretty much the same as I have previously written in this blog:

Over the past few weeks I have written a few different scripts. While they have worked, they have not been ideal solutions to the problem I am trying to solve. If you read the blog post from yesterday, July 2nd, you'll see that I've gotten useful pointers from Peter and that
is what I have been working on today.

What's left to do is:

For wml2po:

• Make sure that the wml and po files are put where they belong.


• Generate hash values for each file before and after converting. Compare these two values and revert if they are the same (i.e. nothing has changed).

For po2wml:

• The code will be pretty much the same as for wml2po and I don't think writing this script will be a problem.

Converting from wml to po, including translations:

• I have already converted the wml files in the repository (including their translations) to po files. I'm just not sure where I should upload these. What do you think?


• I also think that uploading those po files won't do much good until we have a script that can keep them updated automatically.

According to my timeline I should be done with the coding by the end of next week, and then start to write documentation (that is, document the code properly and write a readme). I am confident that I'll be able to stick to that plan. Let me know if you have any questions.

A few bumps in the road

Me and my code have been hitting a few bumps in the road over the past couple of weeks. Either because the solution I came to simply wasn't the best solution out there or because I misunderstood something. Peter Palfrader has been really nice and given me a few pointers, and I am confident that I am on the right track.

So, the new approach will be to use a proper lockfile (and not svn lock), comparing files with hash values, converting all of the wml files, reverting the ones that haven't changed since last time and committing the ones that have.

There is also one more thing on the todo-list, other than writing a script that will keep all the files updated automatically, and that is to convert all the wml files to po files. This can be done in just a few minutes (given that all the files can be converted without problems). I figured I'll upload these to the repository when:

1. The fancy, automatic wml-to-po-and-back-script is done.
2. I know where to put the po files.

I also plan on writing a longer progress report tomorrow and email this to my mentor, Jacob Appelbaum, as well as post it here.