One of the first things I noticed when I began converting files with po4a, was the fact that po4a would sometimes exit with an error, even if the file was valid wml. This is a problem for files like download.wml where a special link-format is used to make sure every page shows the same version number (and points to the latest release).
Also, when converting a file from .wml to .po for the first time, a header will be added containing the following information:
# SOME DESCRIPTIVE TITLE
# Copyright (C) YEAR Free Software Foundation, Inc.
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR , YEAR.
It would be nice to have a header that doesn't assign copyright to FSF (or not have a header at all). I have talked to the maintainer of po4a who said he would take a look at these two issues during his two weeks at Debconf.
I have also written two bug reports:
#537236: please add support for valid wml
#537245: modify header in .po
Thursday, July 16, 2009
My current todo list
For those who would like to know what I am working on right now, here is my current todo list (also known as list-of-issues-that-need-to-be-sorted-out). That said, progress is a little bit slow these days as I'm in bed with the flu (again).
Converting files
I am working on converting the translated .wml files to .po files. The files are being uploaded to translation/trunk/tools/gsoc09/po/.
I have run into a little problem that I'm not sure how to deal with. When converting a .wml to a .po, including the translated .wml, po4a will sometimes exit with one of the following (or both) errors:
1. Original has less strings than the translation
2. Structure disparity between original and translated files
There are two types of translations that cause such problems. The first one is where the translator has simply skipped a paragraph or two and moved on to the next. This type of translation shouldn't be too hard to fix and convert. In most cases, anyways.
The second type is where the translator has written his or her own, translated version of the page, and not a direct, translated copy. This type of translation can be hard to fix unless you know the language.
I guess that one possible solution would be to ask the people on the or-talk list for help, or contacting previous translators directly.
No translated .wml in svn
Not having translated .wml in svn sounds like a good idea. After all, one can just generate the .wml from the translated .po when needed. However, we do need to have a few of the translated .wml files in svn. When I say "a few" I mean the translated versions of the donate-page as well as the download-pages.
The reason for this is that these files can not be converted to po (for now, at least). I suggest that we keep those files in svn and have po2wml generate the rest when we need them, i.e. before the website is built and published.
A small note-to-self would be to fix po2wml so it no longer adds and commits files.
Converting files
I am working on converting the translated .wml files to .po files. The files are being uploaded to translation/trunk/tools/gsoc09/po/.
I have run into a little problem that I'm not sure how to deal with. When converting a .wml to a .po, including the translated .wml, po4a will sometimes exit with one of the following (or both) errors:
1. Original has less strings than the translation
2. Structure disparity between original and translated files
There are two types of translations that cause such problems. The first one is where the translator has simply skipped a paragraph or two and moved on to the next. This type of translation shouldn't be too hard to fix and convert. In most cases, anyways.
The second type is where the translator has written his or her own, translated version of the page, and not a direct, translated copy. This type of translation can be hard to fix unless you know the language.
I guess that one possible solution would be to ask the people on the or-talk list for help, or contacting previous translators directly.
No translated .wml in svn
Not having translated .wml in svn sounds like a good idea. After all, one can just generate the .wml from the translated .po when needed. However, we do need to have a few of the translated .wml files in svn. When I say "a few" I mean the translated versions of the donate-page as well as the download-pages.
The reason for this is that these files can not be converted to po (for now, at least). I suggest that we keep those files in svn and have po2wml generate the rest when we need them, i.e. before the website is built and published.
A small note-to-self would be to fix po2wml so it no longer adds and commits files.
Monday, July 13, 2009
Bug fixed in po4a
Last week I ran into a problem when po4a-translate seemed to have a problem recognizing new, translated strings. This turned out to be a bug in the wml module and it has now been fixed in the po4a repository. This email to the po4a-list explains the bug and the quick fix.
Friday, July 10, 2009
The website, po2wml and po4a-translate
I had to deal with a few things at the beginning of the day, but since then I have:
Built the website
I managed to build the website with my own, translated files and everything looks ok. I haven't published it simply because I am a bit unsure how I can put it on www.xcde.net/torproject using the publish-script.
Fixed po2wml
I noticed that, when running, po2wml would also convert empty .po files. To deal with this, I included an if statement that checks for a certain type of comments (of the form "#."). Those comments say something about the following string, if you're translating a part of a list or a paragraph. So, that way the script knows whether or not to convert the file. No comments of the form "#." means no content.
Looked at po4a-translate
The script, po2wml, is using po4a-translate to convert the translated .po files back to .wml files. It seems like po4a-translate has a problem with recognizing new, translated strings. I copied en/30seconds.wml and de/30seconds.po into a new folder, translated 27 of the 32 strings in 30seconds.po and ran po4a-translate.
I got the following output:
We found translations for 84.37% (27 from 32) of strings.
Then I decided to translate the rest of the .po. I did that, ran po4a-translate again and got the exact same output. It says that only 27 of the 32 strings have been translated, but that is not correct.
I'm not really sure why it's doing this, if I'm doing something wrong or if there's a bug somewhere. I have sent an email to the po4a-devel mailing list and hopefully someone will reply in not too long (that or I'll find someone on irc and give them a friendly poke).
Built the website
I managed to build the website with my own, translated files and everything looks ok. I haven't published it simply because I am a bit unsure how I can put it on www.xcde.net/torproject using the publish-script.
Fixed po2wml
I noticed that, when running, po2wml would also convert empty .po files. To deal with this, I included an if statement that checks for a certain type of comments (of the form "#."). Those comments say something about the following string, if you're translating a part of a list or a paragraph. So, that way the script knows whether or not to convert the file. No comments of the form "#." means no content.
Looked at po4a-translate
The script, po2wml, is using po4a-translate to convert the translated .po files back to .wml files. It seems like po4a-translate has a problem with recognizing new, translated strings. I copied en/30seconds.wml and de/30seconds.po into a new folder, translated 27 of the 32 strings in 30seconds.po and ran po4a-translate.
I got the following output:
We found translations for 84.37% (27 from 32) of strings.
Then I decided to translate the rest of the .po. I did that, ran po4a-translate again and got the exact same output. It says that only 27 of the 32 strings have been translated, but that is not correct.
I'm not really sure why it's doing this, if I'm doing something wrong or if there's a bug somewhere. I have sent an email to the po4a-devel mailing list and hopefully someone will reply in not too long (that or I'll find someone on irc and give them a friendly poke).
Thursday, July 9, 2009
So far, so good
Today has been the day of testing code, fixing minor bugs and testing some more. I had 9 items on my list and I have managed to do 8 of them. The items are pretty much the same as the ones described in the previous post. I figured I'd do the last item tomorrow and then go through the list one more time.
Step 1:
Set up a repository with a website module and a translation module.
Step 2:
Put wml files in the english directory in the website module.
Step 3:
Make sure pootle reads the translation module.
Step 4:
Run wml2po to convert the wml files to po files and put them in the translation module.
While converting and updating the files, I noticed that files were being committed to the repository even though they had not changed. Or so I thought. It turns out that a few of the comments in the files would change every time I ran the script. The solution was to modify one of the regular expressions in the script to exclude a certain type of comments as well.
Step 5:
Restart pootle so it can detect the new files.
Step 6:
Translate a few strings using the pootle interface.
Step 7:
Commit using the pootle interface.
After a little back and forth I managed to give my user the right permissions. Translating and committing files using the pootle interface went without any further problems.
Step 8:
Run po2wml to convert the po files back to translated wml files.
I noticed that the script would convert the english po files back to english wml files. This certainly doesn't make much sense, and so the problem was fixed with an if statement.
Step 9:
Build and deploy the website.
Building the website is the one thing I haven't done today, and it is the first thing I'll do tomorrow.
Step 1:
Set up a repository with a website module and a translation module.
Step 2:
Put wml files in the english directory in the website module.
Step 3:
Make sure pootle reads the translation module.
Step 4:
Run wml2po to convert the wml files to po files and put them in the translation module.
While converting and updating the files, I noticed that files were being committed to the repository even though they had not changed. Or so I thought. It turns out that a few of the comments in the files would change every time I ran the script. The solution was to modify one of the regular expressions in the script to exclude a certain type of comments as well.
Step 5:
Restart pootle so it can detect the new files.
Step 6:
Translate a few strings using the pootle interface.
Step 7:
Commit using the pootle interface.
After a little back and forth I managed to give my user the right permissions. Translating and committing files using the pootle interface went without any further problems.
Step 8:
Run po2wml to convert the po files back to translated wml files.
I noticed that the script would convert the english po files back to english wml files. This certainly doesn't make much sense, and so the problem was fixed with an if statement.
Step 9:
Build and deploy the website.
Building the website is the one thing I haven't done today, and it is the first thing I'll do tomorrow.
Step by step
We now have a way to convert from .wml to .po and back to .wml again, but what exactly is the process from start to finish?
Here's how it will work, step by step, from a new .wml, to a translated .po, to .wml again, to a part of the website.
Step 1:
I have written foo.wml, a new .wml file. I commit this to the repository.
Step 2:
I need to convert foo.wml to foo.po. I run wml2po.
Step 3:
I now have foo.po in the repository as well, but pootle doesn't know about it. Pootle needs to be poked to learn about new files. Restart pootle.
Step 4:
Pootle now knows about foo.po and people can begin to translate.
Step 5:
It is soon time to build the website and I want to include translated pages of foo.wml. People forget to click on the 'commit'-button in the pootle interface when they are done translating foo.po. I log on to the pootle admin interface and force pootle to commit foo.po for all languages.
Step 6:
All work in pootle has been committed and I can update the translated documents, for example website/de/foo.wml. Run po2wml to convert translation/projects/website/de/foo.po to website/de/foo.wml and commit.
Step 7:
I build the website and deploy it.
Here's how it will work, step by step, from a new .wml, to a translated .po, to .wml again, to a part of the website.
Step 1:
I have written foo.wml, a new .wml file. I commit this to the repository.
Step 2:
I need to convert foo.wml to foo.po. I run wml2po.
Step 3:
I now have foo.po in the repository as well, but pootle doesn't know about it. Pootle needs to be poked to learn about new files. Restart pootle.
Step 4:
Pootle now knows about foo.po and people can begin to translate.
Step 5:
It is soon time to build the website and I want to include translated pages of foo.wml. People forget to click on the 'commit'-button in the pootle interface when they are done translating foo.po. I log on to the pootle admin interface and force pootle to commit foo.po for all languages.
Step 6:
All work in pootle has been committed and I can update the translated documents, for example website/de/foo.wml. Run po2wml to convert translation/projects/website/de/foo.po to website/de/foo.wml and commit.
Step 7:
I build the website and deploy it.
Wednesday, July 8, 2009
Old code
I have uploaded the code that I have previously written to show that I have been hacking away on stuff, and also what I have been hacking away on. Most of the scripts there should work, but I can't promise anything.
Anyways, they are there for everyone to look at and maybe also learn from (kids, don't use PATH as a variable name). Also, some people might like to see how the ideas and the code have evolved over time.
The code is in the repository, translation/trunk/tools/gsoc09/other.
Anyways, they are there for everyone to look at and maybe also learn from (kids, don't use PATH as a variable name). Also, some people might like to see how the ideas and the code have evolved over time.
The code is in the repository, translation/trunk/tools/gsoc09/other.
Subscribe to:
Posts (Atom)