Boonex Discovers Error in Translated Language Files!

mydatery posted 27th of October 2009 in Community Voice. 7 comments.

In the process of doing a build, we realized that with over 2500 Language Keys, purchasing a translation would be useless as a base Dolphin has nowhere near that many Language Keys (which is what a Language File comes with is the base LK's) and if we did then we'd have to delete and re-add approximately 600 English LK's to the system due to how Dolphin created them.

 

So, we did the next best thing, we stopped all development in other areas and began the translation to the other languages.  The first Language up was Italian, just because it's a fun Language and would be a good test to see how UTF-8 worked in regards to the unique characters.  We're now in the German Translation and finding that UTF-8 works awesome.  But that wasn't the issue, the issue was an error that happened only in Italian, and no other language.

 

When a member was on a "Profile Page" and attempted to leave a comment, they would click the _Add a comment LK and instead of opening up the little JS window to write a comment, it would instead take them to the home page.  What?  How did that happen?  Especially since everything was working perfect before the translation and when the site was put into English it worked great also (the site has a language switcher on it that members can click, but not the issue here either).  So how in the world can this happen?  After all, the code is the same regardless of the Language, we're only altering LK's here.

 

But, that is not a true statement, because when you alter LK's you do alter code in some of them if your not careful.  So off we go to start checking LK's.  We checked 2,558 LK's and everyone of them came back as being correct.  No href's we're altered, all compared to their English counterparts and now this really did not make sense.  We've actually worked on this since Friday not realizing the issue was right in front of our faces and it's even been used multiple times in this post.  Actually, everytime I've typed LK's (like that) the issue has been right in front of me.  The most amazing thing, it didn't even have to do with comments so to say.  Here was the translation causing the issue, it's actually kind of ironic on the one that caused it:

 

L'errore ha accaduto

 

Note the apostrophe ( ' ) that was used in the translation?  That was the issue.

 

Instead it should of been:

 

L`errore ha accaduto

 

Utilizing an accent ( ` ) instead of an apostrophe in the new translation.  This key had no href on it, it had no {0} or such on it, nothing.  Just a key saying an error has occured.  Nothing more and nothing less.

 

In the end, I nor my team found this problem.  Instead, after looking at it all weekend contact was made Sunday evening (my time, Monday workday Australia time) with Boonex who found it to be an interesting problem so they took a look at it for us.  The Boonex team is the one who discoverd this issue, one that we spent countless hours repeatedly looking for and it was so tiny yet so blatantly obvious that it was completely overlooked. 

As it turns out, you can not use an "apostrophe"  ( '  ) in the LK's without taking precautions for it or anticipating errors happening.  I took a look at the language files to see the reason, it's literally a blatant fact that it's not a good idea to do this. 

My Italian Language file has 2,558 LK's in it, to display the reason I'm going to show you what just one of them in the file langs/lang-it.php looks like:

 

'_Block' => 'Blocco',

 

Notice all the apostrophe's in that line?  There are 4 of them, signalling the beginning and end of the items.  So if we we're to type

 

'_Block' => 'B'locco',

as an example, then that LK would look like the letter "B" and then an error after it as   

 

locco',

 

would not have a key to reference.  Now what is the proper way to use an apostrophe?  Not sure, as I know that Boonex has done it but I'll have to look at LK's to figure that out when it comes to other ones.  We see the word member's all the time.  But in this instance, it was the error.

 

Just wanted to share what happened with everyone and give a huge thanks to Boonex for finding this error.  It really was a small item, but it was a huge issue also.  I can't say thank you enough for this to Boonex and it's saved us countless more hours in looking for the issue. 

 

 

 
Comments
·Oldest
·Top
Please login to post a comment.
Great find. We also started to manually work on lang translations for D7 a few weeks ago, and were a bit concerned about apostrophe's as well. Since our first lang translation was into Spanish, however, all accents and "ñ" letters were parsed perfectly.

So, when you mention the need to "tale precautions", the fix is to escape apostrophe's using the "/", right?
therock
Yes the fix is to escape out the apostophe's "/" but the question is do we do it directly in the lang keys through the admin panel or do we do it after the new lang file has been created and translated in the example; lang-it.php
mydatery
It would have to be done in either the Admin Panel as the LK is written or by going into DB and directly editing them there.

You can't adjust the language file (in this case langs/lang-it.php ) because as soon as you do it via a code editor your fine until the next time you compile the lanugage on that file. Then the DB will just overwrite.

In this instance, I had to open langs/lang-it.php and do a search for /' and replace all occurrances via Admin-> Settings -> Language Settings see more and doing a search for each key that came up. We found 376 occurances and yes, it took hours to fix them all.
lancashiredates
I type ' or ' instead of an apostrophe, that does the trick for me..
houstonlively
If things are coded correctly, shouldn't you be able to use whatever text string you please? If an apostrophe in a language string is being interpreted as code, I would think the correct solution would be to fix the code, not apply a work-around to the language string.
mydatery
If you have a permanent fix for the code, that would be the way to do it. But since we just found this out last night and spent hours making sure things work, we've had no time yet to look at the code itself.

Perhaps you have a solution that works for this? After all, it would seem that it's a universal issue since looking through the DB I notice that the English Language utilizes /' when it needs to insert an apostrophe into the code. Quotation marks are often placed in using "whatever see more your quoting"

So I'd say it's not just a Dolphin issue, but more than likely a net wide item. Looking at phpBB3 I'm seeing the same issue in it's DB.
houstonlively
There was a similar issue in an early D7 beta, where apostrophes in comment text were not properly escaped, and it was fixed. I'm sure they will fix this one too sooner or later.

Things like this are everywhere on the internet. One that Major League Baseball hasn't figured out how to fix yet, is posters on MLB message boards defeating the censored word filter by preceding text with the '<' character. Try it, you'll be able to post whatever obscenities you like. MLB message boards are hosted see more by Prospero, and you can do the same thing to any message board hosted by them... MLB... marthastewart.com... etc. Of course, you'll probably get banned by doing so, but I think it's kinda funny that they haven't been able to fix this for at least 3 years.
 
 
Below is the legacy version of the Boonex site, maintained for Dolphin.Pro 7.x support.
The new Dolphin solution is powered by UNA Community Management System.
PET:0.057358980178833