Translating Shijro into 19 languages, and what the AI could not fix

Shijro 3.0 is in 19 languages. The app has about 1,600 sentences, buttons and labels, and AI models drafted every one of them. I read Urdu and Sindhi, so I checked those two myself, line by line. The other seventeen have not been read by a native speaker yet.

I expected the hard part to be choosing the right model. It wasn’t.

Three models, then one

French went first. One model wrote the drafts, a second read them and flagged anything a French reader would stop at, and a third looked at each flag and decided: keep the draft, take the suggested fix, or write its own. The whole French app came to about $3.30 and an hour and a quarter.

The flagger caught about half of the lines I later changed in Urdu and Sindhi, which sounds useful until you look at its fixes. I almost never picked them. I tried two other flagging models as well, and one of them flagged a single line out of fourteen that I had rewritten.

Then I ran a blind comparison between two drafting models on the same 60 lines in each language. In Urdu I preferred one model’s version 11 times, the other’s 9 times, and liked them equally 21 times. Neither was clearly better. I kept one model and dropped the flagging and deciding steps.

What I actually corrected

Most of my corrections weren’t mistakes of meaning. The Urdu was correct and still nobody would say it. “Continue with email” came out as a perfectly good translation of “continue”, and what an Urdu speaker would write is just “use email”. “Have an invite code?” became “Has someone given you a code?”, because nobody in Urdu or Sindhi talks about an invitation code.

Some words were right in the dictionary and wrong on a phone. The everyday Urdu and Sindhi word for “information” was used for notifications, where people actually say notification. The Sindhi word picked for “account” is an account in a shopkeeper’s ledger.

And a few were insulting without meaning to be. Shijro has a made-up sample family, the Hartleys, that you can look around before signing up. The Urdu and Sindhi words chosen for “sample” read like mockery, and the Sindhi word for “made-up” means fake. That one I would never have caught by checking whether the translation was accurate.

Each of these went into a list of lessons that every language now reads before drafting. When I took the Sindhi lessons out and kept only the shared ones, the drafts still avoided every mistake I had fixed.

The mistakes that were in my code

The corrections that took longest weren’t translation at all.

In Activity, some sentences were handed a name with the English possessive already on it: “Ishtiaq’s”. Urdu doesn’t have an apostrophe-s. Its possessive changes with the thing that is owned, and it sits after the name as a separate word. So the Urdu sentence came out with “Ishtiaq’s” dropped into the middle of it, which isn’t Urdu or English. No model can fix that from the words it is given. The fix was to give every sentence the plain name and let each language put the possessive where it belongs.

The same thing happened with gender. “Joined the family” in English is the same for everyone. In Urdu the verb changes for a woman, and so does “turns 5”. The sentences didn’t say whether the person was a man or a woman, so the translation had to guess. Now they do.

“And 3 others” was built once and stuck on the end. Sindhi needs a different form of “others” depending on where it sits in the sentence.

Names in Latin letters inside an Urdu or Sindhi sentence pulled the words around them out of order, because the text didn’t mark where the name began and ended. And on Android the family tree drew Arabic-script and Hindi names with their letters unjoined, as separate shapes, because the tree is drawn in a way that skipped the text shaping the rest of the app gets for free.

There was even one mistake that wasn’t there. My editor drew two Sindhi letters, ٿ and ڀ, with each other’s dots. Correct drafts looked misspelled until I checked the actual characters.

If you read one of the other seventeen

I have only read two of these languages properly. If something in Shijro reads oddly in yours, or is correct but not what anyone would say, please tell me at contact@shijro.com.