Xbox LIVE Indie Games
Sort Discussions: Previous Discussion Next Discussion
Page 1 of 1 (20 posts)

Doesn't peer review also include languages check?

Last post 9/8/2009 7:13 PM by Spyn Doctor. 19 replies.
  • 8/28/2009 6:28 PM

    Doesn't peer review also include languages check?

    Hi,

    I just failed three games (Duologue, Gitigiti and Project Alpha) because they had supplied a game description in various languages, without also checking these languages during submit.

    For example, Project Alpha was submitted with only "English" language marked, but it contains descriptions for all 6 supported languages. But since it was submitted with only "English", the peer review framework will only require English reviews of this game. So it may be that any forbidden content in the other description slips through, because the reviewers understand only English and are therefore unable to review these other descriptions. And since the review framework was never told about these other languages, it also doesn't require any reviews in these languages.

    Similar for the other games.

    I think all of these games already had passes though. Don't reviewers check this???

    Or am totally wrong with my interpretation of the rules and everybody else does it right?

    Doc
  • 8/28/2009 6:45 PM In reply to

    Re: Doesn't peer review also include languages check?

    Answer
    See the Language FAQ - if they provide a description in another language the game will still need to be reviewed by those languges even if they don't check the boxes. So its OK.

    The check boxes indicate if the entire game is translated - it is apparantly ok with Microsoft for a description to be localised even if the game is not. So you were incorrect to fail for this. Though Microsoft have not done a good job of making this clear - it took me a few days to get a clear statement when I wrote that FAQ. Its been a while since anyone asked about this though so I suspect that many reviewers dont check it... so I applaud your diligence even though I wish you had asked before you failed them ;-)
  • 8/28/2009 6:55 PM In reply to

    Re: Doesn't peer review also include languages check?

    I see. Very confusing indeed. Guess I'm gonna have to apologize to the game creators then, for failing incorrectly.

    Well, at least I have to apologize to the creators of Duologue, because as I see in your FAQ, the fails for the two other games were still correct then, because you wrote:

    Valid language fail reasons include:
    Bad translations - e.g. using automated processes - in the binary or the description


    And it is obvious that the German descriptions of Gitigiti and Project Alpha were done by Google or whatever. They are total nonsense.
    (The German description of Duologue is fine though, just a few very minor grammatical mistakes, but certainly no worse German than my English. ;-)

    [EDIT: ZMan, I see that you too already mentioned this point in the relevant peer review threads, so at least 2 out of 3 of my failures were correct anyway...]

    Doc
  • 8/28/2009 7:24 PM In reply to

    Re: Doesn't peer review also include languages check?

    Indeed, if people use google or provide a bad translation then as a native speaker of that language its perfectly acceptable to fail it.

    Many of the Japanese games should be getting failed for bad English translations IMO - however I am guessing that Japanese reviewers who speak English are passing them at the same time as Japanese (valid if they speak both languages) and so the bad English is getting through. Whilst I know many English speakers enjoy the bad translataions as part of the culture of playing Japanese games it should still be getting checked.
  • 8/28/2009 9:55 PM In reply to

    Re: Doesn't peer review also include languages check?

    Well, at least I have to apologize to the creators of Duologue, because as I see in your FAQ, the fails for the two other games were still correct then

    Heh, honestly, as one of the creators of Duologue I thank you for settling it... because we didn't know.

    We were unsure as to whether or not non-localized games were allowed in a given region. But the weekend before we decided to upload our release we saw "Jump" (コロンでジャンプ) had been cleared for non-Japanese regions... (and we were happy for it because we liked the game, English localization or not :-)

    So we decided to go ahead and do the same with Duologue. Our logic was that if we weren't supposed to release them non-localized we'd be rejected and we'd know for future games/releases.

    So thank you for helping us settle the matter :-)

    The German description of Duologue is fine though, just a few very minor grammatical mistakes,


    Actually, I'm really glad that it's acceptable. Each of our translators were not gamers, and they struggled with the game terms.

    As a complete aside, I've actually changed my mind as to whether or not non-localized games should be allowed in a given region. Before seeing "Jump" (コロンでジャンプ), I felt the same way you originally did- that non-localized games shouldn't be allowed. However, now I think it's a good thing because, as a gamer-myself, I really enjoy seeing some of these quirky titles that I otherwise wouldn't have.
  • 8/28/2009 10:08 PM In reply to

    Re: Doesn't peer review also include languages check?

    While you can release in any territory its not madnatory to translate even the descriptions. In fact doing so will slow down your review becuase you will now need several reviewers who speak each language. The last game that tried to hit all 4 languages ended up timeing out the review! Now we have German and Japanese as well.
  • 8/28/2009 10:24 PM In reply to

    Re: Doesn't peer review also include languages check?

    The ZMan:
    While you can release in any territory its not madnatory to translate even the descriptions. In fact doing so will slow down your review becuase you will now need several reviewers who speak each language. The last game that tried to hit all 4 languages ended up timeing out the review! Now we have German and Japanese as well.

    Ah, I had no idea. The process is a bit opaque to people on their first game release :-)

    At any rate, we figured it couldn't hurt to have at least localized descriptions to the game. But now we know.

    For our next title we've actually been working on getting the in-game menus localized as well... If that causes a potential timeout on the review process, maybe we don't actually want to do that.
  • 8/29/2009 3:01 AM In reply to

    Re: Doesn't peer review also include languages check?

    The ZMan:

    however I am guessing that Japanese reviewers who speak English are passing them at the same time as Japanese (valid if they speak both languages) and so the bad English is getting through.


    Hmm. I think that this is over-guessing unless you provide a valid evidence though you don't have to have one for just making a guess.
  • 8/29/2009 5:05 AM In reply to

    Re: Doesn't peer review also include languages check?

    The same developers making Engrish come from the same pool that are passing it. I might have to rethink my plan, for great justice, to include a badly-translated Japanese description; darn.
  • 8/29/2009 5:23 AM In reply to

    Re: Doesn't peer review also include languages check?

    Runway 360:
    The ZMan:

    however I am guessing that Japanese reviewers who speak English are passing them at the same time as Japanese (valid if they speak both languages) and so the bad English is getting through.


    Hmm. I think that this is over-guessing unless you provide a valid evidence though you don't have to have one for just making a guess.


    Most of the regular English speaking reviewers post to the forum when they pass or fail - I see few english comments in any of the threads. There are at least 3 Japanese games out there that passed very fast, mostly during Japanese hours which have terrible english translations in the description - most of them pass so fast I don't even get a chance to look. English reviewers would have raised those as an issue at least in some cases and I've seen no comments along those lines

    Yes at the end of the day I am guessing since mods don't have access to the actual data but I've spent the last 6 months in these review threads and you would be amazed at the trends I can spot as a result. Lets call it an educated guess.

    Don't think we're picking on Japanese games - we've had some European games that used google translate for their English and we made the same observations and fails. When English people use translation tools to get French or German those countries also can fail games. It very important to get a good translation wherever possible.

    As one of the few Japanese/English speakers in here I hope you offer to help them translate at least their descriptions. I would be happy to improve them for people too - please let them know.
  • 9/7/2009 11:24 AM In reply to

    Re: Doesn't peer review also include languages check?

    In this thread, I learned that bad translation can be a valid fail reason. My question is ...
    What specific procedure do you think I should take to fail a game when I spot bad translation?
    How actually did some of you fail games for having terrible translation?

    Should I cancel reviewing by closing the browser since I can't review the translated description in the first page
    and thus can't check the checkbox in the same page which says "I have downloaded and played
    this game and reviewed the content on this page."

    Or should I fail the game by checking either one of the following checkboxes in the Game Defects section?:

        "Grossly Misrepresents Content in Content Info" - Bad translation fails in representing what the game is about.
        "Inappropriate for Xbox LIVE" - No way! We shouldn't force Xbox Live members to read this terrible translation.

    Or should I approve (complete reviewing) the game on the condition that, in the final "Summary" page, I don't check
    the checkbox of the language which matches with one in which bad translation was made.

        Example: Let's assume that I understand Japanese and English. I am reviewing a Japanese-only game which has
                    Japanese and English descriptions. The game is O.K. The Japanese description is O.K. However, the English translation
                    is so terrible that it makes me wonder if it has any meaning. So, I complete my review by checking only the "Japanese"
                    and leaving "English" unchecked.

  • 9/7/2009 7:21 PM In reply to

    Re: Doesn't peer review also include languages check?

    I would use the 

    "Grossly Misrepresents Content in Content Info" - Bad translation fails in representing what the game is about. And put notes in the reivew notes and the fourm explaining the issue.

    If you see a fail reason then don't remove that language and pass it as your vote might be what puts a badly translated game into production.
  • 9/8/2009 6:54 AM In reply to

    Re: Doesn't peer review also include languages check?

    Thanks. I mentioned the translation issue in the Japanese forum.
  • 9/8/2009 2:14 PM In reply to

    Re: Doesn't peer review also include languages check?

    Since I don't know how many further questions I will have regarding peer-reviewing games, I should reuse
    this thread.

    I know some Indie Games require bilingual peer-reviewers. A Japanese-language game with English and Japanese descriptions
    can't be passed only with Japanese-speaking peer-reviewers. It also requires some reviewers who speak two languages.

    My question is if there are any cases of submissions which require trilingual reviewers or reviewers who speak more languages.

    In my understanding, peer-review process requires bilingual peer-reviewers when the number of supported languages in a game
    doesn't equal the number of languages for which descriptions are prepared, in other words, when the same peer-reviewer has to
    simultaneously review a game in language A and its description in language B.
  • 9/8/2009 2:25 PM In reply to

    Re: Doesn't peer review also include languages check?

    No games require bilingual reviewers... they require reviewers from each language as part of the review pass.

    To be approved a game must get at least N passes with at least M passes in each language. (N and M are numbers which are not public and can be changed at any time)


    e.g. "My Cool Game" is marked as English, French and German and includes descriptions in English, French, Spanish and Japanese. (yes I know that's odd)
    • This means the required review languages are English, French, German, Spanish and Japanese. (the union of the game and description languages)
      [NOTE: Some of this has been clarified by Microsoft and was wrong... it was my understanding at the time based on previous discussions with Microsoft. See http://forums.xna.com/forums/p/38493/224379.aspx#224379 for clarification.]
    • Lets say N is 4 and M is 1 (they are not that's just an example)
    • This means we need at least 5 passes total with at least one person from each language
    • If the first 5 reviewers only speak English then we have our 5... but we don't have the one from each language. At this point any further English reviews are wasted.
    • Then we get a bilingual French/Spanish speaker which checks off those languages
    • Then we get a English/German speaker who checks off both languages (but the English one has no effect)
    • Finally we get a Japanese reviewer and the game passes
    • In this case it took 8 people instead of the normal 5.
    If you speak one language then your 'pass' counts only for that languase. If you are a bilingual person then your 'pass' vote will count for both languages. If you are trilingual then you can mark 3 etc (though rare I assume). However you should only mark that language if you are native speaker or VERY fluent.

  • 9/8/2009 3:21 PM In reply to

    Re: Doesn't peer review also include languages check?

    The ZMan:


    e.g. "My Cool Game" is marked as English, French and German and includes descriptions in English, French, Spanish and Japanese. (yes I know that's odd)
    • This means the required review languages are English, French, German, Spanish and Japanese. (the union of the game and description languages)
    • Lets say N is 4 and M is 1 (they are not that's just an example)
    • This means we need at least 5 passes total with at least one person from each language
    • If the first 5 reviewers only speak English then we have our 5... but we don't have the one from each language. At this point any further English reviews are wasted.
    • Then we get a bilingual French/Spanish speaker which checks off those languages
    • Then we get a English/German speaker who checks off both languages (but the English one has no effect)
    • Finally we get a Japanese reviewer and the game passes
    • In this case it took 8 people instead of the normal 5.



    Thank you The ZMan, as always.

    In your example, a Japanese reviewer casts a final vote and the game passes. I could be an idiot but I'm still wondering how the Japanese
    could do reviewing even though the person is not fluent in any of supported languages in the game. Does the system deem that the Japanese
    fulfilled duty if in this case he/she checked the Japanese description without checking the game?

    Also, you wrote that no games require bilingual reviewers (well bilinguals occasionally accelerrate the peer-review process).

    If you don't mind, I'd like to discuss what I came up with as a game which could possibly require trilingual reviewers.

    I can think of two types of games which include three languages. The fist type is a game showing only one language which
    the console's language setting automatically chooses from the three.

    The second one is something like an RPG in which text is written in Japanese, warriors speak French and wizards speak English.
    In short three languages are mixed. Previously, I thought this game require trilingual peer-reviewers. Now, I think a Japanese-
    speaking reviewer only has to concentrate on reviewing the Japanese part.

    [EDIT]
    Could you please read this thread (Spyn Doctor's comment)?
    http://forums.xna.com/forums/t/38454.aspx

    Didn't it discuss cases of games which require bilingual reviewers?

  • 9/8/2009 3:51 PM In reply to

    Re: Doesn't peer review also include languages check?

    The ZMan:
    No games require bilingual reviewers... they require reviewers from each language as part of the review pass.


    I think that is not entirely correct. Or more precisely: It is technically correct, but in reality, you will need bilingual reviewers in some cases. For example the following case:

    The game itself is Japanese only, but the game description is both Japanese and English.

    As per your (ZMan's) description above, this would simply mean that you need N reviews, of which at least M are Japanese and M are English.

    The problem is, if you have only reviewers that are monolingual (is that a word?) either in Japanese or in English, then you will never reach this goal: You will easily get your M Japanese reviews, and M may even be >= N. However, you will not get the required M English reviews, because an English reviewer who only speaks English can not review a game where the only in-game language is Japanese. There simply is no review option for such a reviewer to say: "Of this game I have reviewed only the (English) description". The reviewer will have to say "I have played and reviewed the whole game", which is simply impossible if he understands only English (because how would he know that the game did not contain any forbidden text if all in-game text is Japanese?).

    So what you essentially need is at least M reviewers which are bi-lingual in English and Japanese.

    EDIT: So at worst case, if you augment your Japanese-only game with descriptions in all 6 languages, then essentially you need reviewers for Japanese, plus bi-lingual reviewers for all the combinations Japanese/English, Japanese/German, Japanese/French, Japanese/Spanish and Japanese/Italian. (Or if you are lucky you have someone who speaks several of these languages - but I wouldn't count on it.)

    Doc
  • 9/8/2009 5:30 PM In reply to

    Re: Doesn't peer review also include languages check?

    You are giving wrong info. I don't want to cross-post so please have a look here.

    Cheers,
    Pino
  • 9/8/2009 5:45 PM In reply to

    Re: Doesn't peer review also include languages check?

    Probably best to continue the discussion in that thread, although it doesn't look like it needs to continue from the answer there.
  • 9/8/2009 7:13 PM In reply to

    Re: Doesn't peer review also include languages check?

    We can surely continue the discussion in the other thread, although I too don't think that this is necessary, as everything was explained there already.

    However, as a clarification for anyone who should read this thread at a later time, I can not let this stand without a reply:
    PinoEire:
    You are giving wrong info. I don't want to cross-post so please have a look here.

    Because this is not true. The information I gave in my posting above is not wrong, but is totally correct. I simply explained (and gave an example of) how there may indeed be situations where bi-lingual reviewers are required to pass a game. The fact that this is so is confirmed officially by Julien Ellie from the XNA team in this post in the other thread you linked to:

    http://forums.xna.com/forums/p/38493/224390.aspx#224390

    (It's not a confirmation of my posting explicitely, but simply explains the same facts as I explained.)

    Doc
Page 1 of 1 (20 posts) Previous Discussion Next Discussion