Skip to content
Locale Lab

Chapter 20·Shipping Translations·~12 min·

Scoring a Translation Quality Review (LQA)

Multidimensional Quality Metrics (MQM) standardizes linguistic quality assurance by classifying translation errors across dimensions and weighted severities. Normalized penalty scores per 1,000 words provide an objective release gate.

MQMLQAqualityreview

Overview

Linguistic Quality Assurance (LQA) evaluates translated text systematically using the Multidimensional Quality Metrics (MQM) error typology. Reviewers classify errors by dimension and severity, then compute a normalized penalty score per 1,000 words to make an objective pass-or-fail release decision.

Core rule

Classify every translation error by MQM dimension and severity. Report the normalized penalty score together with the severity weights, pass threshold, and word sample size.

Specification note: Under standard MQM weights (0, 1, 5, 25), a single critical error carries the penalty of 25 minor errors and automatically fails the review round.

Failure mode

Subjective feedback such as “the German translation reads awkwardly” cannot guide vendor corrections or compare quality across releases. Multidimensional Quality Metrics (MQM) replaces subjective impressions with a structured error typology, four severity levels with defined penalty weights, and a normalized penalty formula per 1,000 source words. Teams use the resulting score to enforce vendor service-level agreements, benchmark machine translation engines, and gate production releases.

Stakeholder Teaching Aid & Novice Guide
Cross-functional alignment tool

Judging translation quality based on ad-hoc opinions from bilingual colleagues causes subjective disputes. Multidimensional Quality Metrics (MQM) provides objective scoring.

Business stakes & user risk

Arguing with translation agencies without objective quality metrics delays releases. Teams either accept substandard translations or waste budget re-translating good work.

Rule of thumb

Score translation batches using MQM error typologies (accuracy, fluency, terminology) and weighted severity thresholds.

The Novice Trap vs. Production Standard

Contrast the intuitive assumption with the architecture required for global releases.

Naive mental model

“Translation quality is subjective; you just ask an in-house native speaker if the translation 'sounds good'.”

Naive approach: Subjective review feedback

// Reviewer note: "This German translation feels unnatural. Please rewrite."

Defect:Vendors cannot take actionable corrective steps, and teams cannot measure whether translation quality is improving over time.

Production standard

Objective localization quality assurance categorizes errors by dimension and severity (minor, major, critical) to calculate a reproducible quality score.

Production standard: MQM scoring scorecard

{ error: "Accuracy/Mistranslation", severity: "Major", penalty: 5, sampleWords: 1200, score: 98.2 }

Benefit:Calculates normalized penalty points per 1,000 words against a contractual quality pass mark (such as 98.0+).

Cross-Functional Review Questions

Questions for product managers, designers, engineers, and testers to ask during sprint reviews.

For ProductQuestion #1

“Do our Master Services Agreements (MSAs) with translation agencies specify objective MQM pass thresholds?”

What to look for: Contracts defining maximum allowable penalty points per 1,000 words before payment penalties apply.
For QAQuestion #2

“Are linguistic bug reports logged with standardized MQM categories and severity ratings?”

What to look for: Bug tickets specifying dimension (Accuracy, Fluency, Terminology) and severity (Minor, Major, Critical).
For DesignQuestion #3

“Are design layout issues separated from linguistic quality defects during LQA cycles?”

What to look for: Truncations and text overflows classified under Design/Layout rather than blaming translation grammar.
For EngineeringQuestion #4

“Are resolved linguistic corrections fed back into the centralized Translation Memory?”

What to look for: A feedback loop ensuring corrected terms are permanently updated in TM and glossaries.

Mechanism

Error typology and defect routing

Unstructured translation reviews produce subjective feedback that vendors cannot action and quality scorecards cannot track. A standardized error typology requires reviewers to classify every issue into a defined category. Two reviewers evaluating different batches then produce comparable quantitative metrics.

Multidimensional Quality Metrics (MQM) defines seven top-level dimensions: Terminology, Accuracy, Linguistic conventions, Style, Locale conventions, Audience appropriateness, and Design and markup. Earlier MQM versions named Linguistic conventions "Fluency". Each top-level dimension contains specific child error types.

The error dimension routes the defect to the responsible team. Terminology errors go to the termbase owner. Locale conventions errors usually indicate an engineering defect in date or number formatting that a translator cannot modify. Design and markup errors go to UI engineering, while Accuracy and Linguistic conventions errors return to the translation vendor.

Interactive demo

Guided exercises

  1. 1
    Leave every segment set to No error and click Score my review. The review misses all 8 seeded errors but reports a score of 100 and a PASS verdict. Quality programs track reviewer recall separately because an empty review always scores 100.
  2. 2
    Flag segment 1 (the 3/4/26 date) and test different dimensions before checking the answer key. The correct dimension is Locale conventions because German short dates use DD.MM.YY (04.03.26). Filing this defect under Locale conventions routes the fix to engineering rather than the translator.
  3. 3
    Set segment 6 (the dropped 'on orders over $50' condition) to Accuracy and change its severity from minor to critical. The segment penalty jumps from 1 point to 25 points. At default weights, one Critical error outweighs all seven other seeded errors combined (23 points).
  4. 4
    Flag a clean segment (segment 3 or segment 9) with any error dimension. The scorecard marks the flag as a false positive and lowers reviewer accuracy. False positives on clean segments generate vendor disputes.
  5. 5
    Score all segments accurately and inspect the normalization formula in the arithmetic panel. 48 penalty points across 39 evaluated words normalizes to more than 1,200 points per 1,000 words, clamping the raw score at 0. Small word samples require raw penalty reporting rather than per-1,000-word normalization.
  6. 6
    In the Weight lab below the scorecards, drag all three severity sliders to 0. The raw score rises to 100, but the verdict remains FAIL. The critical-error veto checks the Critical error count directly rather than the weighted score.

Review brief

Ten checkout segments, English into German, 39 source words. Give each error an MQM dimension and a severity, and leave clean segments on No error. Flagging a clean segment counts as a false positive.

Approved termbase

  • cart→WarenkorbApproved. Do not use Einkaufswagen or Korb.
  • sign in→anmeldenApproved. Do not use einloggen.
  1. 01Order confirmation banner

    Source (en-US)

    Your order will arrive on 3/4/26.

    Target (de-DE)

    Ihre Bestellung kommt am 3/4/26 an.

  2. 02Cart totals row

    Source (en-US)

    Total: $1,234.56

    Target (de-DE)

    Gesamt: $1,234.56

  3. 03Primary button, 20 character limit

    Source (en-US)

    Add to cart

    Target (de-DE)

    In den Warenkorb

  4. 04Empty state

    Source (en-US)

    Your cart is empty.

    Target (de-DE)

    Ihr Einkaufswagen ist leer.

  5. 05Section heading

    Source (en-US)

    Order summary

    Target (de-DE)

    Order summary

  6. 06Promotional line under the totals

    Source (en-US)

    Free shipping on orders over $50.

    Target (de-DE)

    Kostenloser Versand.

  7. 07Auth gate

    Source (en-US)

    Sign in to continue.

    Target (de-DE)

    Anmelden um fortzufahren.

  8. 08Promo badge

    Source (en-US)

    Save 20% today

    Target (de-DE)

    Sparen Sie 20% heute

  9. 09Payment error toast

    Source (en-US)

    We could not process your payment.

    Target (de-DE)

    Wir konnten Ihre Zahlung nicht bearbeiten.

  10. 10Banner with inline markup

    Source (en-US)

    <b>Limited time</b> offer

    Target (de-DE)

    <b>Angebot für kurze Zeit offer

Implementation rules

  • Define severity criteria before starting review. Train reviewers to log preferential rewrites as Neutral (0 penalty points). Treating stylistic preference as a weighted error distorts quality scores and triggers vendor disputes.
  • Enforce an automatic critical-error veto. A single Critical error (such as a corrupted placeholder, wrong medical or legal meaning, or exposed PII) fails the delivery immediately regardless of the numeric score.
  • Audit a statistically valid word sample. Review 1,000 to 2,500 words per language, or 10% to 20% of the batch per ISO 5060. For small batches under 500 words, evaluate raw penalty totals rather than normalizing per 1,000 words.
  • Publish the typology, severity weights, and termbase prior to handoff. Provide translators and reviewers with the exact glossary, style guide, and pass threshold before translation begins.

Operational context

Roles

LocalizationVendor ManagementQA

Workflow stages

  • ·Establishing vendor SLAs and quality scorecards
  • ·Arbitrating linguistic review disputes
  • ·Evaluating machine translation post-editing quality

Production incident

When reviewers log stylistic preferences as Minor errors instead of Neutral notes, valid translations fail the quality gate and trigger vendor disputes. Reviewer calibration tracks false-positive flags alongside missed errors.

ISO 5060:2024, Evaluation of translation output ↗

Terms in this chapter

Further reading

Related chapters