Two English translations were prepared, one by a professional human translator and the other generated by ChatGPT. Sixteen Professor of English Literature were shown only the original Korean text and the two English versions, without being told who translated which.
In this blind review, most professors ultimately favored the AI-generated translation, while a small number preferred the human version or found the result inconclusive.
Why Professors Rated the ChatGPT Translation Higher
Professors who favored the AI translation cited its strong grasp of Korean historical and cultural context, as well as its ability to preserve the rhythm, tone, and rhetorical structure of the original poem.
Many evaluators emphasized that the AI version felt more “literary” in English, less explanatory, more restrained, and closer to how moral verse is traditionally rendered in Anglophone poetry.
At the same time, even professors who preferred the human translation or chose “undecidable” acknowledged that the qualitative gap between AI and human translation has narrowed to an unprecedented degree.
What Makes a Translation Truly Excellent?
The test highlighted a crucial point: translation quality is not determined solely by grammatical accuracy. Instead, professors evaluated translations based on whether they:
- Convey the author’s worldview
- Preserve structural balance and rhythm
- Make conceptually appropriate lexical choices
In literary translation especially, correctness is a baseline. What matters is whether the translation interprets meaning rather than merely transferring words.
1) Understanding Context
Context emerged as the most decisive factor.
A widely discussed example comes from the line:
“위로 하늘 / 아래로 땅이 / 내가 한 일 모를 거라 여긴다면 / 이는 누구를 속이려는 것인가?”
The human translation rendered “하늘” as Sky, treating it as a physical space.
ChatGPT translated it as Heaven, capturing its moral and metaphysical significance.
Given that Jang Yu was a Confucian scholar, professors agreed that Heaven, as a moral authority and ethical witness, was a more faithful interpretation than a literal reference to the sky.
This single choice demonstrated how historical, philosophical, and cultural context can reshape meaning.
2) Vocabulary and Lexical Precision
Vocabulary choice proved to be the clearest technical differentiator.
Professors noted that the AI translation:
- Used fewer, more deliberate words
- Maintained parallelism (“Above is Heaven, below is Earth”)
- Preserved the poem’s didactic and aphoristic tone
By contrast, the human translation was described as:
- More prosaic
- More verbose
- Safer, but stylistically conventional
Several professors commented that the AI version better reflected the poetic restraint characteristic of classical moral texts.
This reinforces a broader point that emerged throughout the evaluation: without contextual understanding, even grammatically flawless translations can miss the core of the original text. Especially in literary translation, meaning is embedded in cultural assumptions, ethical frameworks, and historical usage of language.
Flitto’s Strength: Natural and Accurate Translation Through Context
At Flitto, this result reflects what we have observed for years in AI training data.
High-quality translation models do not emerge from algorithms alone. They are shaped by context-rich datasets that reflect how humans judge nuance, intent, and appropriateness across languages.
Flitto has spent over a decade building such datasets for global AI companies. focusing on why one translation is preferred over another.
Why Flitto’s Context-Based Translation Is Possible
1. Large-Scale, High-Quality Parallel Corpora
Flitto has built multilingual parallel corpora through a crowdsourced platform where:
- Multiple translations of the same source are compared
- Context, tone, and nuance are evaluated
- Meaning preservation matters more than literal matching
This allows AI models to learn that translation is choice-driven, not deterministic.
2. Multi-Layer Review: AI + Human Judgment
Before data is finalized:
- AI models filter out outputs that fail minimum quality thresholds
- Human reviewers assess contextual accuracy and stylistic suitability
- Internal experts conduct final validation
This layered process encodes human evaluation logic directly into training data.
3. Teaching Models to Choose Meaning
The Sky vs. Heaven distinction is exactly the kind of judgment Flitto’s datasets are designed to teach.
By training AI on data that reflects cultural, philosophical, and situational context, models learn when literal accuracy must give way to conceptual fidelity.
A Turning Point for Translation
This blind test does not signal the end of human translators. Rather, it marks a shift in where human expertise is most valuable, from direct translation to data design, evaluation, and ethical oversight.
As Rep. Min Hyung-bae noted: “AI is already an irreversible reality. The task now is to use its efficiency wisely, while reconsidering the cultural context and ethics that only humans can define.”
The future of translation will not be human versus AI, but human judgment embedded within AI.