After Babel, The Machine?

After Babel the world speaks many languages, and someone has to translate; George Steiner took the phrase as the title of his 1975 study of translation. Here the translator is a machine. When it translates a novel, does it make the choices a human translator makes? And if it reads that translator’s work first, does it learn them?

Was für eine Süße …

Tergit’s German, 1951

What sweetness …

JuneOctoberall seventeen machine versions (Claude Opus 4.7 and Opus 5.5)

How sweet the air was …

Sophie Duvernoy’s English, 2025

Across the chapter, at 240 points where the German leaves a choice, any two machine versions take the same option at 63–⁠93% of them; a machine version and Duvernoy’s at 38–⁠52%. Reading twenty of her earlier chapters first, with an instruction to continue her voice, raised the machine’s agreement with her from 43% to 50%, mostly where her chapter repeats her earlier renderings. Figure 5, Figure 3

Chapter 25 of Gabriele Tergit’s Effingers opens with a refrain whose second sentence, shown above, returns four times. It is one of those points, with four listed options: keep the verbless exclamation, as all seventeen machine versions do; expand it into a clause with a subject and a verb, as Sophie Duvernoy does in her published translation; or turn it into one of two other kinds of exclamation. Nothing in her earlier chapters shows how she would render it.

The seventeen machine versions

June 2026, Claude Opus 4.7, six briefs

  • A, persona built from her own writing about Tergit
  • B, persona self-built from Tergit's books and their German world
  • C, the control: no persona, no style brief
  • C-let, the control with permission to domesticate
  • D-let, B's sources plus four English novelists, with permission to domesticate
  • D-aim, the same persona and reading as D-let, told to follow a domesticating school

October 2026, Claude Opus 5.5: C0 and branches of one saved reading, each named by its dose, the number of her chapters it read first

  • C0, the first study's control brief on the newer model
  • dose 0, run 1
  • dose 0, run 2
  • dose 0, run 3
  • dose 1
  • dose 3
  • dose 10
  • dose 20, run 1
  • dose 20, run 2
  • dose 20, run 3
  • dose 24

Two experiments

Both use the same chapter: Chapter 25, “Spring”, of Effingers, Tergit’s novel of Berlin families, published in 1951. A refrain marks the hours of one Saturday in March 1887, from ten in the morning to three the next morning; a girl quotes a Schumann song in a love letter; the street talks Berlin dialect. Sophie Duvernoy’s English translation of the novel was published by New York Review Books in 2025. It is the one human translation compared here, and “her” on this page means Duvernoy.

The first study, in June 2026, tested the brief. Six instances of Claude Opus 4.7 translated the chapter under six briefs, each in two parts. The first, a persona, was a translator’s profile given to the instance or built by it before translating; four briefs had one and two did not. The second, the style brief, was empty, gave permission to domesticate (to turn the German into ordinary English) or told the translator to follow a domesticating school (in Schleiermacher’s terms, moving the author toward the reader).

PersonaNo style briefPermission to domesticateA domesticating school
Built from her own writing about TergitA
Self-built, from Tergit’s books and their German worldB
The same, plus four English novelistsD-letD-aim
NoneCC-let
The first study’s six briefs. Each label is one version of Chapter 25: C is the control, -let marks the permission to domesticate, -aim the instruction to follow a domesticating school. D-let and D-aim share one persona and one reading of the novel, branched at the moment of translation; the other four each read the novel separately. The four English novelists are Wharton, Powell, Mitford and Isherwood.

The second, in October 2026, tested her example. One instance of a newer model, Claude Opus 5.5, read the whole German novel under a minimal translator’s brief, which held back the translation instructions until after the reading, and kept notes; by the end, the notes held a provisional plan for Chapter 25. The conversation, notes included, was saved and copied into branches, so that every branch began from the same reading. Each branch then read a number of her chapters, counting back from Chapter 24 (none, one, three, ten, twenty or all twenty-four), and translated Chapter 25; that number is the branch’s dose. Branches that read her chapters were also told, in two added sentences, to continue her voice “seamlessly”. The runs used the model’s highest reasoning setting, with file tools and no internet. A reference run, C0, repeated the first study’s control brief on the newer model, with a reading of its own.

The second study’s branches. Each rust square is one chapter of her English, read start to finish before translating; the blue square is the branch’s own Chapter 25. Every branch began from one saved reading of the German novel. Claude Opus 5.5, 13–⁠15 October 2026.

Both studies compare versions point by point, at places where the German leaves a real choice. The first coded 45 such points. For the second, another Claude instance, working from the German and her translation alone and without seeing any machine version, built an instrument: a list of 240 points, each with its options, the ways it can be rendered. Every version, June and October, was scored on both sets of points. On the 240 a script decided wherever the wording settles the option, and a Claude instance coded the rest blind, with the texts under neutral labels; the October versions’ 45 points were coded blind throughout, while the June versions’ first coding was made with their labels known. Two versions agree at a point when they take the same option. The studies measure who chose what, not quality.

Why it matters now

Since 2024, publishers have begun to try machine translation for books, or have said they are testing it. In November 2024 Veen Bosch & Keuning, the largest Dutch publisher, confirmed a trial of AI-assisted translation into English for fewer than ten commercial novels whose English rights had not been sold (Publishers Weekly). In December 2025 the French literary translators’ association ATLF and the collective En chair et en os said that Harlequin in France was dropping several dozen regular translators for an agency that machine-translates and has freelancers post-edit; the publisher called it a test and said no series had been translated by machine alone (ATLF, Publishers Weekly). In September 2026 Publishers Weekly reported that HarperCollins’s chief executive had said the company aims to automate its overseas translations.

Translators and the institutions around them have answered with surveys and rules. In a survey released in October 2026 by CEATL, the European council of literary translators’ associations, over 77% of 1,785 literary translators took a negative view of generative AI, roughly 30% reported fewer assignments or clients because of it and 37% had considered changing careers (CEATL). In CEATL’s previous survey, in 2023, half had used AI tools on a literary project by their own choice, mostly for single phrases or sentences (CEATL). Spain’s 2025 grants for translation between its languages exclude work translated with generative AI (Ministerio de Cultura); in Denmark, translators, authors and publishers agreed that post-editing a machine translation should not be called translation (Dansk Forfatterforening); a new translation prize asks for fully human work (Janklow & Nesbit).

Much of this debate is about work, credit and whether a machine translation is good enough. A literary translation is also a set of choices that belong to one person. The two studies test three assumptions: that a brief steers a model, that examples teach it one person’s manner and that one output stands for what the model does.

The results in brief

The machine versions resemble one another more than any of them resembles Duvernoy’s.
The two machine versions furthest apart still take the same option at 63% of the 240 points; the machine version closest to her takes her option at 52%. On these points, no difference of brief, model, reading or run puts two machine versions as far apart as any of them is from her. Figure 5
Neither a domesticating school nor a persona brought a version much closer to her choices.
Under the school’s instruction D-aim scores 46 on a 0–⁠100 scale of how far a version moves the German into English, against 17 for its twin D-let and 75 for hers; yet D-aim matches her at 12 of the 45 points, D-let at 11. The four June versions with a persona, one of them built from her own writing about Tergit, match her at 11–⁠13 of the 45 points, the two without one at 17 and 18. Figure 1
Her chapters, with the instruction to continue her voice, moved the machine toward her earlier renderings, not toward her new choices.
After twenty of her chapters the branches use one of her earlier renderings at 118 of the 155 points where her chapters 1–⁠24 show one, against 90 without her chapters and 115 in her own Chapter 25. Where her Chapter 25 departs from her earlier renderings or has none to follow, the branches barely change. Figures 2 and 3
The same setup, run twice, gives a different translation.
Two runs from an identical saved reading choose different options at 18–⁠34 of the 240 points, fewer than the 116 or more at which any machine version differs from hers. Figure 5

The first study: six briefsJune 2026, Claude Opus 4.7

The first study’s 45 points cover proper nouns, loanwords and titles, forms of address, idioms, the songs, the Berlin dialect, the refrain and the paragraphing. At each, the coding records which rendering a version chose; at 36 of them, and at two further loanwords, it also records whether the rendering keeps the German or moves it into English: English names, titles and idioms, and for the dialect an English colloquial register, where standard English counts as keeping the German. The share moved into English places each version on a scale from 0 (foreignizing, in translation studies’ terms) to 100 (domesticating).

Every June version keeps more of the German than hers does, whatever its brief.

June 2026six briefs, one model

D-aimD-let

17–⁠46

Her translationSophie Duvernoy, 2025

75

keeps the Germaninto English

Figure 1. How far each version moves the German into English, over the 38 coded items that record it (36 of the first study’s 45 points and two further loanwords). For each kind of item the measure is the share moved into English; the score averages the measures, each weighted by how many items it covers. The shaded band spans the June versions; dots are stacked only where they would overlap.
The numbers
Each version’s score, 0 to 100
VersionScore
A, persona built from her own writing about Tergit27.6
B, persona self-built from Tergit’s books and their German world19.7
C, the control: no persona, no style brief29.0
C-let, the control with permission to domesticate30.3
D-let, B’s sources plus four English novelists, with permission to domesticate17.1
D-aim, the same persona and reading as D-let, told to follow a domesticating school46.1
Her translation75.0

Any two June versions made the same choice at 47–⁠80% of the 45 points, 64% on average; a June version and Duvernoy’s at 24–⁠40%, 31% on average.

Two pairs of versions differ by a single instruction: C and C-let (which also read the novel separately) and D-let and D-aim. Adding permission to domesticate to the control brief took the score from 29.0 (C) to 30.3 (C-let), a step no larger than the spread among three October runs from one saved reading, on the newer model (26.3–⁠28.9). A domesticating school, given at the moment of translation, took it from 17.1 (D-let) to 46.1 (D-aim), the largest shift in either study. Agreement with her moved from 17 to 18 of the 45 points (C to C-let) and from 11 to 12 (D-let to D-aim): less than the spread among those three October runs (16–⁠19). Each June brief ran once.

Version A, whose persona was built from her own critical and biographical writing about Tergit, matched her at 13 of the 45 points, fewer than either control. On the second study’s 240 points the two groups’ agreement with her overlaps: 90–⁠100 points with a persona, 94–⁠104 without.

Run again on Chapter 26, with that chapter’s own coding (made with the labels known), the six briefs again kept the machine versions together and hers apart: a band from 28 to 53 on its scale, with hers at 69; the versions agreeing with one another at 46% of its coded points on average, and with hers at 13%. That chapter’s points are different, so its percentages do not compare with this one’s (the replication).

The second study: her chapters firstOctober 2026, Claude Opus 5.5

The saved reading’s notes already held a provisional plan for Chapter 25. It fixed a frame: keep the verbless fragments verbless, keep the present-tense Wollmarkt passage in the present, set the wording of each return of the refrain. It also fixed conventions: German street names with their ß, titles such as “Figaro” in quotation marks, the street talk in colloquial English with dropped endings. The study therefore asks whether her chapters move a reading that already has a plan.

Branches that read twenty of her chapters use more of her earlier renderings than branches that read none, about as many as her own Chapter 25 does.

points, of 1558090100110120her own Chapter 25: 115013102024mean 90mean 118

her chapters read before translating (doses, not to scale)

Figure 2. Of the 155 points where her chapters 1–⁠24 show how she rendered the same item, a comparable one or a looser analogy, the number at which a version uses an option she used there. Dose 0 and dose 20 were run three times; the line joins the means, and the axis begins at 80. Her own Chapter 25 is not the ceiling: at 40 of these points its option is not one her earlier chapters document, including the 25 where it departs from all her earlier renderings.
The numbers
Points, of 155, at which a version uses one of her earlier renderings
VersionPoints
Dose 0, run 190
Dose 0, run 289
Dose 0, run 392
Dose 1101
Dose 3104
Dose 10105
Dose 20, run 1118
Dose 20, run 2119
Dose 20, run 3117
Dose 24115
C0, its own reading, none of her English94
Her own Chapter 25115

Without any of her English, the branches already use one of her earlier renderings at 90 of the 155 points where her chapters 1–⁠24 show one: many of her choices are ones the model makes on its own. After twenty of her chapters they do so at 118 (runs 117–⁠119), slightly more often than her own Chapter 25. Every point gained at dose 20 uses a rendering found in the chapters that branch read, so the gains could have come from them. At doses 1 and 3, most gains use renderings the instrument documents only in chapters the branch had not read (6 of 9, 7 of 11), yet they are not noise. Most are the plan’s conventions giving way to hers: at all six street names whose German spelling the plan kept, every branch that read her chapters spells them -strasse, as she does and as her Chapter 24 already spells others, while no dose-0 run does. The dose-0 runs’ own extra points (1–⁠5 per run) fall at different points each time.

After twenty of her chapters, the branches match her more often where her Chapter 25 repeats her earlier renderings, and hardly more often anywhere else.

Where her Chapter 25 …

Repeats all her earlier renderings78 points

65% → 83%

Repeats some of them52 points

37% → 42%

Departs from all of them25 points

25% → 17%

Has no earlier rendering85 points

33% → 35%

Figure 3. Share of points at which a version takes her Chapter 25’s option, means of three runs; hers is 100% by definition. The rows sort the 240 points by whether her Chapter 25 repeats her earlier renderings, a classification made by the Claude instance that built the instrument, re-examined at 111 points by later Claude reviewers, who changed some of it, and not checked by a person. Together the last two rows hold 110 points; the branches match her at 35 of them at dose 0 and 34 at dose 20.
The numbers
Points at which a version takes her Chapter 25’s option, mean of three runs
Where her Chapter 25PointsDose 0Dose 20
Repeats all her earlier renderings7850.364.7
Repeats some of them5219.022.0
Departs from all of them256.34.3
Has no earlier rendering8528.330.0

The gains come where her Chapter 25 repeats her earlier renderings: from 50 to 65 of 78 such points. Where it departs from all of them, the dose-20 runs match her at 4–⁠5 of 25 points, the dose-0 runs at 6–⁠7; where her earlier chapters hold no rendering of the point, the branches match her at 25–⁠33 of 85, at every dose. A low score on her departures can also mean that the instrument matched a point to a loosely comparable earlier one, or that her choice belongs to this chapter’s context; the counts cannot tell these apart.

The street dialect is one of those conventions, and here the change moved the machine away from her Chapter 25. The plan wrote the street talk with dropped endings, and the three dose-0 runs do so (goin’, nothin’); C0, with a reading of its own, has a few such spellings. Her earlier chapters render dialect both ways: the instrument lists standard English in six of them (four only by loose analogy) and marked spellings in three. Her Chapter 25 writes the street talk as heavy eye-dialect (respelled words that mark speech as dialect), which the instrument reads as extending the marked speech of Chapter 6. No June version uses eye-dialect, in the first study’s coding. Of the seven branches that read her chapters, only dose 3 keeps the dropped endings clearly, in October’s blind coding, and three others have one marked spelling each: the branches gave up the plan’s convention for standard spelling. Dose 1, which read no chapter with a listed dialect example, gave it up too, so the change cannot be traced to her examples.

One of her choices no machine version makes; others vary between June and October, or change once the machine has read her chapters.

Her Chapter 25 …June 2026six versionsOctober, without her EnglishC0 and the three dose-0 runsOctober, after her chaptersdoses 1 to 24, seven branches
Expands the refrain’s second sentence into a clause at its first appearance0 of 60 of 40 of 7
Keeps the Schumann song rhymed0 of 62 of 45 of 7
Writes the street dialect as eye-dialect0 of 64 of 41 of 7
Joins or divides some of the German’s paragraphs3 of 60 of 40 of 7
Completes 42 of the German’s verbless fragments9–⁠1411–⁠129–⁠14
Sets the tense at 6 coded places1–⁠322
Figure 4. How many machine versions make each kind of choice she makes; for the fragments and the tense, how many of hers each version shares, as a range. The song and the dialect come from the first study’s 45-point coding, blind for the October versions and made with the labels known for June’s; the refrain, the paragraphs, the fragments and the tense from the second study’s scoring.

The plan’s frame, by contrast, held. Of the six places coded for tense, every branch matches her at the same two; at the two passages the German narrates in the present, she uses the past and no branch does. Every branch keeps the refrain’s verbless What sweetness and all 179 of the German’s paragraphs, while she joins or divides paragraphs in three places. The plan cannot be shown to be the reason: C0, on the same model without the plan, makes the same choices. June’s six mostly do too, though D-let puts the Wollmarkt passage in the past, as she does, and three of the six join or divide some paragraphs.

The branches that read her chapters also take on the house style of the published translation: American spelling and the closed em dash, where two of the three dose-0 runs spelled British and set spaced en dashes. They then use the dash more often than she does, about 8 times per 1,000 words at dose 20 against about 4 in her Chapter 25. These conventions may be the publisher’s as much as hers.

The model and the reruns

Even without her English, the four October versions (C0 and the three dose-0 runs) differ from June’s six. They match her somewhat more often on the 45 points (16–⁠21, against 11–⁠18 in June). They agree with one another more closely (85–⁠93% of the 240 points, against 71–⁠88% among June’s six); three of the four share one reading, but C0, with a reading of its own, still agrees with them at 85–⁠88%. And some keep the song’s rhyme or write the dialect as eye-dialect, which no June version does. On the first study’s scale, though, every October version, with or without her English, falls between 26 and 38, inside June’s band. Between June and October the model, the brief, the reading, the run settings and, on the 45 points, the coding all changed at once, so these comparisons describe outputs, not causes.

Two runs from the identical saved reading, with identical instructions, choose different options at 18–⁠34 of the 240 points, and the spelling can switch between British and American. The points where repeat runs split gather in period terms, idioms, dialect and verse (22 and 30 of the 129 points in those categories, in the two sets of repeat runs) and are rare in names, titles and foreign words (1 and 1 of 47). In process research, human translators’ renderings also diverge most where the source is hard (Dragsted 2012).

What the two studies show

Any two machine versions agree more often than any of them agrees with hers.

Two runs of dose 0identical reading and instructions; 3 pairs

91–⁠93%

Two runs of dose 20the same, each after twenty of her chapters; 3 pairs

86–⁠90%

C0 and a dose-0 runone model, two briefs, two readings; 3 pairs

85–⁠88%

Two branches at different dosesone reading, the first run of each dose; 15 pairs

76–⁠91%

Two of June’s sixone model, six briefs, five readings; 15 pairs

71–⁠88%

June’s C and October’s C0one brief, two models, two readings

79%

Any two machine versionsevery pair, one tick each; 136 pairs

63–⁠93%

A machine version and hersevery machine version; 17 pairs

38–⁠52%

Figure 5. Share of the second study’s 240 points at which two versions take the same option, one mark per pair. The options come from the instrument another Claude instance built from the German and her translation alone, without seeing any machine version. Two renderings that both fall outside the listed options count as the same choice, which accounts for at most 11 of the 240 points in any pair of machine versions. The points are not independent: repeated terms and related choices move together.
The numbers
Share of the 240 points at which two versions take the same option
Two versionsSame optionPairs
Two runs of dose 091.2–⁠92.5%3
Two runs of dose 2085.8–⁠90.0%3
C0 and a dose-0 run85.4–⁠88.3%3
Two branches at different doses75.8–⁠90.8%15
Two of June’s six71.2–⁠87.5%15
June’s C and October’s C078.8%1
Any two machine versions62.9–⁠92.5%136
A machine version and hers37.5–⁠51.7%17

A brief steers the model, but not toward her choices. Briefs, model generations, readings and reruns all change a machine translation. Part of the machines’ resemblance is shared structure: every machine version keeps the refrain’s What sweetness verbless, and none completes more than 14 of the 42 verbless fragments she turns into sentences. The five least alike pairs all include D-aim, the version told to follow a domesticating school.

Her chapters, with the instruction, carried over her earlier renderings, not her next decisions. With her chapters and the instruction to continue her voice, the machine took on her documented earlier renderings and the published translation’s conventions, giving up its own plan’s. Where her Chapter 25 departs from them or has none to follow, it matched her there no more often than without her chapters (34 of 110 points after twenty chapters, 35 without); with the dialect it moved away from her Chapter 25’s eye-dialect. After twenty of her chapters the branches still chose differently from her at 119 of the 240 points on average, 76 of them where her Chapter 25 departs from her earlier renderings or has none to follow.

One output does not stand for the model. Next to her, though, the spread between runs is narrow. As far as these measurements reach, a publisher who hands this chapter to one of these models gets, before any editing, one draw from a close family, nearer to what another run, another brief or the other model generation would give than to her translation.

Where this sits

What readers prefer

Readers’ verdicts depend on what they are shown. Expert literary translators comparing paragraphs of novels preferred the published human translation to Google Translate’s 84% of the time (Thai et al. 2022). With current systems, avid readers still preferred published human translations of recent novels to translations made with Claude Opus 4.6 and GPT-5.4, yet picked out the human version in only 17 of 30 comparisons and tended to prefer whichever they believed was human; automatic measures, AI judges among them, preferred the machine versions (Ferstler et al. 2026). Readers who saw only the English preferred a multi-agent system’s translations of Chinese web fiction to the human ones (Wu et al. 2025), and in a reception study of a Vonnegut story Catalan readers rated the human translation highest, Dutch readers a post-edited machine version (Guerberof-Arenas and Toral 2024).

How translators choose

Translation studies has long described translating as a series of decisions among the options a text leaves open (Levý 1967), and computational work has found that a single ordinary sentence can have billions of acceptable translations (Dreyer and Marcu 2012). Corpus studies describe a translator’s style as recurring patterns of choice rather than one-off interventions (Baker 2000). Five literary translators of the same Hemingway story into German resolved its ambiguities differently, according to their preferences, knowledge and habits (Kolb 2013); two German translations of one Fitzgerald novel, published the same year, differ in small recurring choices, enough to shift the book’s emphasis (Winters 2007; 2009); and classifiers tell two published English translations of the same novel apart, and from machine output, with 86% accuracy on average (Scalena et al. 2026).

Other studies follow translators who work from machine drafts. Reviewers rated unaided human translation of a Vonnegut story more creative than post-edited machine translation, which scored above raw machine output (Guerberof-Arenas and Toral 2022); four literary translators post-editing large-language-model drafts of a novel worked faster than when translating from scratch, with similar creativity (Castaldo et al. 2025).

What machines can do

Capability research has moved from sentences to documents, and from errors to creativity. Translating literary paragraphs whole improved GPT-3.5’s output over sentence-by-sentence translation, but critical errors persisted (Karpinska and Iyyer 2023). Across four language pairs, German–English among them, published human translations outperformed large language models, whose translations were more literal and closer to the source’s syntax; they also shared more vocabulary with the other systems’ translations than the human translation did (Zhang, Zhao and Eger 2025). A 2026 benchmark of 23 models found that they understood the source well but often rendered metaphors and wordplay literally, with only one model approaching human creativity (Zhang et al. 2026). Outside translation, language models’ answers to standard creativity tasks resemble one another far more than people’s do (Wenger and Kenett 2026).

Given a few translation pairs in its prompt, a model adapts to their terminology and style (Moslem et al. 2023, on non-literary text). In the nearest earlier test of imitation, small open models shown twenty examples of one translator’s work moved toward that translator’s style, but the authors conclude that replicating an individual translator’s style remains highly challenging (Scalena et al. 2026). On other texts and measures, machine and human translations come out close: GPT-4’s English versions of Chinese web fiction mirrored human ones in lexical, syntactic and content features (Yao, Kang and McCosker 2025).

What these studies add

First, they measure resemblance decision by decision, within one model family, across briefs, two model generations and repeated runs, where earlier work measured shared vocabulary across different systems. Second, they test imitation of one named translator at book scale: a frontier model given up to twenty-four of her chapters, where the nearest earlier test gave small open models twenty examples. What the machine took on, with her chapters and the instruction, was her documented earlier renderings, many of them spelling and naming conventions. Third, they show how much one machine workflow varies from run to run, which an evaluation of a single output cannot see; the second study’s literature search, not exhaustive, found no other measurement of it in literary translation.

Limits and open questions

Open questions

Read further

References

The studies’ own pages carry their full references. The second study’s translations and the instrument that scored them are described there; her translation is published by New York Review Books and is not reproduced on this site.