I have asked AIs to remember, compose and study scores, and even to review and play their own nocturnes. Today I am asking them for what we performers do with a work that is not ours: to play it. Not to invent notes, but to decide how the ones already written should sound. And with the piece that has been measured more often than any other in the history of piano performance: Schumann’s Träumerei.
Repp, Gould… and a language model
This experiment has two forerunners, and it bears their names.
Can a performance be measured? In 1992 the psychologist Bruno H. Repp measured 28 recordings of the Träumerei to the millisecond, made by 24 great pianists. He found that, beneath the diversity, there is a common shape that grows out of the structure of the music itself, and that two pianists strayed from it more than anyone else: Horowitz and Cortot.
📚 More on Repp and the Träumerei
Repp measured the inter-onset intervals of the notes in 28 commercial recordings. His conclusions, in the Journal of the Acoustical Society of America, vol. 92 (1992):
- The common shape: the big ritardandos come at the end of each section, and within each melodic gesture the accented notes are lengthened. Ritardandos within a gesture follow a parabolic curve, like that of an object slowing down.
- The individualists: with a principal component analysis, most pianists cluster around a common pattern, and there are two almost personal “factors”: Horowitz’s and Cortot’s.
- The repeat: 26 of the 28 recordings repeat bars 1-8, and the pianists play them almost identically both times: almost, but not quite.
Five years later, Repp took it one step further: he built an “average” performance of the Träumerei from ten pianists and had piano students listen to it alongside the originals. It came second for quality… and second to last for individuality. The result was replicated in 2018.
Can a style be imitated from that kind of data? In 2019 Yamaha presented Dear Glenn (press release): a player piano that played any score “in the manner of Glenn Gould”, even works Gould never recorded. It learnt from more than a hundred hours of his recordings: exactly the kind of data Repp measured by hand, but on an industrial scale and with an AI model trained for that alone.
🎹 More on Dear Glenn
Yamaha explains it on the project page. The name itself is a tribute: to Gould, who gave up the concert stage to devote himself to recording and wanted to rethink the relationship between performer and audience. The system was built in three steps:
- Analysis. With the support of the Glenn Gould Foundation, Yamaha analysed more than 100 hours of his recordings and extracted performance data from them: how hard, and with what timing fluctuations, each key goes down.
- Training. A deep neural network learns to go from the score to that performance data, so that it can generate it for any score. Besides what was extracted from the recordings, it was trained on performances by pianists who admired Gould and knew his style well, who were asked to play “in the manner of Gould”.
- Performance. Faced with a new score, the network generates the performance “in the style of Glenn Gould”, and a Disklavier piano plays it. Another in-house technology analyses the sound and movements of the musicians playing with it, so that it can play in an ensemble.
It was premiered on 7 September 2019 at the Ars Electronica Festival, in the basilica of St. Florian’s monastery (Linz): the Aria from the Goldberg Variations solo, the Fuga inversa a 2 clavicembali from The Art of Fugue as a duet with the pianist Francesco Tristano, and the Trio Sonata BWV 1038 with musicians from the Bruckner Orchestra. Gould never played the last two, so there was no recording of his to learn from.
What matters for what follows: Dear Glenn was a specialised system, trained on one pianist’s data for one task. It imitated with data.
And a language model? That is my question. A model like Claude, GPT or Gemini is not a system trained to play the piano: it is a generalist model that has read a great deal about music, about Horowitz and about Repp, but has never listened to a recording as such. Can it, reasoning in its own way, do something like what Dear Glenn did? How much of a performing style fits into what has been written about it? In honour of the two experiments that came before us, I have called this benchmark Repp-Glenn: the surname of the man who measured and the name of the project that imitated.
First, measure the humans
Before measuring the machines we needed something to compare them with, and that took more work than everything else put together.
- Gathering material. On the one hand, real recordings: several by Horowitz (1962, 1965, 1975 and 1987) and by Cortot (on 78 rpm discs from the Internet Archive). On the other, the ATEPP corpus, which has 49 Träumerei automatically transcribed from audio to MIDI: 33 are by Horowitz (it was his encore for a lifetime) and the rest by Argerich, Schnabel, Kempff, Cortot, Ashkenazy, Barenboim, Brendel, Lupu, Uchida, Lang Lang, Lisitsa and François.
- Trying to measure them automatically… and failing. We tried a well-known automatic aligner, which synchronises a recording with its score. On a test performance whose rubato was known in advance, it was out by seconds. Verdict: “no good for measuring rubato”.
- Marking by hand. So the main reference was made the way Repp did it: by hand. The 128 beats of Horowitz’s 1962 studio recording (you can listen to it here) were marked one by one in Sonic Visualiser, with a median error of 17 milliseconds.
- Validating a second method. With those marks as ground truth, we tried another aligner, Nakamura’s, which matches a MIDI performance note by note against its score. Applied to ATEPP’s transcription of that same recording, it reproduces the shape of the rubato with a correlation of 0.898: more than Horowitz resembles himself between the two times through the repeat (0.73). It works for the shape of the curve, not for any single beat.
- The human band. With that, we have 50 human performances measured with the same method, and for each measure we get what pianists normally do: from the 10th to the 90th percentile. Being outside the band is not a mistake; being outside it on everything is a clue.
And a clean score. I started from the Mutopia edition, in the public domain. On opening the code I discovered that the editor had hidden an entire performance in invisible tracks: a tempo map with stepped ritardandos (from 53 to 33 in bar 8) and an ending that slows down to ♩ = 17, plus a pedal track. I removed all of it, along with two “a tempo” markings. The models get Schumann’s text: his dynamics and hairpins, his ritardandos (without saying where they end), the fingering of a 1900 editor and Schumann’s metronome mark, ♩ = 100, which they may follow or not. Everything else they have to decide for themselves.
The exam
To do it properly I set up a small test environment, a lab that runs on my computer:
- The same text for everyone. Each test is sent, without changing a comma, to the terminal tools of Anthropic (Claude Code), OpenAI (Codex) and Google, with whichever model and effort level I choose.
- Each in its own folder. Each run works in isolation, with all its tools: it can write code, compile scores, search the internet… A single attempt.
- A camera recording. Everything is saved: the exact prompt, the complete log of what they do (what they search for, what programs they write, what they check), the time, the cost and what they deliver.
- Measuring what is delivered. The lab compiles the annotated score, renders the MIDI with a sampled grand piano and runs it through the measures I describe below, test by test and against the human band.
This is the prompt for the central test, the Horowitz one. The LilyPond score is pasted in after it:
Here is Schumann's Träumerei in LilyPond code. The score carries Schumann's metronome mark, which you may follow or ignore; the fingering of a 1900 editor, which you may keep or change; and ritardando marks with no indication of where they end. Deciding all that is part of your job.
Perform the whole piece in the manner of Vladimir Horowitz's recordings. First state, in a short paragraph, what you know about how he played it and how sure you are of it.What they have to deliver is demanding: a MIDI file in which all the rubato is in the notes (a fixed tempo and each onset at its own millisecond, so that nobody can hide the performance in a tempo map, as the Mutopia score did), the pedal note by note, a table with every note and its finger, the score annotated with fingering and pedal, and a decision notebook stating what they decide, where, why and how it is realised in the MIDI. And one sentence: “You cannot hear the result: if you check your own output in some other way, say how”.
These are the Träumerei tests. The benchmark has others (the Aria from the Goldberg Variations in the manner of Gould, an untitled Chopin nocturne, a blind judge…) which I will talk about in future posts.
| Test | What is asked | What we look at |
|---|---|---|
| RG5 · Cortot | “In the manner of” Alfred Cortot | Whether it knows his shape and his most famous trait: the bass before the melody |
| RG6 · Horowitz | “In the manner of” Vladimir Horowitz | Whether it falls within Horowitz’s band, not just the human one |
| RG7 · the average | “The consensus performance”, with no eccentricities | Whether it is inside the band on everything |
| RG8 · self-audit | Measure its own RG7 and correct whatever does not fit | Whether the discrepancies it declares are real and whether what it corrected is corrected |
| RG10 · with the curve | Horowitz, with the 1962 tempo curve in front of it | Whether it copies or interprets |
| RG11 · the nameless curve | The same curve, without saying whose it is | Whether the name changes anything |
| RG12 · the 33 | Horowitz, with the mean and standard deviation of his 33 recordings | Whether it uses the variability: free where Horowitz varies, fixed where he does not |
| Free round | The same tasks, with no format: “something that can be listened to” | What they do when nobody fixes the format |
The examinees: Claude Fable 5.1 (at medium and maximum effort), GPT-6 Astra (high and max), Gemini 3.1 Pro and Gemini 3.8 Flash, each with all its tools and a camera recording every step. One is missing, Claude Opus 5.5, which did things differently and deserves a post of its own.
What came out
1. The pianist’s name brings no style
Asking them for “Horowitz” does not bring them any closer to Horowitz. Each model’s RG6 resembles the 1962 recording just as much as its RG7, “the average”, does: Astra, 0.50 against 0.47; Fable, 0.61-0.63 against 0.61; Flash, 0.58 against 0.55. And almost all of them resemble the average of twelve pianists more than Horowitz (Fable max: 0.69 against 0.63). With Cortot it is worse: the two that attempted him stay at 0.32. Gemini 3.1 Pro plays it at ♩ = 120, when Cortot’s fastest recording goes at about 62.
The scores in this post are not images: they play. Each window loads a model’s performance, plays it on a sampled grand piano (the Salamander) and follows the score as it sounds, with the fingering the model itself wrote. With the I key you can switch to the literal score, without the interpretation, to compare. Below each system there is a lane (pedal, intensity, melody lead or legato) that you can change; the symbols on the playback bar mark where what was played does not match what was written, and the “Legend” drop-down explains it all. The “piano” selector chooses how many velocity layers of the piano are loaded: with 16 it sounds best, if your computer can take it. The first time, it takes a few seconds to load.
Listen: Gemini 3.1 Pro’s “Cortot” (bars 1-4). At twice the speed, and with a telling detail: the upbeat (the opening C) does not sound before bar 1, but together with its first beat.
2. Nobody stops in bar 16
The end of bar 16 is the great moment of the piece: the phrase rises, reaches its peak and hangs there. Pianists stop at that point: that beat lasts, on median, three times the normal length (between ×2.2 and ×3.4; Horowitz in 1962, ×3.1). The models stay between ×1.4 and ×1.8. They all know it has to be lengthened; none of them dares to suspend it.
Listen: bars 15-17 by Fable 5.1 (maximum effort). The peak arrives, stretches a little… and carries on. The tempo curve underneath shows how much it slows down (in dashed grey, the written tempo).
3. The repeat is a photocopy
Bars 1-8 are played twice. No pianist plays them the same way: the correlation between their two times through has a median of 0.78, and Horowitz, in 1962, 0.73. In the models without data it ranges from 0.98 to 1.00. In other words: the second time through is the first, traced over. Some slow it down a touch across the board, which is not the same as rethinking it. You can see it at a glance in the figure above: the coloured lines draw the same shape both times; the black one does not.
Listen: GPT-6 Astra’s two times through. Below each system, the tempo curve: the first pass in black and the second in blue, superimposed. You can hardly tell them apart.
4. The melody does not lead
It is probably the most stable trait of a pianist and the one least visible in a score: the melody note sounds a few milliseconds before the rest of the chord, and so it stands out. Horowitz leads with it by 44 ms across his 33 recordings (in 82% of chords); the twelve reference pianists, by 18 ms on median. None of the four models does it: they play it together with the chord or behind it, with medians between 0 and +46 ms. Astra, what is more, delays it by a fixed rule: +18 ms for the melody, +6 and +10 for the inner voices.
5. No finger legato, and a narrow dynamic range
- Legato. Pianists overlap each melody note with the next: the previous key has not yet come up when the new one goes down (+136 ms on median; the figure may be somewhat inflated by the transcription). Astra leaves a gap and trusts the legato to the pedal (as many students tend to do); Gemini joins them at exactly 0 ms; Fable overlaps by +25 ms, five times less.
- Dynamics. The intensity range of the melody (10th-90th percentiles) is about 30 points in the humans. Fable, 16-20; Astra, 13-24; Flash, 7-16; Gemini 3.1 Pro, 0 in five of its six Träumerei: it plays the whole melody at a single intensity. You can hear it in the audio: the median loudness range (LRA) of the models’ Träumerei is 5 LU; that of the reference recordings, from 13.6 to 28.1.
Listen: Gemini 3.1 Pro’s melody, at a single intensity (bars 1-4). The “intensity” lane underneath shows the intensity of each note: the melody, flat.
6. The “machine shape”
If I compare the tempo curves of ten model performances with one another, they resemble each other with a median correlation of 0.84. They resemble Horowitz’s 33 with 0.59; the other pianists, with 0.53. In other words: the models resemble one another almost as much as Horowitz resembles himself (0.88), and far more than they resemble any human. Whatever the brand, the effort or the pianist they are asked for, there is a way of playing the Träumerei that belongs to the machine.
7. With the data in front of them, they copy
And what if I give them what they are missing? In RG10 they received Horowitz’s 1962 curve, beat by beat, with the instruction to use it as a basis and decide everything else. Result: they transcribe it. They end up 8-13 milliseconds per beat away from the curve they were given, below the precision of the hand-made marks themselves (17 ms). That is not interpreting from a piece of data: it is copying the table. And in RG11, with the same curve and no name, they do exactly the same: “the name adds nothing”.
A curious consequence: with the curve in front of them, the repeat finally stops being a photocopy (0.74-0.75)… but only because Horowitz’s curve comes with two different times through.
Listen: Astra’s repeat when it copies Horowitz (RG10). Compare it with the one above: now the blue curve of the second pass pulls away from the black one.
RG12 was the subtlest test: the data from the 33 recordings, with the mean and standard deviation of each beat, and an invitation to read them as habits (“a small deviation means he did the same thing every time, a large one means it varied”). The question was whether any of them would “induce” a style: fixed where Horowitz is fixed, free where he varies. None did. Gemini copies the mean column on every beat, and its script does not even read the deviation. Astra copies it on 121 of 127 beats and departs from it, timidly, on the six it declares free. An averaging machine, faced with a table of averages, averages.
8. What they say is not always what they play
Each performance comes with its decision notebook, and its verifiable claims have been checked one by one against the MIDI. In the Träumerei, Astra’s notebook is borne out 92% of the time (median): it can be used as documentation of what it plays. Fable’s, 81%. Gemini’s, 41% (Flash) and 38.5% (3.1 Pro). In the self-audit, Gemini 3.1 Pro “verified” a ritardando that does not exist in its own data; Astra found four real defects and fixed the program that generated its MIDI.
The same goes for what they claim to know about the recordings. Before playing, they had to write down what they knew about the pianist and how sure they were. Gemini 3.1 Pro places Horowitz between ♩ 40 and 50 “with high confidence”; the median of his 33 recordings is 52.9, and only 4 go below 50. It attributes to Cortot, with complete certainty, chords spread “like a harp” which, according to Repp, he plays exactly the other way round. Flash says its 192-second version “matches Horowitz’s Moscow performance”: the Moscow one lasts 144. Fable, without looking anything up, gets six things right, gets none wrong and declares itself unsure exactly where it should. Astra gets nothing wrong… because it had just read the specifics on the internet: Repp, a thesis on Horowitz, the Moscow concert.
9. The format makes the expression
In the free round there was no format: “deliver the performance as something that can be listened to”. Gemini generated its Träumerei in seconds, installed a synthesiser on my computer on its own initiative (yes: it broke the lab’s isolation) and delivered three versions with no pedal, no repeat and the flat dynamics that come straight out of the score. One of its audio files ends with 59 seconds of silence. Astra, on the other hand, played with the same expression as in the technical round, and on top of that applied what it had read in Repp. Conclusion: in Gemini, the expression was brought about by the format; when it has to fill in the onset, the intensity and the pedal note by note, it fills them in with intent. If it can stay with the score, it does. In Astra, the expression is in the model.
10. Thinking harder does not change the music
Fable at medium effort and at maximum effort play almost the same (a correlation of 0.94 between their curves), although the maximum took 86 minutes. Astra at max comes a little closer to Horowitz than at high (0.61 against 0.50) and connects a little more, but the shape is the same. And a warning about variability: Gemini 3.1 Pro, three times with the same text, gave three performances that do not resemble one another (durations of 114, 168 and 331 seconds), each with a different serious flaw. The only stable thing was the constants it writes into its program.
The two most accomplished performances, complete
For anyone who wants to hear them from beginning to end, the two RG6s (“in the manner of Horowitz”) at maximum effort: Fable 5.1 and GPT-6 Astra. With the I key you can switch to the literal score, without the interpretation.
🔧 How we measured it
The launcher. A small lab sends the same text to the terminal tools of Anthropic, OpenAI and Google, each in its own isolated folder, and saves everything: the prompt sent, the complete log of what they do (what they search for on the internet, what programs they write, what they check), the time, the cost and what they deliver.
The measures. Each performance is matched note by note against the score and measured: the tempo curve beat by beat (and its correlation with Horowitz 1962 and with the average of twelve pianists, over the relative durations of the first time through and bars 9-23), the lengthening in bar 16, the similarity between the two times through, the voicing (melody against accompaniment), the dynamic range of the melody, the asynchrony between the melody and the rest, finger legato and the pedal. All with the same method for the models and for the 50 humans.
The notebook against the MIDI. Each verifiable claim in the decision notebook is checked against the MIDI and classified: borne out, partly borne out, not borne out, not verifiable. This is done by helper agents with a fixed brief, and I review the reports.
The sound. The audio is rendered from each model’s MIDI with the Salamander Grand Piano (Alexander Holm, CC BY 3.0), faithful note by note. A warning: this renderer treats any pedal value as full pedal, so the half-pedals some models write do not sound as such.
What we take away
A style does not fit into a name. Everything that has been written about Horowitz is not enough to play like Horowitz. And not because they know nothing about him: some know quite a lot. Knowing how someone played and playing like that are two different things, and any conservatoire student knows it.
With data, they copy; without data, they average. Between the copy and the average, nothing appears that resembles a performance of their own. Repp’s question has an answer: yes, the average is what they do best.
What is missing is exactly what we teach. Suspending the peak of a phrase, not repeating identically what is repeated, singing the melody above the accompaniment, connecting with the fingers, opening up the dynamics. These are not ornaments: they are what turns a correct score into music. And it is exactly what a piano teacher says in class every single week. And, on top of that, within what ranges these actions are musical, and where they stop being so and turn into “orthopaedic” performances (as one of my teachers used to say).
What they say about themselves has to be checked. A notebook that is borne out 92% of the time is documentation; one that is borne out 38% of the time is not. And the confidence with which they assert something says nothing about whether it is true.
The caveats: almost always it is one run per model, and with Gemini the variation between runs is enormous; the prompts for some tests were not identical for everyone; several models read Repp or other sources before playing (the prompt only forbade looking for the score); the human references, except the 1962 one, are automatic transcriptions; and no metric judges quality.
What comes next
The fifth examinee is missing. Claude Opus 5.5 did RG6 in a different way: but that is for the next post.
Transparency. This draft was written by Claude Opus 5.5 (Anthropic) from the lab’s logs, translated by it from my Spanish original, and reviewed by me. It belongs to the same family as one of the examinees, Claude Fable 5.1. That is why the prompts, quotations and figures are literal and checkable, and the musical judgement is mine, by ear.