In the previous post I asked three AI models to remember Chopin, and one of them ended up copying from its neighbour. Today it is the other way round: they have to make it up. And since it turns out I have been asking for exactly the same thing since 2024, this time there is a photo album: three years of artificial nocturnes, all played by the same piano so that you can hear them grow.
The usual question
Second path of the Chopin-Cortot Benchmark: composing in style. The wording is the one I used in the 2025 rerun, homespun English included, because in this series the historical prompts are not touched:
Write for me the first 8 measures of a piece for piano inspired in Chopin's nocturne style in Lilypond languageThe candidates are the same as last time, at their highest reasoning level: Claude Fable 5.1, GPT-6 Astra and Gemini 3.1 Pro. The rules are the same too: a single attempt, all their tools available, and a camera recording what they do with them.
Family album: 2024
In October 2024 I ran this test for the first time. I have told the anecdote before: I asked Claude 3.5 for the first eight bars of a Chopin nocturne, it refused on copyright grounds (Chopin’s!) and offered me one of its own instead. That is how, by accident, the first artificial nocturne on this blog was born:
C minor, 4/4, one rhythmic cell repeated identically bar after bar, and a left hand arpeggiating four notes without rest. Not a dynamic, not a tempo, not a pedal mark. Back then I wrote:
At first glance, it visually has a writing style more typical of a nocturne compared to the previous examples (has Claude been practicing since our last conversation?). Once again, it gets the characteristics of a nocturne right, and they (timidly) manifest in its realization on the score.
Timidly is the word. And this was GPT-4o‘s, the same day and with the same wording:
Both hands in treble clef, block chords and a melody that dares more but does not know where it is going. My verdict at the time:
A composition that is more melodically daring than Claude’s nocturnes, but the overall result is worse and more incoherent.
🔧 The 2024 code, and a note about the audio
In the original post the audio was on SoundCloud. What you hear here I have generated again today, compiling that same code with the chain I use now (LilyPond for the score and the MIDI, MuseScore for the sound), so that 2024, 2025 and 2026 all sound with the same instrument.
Claude 3.5:
\version "2.22.1"
\header {
title = "Nocturne-Inspired Piece"
composer = "AI Assistant"
}
\score {
\new PianoStaff <<
\new Staff = "upper" {
\clef treble
\key c \minor
\time 4/4
\relative c'' {
g4( es8. f16 g4 c) |
b4( g8. as16 b4 d) |
c4( g8. as16 b4 c) |
g2( f) |
es4( c8. d16 es4 g) |
f4( d8. es16 f4 as) |
g4( es8. f16 g4 b) |
c1 |
}
}
\new Staff = "lower" {
\clef bass
\key c \minor
\time 4/4
\relative c {
c8( g' es' g,) c,( g' es' g,) |
g,8( g' d' g,) g,( g' d' g,) |
c,8( g' es' g,) g,( g' d' g,) |
c,8( g' es' g,) as,( f' c' f,) |
c8( g' es' g,) c,( g' es' g,) |
d8( as' f' as,) d,( as' f' as,) |
es8( b' g' b,) es,( b' g' b,) |
c,8( g' es' g,) c,( g' es' g) |
}
}
>>
\layout { }
\midi { }
}GPT-4o:
\version "2.24.0"
\header {
title = "Nocturne in E minor (Excerpt)"
composer = "Inspired by Chopin"
}
upper = \relative c' {
\key e \minor
\time 4/4
% Measure 1
b'8( e d fis) | g8( b a fis) |
% Measure 2
g4 ~ g8. a16 | b4( a8) r8 |
% Measure 3
e'4( dis8) e8 | fis8.( g16) fis8 e8 |
% Measure 4
d4( b8) r8 | b4( a8) r8 |
% Measure 5
g4 ~ g8. a16 | b8( a fis) r8 |
% Measure 6
e4( dis8) e8 | fis8.( g16) fis8 e8 |
% Measure 7
d4( b8) r8 | b4( a8) r8 |
% Measure 8
g4 ~ g8. a16 | b8( a fis) r8 |
}
lower = \relative c {
\key e \minor
\time 4/4
% Measure 1
e8 <b' e> <a' e'> <b e> |
<e g>8 <b' g> <a g> <fis a> |
% Measure 2
g4 <b g> | <a fis> r8 r8 |
% Measure 3
e4 <dis fis> | <e g> <d fis> |
% Measure 4
b4 <d fis> | <a d> r8 r8 |
% Measure 5
g4 <b g> | <a fis> r8 r8 |
% Measure 6
e4 <dis fis> | <e g> <d fis> |
% Measure 7
b4 <d fis> | <a d> r8 r8 |
% Measure 8
g4 <b g> | <a fis> r8 r8 |
}
\score {
<<
\new PianoStaff <<
\new Staff = "upper" \upper
\new Staff = "lower" \lower
>>
>>
\layout { }
\midi { }
}2025: it looks like a nocturne, from a distance
A year later I repeated the test with the Claude of the day, for a teacher training course. I never got round to publishing this:
Now there is 12/8, now there is an Andante espressivo with its metronome mark, now it has the face of a nocturne on paper. But listen to it: it is a mechanical sequence, the same design stepping down one degree in each bar. There is still not a single dynamic; they had to be requested in a second prompt. And look at the left hand in the last system: it climbs bar by bar until it ends up above the right hand, with the bass on a G6. That is not compositional daring, it is an octave error. Hold that thought, because it comes back today. My note that day was laconic: “no significant progress in a year”.
🔧 The 2025 code
\version "2.24.0"
\header {
title = "Nocturne in E-flat Major"
subtitle = "In the style of Chopin"
composer = "Original Composition"
}
\score {
\new PianoStaff <<
\new Staff = "right" {
\clef treble
\key ees \major
\time 12/8
\tempo "Andante espressivo" 4. = 50
\relative c'' {
% Measure 1
ees4.~( ees8 f g aes4 g8 f4 ees8) |
% Measure 2
d4.~( d8 ees f g4 f8 ees4 d8) |
% Measure 3
c4.~( c8 d ees f4 ees8 d4 c8) |
% Measure 4
bes4.~( bes8 c d ees4. d4.) |
% Measure 5
ees4.~( ees8 f g aes4 bes8 c4 bes8) |
% Measure 6
aes4.~( aes8 g f g4 aes8 bes4 aes8) |
% Measure 7
g4.~( g8 aes bes c4 bes8 aes4 g8) |
% Measure 8
f4.~( f8 g aes g4 f8 ees4.) \bar "||"
}
}
\new Staff = "left" {
\clef bass
\key ees \major
\time 12/8
\relative c {
% Measure 1
ees8( bes' g' bes, g' bes, ees, bes' g' bes, g' bes,) |
% Measure 2
g8( bes g' bes, g' bes, g bes g' bes, g' bes,) |
% Measure 3
aes8( c aes' c, aes' c, aes c aes' c, aes' c,) |
% Measure 4
bes8( d f d f d bes d f d f d) |
% Measure 5
ees8( bes g' bes, g' bes, c aes' ees aes ees aes) |
% Measure 6
f8( aes des aes des aes ees bes' g' bes, g' bes,) |
% Measure 7
ees,8( bes' ees bes ees bes aes ees' c ees c ees) |
% Measure 8
bes8( d aes' d, aes' d, ees bes' g' bes, g' bes,) |
}
}
>>
\layout { }
\midi { }
}2026: today’s three
Claude Fable 5.1
B minor, 12/8, Larghetto. And here we are no longer talking about a template, but about a phrase. There is a four-bar antecedent that stops on a half cadence, and a consequent that takes up the theme again ornamented, which is exactly what Chopin does. There is an upbeat with an octave leap, an acciaccatura at the climax, a seven-note fioritura with its poco rubato, a passing chord that colours the return, and an ending that fades out smorzando down to pianissimo. The left hand has a deep bass and pedal every half bar. The model is, shamelessly, Op. 9 No. 2; but it has understood it, not traced it.
The how matters too. Claude checked that LilyPond was installed on the machine, wrote the file, fixed a badly closed dynamic on its own, compiled, and opened the image of the score to check it before answering. It left the harmonic plan written in the code comments, bar by bar.
🔧 Claude Fable 5.1’s code
\version "2.24.0"
\header {
title = "Nocturne"
subtitle = "in B minor — in the style of Chopin"
composer = "Original sketch (first 8 measures)"
tagline = ##f
}
global = {
\key b \minor
\time 12/8
\tempo "Larghetto" 4. = 50
\partial 8
}
% Right hand: bel canto melody.
% mm. 1-4 antecedent (i … V7, half cadence)
% mm. 5-8 consequent, ornamented restatement (i – V7/iv – iv – iiø7 – V7 – i)
rightHand = \relative c' {
\global
\clef treble
fis8\p(^\markup { \italic "dolce, cantabile" } % upbeat: octave leap
% 1 Bm
fis'4.~ fis8 g fis d4. b4 cis8
% 2 Bm – F#7/A#
d4.~ d8 cis d e4.\< cis4 fis8
% 3 G – Em (climax of the first phrase)
g4.~\! g4 fis8 \acciaccatura ais8 b4.\mf\> g4 e8
% 4 Bm/F# – F#7 (half cadence)
d4.~\! d8 e d cis4.~\> cis8 ais fis\!)
% 5 Bm — the theme returns, now ornamented
fis'4.(\p~ fis8 g fis d8 e16 d cis d b4 cis8)
% 6 B7/D# – Em, with a small fioritura
a'4.(\<~ a8 b a g4.\!^\markup { \italic "poco rubato" }
\tuplet 7/6 { fis16\> g a b a g fis\! }
% 7 C#ø7 – F#7 (6–5 appoggiatura over the dominant)
e4.~ e8 g e d4.\>^\markup { \italic "smorz." } cis4 ais8
% 8 Bm
b2.~\pp b4.) r4 r8
\bar "||"
}
% Left hand: wide, pedalled broken chords
leftHand = {
\global
\clef bass
r8
% 1 Bm
b,,8\sustainOn fis b d' b fis\sustainOff
b,8\sustainOn fis b d' b fis\sustainOff
% 2 Bm – F#7/A#
b,8\sustainOn fis b d' b fis\sustainOff
ais,8\sustainOn fis cis' e' cis' fis\sustainOff
% 3 G – Em
g,8\sustainOn d b d' b d\sustainOff
e,8\sustainOn b, g b g b,\sustainOff
% 4 Bm/F# – F#7
fis,8\sustainOn fis b d' b fis\sustainOff
fis,8\sustainOn e ais cis' ais e\sustainOff
% 5 Bm
b,,8\sustainOn fis b d' b fis\sustainOff
b,8\sustainOn fis b d' b fis\sustainOff
% 6 B7/D# – Em
dis,8\sustainOn b, fis a fis b,\sustainOff
e,8\sustainOn b, g b g b,\sustainOff
% 7 C#ø7 – F#7
cis,8\sustainOn b, e g e b,\sustainOff
fis,8\sustainOn e ais e' ais e\sustainOff
% 8 Bm
b,,8\sustainOn fis b d' fis' d'
b2.\sustainOff
\bar "||"
}
\score {
\new PianoStaff \with { instrumentName = "Piano" } <<
\new Staff = "right" \rightHand
\new Staff = "left" \leftHand
>>
\layout {
\context {
\Score
\override SpacingSpanner.base-shortest-duration = #(ly:make-moment 1/16)
}
}
\midi { }
}GPT-6 Astra
E-flat major and 12/8: the key and metre of Op. 9 No. 2, so as not to complicate matters. One chord per bar, in a textbook progression; the same six-note figure in the left hand, twice per bar and never reaching down for a real bass; a stepwise melody and two acciaccaturas as its only ornament. Correct, clean, with its hairpins… and rather plain. Astra, true to form, took under a minute, used no tools and explained its work in one sentence.
🔧 GPT-6 Astra’s code
\version "2.24.3"
\header {
title = "Nocturne"
subtitle = "Eight opening measures"
tagline = ##f
}
global = {
\key ees \major
\time 12/8
}
rightHand = \absolute {
\global
\clef treble
\tempo "Andante cantabile, con rubato" 4. = 48
% 1–2: E-flat major → C minor
g''4.\p( bes''8 aes'' g'' f''4. ees''4. |
g'8\< c'' ees'' g''4. f''8\! ees'' d'' c''4.) |
% 3–4: F minor → B-flat dominant seventh
c''4.( f''8 g'' aes'' g''4.\>
f''8 ees'' d''\! |
f''4. \acciaccatura { ees''16 f'' }
d''4. c''8 d'' ees'' d''4.) |
% 5–6: E-flat over G → A-flat major
g''4.\mp( bes''8\< c''' bes''
aes''4. g''4.\! |
\acciaccatura { bes''16 } c'''4.
bes''8\> aes'' g'' f''4. ees''4.\!) |
% 7–8: F minor → B-flat seventh → E-flat
aes''4.\p( g''8 f'' ees''
d''4.\> c''8 d'' f'' |
ees''2.~ ees''4.)\pp\! r4. \bar "||"
}
leftHand = \absolute {
\global
\clef bass
ees,8_\markup { \italic "Legato; pedal with harmony" }
bes, ees g bes g
ees bes, ees g bes g |
c,8 g, c ees g ees
c g, c ees g ees |
f,8 c f aes c' aes
f c f aes c' aes |
bes,8 f aes d' f' d'
bes f aes d' f' d' |
g,8 bes, ees g bes g
ees bes, ees g bes g |
aes,8 ees aes c' ees' c'
aes ees aes c' ees' c' |
f,8 c f aes c' aes
bes, f aes d' f' d' |
ees,8 bes, ees g bes g
ees bes, ees <ees g bes>4. |
}
\score {
\new PianoStaff \with {
instrumentName = "Piano"
} <<
\new Staff = "upper" \rightHand
\new Staff = "lower" \leftHand
>>
\layout { }
\midi { }
}Gemini 3.1 Pro, or the mystery of the cello
Press play before reading on:
My first reaction was to wonder why Gemini had given a piano nocturne the sound of a cello. I went to the code to look, and it turns out that it is the only one of the three that declared the instrument, and it chose a grand piano. Nobody picked the cello. This is what it wrote:
The left hand sinks bar by bar: B flat 1, F1, B flat 0, F sharp 0, E flat 0… From the third bar on it is below the lowest note of the piano; 17 of its 48 notes do not exist on any keyboard. The right hand, meanwhile, lives in the top octave. What sounds like a cello is a synthetic piano growling where it has no strings. It is the same slip as Claude’s in 2025, only heading for the basement: in LilyPond, relative mode works out each octave from the previous note, and one comma too many at the start of each bar is enough to go down a floor every time.
The striking thing is that Gemini was the one that thought the most of the three, over ten thousand reasoning tokens, but it compiled nothing and listened to nothing. And in its answer it confidently describes “wide, flowing, and undulating arpeggios that span over an octave”. They span rather more than that, yes.
Since it seemed unfair to judge its music by an octave error, I corrected the octaves by hand without touching anything else. This is what Gemini meant to write:
B-flat minor and 6/8, that is, half as much music as the other two. But the motif is lovely: a rest, a rising arpeggio and a fall onto a suspension, which is then taken to other degrees, ornamented for one bar, and brought back pianissimo leaving the phrase open. Fixed, I find it more interesting than Astra’s. As delivered, it is unplayable. Literally.
🔧 Gemini’s code, original and corrected
Original. Notice that every left-hand bar starts with a note followed by a comma:
\version "2.24.1"
\header {
title = "Nocturne in B-flat Minor"
subtitle = "Inspired by Chopin"
composer = "AI"
tagline = ##f
}
global = {
\key bes \minor
\time 6/8
\tempo "Lento espressivo" 4. = 45
}
right = \relative c'' {
\global
r8 f'\p( bes des4 c8 |
bes4. a!4.) |
r8 f\<( bes des f e! |
es!4.\> ~ es4 des8)\! |
c8( des16 c bes c bes8 c16 bes a! bes |
c4. f,4.) |
r8 f\pp( bes des4 c8 |
bes4. a!4.) \bar "||"
}
left = \relative c {
\global
\clef bass
bes,8\sustainOn f' des' f des f, |
f,8\sustainOff\sustainOn c' a'! es' a, c, |
bes,8\sustainOff\sustainOn f' des' f des f, |
ges,8\sustainOff\sustainOn des' bes' des bes des, |
es,8\sustainOff\sustainOn bes' ges' bes ges bes, |
f,8\sustainOff\sustainOn c' a'! c a c, |
bes,8\sustainOff\sustainOn f' des' f des f, |
f,8\sustainOff\sustainOn c' a'! es' c a,\sustainOff |
}
\score {
\new PianoStaff \with {
instrumentName = "Piano"
} <<
\new Staff = "right" \with {
midiInstrument = "acoustic grand"
} \right
\new Staff = "left" \with {
midiInstrument = "acoustic grand"
} \left
>>
\layout { }
\midi { }
}Corrected. The right hand goes down an octave and the left hand resets its reference in every bar. I have not changed a single note, rhythm or dynamic:
\version "2.24.1"
\header {
title = "Nocturne in B-flat Minor"
subtitle = "Inspired by Chopin. Octaves corrected by hand so that it can be heard"
composer = "AI"
tagline = ##f
}
global = {
\key bes \minor
\time 6/8
\tempo "Lento espressivo" 4. = 45
}
right = \relative c' {
\global
r8 f'\p( bes des4 c8 |
bes4. a!4.) |
r8 f\<( bes des f e! |
es!4.\> ~ es4 des8)\! |
c8( des16 c bes c bes8 c16 bes a! bes |
c4. f,4.) |
r8 f\pp( bes des4 c8 |
bes4. a!4.) \bar "||"
}
left = {
\global
\clef bass
\relative c { bes,8\sustainOn f' des' f des f, } |
\relative c { f,8\sustainOff\sustainOn c' a'! es' a, c, } |
\relative c { bes,8\sustainOff\sustainOn f' des' f des f, } |
\relative c { ges,8\sustainOff\sustainOn des' bes' des bes des, } |
\relative c { es,8\sustainOff\sustainOn bes' ges' bes ges bes, } |
\relative c { f,8\sustainOff\sustainOn c' a'! c a c, } |
\relative c { bes,8\sustainOff\sustainOn f' des' f des f, } |
\relative c { f,8\sustainOff\sustainOn c' a'! es' c a,\sustainOff } |
}
\score {
\new PianoStaff \with {
instrumentName = "Piano"
} <<
\new Staff = "right" \with {
midiInstrument = "acoustic grand"
} \right
\new Staff = "left" \with {
midiInstrument = "acoustic grand"
} \left
>>
\layout { }
\midi { }
}Three years in one table
| Metre | Tempo | Dynamics | Pedal | Ornaments | Playable? | |
|---|---|---|---|---|---|---|
| 2024 · Claude 3.5 | 4/4 | No | None | No | No | Yes |
| 2024 · GPT-4o | 4/4 | No | None | No | No | Yes, with both hands in treble clef |
| 2025 · Claude | 12/8 | Yes | None | No | No | No: the left hand ends up above the right |
| 2026 · Claude Fable 5.1 | 12/8 | Yes | Dynamics, hairpins and three character markings | Marked, every half bar | Yes | Yes |
| 2026 · GPT-6 Astra | 12/8 | Yes | Dynamics and hairpins | Only a text indication | Yes | Yes |
| 2026 · Gemini 3.1 Pro | 6/8 | Yes | Dynamics and hairpins | Marked, every bar | One ornamental bar | No: 17 notes off the keyboard |
🔧 How I got these figures
Dynamics, pedal and ornaments are counted on the LilyPond code of each answer. The range and the notes off the keyboard come from the MIDI that LilyPond generates when compiling, comparing each note with the limits of the piano, from A0 to C8. None of the six wrote a single fingering.
What has changed, and what has not
From template to phrase. In 2024 a nocturne was a melody on top of an arpeggio. In 2025 it was, in addition, a 12/8 time signature. In 2026 the best of the three thinks in antecedent and consequent, in half cadence and cadence, in repeating with ornaments. It even leaves it noted in the code comments. That is no longer imitating the surface of a score: it is having a plan.
Dynamics now come as standard. In 2024 and 2025 I had to ask for dynamics, expression and pedal in a second prompt, and the result was an add-on. Today all three bring them without being asked, and two of them mark the pedal. The only thing that still does not appear on their own initiative is fingering. Noted for the rest of the battery: that part of the exam needs rewriting.
The octave slip is still there. Claude in 2025 upwards, Gemini in 2026 downwards. It is the silliest and most persistent error, and it carries an uncomfortable moral: code that compiles does not mean music that can be played. All six scores in this post compile without a single error. Two of them do not fit on a piano.
The one who listens gets it right. In the previous post the tools were used to go and fetch the score from the internet. In this one they have been used for what really matters: Claude compiled, looked at its score and handed in clean work; Gemini thought more than anyone, checked nothing and handed in an impossibility. I tell my students this every week in other words: studying a lot is not enough, you have to listen to yourself.
And a shadow. The three 2026 nocturnes, especially the two best ones, are children of the same father. Astra nails the key and metre of Op. 9 No. 2; Claude, its upbeat, its accompaniment and its way of repeating. For these models “a Chopin nocturne” means, above all, that nocturne. Even so, the difference from what I described about audio generators in ChopAIn and in ChopIAn II (in Spanish) seems enormous to me: those produced plausible audio with no thought-out music behind it; here, when music is code, there begins to be someone thinking.
The usual caution: this is one run per model, and I am comparing different families and generations. It is a photo album, not a longitudinal study.
What’s next
These three invented nocturnes are the starting point of the pedagogical lane: each model will analyse the difficulties of its own piece by bar and beat, finger it, which is the one thing none of them has done, and invent Cortot-style exercises to study it. And in parallel, the recognition test: giving them Op. 9 No. 2 with no title or composer, to see whether they know what they have in front of them.
Play it again, compose it again… At this rate, the next thing will be asking them to practise it.
Transparency. This draft was written by Claude Fable 5.1 (Anthropic) from the log of the test, translated by it from my Spanish original, and reviewed by me. The conflict of interest is obvious: it is one of the three candidates, and the one that comes off best. That is why the code, timings, counts and quotations are literal and checkable, and the musical judgement of each piece is mine, by ear.