This sort of “autocomplete” is actually fundamental to how classical composers were trained.

For anyone interested, I’d highly recommend reading Robert Gjerdingen’s article Gebrauchs-Formulas. https://www.researchgate.net/publication/259731561_Gebrauchs...

You can also listen to the transcript of four Russian composers, including Rachmaninoff, playing this pattern recognition and generation game at a dinner party in the late 1800’s: https://youtu.be/PlFPOWuwBHI?is=EKBK7QQkJs4MsTCU

Composers at the time could do this just by looking at sheet music and audiating, without using a piano.

  This time Steibelt had prepared a brilliant fantasy for piano and strings, the theme coming from the Beethoven trio he had heard the previous week. Steibelt's admirers were in raptures. Now the issue was joined, and Beethoven had to show his strength. He walked to the piano, grasped the cello part of the Steibelt work en route, put it upside down on the piano and insultingly drummed out a theme with one finger. Then he improvised, and—angry, excited, on his mettle—how he must have improvised! Before Beethoven had finished, Steibelt stole from the room. He never again would meet Beethoven and made it a condition before going anywhere in Vienna that Beethoven not be invited.

I think this is a great project and very HN. Not sure why the comments are so focused on the deliverable- you learned way more and had a much more interesting experience.

One think I didn't see mentioned in the post- maybe I missed it- how large was the data? How many samples did you use to pretrain and post-train

Classical pianist and software product designer here.

I see so much in common with this project and the numerous AI-based UX design tools out there. Whether it's music or UI, now that the "generation" portion of the work costs zero, all that remains is taste.

And so much of taste comes from exploring and killing off possibilities that turn out to be dead-ends. I love the idea that models like these will help us find the dead ends faster, or even produce a gem here and there.

P.S. if you want another uncanny version of Fur Elise, listen to Beethoven's own 1822 revision: https://www.youtube.com/watch?v=s24TtiGgb6k. His 1810 version that we all know was simpler and more balanced. But for what it's worth, Beethoven didn't publish either of them.

Reminds me of this project to generate every melody possible algorithmically in order to fight music copyright lawsuits. https://allthemusic.info/

Hearing the start of Für Elise, and then it being taken in an incredibly different direction, is surprisingly disconcerting.

Really cool! It reminds me of a demo I saw live at NeurIPS: https://www.youtube.com/watch?v=8s3V922h3CU (paper at https://openreview.net/pdf?id=3yeBer3J5z).

If you like such creative AI work, I would recommend looking at the other "NeurIPS Creative AI Track 2025" submissions as well.

Reminds me of Francois Pachet’s Continuator (all the way back in 2003, using hierarchical markov models)

https://www.francoispachet.fr/continuator/

The end of the AI's first sub-phrase in the video is wrong.

You started by playing a simple I - vii - I as the first sub-phrase. Then the AI started its response with a dominant 7th-chord. In the vast majority of cases I can think of from the Classical era, the AI's sub-phrase would end with a half-cadence. So, including what you played, the first two sub-phrases would look like this:

I - vii - I

V(7) - I - V

But instead, the AI took the unusual step of having a full cadence for that 2nd sub-phrase. This in music is what someone might call a "double beat" in screenwriting.

Moreover, there are no half cadences in the phrase after that. It's all full cadences in the tonic key, plus two plagal cadences over a tonic pedal there in the 2nd phrase. Each sub-phrase just extends the tonic. There are idiomatic ways to just stay in the tonic for awhile, like "Horn fifths." But this excerpt isn't idiomatic and sounds like the composer accidentally forgot to develop the harmony.

I'm going to claim that this excerpt would sound like the musical equivalent of a run-on sentence to any composer who lived from the time of J.S. Bach to Robert Schumann. And from Mahler on they'd assume you're thumbing your nose at convention.

Put another way: the kind of student who'd use this to finish a harmony exercise would end up getting the same grade they would have gotten doing it on their own.

Edit: clarifications

Interesting. I do feel letting a machine generating notes is taking the joy out of improvisation, is it not?

I think one of the great, early, joys of learning a piano is gaining the following intuitions: The seemingly harder path of learning sheet is actually faster. Your mind _should_ learn to think in two dimensions Spatial — where fingers go — and Time — pitch and tempo – ** when learning. The _internalization_ of Space and Time queues guide the fingers in a dance that is vastly satisfying. This skill leads you to the final part of the journey that is improvisation and the one more exciting than what i am on now.

---

**

Space: Your finger placement on keys right, e.g. knowing how to go from landmark/anchor notes(mid-c, G, F etc) and then go to the others above and below it. Crudely this is some what like typing from your landmark f and j qwerty keyboard

Time: The out singing/verbalizing of the notes/beats on a time measure as you play them(per the time measure). e.g. you can say out loud 1-2-3-4 for 4/4 measure, if the measure has quarter notes say out loud. And `1-e-and-a-2-e-and-a-3-e-and-a-4-e-and-a` for a 4/4 with 1/16th note granularity. Do this as you play the notes and you get a sense of tempo.

I would love something like that, except that I play the melody, and it produces proper 3-4 part accompaniment, preferably in good baroque style. Extra bonus if it could also write it into a file in a format suitable for music editing programs.

Ah, MIDI files. The only type of music you could realistically download from the internet back in the day, and you had to wake up at ungodly hours so that your dialup modem would not rack up a massive phone bill.

The results strike me as comparable or worse than you could get with a Markov model. I think it reveals the gap in understanding between LLMs and music. I think you need to either: - Set up a pipeline to decompose music into, say, harmonic sequences and melodic sequences, and then have the LLM work on some more fundamental or more high-level layer of musical composition and then re-translate it back into actual sounds. - Develop a better dataset and train the LLM more natively on musical examples.

Does anybody know of a project that has produced more convincing results?

This is really fun. Scaler 3 starts with a chord progression and lets you break it down into musical performances and parts. Useful for ideation when producing.

Would be fun to get a midi clock going and play some chords on my piano and have my synth start jamming along with the bass and my keyboard doing some performance. Or any combination of the above.

Oh, a cool idea! I just tried it, works pretty well. Kudos!

One feature request:

Instead of playing the AI-generated audio solely through the iPhone's speakers, add an option to send the audio as midi notes to a device (probably the same one you received the mini notes from).

You could also distill the fabulous Anticipatory Music Transformer from Stanford. https://crfm.stanford.edu/2023/06/16/anticipatory-music-tran...

Also consider checking their decisions about representation, etc

The idea is awesome! :) However there's definitely much room for improvement, first of all rythm and composition (so there's some sense of musical form).

Even after a few years deep into AI, I find your application absolutely magic. This is very inspiring, thank you for sharing.

Für Elsevier Journal Access Library pass out candy for safety this halloween your baby off milk shake it off.

This is amazing. Great inspiration to train a model for a midi electric guitar solo. Though with bends, vibratos, mutes, harmonics, dive bombs etc as part of the event feed, it may possibly create way more events than this which means more training I suppose.

Amazing idea! Gonna hook this up to my little synthesizer and blast some square wave arpeggiated ML music!

How would you expand this to support elements like attack ("velocity of the key-down" in piano speak), grace notes, timing etc. Would each of those be part of this model or another model? How would you model an arbitrary element (pedal, duration, etc...)

  NOTE(
    pitch,
    delta_onset,
    duration,
    attack_velocity,
    release_velocity,
    ...
  )

  NOTE(...)
  | CONTROL_SUSTAIN(delta_onset, value)
  | CONTROL_TEMPO(delta_onset, bpm)
  | CONTROL_PROGRAM(delta_onset, instrument)

It occurres to me that the relationship between pitches usually matters more than the pitches themselves. Perhaps the pitches could be encoded in a vector like what is typically done for position.

This is so amazing, can you improve the quality of generation at the cost of notes per seconds ? No one can play 108 notes/sec anyways, maybe you can train the model to do CoT for better quality

Question as a musician: Do you find satisfaction in the piece it created in-and-of-itself (subtracting the satisfaction from making the tool itself)?

> I trained a 125M model to autocomplete piano on-device

I first parsed the "on device" as _on the piano_ and was intrigued how that worked :) maybe someone can make "dynamic" rolls for a player piano somehow (like one of those braille "screens")?

Running a 125M model on-device at that speed is impressive. How much did you have to optimize the model to get that performance on an iPhone?

I don’t have MIDI. How about whistling or playing the piano via microphone? Sounds easy. Another 6 month rabbit hole? :)

I have about 60 hours of my own playing in midi data. I wonder if I could fine-tune this model on that?

Talking about AI music with some live human MIDI inputs, Magenta Realtime 2 was released a few weeks ago and is pretty fun.

https://magenta.withgoogle.com/magenta-realtime-2

This would be great to be able to use via Web Midi. That size LLM will run well in a browser.

Looks neat! I'd love for it to feed the MIDI notes back into my player piano instead of playing out of my iPhone's (comparatively tinny) speakers though.

Prompting and having the model fill out the rest makes it feel like jazz or at least an improvised result of a song

This is almost exactly what Jordan Rudess is doing now with some folks from MIT or Stanford right now. Google his interview with Rick Beato.

Congratulations! It is amazing!!! Can you do the same with a song and give different drums to see which one fits better?

Are the weights of the model available?

> Eventually I used Gemini 3.5 Flash for pairwise evaluation

But, but… wouldn't that be… (gasp) DISTILLATION?

Fun project!

I would love to see a jamming partner. So I could play along him on the same piece.

Very cool! Can you say a little bit about the size of the DPO training examples and how long training took?

This is really awesome thanks for sharing

Would you be willing to share about how much it cost to train a model like this?

I wonder if training SOTA models on this will make them more humane?

Like... why.

Any way to share the actual model? I don't have an iPhone to run the app on.

Cool work. I tried using LLMs to parse sheet music and they are really bad.

This would make a brilliant VST or Max 4 Live device if you would ever consider doing so.

Should tokens be multiple Midi notes? Why or why not?

next step is a model that autocompletes the part where you actually practice. mine just sits there judging my scales.

This is honestly astonishing and the first "AI music" I've heard that has the potential to sound beautiful. I always thought that MIDI would be a perfect format for this. Glad to see this person make it happen!

really incredible work! great use case, impeccable learning strategy, congrats!

Pretty Impressive!!

Gemma 4 E2B was too heavy for your needs?

I too am currently learning to play Sonatina in G Major, it's a fun one! the end is surprisingly tricky

[flagged]

[flagged]

[flagged]

[flagged]

  What is the goal here?