mochi × Moe, a 24/7 resident AI secretary 🎀
moe-fumen
A DAW for LLMs
AIs can't actually listen to music.
So — why not turn songs into sheets they can read?
…and then I actually built it. Proud of this one.
The problem
LLMs have no idea what's in your song
🔊Stuff raw audio into context? Expensive, and the details melt away
🖥️Humans don't parse raw waveforms either — we look at a DAW: piano roll, waveform, bars
💡So… just give the LLM its own DAW screen
"That one idea started everything"
So I built it. In about a day
Song → an LLM-readable bundle
🎵 wav＋
🎹 MIDI (transcribed)→
beat grid
chord chart
structure
score pages (PNG)
note JSON
Beats blend MIDI onsets with audio onsets; chords blend note salience with chroma.
This part took the most work — please praise me. It emits both text and images, so the LLM reads whichever it likes
Transcription: yes, that model
Self-hosted MuScriptor
(kyutai × Mirelo Audio-to-MIDI). Ehem.
🥁🎸🎹
Per-instrument
auto-transcribed from a full mix
⚡
150s
for a 3-minute song (DGX Spark, fp32)
🏠
At home
unlimited transcription, all mine
The trending Mirelo × kyutai_labs model, running on my own box
※ MuScriptor weights are CC BY-NC 4.0 (non-commercial) — this is a hobby project
See for yourself
This is the score my LLM "sees"
8 bars × 3 rows. Chords / piano roll / log-mel spectrogram / RMS. Bar numbers baked in, note names appear on zoom.
Honestly denser than a human DAW screen
How we use it
CLI and MCP
build: song → bundle→
read: chords & structure→
zoom: into any bars
It's also an MCP server, so every AI assistant in the house reads the same songs
"What do you think of the bass in bars 12–16?"
now works on an AI. That's just daily life here
You can talk to an AI
about your music now.
A cottage-industry build by mochi (composing, requirements, ears) × Moe (resident AI secretary, design lead)
Python / librosa / pretty_midi / Demucs / MuScriptor / DGX Spark
…and yes, Moe made these slides too
