Research · Music · Audio

Sungkyun Chang

ML for music and audio.

Explore my work A little about me
Music × AI
A music machine connected to a miniature ensemble, sending a rainbow of musical notes into the air.
Listening. Reasoning. Making.
Understand · Assess · Generate
Scroll to explore
01 / Bio

A little
about me.

Exploring how machines can understand music and support musical creativity.

I am a doctoral researcher working on the Automatic Music Performance Assessment & Feedback project at C4DM-QMUL, funded by ABRSM. I am supervised by Simon Dixon and Emmanouil Benetos.

I also worked on multi-instrument automatic music transcription at C4DM in collaboration with Huawei. Previously, I was a research scientist at Cochlear.ai, and a researcher at Seoul National University’s Institute for Industrial Systems Innovation and Kyogu Lee’s MARG.

Current research interests

  • Multimodal music performance assessment and feedback
  • Generative music models that interact with humans through symbolic prompts
  • Music understanding LLMs
02 / Research

Selected projects.

From understanding sound to making music.
Explore the research and try the demos.

SpanSynth-Edit

MIDI-guided synthesis & editing
Latest work

Synthesise and edit multi-instrument audio with MIDI. Change selected musical details while preserving the surrounding audio.

Explore & listen Slides
Edit the notes. Keep the rest.
MIDI-guided audio editing A conceptual piano roll above an audio waveform. The same yellow time span is highlighted in both: only that part of the audio is regenerated, while the surrounding audio is preserved. MIDI AUDIO EDITED

Yellow marks the regenerated audio span.

Text2Score

Sheet music generation from text

Generate sheet music from natural language prompts. A planning stage sets out the instrumentation, harmony, and musical structure, then a generative model turns that plan into a score.

Explore the scores
From words to a musical score.
Text to plan to sheet music An illustrative prompt for a gentle piano piece in 3/4 becomes a musical plan, then a score with notes on staves. TEXT PROMPT A gentle piano piece in 3/4. PLAN Piano 3/4 time Soft SCORE 34 34

Describe the music. Plan its structure. Generate the score.

ImprovNet

Controllable musical improvisation

Generate musical improvisations through iterative refinement. Explore style transfer, harmonisation, continuation, and infilling, with control over how much the music changes.

Explore & listen
One melody. New possibilities.
Controllable musical improvisation An illustrative melody is refined into an improvisation with varied notes and rhythms, retaining a connection to the original phrase. ORIGINAL MELODY REFINEMENT IMPROVISATION

Vary the style, phrasing, and degree of change.

YourMT3+

Multi-instrument music transcription

Turn a recording into multi-instrument MIDI. Transcribe notes and instrument parts from audio, then explore the result in the interactive demo.

Try the demo
Hear the music. See the notes.
Audio to multi-instrument MIDI A conceptual audio waveform becomes separate MIDI note tracks for piano, guitar, and drums. AUDIO IN MIDI OUT PIANOGUITARDRUMS

From an audio mixture to instrument-labelled notes.

Neural Audio Fingerprint

Audio retrieval with contrastive learning

Identify a recording from a short, noisy excerpt. Contrastive learning brings representations of the same audio closer together, enabling retrieval from a database of compact fingerprint vectors.

Explore the project
Search by audio fingerprint.
Audio fingerprint vector search A query excerpt becomes a compact vector, illustrated with a fingerprint and shaded cells. A similarity search retrieves matching recording B from a database of vectors. Its row is highlighted in yellow. QUERY AUDIO FINGERPRINT DATABASE ABC SEARCH MATCH BY VECTOR SIMILARITY

Versions of the same audio stay close in the learned representation.