Designing music as a system rather than a fixed soundtrack

AMS, the Adaptive Music System, began in 2019 when Unreal Engine developer Jorge from Coqui Games approached Marc Thornton about an adaptive-music idea.

The technical starting point was an Unreal system that could react to gameplay through Blueprint logic. The creative challenge was harder: how should music be written if the game can change its state at unpredictable moments?

The two began by looking at both sides together. They explored vertical remixing through compatible musical layers, gameplay states such as exploration, tension and combat, transitions between tempos and sections, and ways to avoid the obvious interruption produced by simply stopping one track and starting another.

They also looked at existing approaches, including Elias and adaptive-music products available through the Unreal Marketplace, to understand where a simpler in-engine system might fit.

The collaboration developed a clear division of responsibilities. Jorge handled the core Unreal Engine and Blueprint implementation. Marc focused on composition, production, interactive-audio design and the musical architecture needed to make the technical behaviour convincing.

That meant writing differently.

Instead of composing a finished track from beginning to end, Marc created compatible pieces of musical material that could survive being rearranged by the game. Stems could enter or leave as intensity changed. Sections had to remain convincing if the system arrived halfway through them. Some experiments deliberately constrained material to common properties such as the same key and 4/4 time signature so different sections could be exchanged without breaking harmonic or rhythmic continuity.

The team also experimented with tempo changes, transitions, diagnostic audio and procedural techniques. Test material developed into demonstrations covering exploration, tension and combat, including a fantasy-themed example that showed the soundtrack reshaping itself in real time.

The product idea grew alongside the R&D. Rather than keeping the system inside one game, Marc and Jorge explored packaging it as a reusable Unreal Engine asset. The Blueprint framework could ship with demonstration music, while additional compatible soundtracks could be offered separately.

The proposition was to provide some of the adaptive behaviour associated with specialist systems such as Elias while keeping the workflow directly inside Unreal Engine.

AMS reached a substantial prototype and content-development stage but was never commercially released. Marc attempted to restart the project in 2020, after which Jorge joined Epic Games. They agreed that any possible employment or conflict-of-interest implications around collaborative Unreal work needed to be respected, and the project was placed on indefinite hold.

That does not make the R&D wasted work. It changed the way Marc approached composition for interactive products and reinforced a principle that applies to much of Digital SHIFT’s work: creative and technical systems are stronger when they are designed together from the start instead of one being bolted onto the other at the end.