Draft:Music Source Separation

Music Source Separation (MSS) also known as Stem Separation, Demixing, Audio Source Separation or Unmixing is a technique of separating one audio track into multiple audio tracks by targeting mixed material using Music Information Retrieval (MIR) MSS is a branch of Signal Separation which was established in the mid-1990s as a technology to reconstruct one or more source signals from mixtures of them. The process is generally utilized by music professionals to separate existing recordings for the purposes of enhancing the balance of the mix, remixing or remastering. There are additional use cases where there is no multitrack or session files available of the sound recording so it becomes a necessity to rely on tools that can provide stem separation from a single audio file.

Initial audio source separation for commercial purposes resulted in a file that was non-destructively separated, so that the resulting files could be reconstructed and sound exactly like the original without introducing issues when all tracks were performed simultaneously.

AI Stem Separation and Audio Source Separation

Starting late 2018 commercial tools became available for the separation of four part stems from a single audio file using AI models. These applications separated stems into Vocal, Drums, Bass and Other from a single audio file. Deep learning advancements and new processing models such as Wav-U-Net for neural networks aided in higher quality and less phase in separations from mixed material. Izotope RX, AudiosourceRe Demix, RipX DeepMix where a handful of early providers of specialized stem separation tools. Eventually this process would become more widely adopted by users and commercial application developers, simultaneously the technologies would continue to improve in terms of quality of separation and speed of separation. The company Deezer also made their Spleeter tool openly available in around 2019 for further research and development into the audio source separation process. Companies such as Apple would go on to release four part stem separation tools directly in the DAW (requiring Apple Silicon in this case), branded as Stem Splitter. There are dozens of companies now utilizing the technologies involved with audio source separation to fit a multitude of applications. New separation options such as piano, strings, winds, guitar, acoustic guitar, synthesizer are available from a variety of developers as of 2025. The trend for early 2025 was to include these stem separation capabilities with additional AI-based musically purposeful technologies such as mastering, pitch detection and manipulation, chord recognition, lyric transcription, vocal swapping and similar.

AI Stem Separation in Sync Music

A growing number of companies are providing the ability for both music publishers and clients to utilize the stem separation technologies for their project needs. Utilizing these tools provides editors and agents of the film and TV music industry to quickly have available the ability to adjust and contour songs without the need to reach out to providers which would cause time delays. This improves the potential of a usage because a common issue with sync placements is that certain kinds of sounds can interfere too much with the application of the underscore (for example vocals). This also provides the sync professionals to take the track into unexpected directions and otherwise enhance the mix for the purpose of the application of the track.

AI Stem Separation for the DJ

Quick stem separation is a perfect match for the professional DJ looking to create unique mashups. Generally the track would be rendered into a stem by placing the desired songs into the appropriate folder, when the song is selected it will have the basic four stem groupings available and in some cases individual parts (samples) can be triggered on pads for live performance.

AI Stem Separation in Commercial Products

Hardware

Akai MPC Stems

Native Instruments Traktor Stem

Native Instruments Maschine Stems

Serato Stems

Engine DJ Stems

Rekordbox Stems

DAWs

Logic Pro

Fruity Loops

Studio One

n-Track Studio

RipX DAW

SpectraLayers

Acoustica

Mixcraft (Windows)

Band in a Box (Windows version)

Notable Case Studies of AI Stem Separation

Disney Music Group made use of stem separation technologies to enhance their back catalog of recordings. Beatles recordings where split and enhanced with stem separation technologies and the egineers during this process also helped to progress the development of the technlogy.

Stems vs AI Stem Separations

Stems have been used in the recording industry to mean files bounced during the mixing process, generally a collection of like sounds grouped as a "stem". Stems in the context of the original project files can provide a large number of exported audio files for multiple purposes. These kinds of files generally provide a better quality overall and offer the ability to further isolate project material without introducing artifacts.

AI Stem Separations have generally produced material that is ideally suited for volume adjustments or further effect processing or production. These kinds of stems generally have come in the basic four groupings of vocal, bass, drum and other. New approaches and deeper training of models resulted in the capability to isolate additional material beyond the basic four groupings however these kinds of separations generally have spectral anomalies, blend in additional sounds or change some quality of the original targeted sound.

Sound Design with AI Stem Separation Tools

The process of using AI and other methodologies to target specific kinds of sounds happened to enable a new method of spectral separation based sound design through new kinds of tools to edit with such as those in SpectraLayers and RipX. The instant ability to unmix components such as transient information and time based information into full tracks of unconventional sound creations.

Stem Mastering Tool

Native Instruments created a specialized tool called "Stem Creator Tool" for working with four part stem tracks which is ideally suited for the DJ world as digital DJ consoles and Native Instruments hardware like Traktor and Maschine use the four track stem structure. This tool enables quick mastering and saving of files in a "stem" archival format. The tool is free to use and essential mastering effects applicable to stem-based audio are provided.

Known Issues

The length of time it takes to analyze and separate the sound means that the mixtures generally need pre-rendering or there is a delay in processing.