Best workflow for unmixing a full orchestral MIDI recording

Let’s see what the hive mind can come up with…

A composer friend’s computer just gave up, with all data corrupted. He was working on a couple of big orchestral scores which have a deadline of soon, which are now gone. Luckily he had audio mockups in the cloud, so given that Cubase can extract MIDI from audio, we thought we’d give it a go.

But the pieces are full on, so I’m thinking it might help to separate the instruments as much as possible using Spectralayers, before importing the layers into Cubase. We’re talking full orchestra, so I’m wondering if anyone has experience of this kind of operation, and what might be the best way of approaching this.

Any help gratefully received, thanks!

more info required

was anything written down? any notation?

how were the mock-ups created?

at this current time the “Unmix Orchestra” does not exist; percussion Unmixing has greatly improved from SL11 to SL12.

Realize that Unmix Song separates into 7 layers:

  1. Vocals
  2. Drums
  3. Bass
  4. Guitar
  5. Piano
  6. Sax & Brass
  7. Other

I’d say, at this time in the unmixing dev, you are better off starting over. It’s a bitter pill, I know. SL could be used to display note values instead of freq to give a roadmap for re-writing the parts. The Spectrogram could give some useful note and timing info (although waveform is better at representing time), but I wouldn’t even try to unmix a string section at this time. The time spent waiting for the unmix modules to whittle thru and then y’all listen back, looking at the “other” layer and think “wha?…Why?” and then start trying to manually move each sound to it’s specific instrument? My goodness, it would take forever at this stage. Very likely faster to re-compose.

Unmixing the percussion could give some useful returns…or could have y’all scratching your heads. The source is very important in my testing. I have live and multi-tracked performances of three and four piece “rock” and “jazz” type stereo recordings of original music. One group is mainly cello, elec guitar and a Roland OctaPad. SL just doesn’t cope well with separating strings. More elaborate orchestrations just add to the already difficult task. SL will send legato and “singing” timbres to the Vocal layer; effects to the other layer, as a short example. My textural electric guitar parts often end up spread across guitar, vocal and other layers.

So, it’s probably best to try short selections of a couple of the mock-ups and investigate what you get with what SL does with the sound sets. SL is not good at separating synth sounds. I haven’t tried much sample based music either, so I can’t really speak to that.

So, if y’all can get SL successfully separating out percussion, that would be a win. Separating drums from “melodic” instruments and separating individual drums in SL12 is better than SL11 IME, yet, again, I have not found SL very good at non-rock band instruments.

For example, I have a live performance with vocal, elec guitar, electric bass (MM Stingray) and 5pc drum kit; the FOH recording is way out of balance (drums far too loud) with a rather poor vocal. Well SL is pretty good at removing vocals (not 100%, but very good) and one particular song is very STP sounding. Well the bass entirely goes to the drums layer as the bass timing is married to the kick. Separating out the freq area where the bass and BD are would take ages manually or with “select similar”. That bass could be played in lot quicker than any SL separation techniques.

Similarly, I have a live folkish rock band with 5 vocals, at least 3 string players (guitars, banjo, mando, pedal steel), sax, electric piano (Ensoniq KS-32) and real accordion; recording quality similar to that STP sounding song. Well, the bass and drums separated very well in SL11 (I didn’t try SL12 yet); in this case, the balance is a bit better in the recording (drums not over loud) and the bass is a Fender P-Bass. Sax separates well in some areas, and not well in others: maybe some lead lines could be carved out…but by then, you have played them in. Elec Piano and synth are always troublesome. I haven’t tried any real piano or the accordion.

I’d say if you aren’t already an advanced manual editor using SL, that you’ll waste more time learning SL manual separation skills and how to read the spectrogram than if your friend just starts back at square one using those mock-ups as a guide. You’ll just spend so much time experimenting with SL, which would have been time better spent tracking.

I would take out the hard drive and try to get good professional help to recover as much of the data as possible. I don’t rule out the possibility of getting the entire project back…

1 Like

agreed! first port of call

I had assumed this already happened

OK, this little challenge has inspired me to spend a few minutes trying Unmix Instrument module in SL12 before my trial is up as it is something I wanted to test. I have a Handel cue I made 10 years ago of the piece “Air”…I have the full mix and just now quickly exported the stems; this piece was done in Reason and the sample quality is not pretty low end, so it might be a fair test related to this question. I have a little time today (not really) and will experiment a little to see what this UI module might do with orchestral sources.

I’ll report back after a few processings

Ha! first stumbling block: my mixdown is stereo and my stems were mono…SL will resample for user if user says yes…

ok, I tried registering the bowed bass from an exported stem

The only setting for the Unmix Instr module is sensitivity

so far I tried sens=0.0 and sens=2.0

I will try 1.0, -1.0 and -2.0 as well

the bowed string sound appears to confuse the unmix AI which then identifies all timbres that are bowed as the registered instrument…seems to be unmixing the strings so far

maybe I should say my stems direct with no fx and I’m attempting to unmix the final mix, FX and all

OK, I just tried -1.0 sensitivity; this yields much higher separation, but essentially SL12 is detecting the parts of the string section that were not registered (violas and violins) as my registration source was bowed double bass

that doesn’t mean Cubase couldn’t convert the pitches to midi…I didn’t try that; I have Cubase 13, I don’t know if Cubase13 is capable…but, could be starting point

I’m always saying that about the SL unmix modules…they are mostly a starting point

Sens -2.0 has removed too much for sure…-1.0 seems usable for a starting point at a work around

just tried the flute; no dice, forget that, I could play it in in an hour

this flute register wouldn’t give anything beyond what the dbl bass register gave, which was somewhat useful…I didn’t try Unmix Song/brass

seriously, using SL12 to unmix an orchestra might seem like a cake walk at first glance, but SL just isn’t up to that yet IMO/E

Maybe Cubase can create an overall piano part from the midi conversion and then cut and paste to arrange and then edit midi? After having used SL12 to Unmix Song>Drums to separate the percussion?

I’ll come back to this later to put unmix instr through some more paces

Some nice ideas, thanks.

Yes, the drive looks like it’s toast, already tried.

So, what I did was a quick Unmix Song - Piano, Brass, Drums, Other (i.e. Strings). Tried Unmix Instrument, but nothing useful came of it.

In Cubase 12, which I have, Variaudio is monophonic, so I activated a trial version of Melodyne 5 that came with it to analyse polyphonically as an ARA extension. Got plenty of useful note information, pitch rather than rhythm, which was a bit all over the place. Exported as MIDI files.

As I said, my friend had lost all the files, so re-composing a 16 minute piece for full orchestra in a week was going to be pretty difficult. At least now he has a lot of reference material to use as a springboard, so it’ll be pretty helpful in any case.

It would be great if more variety of instrumentation was developed in Spectralayers, but obviously that sounds like quite a challenge, technically speaking.

2 Likes

@slaughterback thanks for reporting back the process for which you were able salvage some data…I know the suggestion to start over is not acceptable…I’ve had to do that plenty of times…very daunting indeed

sadly, redundant saves and back-ups to multiple drives in various locations is a requirement in this computing line of work or risk losing months or even years of work

good luck

agreed on all accounts

like I said earlier, there is no unmix orchestra module at this time :slight_smile:

folks essentially need to figure out workarounds to arrive at best separation; that said, the some of unmix modules save months of (manual) work and just imagine where we’d be without them!

Yep. His computer was playing up for a while too, so I’ve no idea why he didn’t take the necessary precautions, but now he has at least learnt a very expensive lesson about backing up!

As for Spectralayers, it would be nice if Steinberg actually made a few videos for use cases which didn’t involve dialogue repair, or separating FM rock tracks. The amount of tutorial videos which use mediocre guitar rock as their examples is a bit frustrating, and musical styles such as orchestral or electronic or ambient among others are generally ignored. As is the lack of videos / articles detailing any other possible creative uses of the software, but that’s probably a topic that needs its own thread…

1 Like

Was he able to rescue his files?

No. He learnt the hard way…