Robin
Congratulations on 10.
Is the AI one of these? MUSDB18 Benchmark (Music Source Separation) | Papers With Code
@Raymondo it’s an improved version of hybrid transformer demucs. So it would rank at the top of that list.
2 Likes
So am i right in thinking this is something to do with Meta research ?
Would be good to know what it is using. The best models are DemucsV4 FT and ensembles of MDX flavors, but even on £3K of NVidia GPU they’re probably not practical for any realtime preview or even offline demix work within a normal project day. I pretty much live a day demixing in UVR and other AI tools and then pulling it all into SpectraLayers. To integrate something like UVR5 would actually be a dream to work in, you could preview using low Shifts and Overlap, then when happy up the settings and schedule to run on the GPU overnight or something to do the demix cleanup next day.
1 Like
I want to ‘copy’ one selection e.g. 500 times through a track at specific timecode, I don’t want to select it 500 times, transform or do anything but ctrl+copy like you would in say a photo editor to duplicate a selection 500 times before committing to the cut/paste. Likewise Select Similar will not achieve the result I want (it’s pretty basic tool and half a dozen settings, there’s no black art to it lol, it just doesn’t work for my use case that’s all, it works for everyday generic editing though very well)
Lets take this repeated simulated police siren with harmonic. It’s too feint to use Select Similar and is moving in an out of loudness and left right balance, but I need around 100 selections of what is manually selected. You can’t just copy and paste over to the next repeating pattern so you have create a blank layer, save selection, move selection one pattern over, load the saved item back and now you have two selected. It’s a repeating pattern in time, so then you save those two selection as a new selection, move over two and then load the second back in to have 4 selected and go 8 > 16 > 32 > 64 > 128 and then edit back to accommodate the fades and left right. It would be great if there was a way to select the already selected like you can in e.g. photoshop to bypass the intermediate saved selections and just quickly copy and paste across the timeline, there’s no need to save, just elect all selections and then copy and paste in the blank layer, move, repeat and commit to the audible layer and then demix to new.
@Sam_Hocking Unmix > Song in SL10 on a Macbook Air M1 takes:
-1 minute to unmix 3 minutes in Fast setting
-2 minutes to unmix 3 minutes in Balanced setting
-3 minutes to unmix 3 minutes in Best setting
Yep, that’s why I fear it wont be particularly good, like Spleeter was fast too (even on silicon which isn’t the best at this kind of work compared to Nvidia GPUs really). It would be good to have one of the top SDR models integrated like Demucs V4 and MDX, it’s free, although obviously I don’t know the deal with Demucs being a Facebook project originally iirc but it’s used on various free online services and UVR5 and Google Colabs etc.
@Sam_Hocking I mentioned in a message above what AI it was (an improved version of hybrid transformer demucs - the top unmixing quality you can get today).
1 Like
OK, thaks I didn’t see that comment, so good to know. DemuxV4 FT (Hybrid transformer) is not 2-3 mins unless low settings used though, I use it for demixing everyday to Dolby Atmos professionally, but I’m interested to put SL10 it through an SDR comparison and see how it compares. Thanks!
The following album I demixed from stereo for the Atmos release. A few days and nights spent in SpectraLayers, it was some project turned around in a week including the Atmos mixing!
Will SL10 make use of Nvidia CUDA?
Pretty sure it will have to if it’s relatively close to Demucs code. When running Demucs with pytorch install that has to be cuda-enabled system to run.
@MrSoundman Not at release (the bench I posted is CPU-only), but the next patch will add NVIDIA/AMD/Intel HD Graphics/Apple CoreML hardware acceleration where possible.
7 Likes
Can I ask you a question, how do you believe visual effects works (for big budget blockbuster films)? Do you believe that there is some magical button that visual fx artist use and they apply aftereffects and magic happens? For example, most would say that “The Matrix” is one of the greatest films of all time, but I’m curious to know how you believed all those visual effects came together. Do you believe there was a setting that was sitting in the Wachowski’s computer and they just pushed a button and poof all the visual effects came together in one simple click?
Sorry to tell you this but all those visual effects you see (especially in films) is time consuming work. There’s no magical button to push. As a matter-of-fact a lot of visual effects artists complain all the time that drawing effects is not easy work and is extremely time consuming. Same thing goes for open worlds in Unreal Engine 5, it takes time to build those complex architectures. There is no easy button to do everything, sorry to break it to you.
However that picture of those sirens looks overly simple. Can you please share the actual audio file and I’ll show you (through a video) a simple way to target those sirens (without doing too many mouse clicks). I’ll even narrate and walk you through the process. Can you please share the actual audio file.
Thanks Robin
Dude, i’m not talking about magic anything. You have source audio with a very low volume frequency bound repeating pattern in it like a siren or an alarm, but it has several harmonics, altenating channel shifts and loudness in/out happening. If it’s within music it’s probably a sample or sythnesized so identical pattern, although even natural recordings it’s probably a synced pattern anyway… If really lucky it’s even locked to timecode so even repeats at predicted quantized locations.
All my request is, is when you’ve selected one loop of a pattern, it would be really nice to be able to just ctrl+drag copy (like pretty much all graphics and even text-based editors) to rapidly duplicate that selection onto the next repating (but not identical) loop of the pattern without using the saved selection workflow to build it up which is clunky and slow. Magic wand isn’t able to select this with accuracy when low in the mix.
As for film industry, I am reverse audio engineering not ground-up CGI or even as complex as photoshop or film editing, simply a selection cut, copy and paste duplicate across a timeline we are already familiar with isn’t currently supported in V9 and everyday I wish it was, that’s all.
I can’t share this particular piece of audio, it’s not released and I have NDAs anyway on it.
@Unmixing I certainly wouldn’t mind seeing a video of you demonstrating this technique on an audio example if you’re able.
So, you’re either not adding any hardware acceleration at all or only supporting CUDA.
@BrunoDSL Not sure what you mean. To be clear, SL 10 at launch will be using CPU only for AI processing. However the plan is to add hardware acceleration for patch 1 (or at the latest, patch 2) on Windows using DirectML (which support NVIDIA, AMD and Intel HD Graphics hardware) and on macOS using CoreML (which will mostly benefits Apple Silicon macs, but should provide some gain on Intel macs as well).
I already did some testing with those hardware accelerators, but did not find them stable enough for the launch date. Hopefully the situation will be better in a couple months.
2 Likes
I’m working on it. Writing and emailing someone about using their content is the easy part, the hard part is never receiving a response.
2 Likes