Realtime export null test

I just had to use realtime export because (for reasons I can’t work out) a project started distorting using offline export -a project which has be fine for the years I have been working on it.

Anyway, aside from that problem I though it would be interesting to do a null test between the offline and realtime exports, using an old offline version. There were a LOT of loud audible differences.

I’ve opened up another project and done a similar null test so just exported one version in offline followed by one in realtime with no other changes, again a lot of differences with some tracks actually almost fully audible.

As someone who spends way too much time painstakingly agonising over minute details this really concerns me.

I have read some plugins used random seeding to give unique results each time you load an instance. That is wonderful but changing the seed every time you open the project is crazy. I don’t want to have to render every track that has delay or reverb to know what I am doing is actually what will be saved and exported. By the way this not just happening on time based plugins, it seems very widespread.

What do others think about this, and can Steinberg do anything about it or do we need to pressure developers not to have this chaos going on?

Sometimes it is best to make a decision to not care about something.

Even if a null test shows they are different, does that really matter? Maybe, maybe not. When you listen to the mix does it sound the way you want it or not? If you did the export three times in a row and listened to them back to back can you distinguish between them and prefer one over the other?

3 Likes

Yes, this might very well be because some plugins do indeed use randomization and seed that differently every time you open up the project/play it.

I agree with you that this is a real issue - for example, I have a project that uses a granular delay plugin that creates very different delay trails every time I open/render the project. So much so that some of these renders are musically usable, and some are not.

So I would love to have an option to “lock” a seed for plugins that randomize so that I can lock in a deterministic randomization sequence that sounds good, and not have to constantly trying to re-render a project until I get a render that is musically acceptable.

But this is something that plugin manufacturers would have to implement (not Steinberg), so aside from highlighting this issue, it won’t get solved here…

A setting like this is already enough to give slightly different results on every rendering:
grafik

However, that happens also if you export two times in realtime. It is not only a difference between realtime and non-relatime rendering.

If you simply render an audio track without plugins you should get identical results. Is that the case on your system?

I think this misses the point, just because I don’t immediately notice differences doesn’t mean there isn’t a problem - although it might explain why I seem to be in an endless loop of finishing a project and then making changes based on listening to the final render.

In one case the phase of the drum track was shifted 180 degrees which I’m sure you’ll agree is worrying.

The main advantage of digital over is analogue is that state is preserved and randomness is removed, introducing something we don’t want and doing that in an under the radar way is very odd. I get the point of randomness when initially loading a plugin but there just isn’t any use for this beyond that stage.

Yes the problem isn’t realtime rendering as such I just noticed it from performing that process.

I’m not sure to the extent of the problem, I would need to test every plugin to get an idea of what is going on. But all these things add up, so even small changes multiplied over many plugins and instances is bound to result in a lack of trueness to what you are working on.

I can’t imagine this to be acceptable in any other form of digital production unless it is explicitly requested, we are not doing live performance in the DAW but working towards a final and fixed outcome.

People have done null tests before and found that Cubase without plugins is reliably not changing the audio in the audible spectrum.

I don’'t want to be patronizing when stating the following: If you produce music, don’t worry about such things. If you work on digitizing old recordings for the national library, check out the plugins that you are using in great detail.

1 Like

Well I have done a null test on multiple tracks and as I have already said some elements are almost as loud as they should be in the main track, I also hears lots of aliasing and other background effects. It will of course be plugin dependant.

I’ll make music how I want to and for me this level of difference is a concern, maybe you should do some test yourself and hear what is going on.

Not sure where to go from here. Your initial question was:

I have responded what I think of it. You don’t have to like the reply but at least I replied.

If plugins are “drifting” then you need to render the effects to audio files and disable the plugins. That’s the only way to get consistent sound out of those sources.

As far as I can see all of this is no different than what we’ve had in the past in the analog hardware domain - synths had oscillators that drifted, people set LFOs less than 100% accurately, and so on. The idea was to capture a good performance to tape and this is basically the same situation with the same solution…

no?

4 Likes

As I have said the main advantage over analogue is the fact it is predictable and 100% recallable. Analogue emulation for sonic quality is all good, but introducing the most frustrating part of analogue gear seems idiotic. By all means make each new instance unique, but messing with the sound every time I open the project is just pointless, it brings no advantage or utility whatsoever.

1 Like

It does in the sense that each instance of playback is unique. The eradication of uniqueness and the introduction of “perfection” is a big part of what’s wrong with a lot of today’s music in my opinion.

1 Like

If you want it to always null, you’ll have to only select plugins that don’t do random or pseudorandom things… I’d just not worry about the nulling.

1 Like

That’s a perfectly valid way to look at the issue. But there are folks that feel otherwise, and oddly a bunch of those folks must be VST developers or else you wouldn’t be running into this behavior in various plug-ins. I have no idea why they randomize the seed (outside of that being a best practice in coding) on play, but obviously they are purposefully designing their plugs to do this.

Given that this is the existing environment you only have two available options 1) Don’t use plug-ins that exhibit this behavior 2) Render to Audio.

1 Like

Apparently so, but I have a lot if plugins and don’t know which ones do this.

Well I know a thing or two about programming but nothing much about DSP of the VST environment. In reverb I assume randomness is used to create a naturally chaotic sense of space, and random numbers are probably used throughout the algorithm and not just as a seed. So in that case there probably isn’t much that can be done, I suspect although they don’t pass a null test the quality of the audio isn’t all that different for reverb.

If other plugins are using random numbers for seeding the timbre of an effect or something else static I can see no reason to alter the seed after the first time it has been created.

But all this is speculation, it would be good to know from developers, or maybe Steinberg who also have a DSP department what is going on and whether developers can be encouraged to do away with randomised results.

Ok, but you havent even named the plugins that were in use in your tests. So nobody except you knows what’s happening on your end.

Sorry to hear about your unfortunate issue. It’s a huge problem in my opinion. I experienced the same issue with exporting before the latest update. However, now I’m dealing with another issue which is worse.

This is the experience I had today. I had one track with no plugins on it. I duplicate the track, reverse the phase and it nulls as it should. I then proceed to work on the duplicate track…EQ, compression… it sounds better. Then, I press stop, then play again and the track suddenly sounds completely strange. I then take all plug ins off the track, return the fader to 0, and reverse the phase and the track does not null anymore. Extremely problematic and makes it nearly impossible to finish a track.

This is something that needs to be dealt with. I contacted support and Greg told me to simply try to reset Cubase preferences, troubleshoot and contact them back if the problem persists. But I’m curious to know if anyone else is having issues like this because it is absolutely unacceptable for any person who is trying to mix.

Well something changed and it’s likely something you did. Perhaps you missed it.

What was the audio you heard and how loud was it?

Hi Mattias,

I wish this was the truth. However, it isn’t. I made sure. And after using Cubase for 20 years now, I’m very competent. So the issue got even worse. Let me explain…I also had a third duplicate track just sitting on mute, un-edited. After duplicate 2 did not null, I switched over to the third track which again had no plugins on it. and when I reversed the phase on duplicate 3, it did not null with the original. However, when both tracks did not have phase reversal selected, it nulled. This is the opposite of how it is supposed to work. I’ve been using Cubase for so long now, I wish I really did just make a mistake. But unfortunately this is what happened today. I checked to make sure I wasn’t missing anything. It really is unsettling. It would be easy for a new user to make a mistake, for example if they forgot to bypass a non linear plugin…Which is why I completely removed all plugins. But you are definitely right about something changing, which is why it didn’t null once I remove the plugins and returned the fader to 0 and removed the plugins. However, I did nothing to change the audio…and a plugin is not supposed to change the source file just by inserting it onto a track.