Performance issues with Core Graphics

[ray3]

@Arne Scheffler

Hi Joscha,
first of all the Quartz Debug tool does not work any longer correctly. It does not show the dirty rectangles which gets redrawn since we render into a CALayer. The switch to CALayer showed significant better performance here, but maybe your use-case is different. You can try to turn off using the CALayer code path and see if it is the reason of your performance problem. Fastest way is to disable the following code in nsviewframe.mm:

	auto processInfo = [NSProcessInfo processInfo];
	if ([processInfo respondsToSelector:@selector(operatingSystemVersion)])
	{
		auto systemVersion = processInfo.operatingSystemVersion;
		// on Mac OS X 10.11 we activate layer drawing as this fixes a few issues like that only a
		// few parts of a window are updated permanently when scrolling or manipulating a control
		// while other parts are only updated when the malipulation ended, or CNinePartTiledBitmap
		// are drawn incorrectly when scaled.
		if (systemVersion.majorVersion >= 10 && systemVersion.minorVersion > 10)
		{
			[nsView setWantsLayer:YES];
		}
	}

Cheers,
Arne

I wonder if that means that from macOS 10.11 and VSTGUI 4.6 onwards CLayeredViewContainers become completely obsolete?

I am in fact successfully using CLayeredViewContainers in my UI implementations to come by performance issues on Mac. I have to workaround plenty of stuff, though: e.g. https://sdk.steinberg.net/viewtopic.php?f=5&t=77 which I never received any feedback on, CScrollView and other utility views inherit from CViewContainer, CSplashScreen doesn’t support CLayeredViewContainers either and offsetting a CLayeredViewContainer isn’t supported, out of the box (used for collapsible GUIs e.g. Black Rooster Audio Magnetite, which currently has to be worked around by invalidating the entire view in an idle timer when the GUI is collapsed ). There was also a customer that reported black flickering rectangles / refresh issues on his Mac, that could be removed by falling back to CViewContainers, which only seems to happen with specific graphic hw. With Joschas report on the newly (?) introduced performance issues this all becomes a little puzzling so an official clarification would be extremly appreciated.

[bx_joscha]

@ray

I wonder if that means that from macOS 10.11 and VSTGUI 4.6 onwards CLayeredViewContainers become completely obsolete?

Did you evaluate the performances with and without explicit using of layered view containers? Perhaps you can do some quick hacking to build two versions of a plugin to compare them by commenting out setWantsLayer in the platform code. It would be very helpful to understand if the use of layered view containers really pushes the performance. Currently I am very skeptical, because of the latest experiences with the use of setWantsLayer.

[ray3]

@bx_joscha

@ray

I wonder if that means that from macOS 10.11 and VSTGUI 4.6 onwards CLayeredViewContainers become completely obsolete?

Did you evaluate the performances with and without explicit using of layered view containers? Perhaps you can do some quick hacking to build two versions of a plugin to compare them by commenting out setWantsLayer in the platform code. It would be very helpful to understand if the use of layered view containers really pushes the performance. Currently I am very skeptical, because of the latest experiences with the use of setWantsLayer.

Hi Joscha,

I can do some quick tests probably early next week. My previous findings (this happened when migrating to VSTGUI 4.5 with the Rooster stuff iirc, the same goes for the Fuse plugins which are currently on 4.6) showed that using CLayeredViewContainers instead of CViewContainers really helped performance-wise, e.g. sluggish automation, UI related CPU load bursts and slow DAW meter refresh went away for good, but it took a lot of working around other issues as mentioned above. My systems were on 10.10 back then, currently I only have access to a 10.13.5 and 10.12 system, but I’ll get back to you with further reports asap.

Best,
Reimund

[bx_joscha]

@ray

…showed that using CLayeredViewContainers instead of CViewContainers really helped performance-wise, e.g. sluggish automation, UI related CPU load bursts and slow DAW meter refresh went away for good…

That is very interesting, because our QA also reported performance boosts in some situations, e.g. automation, in few hosts. But in many cases and hosts, e.g. Cubase, the performance went so bad, that the plugin became nearly unusable. I personally was able to reproduce the performance lacks and they were really bad. So, maybe setWantsLayer is still the way to go, but the VSTGUI framework does not handle it right? It would be very appreciated if Steinberg could give some details here, how to proceed to always to get the best performance.

[ray3]

@bx_joscha

@ray

…showed that using CLayeredViewContainers instead of CViewContainers really helped performance-wise, e.g. sluggish automation, UI related CPU load bursts and slow DAW meter refresh went away for good…

That is very interesting, because our QA also reported performance boosts in some situations, e.g. automation, in few hosts. But in many cases and hosts, e.g. Cubase, the performance went so bad, that the plugin became nearly unusable. I personally was able to reproduce the performance lacks and they were really bad. So, maybe setWantsLayer is still the way to go, but the VSTGUI framework does not handle it right? It would be very appreciated if Steinberg could give some details here, how to proceed to always to get the best performance.

So far I didn’t get any customer reports regarding impaired UI performance in the Fuse Audio Labs plugins. Then again, these plugins do not have any really big UIs. I’m not part of the Black Rooster Audio team anymore so I can’t say if there have any support tickets in this regard, but a bigger CLayeredViewContainer based UI implementation for reference would be the Magnetite in the non-collapsed mode (SIC: It seems they went for a CViewContainer since v2.2 to get rid of the mentioned flicker/refresh bug).

Best,
Reimund

[ray3]

Btw. my own investigations/profiling sessions are already a little while ago, so the situation may since have changed macOS wise. There will be a bigger Fuse Audio Labs update soon which involves a full regression test and I’ll ask my Betas to have an eye on this. I’ll also let you know about my own findings asap.

[ray3]

One more idea: Maybe your issues have to do with a memory limit that is hit? I know you you store your GUI graphics in PNG-8, but that doesn’t really matter once they’re decoded into RAM / Graphics RAM where they reside in 32-bits ARGB/RGBA. In terms of memory requirements the bx_console plugins are quite heavy I assume. Maybe try removing graphic resources, successively and asses what happens using the activity monitor and/or instruments? Just a guess…

Best,
Reimund

[ray3]

@Joscha:

Okay, here’s some quick feedback…

On my Mac Book Pro (Retina, Mid 2012, 2.6GHz i7, Intel HD Graphics 4000), macOS 10.13.4, I can’t see any significant difference in terms of CPU load between CViewContainers or CLayeredViewContainers. Which kind of corresponds to what Arne is saying. It’s only when I pass an additional kNoCALayer to my CFrame’s open function like this, that things become slightly worse:

#include "../frameworks/vst3sdk/vstgui4/vstgui/lib/platform/platform_macos.h"

...
CocoaFrameConfig config;
config.flags = VSTGUI::CocoaFrameConfig::kNoCALayer ;
	
if(frame && frame->open(parent, type, &config))
{
	...
}

Which is essentially identical to commenting out the “[nsView setWantsLayer: YES];” portion in nsviewframe.
So it’s actually the other way around in my case. However, this was just a quick test session using a single F-59 instance in REAPER64 and macOS’ built-in activity monitor to measure CPU load.

You already mentioned that it may be restricted to a certain setup / host environment, though.

Maybe it’s something specific your plugins’ code? I don’t recall the BX code too well, but iirc meters were polled and updated in your editor’s idle(). What if you remove that for a try?

Since you can reproduce the issue easily on your machine, can you maybe try installing the Fuse Audio Labs F-59 which has a slightly bigger GUI and see whether it also slows down your host as described?

Best,
Reimund

[bx_joscha]

@ray

…It’s only when I pass an additional kNoCALayer to my CFrame’s open function like this, that things become slightly worse:

…

Which is essentially identical to commenting out the “[nsView setWantsLayer: YES];” portion in nsviewframe.

Thanks for the hint and the info.

@ray

So it’s actually the other way around in my case. However, this was just a quick test session using a single F-59 instance in REAPER64 and macOS’ built-in activity monitor to measure CPU load.

You already mentioned that it may be restricted to a certain setup / host environment, though.

Maybe it’s something specific your plugins’ code? I don’t recall the BX code too well, but iirc meters were polled and updated in your editor’s idle(). What if you remove that for a try?

I cannot reproduce it with your plugins. I can only reproduce the performance problems as soon as there are meters attached that are regularly updated. If I load a VST2 plugin (no silence flag) including meters into Cubase, the bug becomes very obvious.

Here is a video that shows how the plugin UI slows down the whole system including the host playback visualization:

https://pluginalliance-my.sharepoint.com/:v:/g/personal/joscha_rieber_plugin-alliance_com/EaHOjcFSKt9GmSPp0VaxUPIB17WWu0FXbkeAFor_NZAuPA?e=gTlsFE

But it is still possible that you are right and it is something specific for our plugin implementations (BX Editor).

[ray3]

Hi Joscha,

@bx_joscha

I cannot reproduce it with your plugins. I can only reproduce the performance problems as soon as there are meters attached that are regularly updated. If I load a VST2 plugin (no silence flag) including meters into Cubase, the bug becomes very obvious.

That’s good to know, thanks for your report. I assume that also goes for the Fuse plugins that do include meters (VCL-4, TCS-68 and the VCL-373)?

@bx_joscha

Here is a video that shows how the plugin UI slows down the whole system including the host playback visualization:

https://pluginalliance-my.sharepoint.com/:v:/g/personal/joscha_rieber_plugin-alliance_com/EaHOjcFSKt9GmSPp0VaxUPIB17WWu0FXbkeAFor_NZAuPA?e=gTlsFE

But it is still possible that you are right and it is something specific for our plugin implementations (BX Editor).

Oh wow, that is indeed quite obvious. One thing that comes to mind in conjunction with your polled meter update mechanism is the following seemingly unrelated quote from another thread (https://sdk.steinberg.net/viewtopic.php?f=5&t=542):

@Arne Scheffler

Yes, but much better would be if you just don’t call into VSTGUI from different threads. One potential issue with setDirty is that you cannot use overlapping views. Performance may also be worse.

I’m talking about Arne’s hint on setDirty() instead of invalid() on the UI thread. But again, I’m just guessing here.

I would be highly interested in your findings once you finally tackle that issue on your end. Maybe it helps others, as well. At least it seems that using CViewContainers and derived views out of the box is the way to go / the way intended by Steinberg, so I can eventually get rid of those CLayeredViewContainer workarounds I’ll keep my mouth shut now.

Best,
Reimund

[bx_joscha]

Hi Reimund,

@ray

That’s good to know, thanks for your report. I assume that also goes for the Fuse plugins that do include meters (VCL-4, TCS-68 and the VCL-373)?

I cannot reproduce it with VCL-4, BUT look at my screenshot. The GUI draws quite strange here? (non-retina display, Cubase 9.5, VST-2)

https://pluginalliance-my.sharepoint.com/:i:/g/personal/joscha_rieber_plugin-alliance_com/ETouwq1enuNMutJZzg-Kr8MBl5TPKkA1uu6LWYUOqV_s2w?e=tMDAvA

Regards,

Joscha

[ray3]

@bx_joscha

Hi Reimund,

@ray

That’s good to know, thanks for your report. I assume that also goes for the Fuse plugins that do include meters (VCL-4, TCS-68 and the VCL-373)?

I cannot reproduce it with VCL-4, BUT look at my screenshot. The GUI draws quite strange here? (non-retina display, Cubase 9.5, VST-2)

https://pluginalliance-my.sharepoint.com/:i:/g/personal/joscha_rieber_plugin-alliance_com/ETouwq1enuNMutJZzg-Kr8MBl5TPKkA1uu6LWYUOqV_s2w?e=h9YReV (download the image and watch it without a scaling. SharePoint seems to blur the image.)

Hi Joscha,

Thanks for the info!

Two questions:

1. Do you have the UI Size set to “Normal (100%)”? Otherwise, this is a known issue mentioned under https://www.fuseaudiolabs.de/index.html#faq which affects non-highDPI displays on mac os and which will be fixed in one of the next builds.
2. Does the plugin run in 32 bits (check e.g. the “About…” info in the plugin menu)? 32Bits VST2 is Carbon on Mac (unless you support REAPER’s cockos extensions, which I don’t). If so, does the UI look better in VST3 or 64 bits?

Thanks!

Best,
Reimund

[bx_joscha]

@Arne Scheffler

Hi,
yes I’m sure that Quartz Debug does not work any longer when using a CALayer.
By the way, you should find a way to make your UI work with the CALayer stuff as this is the way to go. The next macOS version will implicitly set the layer to be active.
On what macOS version did you see this performance issue ?

Cheers,
Arne

Hello Arne,

unfortunately, you are right. We encounter similar problems in Mojave.


Thanks,

Joscha

[Arne_Scheffler]

Hi Joscha,
please move your Meter View into a CLayeredViewContainer and see if it fixes the issue.

Cheers,
Arne

[bx_joscha]

Hey Reimund,

@ray

1. Do you have the UI Size set to “Normal (100%)”? Otherwise, this is a known issue mentioned under https://www.fuseaudiolabs.de/index.html#faq which affects non-highDPI displays on mac os and which will be fixed in one of the next builds.

Yes, 100 % it is.

@ray

2. Does the plugin run in 32 bits (check e.g. the “About…” info in the plugin menu)? 32Bits VST2 is Carbon on Mac (unless you support REAPER’s cockos extensions, which I don’t). If so, does the UI look better in VST3 or 64 bits?

It is 64 bits in Cubase 9.5 VST2. Switching to VST3 does not resolve the issue. VST3 is also affected.

Regards,

Joscha

[ray3]

@bx_joscha

Hey Reimund,

@ray

1. Do you have the UI Size set to “Normal (100%)”? Otherwise, this is a known issue mentioned under https://www.fuseaudiolabs.de/index.html#faq which affects non-highDPI displays on mac os and which will be fixed in one of the next builds.

Yes, 100 % it is.

@ray

2. Does the plugin run in 32 bits (check e.g. the “About…” info in the plugin menu)? 32Bits VST2 is Carbon on Mac (unless you support REAPER’s cockos extensions, which I don’t). If so, does the UI look better in VST3 or 64 bits?

It is 64 bits in Cubase 9.5 VST2. Switching to VST3 does not resolve the issue. VST3 is also affected.

Regards,

Joscha

Thanks for the info, Joscha!

[bx_joscha]

@Arne Scheffler

Hi Joscha,
please move your Meter View into a CLayeredViewContainer and see if it fixes the issue.

Cheers,
Arne

Hi Arne,

Thanks for the hint. But we also encounter CPU spikes when adjusting a knob in case of explicitly using CALayers. So, not only our meters are affected. But with meters, it is constantly bad. So, basically every kind of animation is a potential problem. But we will try wrapping control bundles into Layered View Containers.

Thanks again to Arne and Ray for the input. We will investigate soon and consider your suggestions.


Regards,

Joscha

[bx_joscha]

Hi Arne and all others,

I have good news. We have managed to fix the sluggishness in our UI implementation. Now, we can revert the setWantsLayer patch. So, we do not comment it out anymore. We can see that the performance is inherently optimized when using the setWantsLayer in the NSViewFrame constructor.


Thanks for all the help,

Joscha

[Arne_Scheffler]

Hi Joscha,
that’s nice to hear. So what was the reason ?

Cheers,
Arne

[bx_joscha]

@Arne Scheffler

Hi Joscha,
that’s nice to hear. So what was the reason ?

Cheers,
Arne

The sluggishness was caused by an obsolete setDirty() for a UI control that was continuously called and caused redundant redrawing. In a former version of VSTGUI without the setWantsLayer patch, we did not notice, but with setWantsLayer the effect was very dramatic.

Regards,

Joscha