On macOS 10.13 our GUIs are mirrored vertically in Digital Performer 9.52. Perhaps also newer macOS versions are affected. This happens in both AU and VST. The same builds are okay in other hosts running on the same system.
This is likely VSTGUI related, but perhaps it helps to mention that this issue was introduced after we updated to VST SDK 3.7.7 and it hadn’t been present with the previous SDK version we were using (3.7.1) - we are always using the respective VSTGUI version that comes with the SDK. There weren’t any changes in the respective client code between the intact and the broken plugin builds.
That’s possible. Unfortunately, I don’t own Digital Performer, so we have to rely on the customer’s feedback. Can you guys investigate on your end, too or did anyone else encounter the same issue in their plugins as well?
It’s kind of weird that this only happens in DP, though.
EDIT: If I understand his last post correctly, the fix is obsolete since the CMultiFrameBitmap update as long as one uses “more quadratic” or smaller bitmap dimensions according to Matthias at BX. Is it possible to override the CALayer / red bitmap glitch changes, easily so I can prepare a test build for the customer?
We had a few e-mails that are not mirrored on the forum. Only both changes fixed the red bitmaps for him.
So it’s unlikely that we change this back as it is also something the Apple engineers recommended to us to prevent future issues.
And I’m sorry we are also not able to investigate this further, no DP license and no macOS 10.13 available anymore.
If you want to check if this is indeed the cause of the issue, you can just revert the specific commit in git (this one).
I also don’t think that this is macOS 10.13 specific. It’s more likely to be specific to the SDK version DP was built with. Do you know which version of DP that is? Is it the current released version?
I see. We weren’t getting any reports about mirrored GUIs up until recently and these builds have been in the wild for a while. So I’m pretty sure this only happens on this very specific setup (or DP in general).
One more thought: I haven’t familiarised myself with the specifics of CALayer so I’m just guessing here. There’s a property named contentsAreFlipped (not sure about geometryFlipped) whereas isFlipped always returns YES in the VSTGUI_NSView class definition. Is it conceivable that this along with the embedded CALayer hierarchy should inherit the parent view’s property instead?
The user has also noted that it’s only the visual contents that are flipped but the controls react to the areas where they would normally be placed.
the user who reported the issue was kind enough to run a test build in DP which outputs the NSView and CALayer hierarchies in which the GUI is embedded (see screenshot)
It seems that both of the CALayers attached to the bottom NSView node (0x608000235180) have their contentsAreFlipped property set to NO, while the respective bottom NSView node has isFlipped set to YES.
Depending on the format and DAW you’re running this in, you will of course observe different hierarchies with different “flippings”, but in all correct cases the CALayers’ contentsAreFlipped will be equal to the isFlipped value of the NSLayer holding the layers.
I wonder why that is? I haven’t inspected the NSViewFrame code deeply, but are these properties touched by VSTGUI at all when the CALayer is created? Could this be in some way related to VSTGUI’s obj C class creator implementation?
One could fix this by inspecting the respective properties and then set the CALayer’s isGeometryFlipped to YES if required, but I wonder how and where this could be fixed elsewhere.
Here are two screenshots of cases with the correct appearance:
VSTGUI does not touch the CALayer properties contentsAreFlipped and isGeometryFlipped. And I just see that contentsAreFlipped is just a method and as we use a plain CALayer it returns the default value. This is the doc about it:
* Returns true if the contents of the contents property of the layer
* will be implicitly flipped when rendered in relation to the local
* coordinate space (e.g. if there are an odd number of layers with
* flippedGeometry=YES from the receiver up to and including the
* implicit container of the root layer). Subclasses should not attempt
* to redefine this method. When this method returns true the
* CGContextRef object passed to -drawInContext: by the default
* -display method will have been y- flipped (and rectangles passed to
* -setNeedsDisplayInRect: will be similarly flipped). */
So this explains it. The drawInContext must flip the coordinates of the CGContext in this case.
Will have a look later.
And I just see that contentsAreFlipped is just a method…
Yes, I am aware of that and hence I was suggesting to use isGeometryFlipped which can be changed on the fly. A simple fix that would work for the scenarios I have observed would be something along the lines of the following code patched below line 964 in nsviewframe.mm and probably something similar using the CALayerDelegate view in caviewlayer.mm:
We tried exactly that first and unfortunately it didn’t solve it. If you inspect the examples from the screenshots, you’ll see that the broken case has the terminal layer “unflipped” while the containing view is flipped and in these cases the last two layers of the CALayer hierarchy have the same flipped state (the first screenshot shows the broken example in DP, the two others are fine).
EDIT: Sorry, we tried caLayer.geometryFlipped = ![nsView isFlipped]; which of course didn’t work. I’ll try your suggestion as well and report back. At least it doesn’t mess up the GUI orientation in the hosts I can test.
Just heard back from the DP user: The code I posted above seems to fix it without messing up the cases in which things were already working. The only pattern I could see was what I was describing about the discrepancy between the view and the layer flipping. I’m still a bit sceptic as to whether this is just a random side effect, so it could still break in other cases. I don’t mind admitting that I don’t yet fully understand what’s going on.
I’ll also provide a build to the user that includes your suggestion and report back. I already checked the setups available to me and at least it doesn’t break that way.
Would you mind elaborating why the two terminal layers added by VSTGUI should have opposite geometryFlipped states and why that would fix the issue?
EDIT: Yep, caLayer.geometryFlipped = ![nsView.layer contentsAreFlipped]; seems to work, too. I’d like to understand why and whether it’s a valid fix in all cases.
The CALayer and the NSView are independent of each other. So the NSView.isFlipped property does not affect the CALayer rendering. At least this is how I understand this.
Yes, okay, that’s what I gathered. But why should the two “terminal” CALayers have opposite geometryFlipped settings and why does that fix the issue. It seems like it does, but I don’t have a clue why.
