Crash in AUv2 Wrapper using VST3 SDK v3.7.7

Developer VST 3 SDK
1

Hi,

We recently updated to the VST3 SDK v3.7.7 and now users are reporting a weird issue. It’s related to the aucocoaview.mm implementation and its recent ObjcClassBuilder style refactoring.

Namely in lines 191/192: If AUView::setFrame() is called when the GUI is opened everything works fine. However, once the user resizes our GUI which invokes CFrame::setFrame(), inst.callSuper<void (NSRect)> gets dispached to [self setFrame...] instead of the super class’ method, which, of course, yields an endless recursion / stack overflow → crash.

The only difference I can see on the call stack for things to go wrong is that AUView::setFrame() is called from the VST3 library which calls into the AUWrapper lib.

To make things even stranger this only happens on Intel or in some circumstances on arm64. You can for instance reproduce the issue in the latest GarageBand on Ventura 13.1, but it’ll only happen there when GB is run in Rosetta mode.

I can work around this by either including a recurse guard or by commenting out the call to the super class’ setFrame function which, of course, isn’t the way to go because the container window won’t get resized that way.

It seems to me that something is wrong with the objcclassbuilder.h implementation. We didn’t change anything in the client code so I’m pretty sure it does have to do with the aforementioned refactoring of aucocoaview.mm. I still can’t figure out what’s the reasoning behind the change to a “dynamic” ObjCClassBuilder based implementation here at all.

EDIT: Please check the latet commit in GitHub - raygit83/vst3_public_sdk: VST 3 Implementation Helper Classes And Examples . It looks like when reverting back to a vanilla Objective C implementation the issue won’t occur at all.

Best,
Ray

2

Hi Ray,
I currently try to reproduce this, but no luck so far. Can you reproduce the issue with the AGain example from the SDK?

3

Hi Arne,

Not yet, but I’ll try ASAP or come up with a minimal example that demonstrates this behavior. The plugin has to call CFrame::setZoom() after the editor has been opened.

Please note that it only seems to occur in some circumstances: It has been observed in REAPER (both platforms) or GarageBand and Logic running on Intel or in Rosetta2 mode. There it strangely enough doesn’t happen on arm64.

EDIT: Of course, it’s the call to IPlugFrame::resizeView() not CFrame::setZoom().

Best,
Ray

4

Hi Arne,

Here’s a minimal example. Tested with the latest SDK (applying the necessary patches to the AUWrapper in lines 2584…) on an MBP M1 2020 running Ventura 13.2.1 and the latest GarageBand version.

Everything works fine on arm64, but the crash occurs once you run in Rosetta2 mode. The same has been reported for intel machines:

        class AGainEditor : public VSTGUIEditor, public IControlListener
        {
        enum
        {
            kWidth = 400,
            kHeight = 400,
            kButton1 = 'btn1',
            kButton2 = 'btn2'
        };
            
        public:
            AGainEditor (void* controller) :
                VSTGUIEditor(controller)
            {
                ViewRect vr(0, 0, kWidth, kHeight);
                setRect(vr);
            }
            
            virtual ~AGainEditor () {}
            
            virtual bool PLUGIN_API open (void* parent, const VSTGUI::PlatformType& platformType) SMTG_OVERRIDE
            {
                auto r = getRect();
                CRect size(r.left, r.top, r.right, r.bottom);
                
                auto oldFrame = frame;
                frame = new CFrame(size, this);
                if(oldFrame)
                    oldFrame->forget();
                
                if(frame->open(parent, platformType))
                {
                    frame->setBackgroundColor(kBlueCColor);
                    
                    CRect buttonSize(0, 0, 120, 20);
                    
                    buttonSize.centerInside(size);
                    buttonSize.offset(CPoint(-buttonSize.getWidth() / 2 - 2, 0));
                    auto button1 = new CTextButton(buttonSize, this, kButton1, "zoomFactor = 1", CTextButton::Style::kKickStyle);
                    frame->addView(button1);
                    
                    buttonSize.centerInside(size);
                    buttonSize.offset(CPoint(+buttonSize.getWidth() / 2 + 2, 0));
                    auto button2 = new CTextButton(buttonSize, this, kButton2, "zoomFactor = 2");
                    frame->addView(button2);
                    
                    return true;
                }
                
                return false;
            }
            
            virtual void PLUGIN_API close () SMTG_OVERRIDE
            {
                if(frame)
                {
                    frame->close();
                    frame = nullptr;
                }
            }
            
            virtual void valueChanged (CControl* pControl) SMTG_OVERRIDE
            {
                int tag = pControl->getTag();
                float value = pControl->getValue();
                if(value > 0.5f)
                {
                    if(tag == kButton1)
                    {
                        frame->setZoom(1.0);
                    }
                    
                    else if(tag == kButton2)
                    {
                        frame->setZoom(2.0);
                    }
                }
            }
            
            bool beforeSizeChange(const CRect& newSize, const CRect& oldSize) SMTG_OVERRIDE
            {
                static bool resizeGuard = false;
                bool result = resizeGuard;

                if(plugFrame && !result)
                {
                    resizeGuard = true;
                    ViewRect vr;
                    vr.right = int32(newSize.getWidth());
                    vr.bottom = int32(newSize.getHeight());
                    result = (plugFrame->resizeView(this, &vr) == kResultTrue);
                    resizeGuard = false;
                }

                return result;
            }
        };

Calling plugFrame->resizeView(this, &vr) will end up in the aforementioned endless recursion in the AUCocoaView implementation where inst.callSuper<void (NSRect)>... calls the class’ own function instead of the super class’ one.

Note that the problem goes away once I use the older AUCocoaView implementation which doesn’t rely on the ObjCClassBuilder, perhaps that helps.

Here’s a different bug that has been recently reported after updating to the VST3 SDK 3.7: The editor is flipped vertically on macOS (AU?) when running in Live 9. This hasn’t been the case with older SDK versions. I don’t have access to Live 9 but perhaps you wanna look into it too at some point.

Best,
Ray