We recently updated to the VST3 SDK v3.7.7 and now users are reporting a weird issue. It’s related to the aucocoaview.mm implementation and its recent ObjcClassBuilder style refactoring.
Namely in lines 191/192: If
AUView::setFrame() is called when the GUI is opened everything works fine. However, once the user resizes our GUI which invokes
inst.callSuper<void (NSRect)> gets dispached to
[self setFrame...] instead of the super class’ method, which, of course, yields an endless recursion / stack overflow → crash.
The only difference I can see on the call stack for things to go wrong is that AUView::setFrame() is called from the VST3 library which calls into the AUWrapper lib.
To make things even stranger this only happens on Intel or in some circumstances on arm64. You can for instance reproduce the issue in the latest GarageBand on Ventura 13.1, but it’ll only happen there when GB is run in Rosetta mode.
I can work around this by either including a recurse guard or by commenting out the call to the super class’ setFrame function which, of course, isn’t the way to go because the container window won’t get resized that way.
It seems to me that something is wrong with the objcclassbuilder.h implementation. We didn’t change anything in the client code so I’m pretty sure it does have to do with the aforementioned refactoring of aucocoaview.mm. I still can’t figure out what’s the reasoning behind the change to a “dynamic” ObjCClassBuilder based implementation here at all.
EDIT: Please check the latet commit in GitHub - raygit83/vst3_public_sdk: VST 3 Implementation Helper Classes And Examples . It looks like when reverting back to a vanilla Objective C implementation the issue won’t occur at all.