Disabled COptionMenu entries on mac Cocoa

Hi there,

We’re currently adding a preset menu to our toolbar in our video effect plugins. These plugins are hosted in Adobe AfterEffects, Premiere Pro, Photoshop and Apple Final Cut Pro X using our proprietary framework.

The GUI portion is developed using VSTGUI (v4.7) and I did not have any issues whatsoever using COptionMenu in various contexts - i.e. in both standalone nor audio plugin hosting environments.

Our implementation works fine in AfterEffects and Final Cut Pro X in which the GUI is hosted in interactive mode. This is not the case in Premiere Pro and Photoshop, though, where the GUI must be handeled in a modal dialog once the host calls into the respective selector after the user has requested to open the effect GUI.

We’re creating our GUI window using the following Cocoa code:

NSWindow *nswindow = [NSWindow alloc];
[nswindow initWithContentRect:windowFrame
[nswindow setReleasedWhenClosed:NO];
[nswindow setLevel:CGShieldingWindowLevel()];
[nswindow makeKeyAndOrderFront:nil];

[nswindow contentView]

is passed to


and the GUI is created in the usual fashion, e.g. the preset menu is populated using something like:

COptionMenu *presetMenu = new COptionMenu(size, this, -1, hBackground, hBackgroundClicked);
preset_menu->addEntry("Preset 1");
preset_menu->addEntry("Preset 2");

The GUI is serviced using the following Cocoa code:

NSApplication *nsapp = [NSApplication sharedApplication];
NSModalSession *modalSession = [nsapp beginModalSessionForWindow:nswindow];

bool running = true;

	// Service main loop
	[[NSRunLoop mainRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.025f]];
	// Run modal session
        running = ([nsapp runModalSession:modalSession] == NSModalResponseContinue);

[nsapp endModalSession:modalSession];

The GUI is fully responsive and the preset menu can be opened. However, all the menu entries are greyed out, although none of them have been marked disabled. This only happens in the context of a modal session. I had other instances in the audio world where a very similar code context using a COptionMenu in a modal session in an activation dialog performed just fine, so it must have something to do with the application environment.

Further investigation reveals that VSTGUI_NSMenu_ValidateMenuItem in nsviewoptionmenu.mm which is responsible for en-/disabling menu items is never called.

Maybe some of the Cocoa experts can help?



this looks like that the apps (both Adobe) are stealing this message. Thus I don’t think that you can fix this on your side. Recently we added internal driven menu support. You can call frame->getPlatformFrame->setGenericOptionMenu (true) to use it. Maybe that’s an option for you.


That solved it. Thanks a lot, Arne!