Issue with IPlatformFrame::setGenericOptionMenu() mode

Developer VSTGUI
1

Hi,

I’d like to use the generic option menu mode using getFrame()->getPlatformFrame()->setGenericOptionMenu(true) but I am currently running in to an issue on Windows.

If any of the menu items triggers something modal, e.g. a message box (via the WIN32 API MessageBox() function), the host will just exit without any exception being thrown. This does not happen in regular COptionMode menu mode. Any ideas? Is this expected behavior?

Best,
Ray

2

Hmm, what happens if you have the debugger attached? Which call is responsible for the exit?

3

That’s my problem. Everything works expectedly, the menu pops up, the text changes to the selected entry, the message box shows up (see valueChanged()), but as soon as I hit “Ok” to close the box, the hosting process is terminated.

The debug console shows some suspicios asserts, though:

c:\program files (x86)\microsoft visual studio 14.0\vc\include\list(230) : Assertion failed: list iterator not incrementable
c:\program files (x86)\microsoft visual studio 14.0\vc\include\list(231) : Assertion failed: “Standard C++ Libraries Out of Range” && 0

Here’s a minimal example, so you can reproduce the issue:

#include <Windows.h>

static const CCoord width = 200;
static const CCoord height = 100;

class TestView :
	public VSTGUIEditor,
	public IControlListener
{
public:
	TestView(EditController* controller) : VSTGUIEditor(controller)
	{
		rect.left = 0;
		rect.top = 0;
		rect.right = int32(width);
		rect.bottom = int32(height);
	}

	virtual ~TestView()
	{
	}

	virtual bool PLUGIN_API open(void* parent, const PlatformType& platformType = kDefaultNative)
	{
		if (frame)
			frame->forget();

		CRect size(0, 0, width, height);
		frame = new CFrame(size, this);

		if (frame && frame->open(parent, platformType))
		{
			auto menu = new COptionMenu(size, this, 0xbadface);
			menu->setFontColor(kWhiteCColor);
			menu->addEntry("Entry 1");
			menu->addEntry("Entry 2");
			menu->setValue(0.f);
			frame->addView(menu);

			// Comment out this line and everything works flawlessly
			frame->getPlatformFrame()->setupGenericOptionMenu(true);

			return true;
		}

		return false;
	}

	virtual void PLUGIN_API close()
	{
		if (frame)
		{
			frame->close();
			frame = nullptr;
		}
	}

	virtual void valueChanged(CControl* pControl)
	{
		if(pControl->getTag() == 0xbadface)
		{
			MessageBoxA((HWND)frame->getPlatformFrame()->getPlatformRepresentation(), "Message Box", "Entry Selected", MB_OK);
		}
	}
};
4

Maybe AddressSanitizer gives you a better clue what’s happening. Otherwise you may could delay calling MessageBox with CFrame::doAfterEventProcessing().
Why are you showing a MessageBox by the way?

5

Thanks Arne. I’ll investigate further using your suggestions. I’m a little puzzled why this only occurs with the new generic menu implementation, though. The message box is just an example demonstrating what triggers this condition.

There is a toolbar with two option menus in our plugins via which users can open a product activation dialog that is serviced in a separate window can and there’s also another entry that brings up an about box with copyright and system/plugin information. The minimal example above shows that the issue isn’t located in our client code, though.

6

How would I go about queueing the event via CFrame::doAfterEventProcessing()? Creating a function that gets pushed to the event queue on valueChanged() (see example above), doesn’t work because pImpl->inEventHandling (see L1894 in cframe.cpp) is false in that case.

Do I need to implement and register an IOptionMenuListener?

EDIT 1: I now get an additional message on the debug console that says

“This sometimes happens, only when we are currently processing a mouse down event and via a callback into the host the window gets destroyed. Otherwise this should never get called. We are the owner of the window and we are responsible of destroying it.”

Does that ring a bell? Is this output somewhere in the VSTGUI code?

EDIT 2: I can see that after the message box has been closed, the generic option menu calls into GenericOptionMenu::removeModalView () where it installs the OptionMenuDone animation, after that the process smokes off.

7

Why don’t you show the activation and about dialog inside your plug-in editor? Showing other top level windows is always an issue if you’re not the main application.
But besides this, I’m out of ideas in your actual problem. Looks like your plug-in editor gets closed while you show the MessageBox and this will of course be a problem.

8

Hi Arne. Well, at least showing a message box or using VSTGUI’s built-in CNewFileSelector (which triggers the same behavior), shouldn’t be an issue if invoked from the GUI thread, imo.

Anyway, I found a workaround that works along the lines of your doAfterEventProcessing() hint. That is I set a flag that e.g. the activation dialog was requested in valueChanged() so the generic option menu implementation can properly tear down the menu overlay and then the actual dialog is brought up in the next idle interval without interfering with the event sequence.

Best,
Ray

9

Here is something related: Do you think you could implement an option to set the menu border color/width and round rect radius in the GenericOptionMenuTheme structure?

Thanks.

10

We are always open for PullRequests on GitHub :wink:

11

Okay, gotcha :wink: …I’ll try to work out something.