Pro Tools hangs at exit when using DirectComposition

Problem: Pro Tools (on Windows) hangs on exit if a plugin using VSTGUI was instanciated at any point during the process lifetime.

The only solution I have found so far is to prevent the call to DirectComposition::Factory::create. Calling disableDirectComposition at a later time does not fix the hang.

I’m not sure if this represents a bug in VSTGUI, Pro Tools, or graphics drivers. I was able to reproduce on a system with the latest AMD Catalyst drivers, but I’ve also seen reports from users with Nvidia drivers.

Sharing in case anyone has further insights or suggestions on potential solutions that don’t require disabling DirectComposition.

Backtrace:

ntdll.dll!NtWaitForMultipleObjects()
KernelBase.dll!WaitForMultipleObjectsEx()
KernelBase.dll!WaitForMultipleObjects()
amdxx64.dll!00007ffc80c5968b()
amdxx64.dll!00007ffc80c26fd7()
amdxx64.dll!00007ffc80c72671()
amdxx64.dll!00007ffc80c72a4e()
amdxx64.dll!00007ffc80d36046()
d3d11.dll!NDXGI::CDevice::DestroyDriverInstance(void)
d3d11.dll!CContext::LUCBeginLayerDestruction()
d3d11_3SDKLayers.dll!CBridgeImpl<struct ILayeredUseCounted,struct ID3D11LayeredUseCounted,class CLayeredObject<class NDebug::CBlendState> >::LUCBeginLayerDestruction(void)
d3d11.dll!NOutermost::CDeviceChild::LUCBeginLayerDestruction(void)
d3d11.dll!NOutermost::CDeviceChild::FinalRelease(void)
d3d11.dll!CUseCountedObject<class NOutermost::CDeviceChild>::`vector deleting destructor'(unsigned int)
d3d11.dll!CUseCountedObject<class NOutermost::CDeviceChild>::UCDestroy(void)
d3d11.dll!CUseCountedObject<class NOutermost::CDeviceChild>::UCReleaseUse(void)
d3d11.dll!CLockOwnerChild<class CDevice,0>::UCReleaseUse(void)
d3d11.dll!CDevice::LLOBeginLayerDestruction(void)
d3d11.dll!NDXGI::CDevice::LLOBeginLayerDestruction(void)
d3d11_3SDKLayers.dll!CBridgeImpl<struct ILayeredLockOwner,struct ID3D11LayeredDevice,class CLayeredObject<class NDebug::CDevice> >::LLOBeginLayerDestruction(void)
d3d11.dll!NOutermost::CDevice::LLOBeginLayerDestruction(void)
d3d11.dll!TComObject<class NOutermost::CDevice>::~TComObject<class NOutermost::CDevice>(void)
d3d11.dll!TComObject<class NOutermost::CDevice>::`vector deleting destructor'(unsigned int)
d3d11.dll!TComObject<class NOutermost::CDevice>::Release(void)
d3d11_3SDKLayers.dll!CLayeredObject<class NDebug::CDevice>::CContainedObject::Release(void)
Redacted.vst3!VSTGUI::COM::IUnknownSafeRelease<ID3D11Device1>(ID3D11Device1 * i) Line 26
Redacted.vst3!VSTGUI::COM::Ptr<ID3D11Device1>::reset() Line 80
Redacted.vst3!VSTGUI::COM::Ptr<ID3D11Device1>::~Ptr<ID3D11Device1>() Line 57
Redacted.vst3!VSTGUI::DirectComposition::Factory::Impl::~Impl()
Redacted.vst3!VSTGUI::DirectComposition::Factory::Impl::`scalar deleting destructor'(unsigned int)
Redacted.vst3!std::default_delete<VSTGUI::DirectComposition::Factory::Impl>::operator()(VSTGUI::DirectComposition::Factory::Impl * _Ptr) Line 3309
Redacted.vst3!std::unique_ptr<VSTGUI::DirectComposition::Factory::Impl,std::default_delete<VSTGUI::DirectComposition::Factory::Impl>>::~unique_ptr<VSTGUI::DirectComposition::Factory::Impl,std::default_delete<VSTGUI::DirectComposition::Factory::Impl>>() Line 3427
Redacted.vst3!VSTGUI::DirectComposition::Factory::~Factory() Line 648
Redacted.vst3!VSTGUI::DirectComposition::Factory::`scalar deleting destructor'(unsigned int)
Redacted.vst3!std::default_delete<VSTGUI::DirectComposition::Factory>::operator()(VSTGUI::DirectComposition::Factory * _Ptr) Line 3309
Redacted.vst3!std::unique_ptr<VSTGUI::DirectComposition::Factory,std::default_delete<VSTGUI::DirectComposition::Factory>>::~unique_ptr<VSTGUI::DirectComposition::Factory,std::default_delete<VSTGUI::DirectComposition::Factory>>() Line 3427
Redacted.vst3!VSTGUI::Win32Factory::Impl::~Impl()
Redacted.vst3!VSTGUI::Win32Factory::Impl::`scalar deleting destructor'(unsigned int)
Redacted.vst3!std::default_delete<VSTGUI::Win32Factory::Impl>::operator()(VSTGUI::Win32Factory::Impl * _Ptr) Line 3309
Redacted.vst3!std::unique_ptr<VSTGUI::Win32Factory::Impl,std::default_delete<VSTGUI::Win32Factory::Impl>>::~unique_ptr<VSTGUI::Win32Factory::Impl,std::default_delete<VSTGUI::Win32Factory::Impl>>() Line 3427
Redacted.vst3!VSTGUI::Win32Factory::~Win32Factory() Line 131
Redacted.vst3!VSTGUI::Win32Factory::`scalar deleting destructor'(unsigned int)
Redacted.vst3!std::default_delete<VSTGUI::IPlatformFactory>::operator()(VSTGUI::IPlatformFactory * _Ptr) Line 3309
Redacted.vst3!std::unique_ptr<VSTGUI::IPlatformFactory,std::default_delete<VSTGUI::IPlatformFactory>>::reset(VSTGUI::IPlatformFactory * _Ptr) Line 3471
Redacted.vst3!std::unique_ptr<VSTGUI::IPlatformFactory,std::default_delete<VSTGUI::IPlatformFactory>>::operator=<std::default_delete<VSTGUI::IPlatformFactory>,0>(std::unique_ptr<VSTGUI::IPlatformFactory,std::default_delete<VSTGUI::IPlatformFactory>> && _Right) Line 3414
Redacted.vst3!VSTGUI::setPlatformFactory(std::unique_ptr<VSTGUI::IPlatformFactory,std::default_delete<VSTGUI::IPlatformFactory>> && f) Line 26
Redacted.vst3!VSTGUI::exitPlatform() Line 40
Redacted.vst3!VSTGUI::exit() Line 21
Redacted.vst3!<lambda>() Line 63
Redacted.vst3!std::invoke<void <lambda>(void) &>(void <lambda>(void) & _Obj) Line 1670
Redacted.vst3!std::_Func_impl_no_alloc<void <lambda>(void),void>::_Do_call() Line 880
Redacted.vst3!std::_Func_class<void>::operator()() Line 926
Redacted.vst3!Steinberg::`anonymous namespace'::sortAndRunFunctions(std::vector<std::pair<unsigned int,std::function<void __cdecl(void)>>,std::allocator<std::pair<unsigned int,std::function<void __cdecl(void)>>>> & array) Line 84
Redacted.vst3!DeinitModule() Line 121
Redacted.vst3!ExitDll() Line 82
Redacted.aaxplugin!VST3::Hosting::`anonymous namespace'::Win32Module::~Win32Module() Line 166
Redacted.aaxplugin!VST3::Hosting::`anonymous namespace'::Win32Module::`scalar deleting destructor'(unsigned int)
Redacted.aaxplugin!std::_Destroy_in_place<VST3::Hosting::`anonymous namespace'::Win32Module>(VST3::Hosting::`anonymous-namespace'::Win32Module & _Obj) Line 324
Redacted.aaxplugin!std::_Ref_count_obj2<VST3::Hosting::`anonymous namespace'::Win32Module>::_Destroy() Line 2118
Redacted.aaxplugin!std::_Ref_count_base::_Decref() Line 1160
Redacted.aaxplugin!std::_Ptr_base<VST3::Hosting::Module>::_Decref() Line 1385
Redacted.aaxplugin!std::shared_ptr<VST3::Hosting::Module>::~shared_ptr<VST3::Hosting::Module>() Line 1690
Redacted.aaxplugin!`GetPluginFactory'::`2'::`dynamic atexit destructor for 'vst3module''()
Redacted.aaxplugin!_execute_onexit_table::__l2::<lambda>() Line 206
Redacted.aaxplugin!__crt_seh_guarded_call<int>::operator()<void <lambda>(void),int <lambda>(void) &,void <lambda>(void)>(__acrt_lock_and_call::__l2::void <lambda>(void) && setup, _execute_onexit_table::__l2::int <lambda>(void) & action, __acrt_lock_and_call::__l2::void <lambda>(void) && cleanup) Line 204
Redacted.aaxplugin!__acrt_lock_and_call<int <lambda>(void)>(const __acrt_lock_id lock_id, _execute_onexit_table::__l2::int <lambda>(void) && action) Line 983
Redacted.aaxplugin!_execute_onexit_table(_onexit_table_t * table) Line 231
Redacted.aaxplugin!common_exit::__l2::<lambda>() Line 225
Redacted.aaxplugin!__crt_seh_guarded_call<void>::operator()<void <lambda>(void),void <lambda>(void) &,void <lambda>(void)>(__acrt_lock_and_call::__l2::void <lambda>(void) && setup, common_exit::__l2::void <lambda>(void) & action, __acrt_lock_and_call::__l2::void <lambda>(void) && cleanup) Line 224
Redacted.aaxplugin!__acrt_lock_and_call<void <lambda>(void)>(const __acrt_lock_id lock_id, common_exit::__l2::void <lambda>(void) && action) Line 983
Redacted.aaxplugin!common_exit(const int return_code, const _crt_exit_cleanup_mode cleanup_mode, const _crt_exit_return_mode return_mode) Line 259
Redacted.aaxplugin!_cexit() Line 322
Redacted.aaxplugin!__scrt_dllmain_uninitialize_c() Line 404
Redacted.aaxplugin!dllmain_crt_process_detach(const bool is_terminating) Line 182
Redacted.aaxplugin!dllmain_crt_dispatch(HINSTANCE__ * const instance, const unsigned long reason, void * const reserved) Line 220
Redacted.aaxplugin!dllmain_dispatch(HINSTANCE__ * const instance, const unsigned long reason, void * const reserved) Line 293
Redacted.aaxplugin!_DllMainCRTStartup(HINSTANCE__ * const instance, const unsigned long reason, void * const reserved) Line 335

The destruction of the module is wrong as far as I can see from the callstack. Is this the AAX wrapper from the SDK?

This is the AAX wrapper from the SDK with some modifications to fit our needs.
Thanks for the tip on module destruction, as you can see from the backtrace we load our wrapped vst3 dynamically and it’s possible this has introduced problems, will be investigating.

This was entirely our problem.

Our modified AAX wrapper uses a static VST3::Hosting::Module to access the wrapped VST3, and this was being destroyed at exit / DLL unload. Explicitly destroying during DeinitModule fixed the issue.

From Preventing Hangs in Windows Applications:

Do not:

  • Do any work in the constructors and destructors for global variables, they are executed under the loader lock

Lesson learned!

1 Like