So I tested a little further and actually it really seems to be a Cubase 12 issue. I did a quick test with the Waves MV2 in an empty session with only one audio track in Cubase 12 and Cubase 11.
The loading time on Cubase 12 is more than 10 times higher than on Cubase 11. The program completely freezes for around 20 seconds while in Cubase 11 it’s more or less immediate. Only while playback you get the usual hickups. Also as described above, once one plugin instance was loaded, the issue is gone with further instances in the same session.
I really hope that someone from Steinberg will look into this. At this time Cubase 11 is so much more stable than 12 on my system. Actually Cubase 12 is the most unstable version I had so far since 10 years. 
Video Proof:
Cubase 11:
Cubase 12:
