That’s weird, I have never seen a Dorico window resize on a mode switch, and I don’t think it’s supposed to either. You don’t use macros or custom keyboard shortcuts to switch modes? I mean, something on your system does this unnecessary step, and I think it’s not Dorico on its own. (I know this possibility is a bit far-fetched: I can’t think of a good reason to replace ⌘1…⌘5 by something else.)
BTW: With the preference to open new windows maximized checked, when I select New Window (or type ⇧⌘T) in an open project, the new window is not maximized, unless the original front window was already maximized. The new window always inherits the maximization (is that a word?) from the front window, regardless of the the setting of the preference.