Yep, that’s the way ASIO guard works. It puts every track/group/buss/FX on a very high latency except the one that is active. That track(s) is running on the nominal latency set in the ASIO settings.
My system is set to 64 samples (old RME Multiface). When I activate a track, it’s running on those 64 samples for recording MIDI and/or audio. Everything else in my project runs with a much higher latency. AFAIK with ASIO guard on “high” it’s 64x the latency, so with ASIO latency set on 64 samples all tracks but the active ones are running @ 4096.
Having said that, the problem we are facing with Cubase 12 is that it takes much more CPU for the activated channel(s) (=running on ASIO/nominal latency) than Cubase 11 did. A peak is to be expected when activating a CPU-heavy track, that is the same behaviour in C11. But in C12 that hit/peaks is/are much higher than before.