It’s a bit more complicated.
First: track / mixer channel dependencies
As soon as you have tracks (mixer channels) depending on other tracks / mixer channels, their processing will have to wait for all tracks / mixer channels they depend on to finish their calculations.
This affects busses (sum, fx, groups) and sidechains.
Second: system latencies
This is even worse - realtime processing also depends on reliable response times / low latencies… and modern OS can’t really provide that, even though there is some progress.
Whenever there is a critical section in hardware driver code (graphics, network, whatever), the CPU can’t issue other critical sections - and this is only ONE of the reasons for having 50% of CPU meaning 100% ASIO load.
What we would need from Microsoft (and Apple, of course) is a mode of operation where device drivers are singled out to a single CPU core and where priorities can be handed out (so a critical section in the ASIO driver is always handled first and probably even breaks other critical section locks - this would of course cause problems with system stability, but it could most probably be controlled, etc…).
So your 6 core CPU will become a 5 core CPU for Cubase, etc… but would run MUCH faster and with much higher available CPU power.
Third: the low latency buffer sizes
It took me some time to wrap my head around this one when I first encountered it back in 2003 or so… smaller buffer sizes are bad for ASIO load. But why?
First, thread / context switches are costly. We’re talking about thousands or even ten thousands of clock cycles here.
Especially problematic: raw CPU performance is highly dependent on locality of data nowadays (because of caching) - but every time you work on a different plugin, different track you basically thrash the L1, L2 and (possibly) L3 cache. Reason: other data sets are used which are seriously not local enough.
Second, smaller buffer sizes mean more interrupts to the ASIO driver, therefore higher sensibility to badly written drivers for all the other hardware drivers in your system.
(ASIO drivers are generally quite good or even outright excellent. Problem is some special archetype of “hardware / software engineer” in typical hardware manufacturer companies. Learned his tools of the trade back in the 80s or early 90s at best and never bothered to really learn new ways of thinking, only on the surface if he has to. And still is considered a “capacity” by many - because they don’t know better. Which is sad. And the source of so much hassle.
Have a look at POS printer drivers -and firmwares - and the insane amount of problems they cause - just to give you one very strong example. Most POS printer drivers are so buggy that when you develop software which uses them, about 80% of the time spent with printing actually concerns workarounds and similar things.
This is just a placative example. WiFi and similar devices are often not much better.)