Latency in Logic Pro when using Dorico + NP

I have a macbook M1. When I start up Logic Pro and listen to my audio input (mic => audio interface => macbook, buffer size set to 32 samples) the latency is so low that I can’t hear it (could be under 5ms).
However when I then open a session of Dorico + NP in order to work on my string arrangement, after some time I start to get serious latency issues in Logic Pro (around 30-50ms), I didn’t find yet when the latency exactly starts manifesting. I only experience this latency issue when Dorico + NP are open, so it must be somehow related to that. When I close everything (also Logic) and startup Logic again, latency is normal again.

Can someone explain me what could possibly affect latency cross-application?

Could be random MIDI signals going over the IAC (Inter-Application … something).

Open Audio MIDI Setup.app, and turn it off if you don’t need it.

There are also ways of disabling it as a valid input within Logic’s settings, I think.