this usually indicates that “Constrain Delay Compensation” is active. Have a look at the bottom left of your project window and click on this icon, please.
Check if the track in question is frozen. You may need to unfreeze it, first, before enabling/disabling/removing plugins… or even adjusting anything on Channel Settings.
You are right, good thinking I admit that I tend to forget that freeze might as well be the culprit in some scenarios. In this case, however, it is unlikely since it only affects a few plugins on the track - the ones that usually come with high latency - and not all plugins. You can see it in the screenshot above.
Nonetheless, good point. I better keep that potential culprit in mind