When you have consolidation turned on, empty bars don’t show their real rests, they show multi-bar rests. Yes, the little rectangular blob may look like a regular rest, but it’s actually a multi-bar rest. It may be illuminating to go to Engraving Options > Rests > Multi-bar rests and change the setting so that single-bar multi-bar rests show an H rest and a “1” above. That way you’ll be able to see which (single) bars Dorico’s treating as multi-bar rests.
Text items that are on the downbeat of bars - whether Staff text, System Text, Chord Symbols or Tempo marks - break the multi-bar rests but don’t actually prevent the bar from being treated as a multi-bar rest. This enables Dorico to use different rules for how much space a bar should take up etc.
As soon as you put a text item midway through an otherwise empty bar, that bar is no longer treated as a multi-bar rest and shows its real rests (unless they’ve been removed, in which case the absence of the rest is respected).
If you turn off multi-bar rest consolidation, every bar, whether empty or not, is spaced according to its contents. If it contains a bar rest, it’ll be spaced to account for that bar rest (he said, massively simplifying). If it contains notes, it’ll be spaced to account for those notes and any rests. If it contains literally nothing, because you’ve removed the rest that would otherwise be the only thing in the bar, there is nothing for Dorico to space, so it squishes the bar up.