The most important of the tips that Lillie has added to the documentation is that you should set the distances on the Vertical Spacing page of Layout Options to the smallest values you would be willing to accept, and then rely on vertical justification to move things further apart. The reason for this is that Dorico essentially adds up the height of the staff plus whatever gap or gaps applies below that staff when determining the height of the unit that it then spreads out with vertical justification. You can imagine it like drawing another staff line however many spaces you have specified for the relevant gap value, below the staff, and unless the frame ends up over 100% full, Dorico won’t allow any part of the system above or below to cross that line. The larger that value, the greater the chance that things that stick out above the staff, like rehearsal marks, will run into that imaginary staff line, thus preventing Dorico from bringing those staves any closer together.
So my advice for part layouts with a single staff would be to set the distance between systems to a smaller value, e.g. perhaps as small as 4 spaces, which should allow Dorico lots of leeway when performing vertical justification, and you should then find that rehearsal marks have much less impact on the final staff spacing.
It’s also worth saying that we have recently made some improvements to the vertical justification algorithm that will make a particular difference when pages are not especially full, which is the case in the example posted in this thread, and these improvements will be in the next major version. The next major version will also include a read-out at the bottom of each frame to tell you how full it is vertically when the staff spacing tool is enabled in Engrave mode.