I was able to solve part of the puzzle: The documentation for the Dolby Reference Player (DRP) or Media Encoder (DME) gives the following layouts for 5.1 and 7.1:
5.1: L, R, C, LFE, Ls, Rs.
7.1: L, R, C, LFE, Ls, Rs, Lrs, Rrs.
According to this overview, both layouts use the same surround speakers (namely Ls/Rs). Internally, however, the renderer uses a different layout:
5.1: L, R, C, LFE, Ls, Rs.
7.1: L, R, C, LFE, Lss, Rss, Lrs, Rrs.
This means that a 5.1 signal in a 7.x.x or 9.x.x bed will be processed differently than in a 5.x.x bed. It would be helpful if this was documented somewhere, as it has a big impact on the mix.
For example, if you route your 5.1 music stem to a 5.x.x bed, you will get a free “upmix” from the renderer when playing back with a layout equal to or greater than 7.x.x. This is also problematic because renderers in consumer devices offer additional settings to further “process” the signal.
This explains why a 5.1 signal is processed differently in a 7.x.x bed than in a 5.x.x bed. But it does not explain why the renderer in a 7.x.x bed sends the 5.1 signal for the two surround speakers to the rear speakers as well. According to the Dolby Atmos Master ADM profile, Ls/Rs and Lrs/Rrs have the same coordinates (x, y, z).
My conclusion so far: You only have full control over the signal in a 7.x.x bed.
It would also be helpful if Steinberg didn’t just use “5.1” for a 5.1 layout. A more precise description like 5.1 Dolby (L, R, C, LFE, Ls, Rs) would be better.