Staff size and time signatures

I generally use large time signatures on each bracket. I currently have a client who wants a small stave on the top (with click track info). When I change the staff size of this instrument, the size of the time signature in that bracket is also shrunk.

Any way to avoid this?

I don’t think so, no: the calculation of the size of the large time signature is always done relative to the staff to which the time signature is attached, which is always the first one in the bracket.