You keep referring to latency which you are not really seeing. The ASIO Guard does not in ANY way show latency on your system.
All you are seeing is how big the ASIO Guard buffer is. This is NOT latency.
Are you pressing a key and the note doesn’t come out until a second later? THAT is latency. The meters do not show and have absolutely NOTHING to do with latency. There is no meter in Cubase at all that shows you latency. You can have latency and have the ASIO Guard not even above 1/4" of the way. You can have latency and never see the CPU meter move, has nothing to do with the meter.