Use up and down arrow, if you want to preview it again use enter button, in previous versions you could use enter to replace samples (more info here: Nice trick while browsing and replacing sounds through mediabay on sampler track or groove agent) if you want to assign that enter to another key you can use macro from Jazzius Retrigger Samples in Media Bay - Important! - #10 by Jazzius
The whole point is to use arrow keys instead of mouse. In practice you will scroll and skip some samples and that delay doesn’t play big role than, but if you preview each sample with mouse than it becomes tedious, so i suggest to use keys instead.
P.S I see you are using right zone media bay did you tried an regular one? (F5)
Also previewing samples with tempo sync and other context will introduce delay for sure. That being said try to turn off all icons right next to the volume button of sample preview section.