How to hear sample preview that is shorter than 1 second in a busy project?

I don’t see any options. If I’m listening to drum samples shorter than a second, I see meters jumping but there’s no sound, e.g. it lags so bad the playback is over before it even starts. If I loop it it starts playing after a few seconds and initial hiccuping. Buffer 128 or 2048, doesn’t matter.

I assume it’s latency related?

If I open a brand new project then I have no problems. But inserting new samples in an established project is extremely difficult. Is there a way to bypass latency to listen to the samples?