There may be an easier way, but not that I’m aware of. I take a similar approach to yours when auditioning kicks. Basically, I’ll run through the Media Browser and drag potential samples to Groove Agent ONE. Then, I create a MIDI track with a pattern, as you did, select all events and use the UP/DOWN arrows to switch samples during playback. It can take a lot of time, but I’ve been building kits and soundsets like this for a while, so I don’t always have to hit the Media Browser when looking for a sample.
I use Windows Media Player to open a bunch of samples at once and narrow down the field before putting those I like into a sample player for final choosing.