Thats weird. I don’t know why you’re hard drive performance is seemingly different than mine…
check out this thread for my one (extra) click workaround 
As I understand it, your problem and FShkreli’s problems are a little different though. FShkreli is experiencing real sluggish performance when navigating through the folders (the Define Locations panel), and you are more concerned about a 2-3 second lag in your search results?