I find that when I do a lot of tempo changes, it becomes difficult to play along and quantizing is not as accurate.
If we we had a virtual conductor to look at, then this would be easier. Please consider this. Normally I end up having to temporarily turn off the tempo track. This is a hack. This is not ideal. A little sympathy for someone who does not make dance floor music?