I believe I have finally narrowed down the issue 
I first had an issue with Quantize which was very similar to this one. I’ve then followed up by testing this case even more thoroughly.
Here is the new thread : Phase drift and glitches with Transpose, Time stretching and AudioWarp/Quantize
Tell me what you think.