Under the hood, there is very little difference between a VST instrument, and a VST effect. Whatever holds for one, usually holds for the other.
There is a small effect in performance, but it’s unclear which direction the effect will have in each case! On the one hand, 64-bit means more bytes in each buffer, meaning the L1 cache will see higher pressure, meaning large sweeping processes may run slower. On the other hand, 32-bit may mean that plugins that prefer 64-bit, have to convert to 64 bit on the way in, and then back to 32 bit on the way out.
Both of these effects are small, and whether you can actually tell a difference, depends very much on the particular project size and specific plugins/instruments used.
I generally don’t top out my workstation, so I haven’t had a problem with 64-bit.
Except for that one NI instrument that just won’t play right because it uses more CPU than my poor 1950X Threadripper can supply … (Straylight, I think? Or Pharlight?) 64/32 bit doesn’t make a difference, there.