Depends what your final product is to be. If you are working, as you say, in 44/32, and you want to make a CD, you MUST make 44/16 files. You can do this without dithering, but it is not wise.
If you are doing work for a client and they want 44/24, same thing. You can do this without dithering, but it is not wise.