On another hand our OS are 64 bits, so it seems also a good idea to avoid more conversion and to work directly in 64 bits.
You make a confusion here. Your argument is not valid.
A 64 bits OS is not an OS that is using specifically 64 bits processing. 32 bits OS can do that since a long time, and even 8 bits micro controllers can do 64 bits processing if you are using the right math library to do it.
When we are talking about the OS bit depth, we are talking about the memory bus addressing capability, 32 bits or 64 bits.
This is the quantity of memory directly addressable (without extension API). This has nothing to do with processing depth.
Since a long time, and well before 64 bits OS, it was possible to use 64 bits float processing without math library emulation, as the Intel Floating Point Unit was 80 bits internally. So with a 32 bits OS, it was possible to make 64 bits hardware processing at the same speed than in a 64 bits OS.
As a side note, with a 32 bits OS, well before 64 bits OS, (since Intel Pentium Pro and AMD Athlon processors) it was also possible to address more than 32 bits of memory, using a system extension (PAE extension for Windows for example). This is not something well known.
From Microsoft :
Certain 32-bit versions of Windows Server running on x86-based systems can use PAE to access up to 64 GB or 128 GB of physical memory
In fact is it possible to enable this PAE extension even on Windows 7 non server OS.
So it is important to not confuse processing depth, physical memory addressing width and OS memory allocation width, that can be extended anyway with a special OS extension for older 32 bits systems.
If 64 bits OSs did facilitate memory management for DAWs and large sessions, they did almost nothing for processing capabilities. Newer processors and compilers did nevertheless, with newer SIMD and vector execution units (MMX, SSE, AVX…).