Zero latency seems to me only necessary for performers monitoring during recording.
Today this is correctly achieved using hardware interfaces that can bypass the workstation during recording to reduce the monitoring latency to a few ms or less. Today most if not all serious digital audio interfaces can do that.
In the other end a digital audio workstation alone cannot reliably deliver a small enough latency for recording. Even without inserted plugins, as soon as you try to reduce hardware I/O buffers to a very small amount to simulate zero latency, you start to have drop outs.
Those interfaces are using fast ADC / DAC and internal digital switching / mixing. Most even have basic EQ / reverb fast internal processing that can be eventually remote controlled from the DAW so that the performers monitoring can be processed for more musician comfort.
Another solution is to use a direct analog path between microphones / instruments and the headphones using a traditional analog mixing desk. Typically this is the case for recording studios when they still have large analog desks.
For mixing zero latency is generally not necessary. Latency can be compensated (to a limit particularly if mixing for live events). This latency compensation mechanism is implemented inside all modern DAW and digital audio mixing desks.
Do you have an idea about the amount of added latency using Cuda or OpenCL in a setup with a consequent amount of tracks / plugins and an optimized thread / buffer management ?
If this value stay inside reasonable limits, and because of the large amount of processing power available inside GPUs, there is no reason to not use this technology for audio processing and or VSTi. Eventually enhanced for audio use by Steinberg, like they did for ASIO drivers where they created a new standard followed by most hardware manufacturers, GPU processing could help to give a more reliable and more powerful DAW.
For video, the amount of available processing power is impressive and does allow to manage multiple full HD video streams in real time with processing enabled on each stream on a Intel PC. So we can imagine that for audio, this could translate to a phenomenal power because audio tracks are nothing compared to full HD video streams bandwidth.