Hi,
I have experienced some latency issue using the MIDI remote API recently so I made some tests to see if this was related to my setup (script/devices…). I figured I should share my experiments on the forum so maybe someone else could try this out on their side and confirm the issue (or not).
BACKGROUND
First, a little background to explain how I ended up making these experiments.
I made a custom MIDI DAW controller, based on the MIDIbox project. I’ve been using it on a daily basis for 2 years now. It’s been connected to Nuendo using Mackie Control protocol and a Generic Remote. But I’ve recently been trying to switch to MIDI Remote because the integration is better.
I’m trying to write a script for a 25 faders console. One bank of 24 channels and a the 25th is for the selected channel.
As well as faders my console has 4 buttons (mute, solo, record, select), one push-encoder and a small LCD.
On top of that, I plan to have 3 Touchscreens (currently have only one for development) running touchOSC. These screens should allow me to display and control most (all) of a channel strip and inserts.
That’s a lot of parameters: at least 176 planned for each channel… plus a few callbacks (channel name, color, send names, parameters value…) sent as sysex. So MIDI maximum bandwidth is a concern. Especially when banking or nudging. To mitigate this, I use three ports, 8 channels per port.
First Issue
The last few month, I’ve spent some time playing around with the api to find the best and most efficient way of writing a script for my controller.
It wouldn’t have surprised me that updating the complete console would take a fair bit of time. But more surprisingly, it seemed that the script would become really laggy long before I could implement a full channel strip. In fact, in its actual version, I’ve only implemented 1 fader :channel volume, 1 button :channel Mute, and 2 callbacks : channel color, and channel title sent twice (once restricted to 8 characters, and one full name).
When nudging or banking, I felt it would already need quite a bit of time to update all parameters so I made a little touchOSC template to measure the latency between the button press and the first then last midi message received when nudging the bank to the right (increasing the bank by one).
The Signal flow is the following:
TouchOSC (rpi4) → MIDIBox → Nuendo (w10) MIDI Remote → MIDIBox → TouchOSC
Note: The connection between TouchOSC and MIDIBox is a wired MIDI connection. There is a 30 ms back and forth latency between the two that I measured before and subtracted from the values below.
Note: All data below are averaged over 20 measures. The number between parentheses is the standard deviation.
- My own script with 24 channels 2 callbacks:
1st message: 253ms (17.25) last message: 446ms(18.87)
As you can imagine, 446ms to refresh the console is pretty long. the worst is no incoming MIDI message would be processed during this time frame.
So I tried to see if my internal functions called during callback were the culprit of such a long delay.
Therefore instead of processing the data internally, I tried to send an equivalent amount of DATA first as CC, then as Sysex.
I also tried to reduce the number of channel to see how much of a difference it would do.
| First message | Last Message | |
|---|---|---|
| 24 channels Actual Data | 253.55ms(17.25) | 446ms(18.87) |
| 24 channels Eq. CCs | 102.2ms(13.40) | 288.75ms(12.06) |
| 24 channels Eq. Sysex | 158.65ms(6.88) | 465.75ms(9.88) |
| 8 channels Actual Data | 145.5ms(11.92) | 201.45ms(11.39) |
| 8 channels Eq. CCs | 73.45ms(10.97) | 208.5ms(7.34) |
| 8 channels Eq Sysex | 112.35ms(13.87) | 220.1ms(4.69) |
| 8 channels NO callbacks | 103.55ms(14.76) | 159.5ms(7.05) |
So, we can see that there is an improvement, Especially when reducing the number of channels. BUT there seem to be quite a delay before any data is sent back by Nuendo.
So I made more test to compare my own script with other scripts. For this I used Bjoluc’s script for the Qcon Pro G2 tested with 24 16 and 8 channels. And alos the built-in scipt “real world device”.
I also measured the latency using Mackie Control Protocol instead of MIDI remote.
| First message | Last Message | |
|---|---|---|
| bjoluc Qcon 24 channels | 156.15ms(13.59) | 261.35ms(23.6) |
| bjoluc Qcon 16 channels | 133.8ms(13.55) | 233.4ms(17.46) |
| bjoluc Qcon 8 channels | 92.6ms(2.93) | 182.2ms(8.45) |
| real world device (8ch) | 113.95ms(14.35) | 152.3ms(9.29) |
| Mackie Control | 0.75ms(1.48) | 88.3ms(9.1) |
I finally measured the latency for a single button press (used transport Start value):
31.75ms (2.00)
I also made all these measure using TouchOSC on my PC with a virtual MIDI port (loopMIDI)
and the results aren’t much better:
| First message | Last Message | |
|---|---|---|
| 24 channels Actual DATA | 110.75ms(8.28) | 139.2ms(10.11) |
| 24 channels Eq. CCs | 110.05ms(8.91) | 159.35(11.77) |
| 24 channels Eq. Sysex | 106ms(11.27) | 134.75ms(10.87) |
| 8 channels Actual DATA | 75.5ms(10.43) | 91.8ms(9.10) |
| 8 channels Eq. CCs | 81.8ms(10.14) | 103.65ms(10.50) |
| 8 channels Eq. Sysex | 74.75ms(8.43) | 92.25ms(8.8) |
| bjoluc Qcon 24 channels | 165.55ms(12.39) | 176.9ms(10.02) |
| bjoluc Qcon 16 channels | 126.95ms(8.62) | 143.4ms(8.29) |
| bjoluc Qcon 8 channels | 90.35ms(8.79) | 105.55ms(8.44) |
| real world device (8ch) | 90.05ms(8.35) | 91.2ms(8.56) |
| Mackie Control | 28.65ms(7.08) | 33.2ms(6.33) |
We can see that the initial delay ahas been reduced but is even worse in other scenarios but it is always present and never lower than 75ms.
Also the delay between first and a last message has been reduced a lot. I think this is because when using virtual MIDI port, the bandwidth is far better.
My conclusions so far:
- There seems to be quite a latency when using the MIDI Remote for daw control. Especially when the script has to process some callbacks.
- Lowering the amount of channels or callbacks to process reduces the overall latency as we would expect (less data to be sent) but also the initial latency. So it’s possible that MIDI remote waits for some internal processing to complete before dumping the data.
- Given that with a comparable amount of channels we have comparable initial delay between all 3 scripts (bjoluc’s is a little bit better btw), I am pretty confident that my script isn’t the source of the problem here.
- Also, given the very good result that i measured using Mackie Control Protocol, I’m pretty confident that my hardware isn’t the culprit.
I’m aware that sending a big amount of DATA will take some time. Especially using a protocol as old as MIDI, and I’m already working on some ways of dealing with this problem. BUT it’s supposed to be a low latency protocol, something like 30 to 253ms before receiving ANY data doesn’t feel low latency to me.
So, is there anything that can be done to reduce this first latency? In most of my tests, the delay before receiving the first DATA is more than half the overall delay.
Also could someone try to replicate those measurements? I could share my touchOSC template.
It is still not 100% excluded that my system isn’t the culprit. And repeatability is the base of any scientific experiment.
Thanks for reading this very long post.
Thomas
PS: all the test were made using Nuendo 13.0.21 on a W10 PC (AMD Ryzen 3600 3.59Ghz | 32Gb RAM)