The existence of this thread is about an issue the has existed since march of 2017. That this is still an issue is telling. It’s a VST3 issue. The company that invented the format hasn’t demonstrated a working host to support that format.
Yes there are workarounds. I’ve been dealing with this for all this time and continue to forge ahead because I understand it. But it frustrates me daily. Those who don’t understand the problem and its cause just have problems and don’t now what to do about it. That’s not a workable situation.
FWIW, I just put VEP 7 in Dorico and Bidule as a monitor in VEP, then used the NotePerformer expression map (which sends a bunch of CCs per note) They arrive with the same timestamp, in the order specified in the expression map:
You’re monitoring the input stream to VEP right? This is not where the problem lies.
VST3 devices (in this case VEP) can process multiple MIDI streams in parallel. Events that occur at the same timestamp can get processed at the same time by different streams. So, if it matters what the order is, even though they come at the same timestamp, there is a potential race condition to compute whether or not a keyswitch, for instance, occurs before a note-on event or vice versa. If it does, then you get the expected result. If it does not, then you get an invalid result and only subsequent notes will act on that keyswitch.
A simple example is if you’re playing Arco on strings, and you get two events at the same timestamp that say switch to pizz. on this note. If the keyswitch processes first then it works as expected and you get a pizz. note playing back. If the note processes first then you get another Arco note followed by a keyswitch for subsequent notes which will be pizz.
You can’t monitor this by looking at the input stream (except to observe that the timestamps are the same). You have to listen to the result and see if it is what you expel to occur in all instances. Sometimes it will work as expected and sometimes it won’t. It is unpredictable because you don’t know what else is going on inside the plugin with those streams.
The only way to resolve this so it works consistently is to timestamp events that need to process first with a lower time value than subsequent events. If it doesn’t matter what order the events are processed, then it is fine to use the same timestamp.
All of this is not an issue for VST2 devices because they only can process a single MIDI stream at a time. So events are always processed in the order received regardless of the timestamp.
No, I’m monitoring the input stream sent from VEP to a plugin within it (Bidule).
There is so much vague with what is being said here I’m afraid it is just going to confuse people. This is a theoretical argument based strongly on a plugin interpreting same timestamps as “parallel”. But no plugin host (e.g. VEP/Bidule et al) should ever do that, as that is a policy that can only be determined by the end recipient plugin. Plugin hosts should be FIFO plumbing and that’s been my experience with VEP/Bidule/PatchWorks.
There is no such thing as “parallel” with plugin communication. A buffer of events is sent, a buffer of audio is returned. No two events can be in the same “place” in that memory buffer, they form a list even when given the same timestamp. Thus a simple host policy is: when on the same timestamp, preserve (FIFO) order. A potential for problems exists in VST2 as well - a host/plugin could reorder events, store things with the same timestamp in a non-linear way etc. Such hosts/plugins are broken.
No sane sound-producing recipient plugin that uses keyswitches/CCs for articulation control change should process them “in parallel” or out of order (shame on you Kontakt). VI Pro and VEP plugins certainly retain order. I have less experience with Synchron player but I have seen no problems, and I highly doubt it rearranges order as its multi-dimensional system is even more dependent on control order than VI Pro.
What my test demonstrates is the order that events will be received by any plugin hosted in VEP7. Thus the Dorico->VEP7->sane-plugin path should work. That’s what I’ve found. If things go awry after that, blame the end plugin. If someone has problems with e.g. Kontakt in VEP, the problem lies with Kontakt, not VEP (and is easily demonstrated using Kontakt’s built in monitor, even in VST2).
I’m no fan of VST3 (and its disdain for MIDI standards) either, but we should avoid FUD.
I ran a test and showed the results. It is anecdotal, but not an opinion. Others can reproduce my results and make informed decisions for themselves. NI has acknowledged Kontakt (VST2) reorders CCs (always sending them after notes on the same tick), but has not fixed it. Again, anyone can reproduce this with Kontakt’s own MIDI monitor.
If you have data that shows VEP7 reordering things when run as VST3, please share it. I’m sure VSL would consider it a major bug, as then none of their own plugins would work well in their own host.
@Rob and @Rich: You want hard data. Here is some hard data. Please read this thread, note the date on this thread, think about what I said there and here. And keep in mind that when I reported this to Pianoteq developer and described what I thought was happening, he thanked me for being so thorough in my reporting of the problem, confirmed that that is exactly what was happening, that the problem was with multiple midi streams and timestamps that did not order the events properly.
In this particular case, The developer at Pianoteq decided to fix the problem on his end as he just introduced a new version of the products and didn’t want to wait months for the host to get fixed.
@Rich, there are many cases that do work. There are some that don’t. I’ve written on this topic many times. Feel free to search for my name and read all abut it. In every case where there was a problem and it was fixed, it was adjusting the time stamp to fix the ordering of events that fixed the problem.
You don’t know what you are talking about, sorry. “reordering midi for a given time stamp is fair game” is not true. Neither is “Midi events are ordered by a time stamp. So multiple cores can operate on events with the same time stamp in parallel. The only way to guarantee the proper order of midi execution is to time stamp it appropriately.” Also, multiple MIDI in/outs don’t target the same sound generating sources, they are there so that tools like Halion/VEP can have more than 16 channels, each target being on a single port+channel.
There were many more credible hypotheses in that thread, including: Pianoteq just had a bug.
Your assertion of “So yes pianoteq is reordering those events. This is correct behavior for a midi device.” is completely wrong. MIDI does not have timestamps. No two MIDI messages can ever occur “at the same time” because it is a serial protocol, defined by a series of bytes. DAWs add timestamps to indicate an intended time for realization, because the data is sent in chunks/buffers and not processed in real time, so the target VI needs to know at what offset in the result buffer to generate audio. But the messages still have order.
Now, your example was of all-notes-off and note-on/offs, and one could argue that Dorico should never send that (nonsense) message stream of conflicting note-on and all-notes-off in the same tick. But this kind of thing can always happen when ppq resolution collapses things and Pianoteq should be graceful/sensible about it. We have been talking about control change (CC) messages, which should always be interpreted in order and are never contradicting note ons.
There is no evidence that this is VST3 parallelism voodoo.
I’ve been writing MIDI and audio software since the 80’s, so I 'll stick with my “opinion”.
When you start using VST3 devices and developing your own expression maps and they don’t work, please report the failure mechanism and how you worked around it so we can all learn from you.
Just to follow up on the discussion yesterday, I found this post to the Juce forum which I believe (based on the content and the username of the poster) is the developer for the issue discussed in the thread I referred.
Make of this what you will, but there was clearly a reordering of events that came into the plugin with the same timestamp.
I read that when you first wrote it, which is not really surprising since I use Pianoteq.
The amount of fluff in that post doesn’t come anywhere close to “hard data” IMO, beyond the fact that “there was a bug in Pianoteq and Modartt fixed it” which tells us nothing interesting.
Just for a start, you might consider that if Pianoteq is screwing up its processing of MIDI events, why should you believe that its own log of MIDI events is going to be guaranteed error-free? Except by some sort of magic of course, but personally I don’t believe computers work using magic.
The problem is this: VST3 does not fully and correctly support MIDI. Full stop. It has its own ideas about events and parameters, and thinks it is smarter than decades of industry standards and is entitled to replace them with its own ideas. I took a look at both the JUCE code and the VST3 SDK. Here’s what’s happening:
VST3 does not directly support MIDI CCs as events. It only supports note on/off, sysex, and non-MIDI stuff they made up, like note expression. This is implemented in the (processData) buffer as the inputEvents list. Note this is a list, in order, and each event has a sample offset and ppq timestamp. Events at the same timestamp retain order.
Parameter changes are delivered (in the same processData buffer) in a separate inputParameterChanges list (also in order, with timestamps). Parameters at the same timestamp retain order.
Steinberg expects all plugin developers to stop using CCs and instead use parameters, and the entire virtual MIDI-based plugin ecosystem is supposed to go through gyrations doing a mapping of one to the other (there are tons of threads about problems with this). Besides the hubris behind this disregard for existing practice (has Steinberg never seen any of the various MIDI processing scripting plugins?), there is a more insidious problem. In splitting the MIDI event stream the VST3 system is no longer capable of expressing the relative order of MIDI events - VST3 breaks the serial semantics of MIDI. I wonder if the guys who did this are even as old as MIDI?
The net result is that a MIDI stream of noteA ccB noteC ccD becomes two lists: noteA noteC (events) and ccB ccD (as “parameter changes”). A kludge inside JUCE, similar to the kludge in every MIDI-based plugin converted to VST3, has to read the parameter changes, convert them back into CC events and insert them back into the event list. But where? It no longer has the information that the order is ABCD - VST3 has lost it. This is not a parallelism or concurrency thing, it is just a bad API design. So plugins have to decide when doing this kludge to either put the params-to-ccs (on same tick) either before or after the notes. All JUCE did for Pianoteq was put them before instead of after.
What does this mean for us poor souls who have to make expression maps with this? You can expect CCs on same tick to retain order, and notes as well, but not an intermixing of CCs and notes. JUCE now puts notes last. I tested this hypothesis with my Dorico->VEP->Bidule (inside VEP) stack, and setup an expression map that interleaved CCs, PCs, and keyswitch notes. Sure enough, VEP7 also puts notes last (I’m not sure if they use JUCE).
Synchron player and VI Pro would not be bothered by this, as they remember CCs. They very much care that keyswitches retain order because not all of their KS articulations/matrices have the same number of KS types/dimensions. If the type/dimension KS comes first it will break.
So, again there’s no parallelism/concurrency issue, nor is it a complete random free-for-all on the same tick. MIDI event stream splitting is just a bad API design of VST3 that is incapable of conveying a MIDI event stream intact to a plugin. For a company with the MIDI heritage of Steinberg (I was using Pro 24 in 1987) this is really sad. The sky is not falling, but it is raining. VST3 caveat emptor.
I hope this helps people understand and move forward.
If you Dorico devs ever get the ear of the VST3 team please plead with them to flow MIDI CCs (and all legacy MIDI) through the event system. I’m not opposed to additive progress (e.g. note expression) but there’s no reason to burn these bridges to existing practice.
The Dorico playback team is likely to face no end of grief forcing everyone through VST3 only to hit the substantial ecosystem of existing orchestral plugins that rely quite heavily on the semantics of virtual MIDI and the correct propagation of keyswitches, CCs and program changes.
Thanks for this discussion. Fascinating for even the untutored like me.
I remember the VST3 controversy. To me it was always preferable because it broke the 16 channel limit per instance. I was ecstatic when VE Pro started supporting a VST3-compatible version.
But also I read the howlings from users of VST2 plugins. Now I know why many did not follow Steinberg’s path.
Very interesting discussion guys. If its true that VST3 has removed the possibility to include CC’s, PitchBend, aftertouch and program change message in the same midi buffer where notes are found, then this is definitely a problem for any upstream host or mid program that is presuming it can intermix the various midi messages in an expected serial order and have expected results. That is a severe oversight by Steinberg if true.
It sounds like NoteExpressions could probably be used to ensure CC, PB, AT and PC messages happen in front of each note, but that only works if you have a VST3 midi signal chain from start to finish. Cubase is probably the only VST3 host or plugin that is doing ANYTHING with NoteExpressions at this time. Maybe someday that would become more commonplace, but as of now, I’d say no.
I think its possible JUCE could consider doing that though. A JUCE host should convert a serial stream of intermixed events into NoteExpressions when handing to a VST3 plugin. But other non-JUCE hosts, would all need to know to do that too. This is quite troubling actually…
Bottom line, if you need to care about intermixing CC, PB, AT or PC messages between notes of a poly-articulation chord…you should not do it with VST3. If you stagger the notes of the chord slightly, I guess it would work fine, but that’s kind of a hassle.
