Unreliable MIDI program change? (working on organ playback configuration)

Hi,

It is great that Dorico now supports separate MIDI channels for separate voices on a single instrument!!

I am trying to set up some playback configuration for a GrandOrgue organ, which uses a somewhat complex combination of MIDI CC messages and program change messages for various actions including engaging couplers for certain divisions and switching between preset combinations. So, I am defining in Dorico the corresponding playing technique symbols, the actual playing techniques, and an expression map assigning the required MIDI events, where each stave of the 3-staves organ system sends all such messages on its own MIDI channel.

This works in principle, and the MIDI CC messages seem to work fine. However, I have some difficulties with getting program change messages (used for switching preset combinations of a organ divisions) working reliably. Perhaps I am simply missing something.

Attached is a Dorico file with just 2 custom playing techniques, “I: 1” is supposed to send a certain program change message, and “I: 2” should send another. The strange thing is that the playback (confirmed with a MIDI monitor) skips the first playing techniques, but outputs the other. I can swap the playing techniques in the score, regardlessly always only the MIDI message of the 2nd is sent.

Any help is greatly appreciated! Thank you!

Best,
Torsten
Organ_MIDI_program_change_problem_demo.dorico.zip (808 KB)

I exported a MIDI file from your project and converted it to text, and both program changes are present and correct:

0, 0, Header, 1, 2, 480
1, 0, Start_track
1, 0, SMPTE_offset, 32, 0, 0, 0, 0
1, 0, Tempo, 500000
1, 480, Time_signature, 3, 2, 24, 8
1, 480, End_track
2, 0, Start_track
2, 0, Title_t, "Organ"
2, 0, Program_c, 0, 19
2, 1440, Note_on_c, 0, 72, 70
2, 1848, Note_off_c, 0, 72, 0
2, 1910, Program_c, 3, 20
2, 2880, Note_on_c, 0, 74, 72
2, 3288, Note_off_c, 0, 74, 0
2, 4320, Note_on_c, 0, 76, 67
2, 4728, Note_off_c, 0, 76, 0
2, 4728, End_track
0, 0, End_of_file

Both program changes are present, but they seem to be sent to different MIDI channels - 0 for the first one and 3 for the second. Does that explain why the GrandOrgue ignores one of them?

Ah, well spotted, Rob. I think they should both be being sent to the same channel.

Thanks a lot for responding even on a Sunday. I did not expect that, much appreciated!

I tried again, and indeed an exported MIDI file does contain both MIDI program change messages, though on different channels.

However, realtime playback does not. I am using MIDI Monitor (snoize: MIDI Monitor) to better understand what is going on, and the MIDI Monitor only shows the second program change event, even when monitoring all MIDI channels (it also shows very many CC events that are not part of the MIDI file, which seemingly initialise various settings, but that is OK).

PS: I am using version 3.0.10.1051 (Demo)

I can’t think of any reason why the program change would be included in the MIDI file but not in the playback. Can you post the dump of the MIDI data from playback as captured by the MIDI Monitor here?

Sure, please see below. For completeness, I include everything, also the unrelated CC messages sent by Dorico for initialisation.

The relevant lines are the last.

Thanks again!

18:11:24.198	From IAC Bus 2	Control	1	123	0
18:11:24.203	From IAC Bus 2	Control	1	121	0
18:11:24.203	From IAC Bus 2	Control	1	64	0
18:11:24.203	From IAC Bus 2	Control	2	123	0
18:11:24.203	From IAC Bus 2	Control	2	121	0
18:11:24.203	From IAC Bus 2	Control	2	64	0
18:11:24.203	From IAC Bus 2	Control	3	123	0
18:11:24.203	From IAC Bus 2	Control	3	121	0
18:11:24.203	From IAC Bus 2	Control	3	64	0
18:11:24.203	From IAC Bus 2	Control	4	123	0
18:11:24.203	From IAC Bus 2	Control	4	121	0
18:11:24.203	From IAC Bus 2	Control	4	64	0
18:11:24.203	From IAC Bus 2	Control	5	123	0
18:11:24.203	From IAC Bus 2	Control	5	121	0
18:11:24.203	From IAC Bus 2	Control	5	64	0
18:11:24.203	From IAC Bus 2	Control	6	123	0
18:11:24.203	From IAC Bus 2	Control	6	121	0
18:11:24.203	From IAC Bus 2	Control	6	64	0
18:11:24.203	From IAC Bus 2	Control	7	123	0
18:11:24.203	From IAC Bus 2	Control	7	121	0
18:11:24.203	From IAC Bus 2	Control	7	64	0
18:11:24.203	From IAC Bus 2	Control	8	123	0
18:11:24.203	From IAC Bus 2	Control	8	121	0
18:11:24.203	From IAC Bus 2	Control	8	64	0
18:11:24.203	From IAC Bus 2	Control	9	123	0
18:11:24.203	From IAC Bus 2	Control	9	121	0
18:11:24.203	From IAC Bus 2	Control	9	64	0
18:11:24.203	From IAC Bus 2	Control	10	123	0
18:11:24.203	From IAC Bus 2	Control	10	121	0
18:11:24.203	From IAC Bus 2	Control	10	64	0
18:11:24.203	From IAC Bus 2	Control	11	123	0
18:11:24.203	From IAC Bus 2	Control	11	121	0
18:11:24.203	From IAC Bus 2	Control	11	64	0
18:11:24.203	From IAC Bus 2	Control	12	123	0
18:11:24.203	From IAC Bus 2	Control	12	121	0
18:11:24.203	From IAC Bus 2	Control	12	64	0
18:11:24.203	From IAC Bus 2	Control	13	123	0
18:11:24.203	From IAC Bus 2	Control	13	121	0
18:11:24.203	From IAC Bus 2	Control	13	64	0
18:11:24.203	From IAC Bus 2	Control	14	123	0
18:11:24.203	From IAC Bus 2	Control	14	121	0
18:11:24.203	From IAC Bus 2	Control	14	64	0
18:11:24.203	From IAC Bus 2	Control	15	123	0
18:11:24.203	From IAC Bus 2	Control	15	121	0
18:11:24.203	From IAC Bus 2	Control	15	64	0
18:11:24.203	From IAC Bus 2	Control	16	123	0
18:11:24.203	From IAC Bus 2	Control	16	121	0
18:11:24.203	From IAC Bus 2	Control	16	64	0
18:11:24.203	From IAC Bus 2	Control	4	7	100
18:11:24.203	From IAC Bus 2	Control	4	10	64
18:11:24.203	From IAC Bus 2	Control	3	7	100
18:11:24.203	From IAC Bus 2	Control	3	10	64
18:11:24.203	From IAC Bus 2	Control	2	7	100
18:11:24.203	From IAC Bus 2	Control	2	10	64
18:11:25.243	From IAC Bus 2	Control	4	7	100
18:11:25.243	From IAC Bus 2	Control	4	10	64
18:11:25.243	From IAC Bus 2	Control	3	7	100
18:11:25.243	From IAC Bus 2	Control	3	10	64
18:11:25.243	From IAC Bus 2	Control	2	7	100
18:11:25.243	From IAC Bus 2	Control	2	10	64
18:11:26.742	From IAC Bus 2	Note On	4	C5	69
18:11:27.167	From IAC Bus 2	Note Off	4	C5	0
18:11:27.231	From IAC Bus 2	Program	4	21
18:11:28.242	From IAC Bus 2	Note On	4	D5	71
18:11:28.667	From IAC Bus 2	Note Off	4	D5	0
18:11:29.742	From IAC Bus 2	Note On	4	E5	67
18:11:30.167	From IAC Bus 2	Note Off	4	E5	0

If one plays around with the position of the playing techniques in the Dorico file, then in the realtime MIDI output their corresponding MIDI program change events are unpredictably skipped or included. E.g., if the two playing techniques are swapped, the other one is output. I could not yet detect a pattern when the program change is output and when it is not, but the behaviour seems to be always reproducible: the same file with the same playing techniques arrangement results in the same realtime MIDI output.

To exclude MIDI Monitor as the problem here, I now tested it by recording the Dorico realtime MIDI output in a DAW. That shows the same result.

You need to add a ‘Natural’ technique to your expression map. Once you do that, the program changes work as expected.

Add a ‘Natural’ technique to your expression map

Ah, that does the trick! Thanks a lot!

I am still struggling to get Dorico and GrandOrgue working together properly.

Does Dorico allow for having multiple playing techniques being effective in parallel for a single MIDI channel at the same time? The Play mode view shows only a single selected playing technique at any time for a staff (if that is just a question of graphics that is fine of course), but I would need quasi need multiple playing techniques being effective in parallel –– at least I cannot think of a way to avoid that for what I want to do.

I want to freely control both couplings (which stave plays currently which manuals/pedal) and combinations (which manual/pedal plays which organ stop combination). So, for any note that is played, both at least one coupling (I, II, III or Ped – but also any combination of those should be possible) and exactly one combination for each currently used manual/pedal must be selected. Each of these must work for any stave, i.e. any MIDI channel involved. An additional problem is that each coupling message must be sent only once (unfortunately, when a coupling MIDI message is resent, GrandOrgue understand that as to be switched off again).

NB: I know that my setup works on the GrandOrgue side, because I have such a playback setup working fine with Sibelius. However, I would prefer using Dorico for multiple reasons.

In Dorico, I got the relevant messages work in isolation now, but their combination does not work, because Dorico either omits some messages, or sends them more than once. All I want is that each playing technique occurrence in the score should result in exactly one associated MIDI message, but Dorico omits some messages and resends others. Seemingly I am still missing something.

Attached is another example Dorico file. For clarity it only defines and uses coupling playing techniques for one manual (+I means add division I; -I is defined but not even used) and just two combination for that division (I: 1, and I: 2). The natural playing technique is set to the MIDI message for grand cancel (G.C.), which disengages all organ stops and couplings.

The initial G.C. works as intended, it sends it message (#CC 30 127) on all MIDI channels (2, 3, and 4). (It outputs those twice, but that happens to be no problem here). Then +I is engaged on all staves, and it also sends its message (#CC 21 127) on all MIDI channels, but unfortunately it also sends the message on channel 4 immediately again (though not the other channels), which now immediately unsets this coupling on the top stave. The example score then sets I: 2 in the upper stave at the beginning (MIDI program change 22 on the channel of that stave, 4), and I: 1 (MIDI program change 21) a bar later. Dorico unfortunately skips the first program change, while the second is output. When I define a mutual exclusion group for the combinations of division I, i.e. I: 1 and I: 2, then both are output, but then the MIDI messages for the couplings are skipped instead.

Again, does Dorico allow for having multiple playing techniques being effective in parallel for a single MIDI channel at the same time? I would need to be able to have for a single stave all the following playing techniques effective at the same time: an arbitrary combination of couplings, and one combination for each currently engaged division should be allow to be effective simultaneously. For example, for coupling selections in an extreme case +I, +II, +III, +Ped. should be effective all together (but also combinations like +I, -II, -Ped should work, removing the previously engaged 2nd division and pedal for the stave and instead engaging division I), and a combination for each of these divisions, like I: 1, II: 1, III: 1, and Ped: 1. Of course, in the end I only need the relevant MIDI messages to be output, so the question of having multiple playing techniques being effective in parallel might be a red herring.

For completeness, I am not only attaching the relevant Dorico file, but also again its output as captured by MIDI Monitor below.

In summary, how do I ensure that for each notated playing technique its associated MIDI message is sent, and how do I ensure that this MIDI message is only sent once?

Thanks a lot for your help!


The MIDI Monitor output for the attached Dorico file:

16:09:36.399	From IAC Bus 2	Control	1	123	0
16:09:36.399	From IAC Bus 2	Control	1	121	0
16:09:36.399	From IAC Bus 2	Control	1	64	0
16:09:36.399	From IAC Bus 2	Control	2	123	0
16:09:36.412	From IAC Bus 2	Control	2	121	0
16:09:36.412	From IAC Bus 2	Control	2	64	0
16:09:36.412	From IAC Bus 2	Control	3	123	0
16:09:36.412	From IAC Bus 2	Control	3	121	0
16:09:36.412	From IAC Bus 2	Control	3	64	0
16:09:36.412	From IAC Bus 2	Control	4	123	0
16:09:36.412	From IAC Bus 2	Control	4	121	0
16:09:36.412	From IAC Bus 2	Control	4	64	0
16:09:36.412	From IAC Bus 2	Control	5	123	0
16:09:36.412	From IAC Bus 2	Control	5	121	0
16:09:36.412	From IAC Bus 2	Control	5	64	0
16:09:36.412	From IAC Bus 2	Control	6	123	0
16:09:36.412	From IAC Bus 2	Control	6	121	0
16:09:36.412	From IAC Bus 2	Control	6	64	0
16:09:36.412	From IAC Bus 2	Control	7	123	0
16:09:36.412	From IAC Bus 2	Control	7	121	0
16:09:36.412	From IAC Bus 2	Control	7	64	0
16:09:36.412	From IAC Bus 2	Control	8	123	0
16:09:36.412	From IAC Bus 2	Control	8	121	0
16:09:36.412	From IAC Bus 2	Control	8	64	0
16:09:36.412	From IAC Bus 2	Control	9	123	0
16:09:36.412	From IAC Bus 2	Control	9	121	0
16:09:36.412	From IAC Bus 2	Control	9	64	0
16:09:36.412	From IAC Bus 2	Control	10	123	0
16:09:36.412	From IAC Bus 2	Control	10	121	0
16:09:36.412	From IAC Bus 2	Control	10	64	0
16:09:36.412	From IAC Bus 2	Control	11	123	0
16:09:36.412	From IAC Bus 2	Control	11	121	0
16:09:36.412	From IAC Bus 2	Control	11	64	0
16:09:36.412	From IAC Bus 2	Control	12	123	0
16:09:36.412	From IAC Bus 2	Control	12	121	0
16:09:36.412	From IAC Bus 2	Control	12	64	0
16:09:36.412	From IAC Bus 2	Control	13	123	0
16:09:36.412	From IAC Bus 2	Control	13	121	0
16:09:36.412	From IAC Bus 2	Control	13	64	0
16:09:36.412	From IAC Bus 2	Control	14	123	0
16:09:36.412	From IAC Bus 2	Control	14	121	0
16:09:36.412	From IAC Bus 2	Control	14	64	0
16:09:36.412	From IAC Bus 2	Control	15	123	0
16:09:36.412	From IAC Bus 2	Control	15	121	0
16:09:36.412	From IAC Bus 2	Control	15	64	0
16:09:36.412	From IAC Bus 2	Control	16	123	0
16:09:36.412	From IAC Bus 2	Control	16	121	0
16:09:36.412	From IAC Bus 2	Control	16	64	0
16:09:36.412	From IAC Bus 2	Control	4	7	100
16:09:36.412	From IAC Bus 2	Control	4	10	64
16:09:37.442	From IAC Bus 2	Control	4	30	127
16:09:37.442	From IAC Bus 2	Control	3	30	127
16:09:37.442	From IAC Bus 2	Control	2	30	127
16:09:37.442	From IAC Bus 2	Control	4	7	100
16:09:37.442	From IAC Bus 2	Control	4	10	64
16:09:37.442	From IAC Bus 2	Control	4	30	127
16:09:37.442	From IAC Bus 2	Control	3	30	127
16:09:37.442	From IAC Bus 2	Control	2	30	127
16:09:37.681	From IAC Bus 2	Control	4	21	127
16:09:37.681	From IAC Bus 2	Control	3	21	127
16:09:37.681	From IAC Bus 2	Control	2	21	127
16:09:37.931	From IAC Bus 2	Control	4	21	127
16:09:37.942	From IAC Bus 2	Note On	2	C3	77
16:09:38.367	From IAC Bus 2	Note Off	2	C3	0
16:09:38.442	From IAC Bus 2	Note On	3	C4	70
16:09:38.867	From IAC Bus 2	Note Off	3	C4	0
16:09:38.942	From IAC Bus 2	Note On	4	C5	70
16:09:39.367	From IAC Bus 2	Note Off	4	C5	0
16:09:39.431	From IAC Bus 2	Program	4	21
16:09:39.442	From IAC Bus 2	Note On	2	D3	71
16:09:39.867	From IAC Bus 2	Note Off	2	D3	0
16:09:39.942	From IAC Bus 2	Note On	3	D4	63
16:09:40.367	From IAC Bus 2	Note Off	3	D4	0
16:09:40.442	From IAC Bus 2	Note On	4	D5	72
16:09:40.867	From IAC Bus 2	Note Off	4	D5	0
16:09:40.935	From IAC Bus 2	Note On	2	E3	80
16:09:41.366	From IAC Bus 2	Note Off	2	E3	0
16:09:41.435	From IAC Bus 2	Note On	3	E4	69
16:09:41.866	From IAC Bus 2	Note Off	3	E4	0
16:09:41.942	From IAC Bus 2	Note On	4	E5	67
16:09:42.367	From IAC Bus 2	Note Off	4	E5	0

Does Dorico allow for having multiple playing techniques being effective in parallel for a single MIDI channel at the same time?

You need an entry for each combination in the expression map.

Again, thanks a lot for your response, much appreciated!

However, if I may, I doubt that creating an explicit playing technique combination can be the right approach here conceptually. This is not a case like, e.g., louré bowing, which while it is notated by a combination of legato and potato, it is actually a playing technique in its own right (that might be represented by its own sample set in a library, which might be triggered by its own MIDI event).

For the organ, if one would really need to explicitly define every combination of couplings and organ stop combinations as its own Dorico technique combination, this would far exceed 10,000 Dorico technique combination to be defined explicitly (for 3 manuals + pedal and 10 combinations for each). Even if not all these combinations might be necessary in practice, it really cannot be the right approach to define them explicitly.

Also, the different playing techniques that should “run in parallel” might be written at different moments in time in the score (e.g., the allocation of the division before the specification of the stops). The allocation of the division should last until it is explicitly changed in the score to a different division (or division combination), and during that time, various combinations for that division might be called for. Similarity, for a string instrument your might allocate a string to be used (e.g., Sul g), and on that string then use different playing techniques, but the string indication should be observed until that is changed.

So, I do not want to assign a special action to a combination of techniques, I want Dorico to output the assigned action for each individual technique of the technique combinations (separate actions for both the couplings and organ stop combinations). Or, more simply, for every notated playing technique I want to be able to ensure that its allocated MIDI event is output, and it should be output only once for each technique.

Nevertheless, thanks again for your suggestion.

For the time being, you do need to assign every combination that you want to send. There’s otherwise no way of achieving what you are looking for. If you have e.g. a Flutes 4’ technique, and then want a Reeds 4’ technique, Dorico will be looking in the expression map for the combination Flutes 4’ + Reeds 4’. It doesn’t know that these can be freely added or removed independently. If you define them as mutually exclusive in the expression map, then Flutes 4’ will be removed when you add Reeds 4’.

In the future we plan to provide better support for organ registrations. As it happens, it’s something Paul and I have been discussing this past week. However, we have to carefully prioritise what we can work on, and I’m not sure that we will be able to work specifically on organ registration in the next little while: enriching the capabilities of expression maps is a top priority, but organs pose specific and unique challenges that are not common to other instruments, so dedicated implementation for organs will follow the work we need to do to make other orchestral libraries work more easily with Dorico.

For the time being, you do need to assign every combination that you want to send.
Thanks a lot for this confirmation. Again, I appreciate that you are responding at this day and hour!

If you define them as mutually exclusive in the expression map, then Flutes 4’ will be removed when you add Reeds 4’.
I planned to only control in the Dorico setting certain combinations (for which the relevant stop combination then is defined by the organ setup, e.g., combination 1 might be Flutes 4’ + Reeds 4’ of a certain division), because then I would need to define the more fiddly setup for Dorico only once, and can later allocate whatever stops I want for those combinations more easily at the organ in question. In such a setup, combinations for a single division are then mutually exclusive. Also, the added abstraction layer by notating combinations instead of actual stops makes the notated music independent of the available stops of a specific organ. The notation primarily specifies where registering changes happen, and gives only some rough indication at the beginning what each combination could mean in terms of stops.

we have to carefully prioritise what we can work on
Fair enough, I fully understand that!

BTW: An additional complication of being forced to use explicit technique combinations is that such an approach requires that the MIDI event for the division is also always sent, and GrandOrgue has the annoying feature that resending such a MIDI event is interpreted as its opposite: switching a division coupler ON multiple times in sequence is instead interpreted as a sequence of ON and OFF signals (which in other use cases indeed simplifies the MIDI setup, but not here). I have to further invest whether there is any way around that, but so far I could not find any, and until I find such a way around, using explicit technique combinations is not really an option, unfortunately.

enriching the capabilities of expression maps is a top priority, but organs pose specific and unique challenges that are not common to other instruments, so dedicated implementation for organs will follow the work we need to do to make other orchestral libraries work more easily with Dorico.

If you can perhaps add some mode that does not try to be smart about interdependencies between playing techniques and simply outputs exactly once the allocated MIDI events per notated technique and stave/MIDI channel, then that would perhaps not only help me but also other use cases as well.

For example, some orchestra libraries allow to control multiple playing parameters independently. E.g., with LASS you might want to control which articulation to use with one Dorico playing technique, and in parallel control the cross-fading between non-vibrato and vibrato sounds, control the real-time tuning, and the speed of portamentos and glissandos – each with its own Dorico playing technique outputting some MIDI CC event (or in the case of a continuation technique ideally even a sequence of CC events). Similarly, many parameters can/must be controlled in parallel with the Audio Modeling libraries.

In a nutshell, these orchestral libraries require the same flexibility as my organ registering here: either multiple playing techniques (actually, playback parameters) can quasi “run in parallel”, or there exists some mode that simply outputs all the allocated MIDI events for each notated technique and stave/MIDI channel.

Anyway, I noticed that MIDI chasing works already, and that would of course still be necessary with such a “direct MIDI output” mode for techniques, if you would add that :slight_smile:

This kind of parallelism is exactly the kind of thing we are working towards. But we are doing it specifically with use cases of orchestral libraries in mind, so although we anticipate that some of these changes might also help with the management of virtual organ instruments, that is not going to be our main focus for this next chunk of work, and we have specific ideas about how best to support virtual organs that we will be happy to discuss with you and other users using Hauptwerk, GrandOrgue et al when we can specifically prioritise them.

This kind of parallelism is exactly the kind of thing we are working towards. But we are doing it specifically with use cases of orchestral libraries in mind

That is great news! Also, your prioritisation makes perfectly sense considering the number of users interested in orchestral vs organ music. Thanks for sharing!