I need default expression maps such that. when I update the maps, any playback template that uses the expression maps automatically gets the latest version.
I can place expression maps in the DefaultLibraryAdditions folder (Dorico 5) and this appears to give me the expression maps in both new projects and existing projects.
In other words, without my addition to DefaultLibraryAdditions, the expression maps don’t appear and with the added file, they all appear (I have 3). This only works when for an existing project if it doesn’t already use a playback template that relies on the expression maps.
If I bring up the expression maps dialog and press Reset to Library Defaults, the expression maps are unchanged—they appear to be the defaults.
Next, I apply the playback template that supposedly uses those expression maps. After I do that, 2 or my 3 expression maps lose a lot of their switches. I can import my expression maps back in, but now the Reset to Library Defaults again eliminates the switches.
It’s as though the playback template references some older, different expression maps that have the same name as the updated versions. If I export the Playback Template, the file is a binary file, not an XML file, and so I can’t check its contents.
I even tried loading my correct expression maps and then creating a new playback template from scratch. Once again, the same 2 expression maps were reset back to a basic version (i.e. the same as the Default expression map). The other expression map picked up some changes I just made to it.
Does anyone have a clue as to what is going on or what I could try?
Yes, your playback template does indeed include your custom expression maps, and apparently it contains older, less complete versions. When you apply your playback template, the required expression maps are added to your project.
Really I think you should remove your expression maps from the DefaultLibraryAdditions folder and only have them in your playback template. That matches the way Dorico is designed to work: expression maps don’t need to be in the project’s library until and unless the playback template that requires them is applied.
What I suggest you do is open the project you use to manage your playback template (i.e. the one from which you originally exported the necessary endpoint configurations) and import the latest versions of the expression maps into this project. Re-save the endpoint configurations that use these expression maps, and that should do it (provided you also remove the expression maps from the DefaultLibraryAdditions folder).
If you then set the default playback template in Preferences to be your custom one, whenever you start a new project, this template will be applied and the required expression maps will be added to the project’s library.
I found the problem. I’ve never used some of the endpoints. I have multiple endpoints with inconsistent expression maps. Each endpoint stores a copy of the expression map it uses. The expression map IDs are the same, but the settings are not.
So I deleted all the endpoints in EndpointConfigs and re-created one of them:
I loaded my expression maps.
I added a player with an appropriate instrument.
I went to VST and MIDI and set up the endpoint and saved it.
I created a playback template and added the endpoint.
My playback template now just has a single instrument, but it seems to work properly.
The expression maps are duplicated within each endpoint. I suspect that the last one loaded with a particular id wins.
I’m using instruments from Audio Modeling. There is just one expression map for all the woodwind instruments, one for all the string instruments, and one for all the brass instruments. Each instrument is a separate VST and it’s unlikely I would use all of them.
To fix the problem, I have to have a project with one of every possible instrument. If I ever make a change to an expression map, I have to save every endpoint. If I fail to save one, and it is the last one loaded, then I get an old expression map or a default one (if I never saved the endpoint for that instrument).
I’m not using every Audio Modeling instrument, but there are a lot of them. You shouldn’t be duplicating expression maps—they should be in a central place and referred to by the endpoint configuration file, rather than included in it.
Right now, it looks to me like whoever designed this wasn’t thinking anyone would ever have the same expression map used in more than one endpoint.
I’m trying to see if I can create a single endpoint with all the Audio Modeling instruments that use one expression map. I’m not having much luck and I’ll have to get back to it later.
I tried to create one endpoint with all my instruments, but it looks as though endpoints are per VST.
This forced me to create a project with 20 instruments and then to load my expression maps and laboriously create the endpoints. With the endpoints created, I could now re-create my playback template.
A quick inspection of the files shows that the each endpoint’s playbacktemplatedeps.doricolib reproduces the entire associated expression map.
If, in the future, I make a change to one of the expression maps but fail to update all of the endpoints, what gets loaded would be anyone’s guess.
I’m thing I’m not sure of is whether, if two expression maps have the same name, the system uses two maps or one, even if it only displays one in the Expression Maps dialog. In that case, of course, one map would be hidden.
By the way, after I do all this work, I try actually playing. I know I’ve gotten it to work. but after hitting play, Dorico locks up. I have to kill it. The system has sent diagnostics back to Steinberg. I can hear sounds in Write mode, but it hangs if I try to play.
The problem could be in my expression maps, since I wasn’t really using them before.
One more to close the loop: It looks like one of my expression maps was sending a CC to an Audio Modeling instrument on Init that was causing it to hang up—the hang was in the VST, not in Dorico.
I’ve looked on the web for “how to report a bug in Dorico” and I see you’ve suggested that the best place to report a bug in right here in the forum.
I see that I didn’t specifically use the word “bug” in referring to the issue I discovered, so I’d like to officially state that I consider the problem a bug in Dorico, and I would like a response as to whether the Dorico team agrees or disagrees. I just want to make sure this isn’t consider just an oddball problem that I’ve resolved and needs no further action.
Having two endpoints using the same expression map ID to refer to different expression map settings looks like an error to me and it’s not an easy problem to track down. To solve the problem, I would up having the inspect various XML files, which no user should have to do.
The Audio Modeling woodwinds consists of quite a few VSTs, but really only requires only one expression map. One might not use all instruments in any given piece, so an update to the expression map and a save of the endpoint configuration can lead to problems when the project is later loaded or when a different set of instruments is used in another project.
There is no easy workaround. Creating and maintaining a separate expression map for each woodwind instrument is worse than just saving the endpoint configuration of every instrument. Both are a pain, but the former is a bigger pain and more error-prone.
The case where one might want to use the same expression map for multiple VSTs is not that rare. Consider the Default and CC11 Dynamics expression maps—tweaking either would result in chaos…
The only real workaround for me would be to write a program that inspects all the endpoints looking for duplicated expression map ids, sorting them by modification date, and updating all endpoints to use whatever is latest. After any session where I modify expression maps and save the endpoint configurations, I would need to remember to run the program.
For that matter, I can’t just modify the expression maps. If I don’t save the changes in at least one endpoint configuration, I could lose all my work… It seems like a crazy system to me.
We can argue the toss about the definition of the word “bug”, of course, but from our point of view we consider something to be a bug when it doesn’t work as we have designed it to work. The fact that we may have designed something to work in a way that seems unhelpful to one or more users doesn’t make that design or its implementation a bug, but at that point it’s all just semantics anyway. The point is, it’s unhelpful for the same expression map to be included in a playback template multiple times and for it to potentially have changes in each one.
We’ll certainly think about this issue and consider whether and how we might change how this works in future.
In the meantime, I believe it should be possible to work with the way Dorico works today without too much hassle. You can save a single endpoint configuration for multiple selected instruments via the VST Instruments section of the VST and MIDI panel, which will only save a single copy of the expression map to the score library file containing the dependencies in the endpoint configurations.
Unless I really misunderstood you, your solution doesn’t work as well as you believe. I have three expression maps, one for woodwinds, one for brass, and one for strings. I save all of these in one endpoint configuration and then I rework my playback template to use that one endpoint. Correct?
Later, I am working on a piece for a woodwind quintet. I find a problem in the woodwind expression map, correct it, and save it as you suggest. I have now wiped out my strings expression map from the endpoint configuration as well as the VST setups for all but the five instruments in the woodwind quintet.
The process you suggest only works if I only update expression maps from a file artificially created to hold one of each instrument type.
As I said, this design is a bit of a minefield. It’s hard to understand and easy to screw things up. I suspect you’ve avoided problems because most people don’t bother trying to create expression maps and those who do probably avoid using the same expression map for multiple VSTs.
My solution would be to not include an expression map in an endpoint configuration except as a reference to a single definition stored elsewhere. I’d be curious to know if there are cases where that creates a bigger problem than the one I’ve run into.
Provided you go back to the original project from which you originally created your playback template, where you have VST instruments loaded with each instrument to which each expression map applies (so if your woodwind expression map applies to, say, flute, piccolo, clarinet, oboe, and bassoon, you have one of each instrument loaded in the project, and likewise, if your brass expression map applies to trumpet, trombone, horn, and tuba, you have one of each instrument in the project, etc.), and make the change to the expression map there, then re-save the endpoint configuration from there, that should do the trick.
The expression map is for all woodwinds. The chances I would have all of them in a single project, even in an orchestral score, is really small.
But I am trying to convince you it’s worse than that. If I write for a wind quintet, I now introduce a French Horn, which falls into the brass family. So I need a file with all the woodwinds and all the brass. Actually, I might as well throw in all the strings, while I’m at it.
So I’m playing around with a piece and I realize the expression map has a problem. To fix it, I would most naturally fix it in a score in which I can test the fix, probably wherever score I noticed the problem in. Then I have to export the fixed expression map, open the artificial project, load the saved expression map, and save the single endpoint with all the instruments.
Yes, it works. That’s about all that can be said for it. Claiming this is “not too much hassle” is a bit of hyperbole.
I can create a similar problem with just one instrument. Some VSTs have settings that are not accessible through CC switches. For the Audio Modeling instruments, I can have a warm flute or a bright flute. I can create endpoints labeled “Warm Flute” and “Bright Flute”. I might have some projects with one and some with the other. and some where I mix the two for a richer sound. I had better only save the endpoint configurations from projects where I have both. Start adding more instruments and more instrument endpoint variations, and the only project where you can safely save stuff will be a very artificial one.
Admittedly, the problem only strikes the people developing expression maps and playback templates and only those who do so where one expression map is shared among more than one endpoint. That group of users is very small, which doesn’t make the architectural flaw any less a flaw, although I could understand it would affect your prioritizing any fix.
One more example. I know my issues with the system are low priority, so I thought I’d see if I could suggest that the problem should have a higher priority.
Audio Modeling creates the expression maps, playback template, and endpoint configurations. I’m not sure how these things are shipped, so I’m guessing it would come as a ZIP file with instructions for how to install or load.
A user notes that the shipped flute’s timbre is “flat”. They change it to “warm” and save it as a new endpoint configuration.
Audio Modeling discovers some errors in their playback system, so they package up everything and send it out as v1.1.
The user dutifully installs it all as instructed, overwriting all the previously shipped expression maps, playback templates, and endpoint configurations with the fixed versions.
The endpoint the user created is not updated since Audio Modeling knows nothing about it.
In the project with the “warm” flute, this endpoint happens to be the last to be loaded. As the last loaded, it overwrites the updated expression map for all woodwinds with the old version.
As an aside, this example exposed a hole in my knowledge: I realize the user can save a “warm” flute endpoint, but is there a way they can assign an endpoint to an instrument? I’m thinking that if I defined “warm” flute and a “bright” flute, I might want to use these in another project without having to re-do the VST and save the endpoint each time.
I could create duplicate playback templates and edit them, but this wouldn’t allow me to have both “warm” and “bright” flutes. The endpoint combines an instrument, a VST setting, and an expression map. If I have the same instrument, same expression map, but different VST settings in an endpoint, is there a way of re-using that? After all, the VST settings are sitting the in EndpointConfigs folder by name, but I don’t see any way to access them without going through the playback template, which wouldn’t know which flutes I want to use which VST settings.
Audio Modeling once told me that they planned to create a Dorico playback template, but nothing has happened that I know of.
The scenario I wrote is not specific to Audio Modeling. It can apply in any case where a VST has some parameters that are useful to alter, but that cannot be altered using a MIDI CC. The user could then create and save endpoints with varying versions of the VST and suddenly one has multiple endpoints using the same expression map. I noted one reasonable action (an update to an existing playback template package) which could create endpoints that now use the same expression map ID to represent different expression maps and could lead to problems.
At every point in my scenario everything the user and the playback template creator is doing seems totally reasonable, but could result in problems that can be hard to figure out (as happened to me).
At this point, I think Dan appreciates that the current design might have its problems, but probably believes that very few people will ever be affected. Most people don’t fool around with playback templates and expression maps, so he would probably be right. I was trying to come up with a scenario that might affect people who aren’t messing with the playback templates and expression maps, so as to increase the chances I’ll see a fix in my lifetime.
Hi.
You are stumbling upon an impossible problem : every library is different and it’s a challenge to build an “expression map” system working for everyone of them. What Dorico team has done covers already a lot of ground, but there are still uncovered possibilities (like keyswitches applied during the note, or with offsets, etc)
The fact that you build some files to create your own endpoints is certainly the only way to control your own expression map and the way a custom playback template would work. It’s not an easy path (using NotePerformer is) but it’s working. Having one file for your woodwinds from library X, another for ypur brass in library Y is the proper way. Yes, you are right, if the editor brings an update on their playback template, it could cause some work to re-do, so make sure you document the choices you make in that file (using staff text or comments).
In any case, the only editor that provides us with workings expression maps, playback templates, etc is VSL.
All the other stuff found here is the work from @John_Barron_2 or users like me. Someone has released an Expression map builder (for Mac IIRC), but I haven’t got a chance to try it and see how useful that is.
When you are editing your expression maps, Antonio, are you remembering to increment the version number in the Expression Map Data section at the top of the dialog?
When you open an existing project, Dorico checks the version numbers of the expression maps in the project’s library against the factory and user libraries, and if it finds a later version in the library, it will update the version in the project with the later one from the factory or user library.
At the moment, however, we don’t do this when applying a playback template, and perhaps we could. You could keep your “master” expression maps in a score library file in DefaultLibraryAdditions, so these expression maps are automatically available in every project. When you apply your playback template, we could decide not to import the expression map if the one already found in the project has a higher version number.
No, I’m not. I thought the version number was simply an annotation. I’m a software engineer, and I generally update a version number when a release is made, not for every small change I might make.
That’s interesting. It implies I can put expression maps into the user library. There is no “Save as default” in the Expression Maps dialog. How would I do that?
Ok, so it sounds like I have to manually place them into DefaultLibraryAdditions. It might be nice to add this feature to the UI.
I’m not sure of all the ramifications, but it still seems simpler to save expression maps by name into one XML file and then have the endpoints only include the ID of the expression map and not the settings. The version number check seems more like patching the problem rather than fixing it, But maybe there’s some value to having the expression map settings saved with the endpoints that I’m unaware of.
A playback template is meant to be shareable, so the intention is that everything required for a playback template gets bundled up into a single archive that can be easily installed on another machine. As such, we determined that it should contain all of the required information, including the expression maps, within itself.
Yes, I agree that with the vast number of VSTs out there, coming up with a system that can handle them all is like trying to herd cats. But that is not the issue I’m talking about.
I’m reporting an architectural “bug” in that Dorico allows endpoints to use the same expression map ID to define the same settings, but uses only the settings of the last loaded endpoint with a given expression map ID.
Got it. That’s fine, but what I propose wouldn’t affect that.
I assume that when you share a playback template, you share the playback template definition plus all the endpoint configurations. Why not also include the expression maps, but as a separate item? Again, the endpoints would only include the expression map by reference. The expression maps would be defined elsewhere and no more than once.
This is just a structural change to what is shared when you share a playback template.