Lua Scripting for music generation

Dorico
#1

Hello,
I’m a Sibelius user, running the 30-day eval of Dorico to see if I should switch. Loving everything so far!

I work in automatic composition using AI, so one important thing for me is to be able to add plugins to help me compose. I’m able to do this (with much effort!) in MuseScore, for example.

Is there a way in the current version of Dorico to run arbitrary code (e.g., to use Lua to interact with some non-Lua external processes) and then make edits to the score automatically? For example, having a Lua script communicating with an external process or sending/receiving from a local webserver would be useful. If not, maybe this could be added to the backlog of feature requests?

An ideal workflow would be something like this

  1. User selects a region of music in Write mode
  2. User triggers the Lua script
  3. Optional: a dialog box is triggered and the user changes some script-specific settings
  4. Script reads the staves/measures/notes in the selected region
  5. Script sends the data to another running process or makes a POST request to a local webserver
  6. Script waits for data received from the external process/webserver
  7. Script makes edits to the selected measures based on the received data

I’d love to know which (if any) of this is currently supported by the scripting engine.

Thanks!

#2

Since you mentioned specifically “edits”, you’ll probably find the scripting capabilities quite limited, so that might be dead in the water. As far as I could gather from poking around, you cannot yet inspect notes properties. But I would love — love — for me to be proven wrong. You can, however, run some basic Lua code already, and you even have an interpreter. But this interpreter is probably also a self-contained distribution, and I don’t know if you would be able to access libraries, for example.

I would also love to know a bit about your setup, if you’re willing to share. In the meantime, why not simply convert to symbolic notation in the code?

#3

You can indeed not access properties or otherwise read much information about the music. But it is possible to input music generated externally into Dorico using scripting. A gentleman in Dresden is doing a very interesting project that makes use of the Lua API to automate some very complex polytempo note input with material that he is generating externally.

We do have plans to enrich Dorico’s scripting API, but it’s a big job and we want to be sure we can devote the necessary time to it in order to come up with a clean, well-structured API. This is something we will hopefully be able to work on in the relatively near future, but we need a longer product cycle to make headway in this area.

#4

Would that gentleman wish to remain anonymous? Was he at the conferene in Salzburg? I’m coming hot from a multi-tempo score that I had to do in Finale because the composer so wished and because I already had the parts inputted. As painful as that was, at times, I definitely would like to try something of the sort in Dorico.

#5

He can sometimes be found here on the forum but I don’t want to out him! I’ll drop him a line and ask him whether he wishes to share anything about his methods here. He well may not, though.

#6

Hi, a gentleman from Dresden here.

There is not much that I can add that hasn’t been said above already. What I can say from experience is that if one makes the effort, it is possible to do things on the entry side that I wouldn’t have thought to be feasible at all when I first looked into scripting about three years ago. Today I have my own custom-built text encoding language that I use to automatically write out whole complex scores from scratch. While I have been doing that for a while now, at times I still watch the process (which happens live, as if someone entered the content manually, at an insane speed) and catch myself being amazed by what can be done.

But it is also true that there is currently no way to get any feedback from Dorico itself. This means that – if one wants to use scripting for sophisticated scenarios – there usually will the need to build some kind of state log, since your script and Dorico will have to fly blind, but still in perfect sync with each other.

I don’t think that my work (at this point) would be much easier if I had the possibility to get information feedback from Dorico, but it would certainly open up some interesting possibilities that, even after quite a lot of poking around, remain closed off for good until there is more work done on the API. That being said, I certainly understand the team’s reluctance to introduce any short term changes. As with each feature, they prefer to do things thoroughly instead by fix-on-demand, and I think this approach has been proven to be wise again and again by now.

With Daniel using the term “relatively near future” above, in all honesty I would not recommend to anyone to start developing anything like the roughly – oh my, I just counted – 30.000 lines of Lua code (not including notation content encodings) that I churned out over the last years. But if someone has a project that might be helped by the tools that I have – admittedly vaguely – alluded to, especially with regards to multi-tempo scenarios, feel free to contact me in my capacity as a for-hire engraver/consultant.

#7

Ha — not the least bit surprised it’s you!

It seems I correctly grasped the limitations, and that there isn’t anything escaping me. Whatever there is to be gained comes from entry, I suppose. I had a similar instinct: I actually started building a kind of parser for pseudo-LilyPond input, but that was the work of a couple of afternoons…

#8

Thanks everyone for the detailed replies. I’ll be on the lookout for an expanded API feature set in the future, but I know it’s a big job. LSalgueiro: to answer your question, I’m interested in interactive composition with AI tools. For example, I published an algorithm that automatically generates piano fingerings for a score; would be nice to be able to hook that in. Similarly, I’ve worked on AI systems to offer suggestions or to rewrite a music segment. So ideally this would all be integrated into a notation system, but that would require analyzing the existing score.

#9

Where is the documentation on the existing interface for scripting entry into Dorico? I don’t mean the work that you have done with it. I mean the semantics of what must be produced in order for Dorico to load it?

#10

There is no documentation, since scripting is officially still unsupported. Pretty much the only way to learn is to record macros and to analyse those. Overall it’s not that hard, though. Making command string generation dynamic will take more effort, obviously.