Brainstorm - How could Dorico use AI?

AI has been a leading topic in the past few months and has developed/improved exponentialy in the past years. Artificial Intelligence models are being trained to do all sorts of things, which makes me think how could a notation software such as Dorico take advantage of it.

I am not talking about things that make musicians redundant - like making the AI write the music for you, or orchestrate a passage for you, or figure out a progression for a melody automatically (although that last one could be helpful I guess, even for a composer that’s stuck).

I am talking about things that are boring, time consuming and nobody wants to do. Like transcribing audio, or making a score out of a mockup, or checking parts one by one to make sure there are no weird stuff in them, or scanning a handwritten sketch and turning into something that’s workable, (unless you enjoy any of those… weirdo.) - heck, even training the AI to recognize handwriting on the fly (a great feature for those on tablets or any hardware that supports a pen).

But I’m a composer primarily, so my idea of what AI could do (or what it SHOULDN’T do) is limited. I am curious to see what other people’s ideas are, or if this whole AI thing is a bad idea in general and should never touch Dorico.

My first reaction is that, as useful as many of those functions are or may be, Dorico is primarily a notation program (with a well developed playback capability), and most of those functions would likely best be supplied by auxiliary products rather than built into the Dorico software.

Now if only we could find a programming team as capable and devoted to support as the Dorico Team for each of those add-ons, we would shortly be in business. But I am not sure another such team exists or at least has been recognized as such.

1 Like

In my (humble) opinion (I work with AI), present Dorico is largely an AI program. This following the general definition of AI. Think at the “human” decisions Dorico should continuously take.

It’s not compulsory the use of neural networks or of big data to have an AI program. And the complexity of an AI algorithm may vary from a small computer to the biggest one on this earth…

This said, obviously everything can become better!

My poor single ç…

1 Like

It seems the goal of Dorico is better and more sophisticated algorithms (“If X conditions, then do Y”). Whether that’s AI, I don’t know.

1 Like

Out of curiosity, I asked ChatGPT to give me an example of music said to represent a sunset.

There are many classical music compositions that are inspired by the sun or depict sunrises, but not many that specifically indicate a sunset. One possible example is Claude Debussy’s Prélude à l’après-midi d’un faune (Prelude to the Afternoon of a Faun), which is based on a poem by Stéphane Mallarmé. The poem describes the sensual dreams of a faun in a forest at dusk, as he plays his flute and tries to seduce some nymphs. Debussy’s music captures the languid and dreamy atmosphere of the poem, using impressionistic harmonies and melodies. You can listen to it here: Claude Debussy - Prelude to the Afternoon of a Faun - YouTube

Another possible example is Joseph Haydn’s Symphony No. 45 in F-sharp minor (Farewell Symphony), which has a famous ending where the musicians gradually leave the stage one by one, until only two violins remain. The symphony was composed in 1772, when Haydn and his orchestra were staying at the summer palace of Prince Nikolaus Esterházy in Hungary. The musicians were longing to return to their families in Vienna, but the prince kept delaying their departure. Haydn decided to express their discontent through music, by making the final movement a symbolic farewell. The prince got the message and allowed them to leave the next day. The symphony can be interpreted as a musical representation of the sun setting and the night falling, as the music becomes quieter and darker. You can listen to it here: - YouTube

1 Like

Ai could be useful to do more complex tasks… It could review your entire body of work, probably all of it in 10 minutes and give you a detailed review concerning any parameter you wanted.

I’m sure it could be set up to eventually do a whole bunch of routine tasks strung together far beyond a few simple commands. But you would definitely want an undo button for each step.

That said, as Chatgpt was trained on weighted word choice not notes or composition. Much better if someone assembled a massive data base of compositions and trained it on that.

Not sure I would want it either, but on the other hand even using the tools that create pictures simply on word prompts without needing to know how to draw, still leaves open for creativity. You can still strive to create something you or others appreciate more than average. Not everyone is going to get the best results from what they do.

What is it that you use it for?

Worked in AI for 30 years professionally FWIW. IMO using modern DNN tech you see in the news doesn’t have a good application in Dorico. Whatever flavor you’re using, all these approaches use enormous amounts of data, literally you almost can’t have enough data (overfitting is another problem). And you need data for training and test sets.

Anyhow the problem here is the copyright laws in music. Whereas humanity is quite happy to upload exabytes of unlicensed visual and textual data every day to the internet, the same is not true for high quality music, for obvious reasons (it’s easy to take a picture of your dog, and hard to write a song for it). Basically, I doubt there’s enough data to make it worth the while, especially considering the market size (musicians = tiny). And you’d have to vet each sample for copyright. Simple example, imagine what would happen to your company if a little Beatles snuck in there? Or something close to the Beatles.

Music copyright is absolutely draconian, and may well save musician jobs and pay (such as it is) from the A.I. meatgrinder. It’s possible there’s enough CC0 high quality labeled training data out there but I kind of doubt it.

5 Likes

IBM Research & Services…

Sibelius has recently announced an “AI” feature, in which it can suggest likely chords within a harmonic context:

What next – figured bass realisation?

“Machine Learning” is, very simply, extrapolation based on large data sets. So as far as composition or harmonic construction goes, it’s going to be a pastiche of common forms.

It’s possible that you could feed AI a large set of high-quality engraving from the ‘golden age’ of hand engraving, and then use that to fix all the slurs and beams in any given score.

But there comes a point when the data that AI draws upon for its reference was itself created using AI, creating some kind of stagnation or weird feedback loop…

4 Likes

As you probably know, the definition of AI is extremely vague and debatable… And in many cases the result is often AS (Artificial Stupidity)…

5 Likes

Good thought but that’s incorrect, actually backwards. The big problem with ML is that it doesn’t extrapolate. Show a toddler a dog and it understands all dogs, and by extrapolating very quickly cats and goats. Show a ML system a billion pictures of dogs and it will detect dogs (not cats or goats) in pictures - most of the time.

Reminds me of my son the first time he saw the ocean down in Big Sur, he looked out the window and suddenly exclaimed “Wa-waa!” (water). It’s probably the domain of GAI (General AI) to be able to do that.

No historically AI is when we figure out a new technique to do something human-like, like the famous A* search of classic AI. But then it shortly becomes not AI as it’s incorporated into general CS (like that A*)

And in many cases the result is often AS (Artificial Stupidity)…

Not much of that anymore, when NN’s get it wrong it’s usually not that wrong. I’ve done detailed looks at my NN failures and most of the time they actually make a kind of sense, but they’re wrong simply because a human wouldn’t make that mistake.

Kind of like when AlphaGo made that famous “god move”. I was watching it live, the 8-Dan (or whatever level he was) host was amused and kept putting it on the ‘correct’ move - the one a human would do. But it won the game and now is revered by go fanatics (I used to be one because it seemed like an impossible problem to solve, back in the pre NN days).

2 Likes

Absolutely riveting documentary, and free on YouTube.

3 Likes

It’s been an exciting 30 years. I spent so much time thinking about how we possibly could teach a computer to recognize patterns, it just seemed impossible. My mistake was giving up on NN’s, what I didn’t see is the explosive growth of gaming, of all things, that gave us cheap, massive SIMD capability (Nvidia basically). All the modern DNN stuff isn’t that much further advanced from what was done in the ‘60’s, it’s all just making use of cheap hardware and cheap data.

But yeah that match was a thrill.

I might could enjoy selecting 8 or so bars of a composition I already composed, and have ai suggest several alternate compositional changes in a shadowy overlay.

(People are still trying to figure out good uses for VR, perhaps a I need a VR workspace that combines composition with a workout. I hurl notes onto a score like a beat saber }

If I remember correctly, AI was mentioned in early Dorico marketing. I don’t think any details were ever given.
I haven’t done a thorough search but there’s a mention in Deceptively Simple: How Dorico Does Things Differently.

1 Like

You do remember well, but those were elements brought by the marketing department, which made the Team quite uncomfortable (as well as other elements like gold standard…)
Dorico has the best automatic output, no need for anything else. Bells and whistles belong to the stage :wink:

1 Like

I’m guessing that some work will be done on this (perhaps building on Spleeter, Demucs and the like – already used in SpectraLayers).
The extraction of metre and tempo would be useful for linking notation to pre-existing audio.

Reorchestration/ transposition?

For the CoMA (Contemporary Music for All) score library you are required to create parts for open score, but actually that means creating parts in Bb, F, possibly Eb and hoping they still lie within range, or else doing some tiresome manual edits. How much could AI help to work out, from the huge corpus of out-of-copyright classical music, how composers ‘doubled’ lines in other specific instruments while respecting their range and technical capabilities? For example, don’t use tremolo - that a string instrument might be playing - on a french horn, and ration the use of flutter-tongue on a wind instrument. Or, as with Brahms’ viola transcription of his clarinet/piano sonatas, consider introducing double stopping that only the string instrument can play, or ‘divisi’. These strategies are really ‘features’ of a kind that deep neural networks have been learning for years. And this is a case where copyright concerns won’t come into play.

It seems to me that would fit well with one of Dorico’s USP’s, that is making the creation of large scores, with possible expansion from a piano score, easier than the competition.

A further step might be ‘instrumentation on demand’ - take the basic 8 instrument score and generate alternative or additional instrumental parts. This could save so much work when flexible ensembles gain or lose members.

Dorico 5 has been most helpful in helping me learn the craft of scoring. In fact, because I find it more intuitive than Notion, Sibelius, MuseScore, etc., it’s become my go to arranger. Sometimes, I even feel like I’m cheating with Dorico!

Another great help to me is tonica fugata. Now, that really feels like cheating, given it creates figured bass for you, as well as other such devices. I can see Steinberg incorporating more of that in the future to garner new users.