Dorico AI "Performance Mode"

It’s my term, Dorico AI “Performance Mode” (PM). I consulted my crystal ball.

Imagine we have two basic modes of playback, first basic MIDI Based playback and then an AI Perfomance Mode. Essentially this is when AI parses some MIDI input and “conditions it” to sound like the real thing. A Quick example, take a MIDI trumpet. On the real thing the transition between one legato note and the next is conditioned by the continious blowing over a set of notes and the nature of valve transitions. These are ideosyncratic, indeed the term legato means widely different things to each instrument, subject to it’s mechanics. Some instruments cannot do it, a triangle or a saxophone would be examples.

Consider a trumpet “hat” mute, the way the hand moves, musically, to craft notes is not easily replicated by samples. AI can replicate this.
Staccatos are another example. I was surpised to find that there are whole ranges of staccatos, on each instrument, when I try the generic Orchestral Package X samples they often end up not suiting the tempo beibg either too long or too short having too much or too little attack.
Take another example. A guitar playing open chords. In real instruments the notes that ring out and sustain are controlled by teh mechanics of the fingers on the board, often notes are left to ring, sometimes they are deliberately dampened. As one moves up the neck the tonality changes not just velocity.

Now we can buy AI guitars and trumpets, but if Dorico had a performance mode which applied intelligent processes to the way a MIDI track is delivered contouring it for example for a Purcellian non vibrato performance, or a Chet Baker cool drool, this would save so much unecessary work. Instead of composers being engineering keysweitch warriors we can concentrate (more so) on the performance.
More than this, AI could do things like take real notice of markings like sostenuto or dolce or it can give better authenticity to beats and crescendos.
Before anyone points out that this is taking away creativity, I want to point out that a real composer, even if they are on a podium conductiing their own work, hands over control of teh actually delivery to the brains of the performers. I think AI wil take us back to this, though I hope we will retain the ability to make fineer edits.
There is no reason why an AI language model could not be used so that the composer could make remarks like give more accent to the first beats, of an orchestra piece, or “play with more vivace”, or “bring back the brass”.
This is my idea of performance mode. A mode of Dorico which intelligently parses each instrument so that the sound is more authentic than a basic MIDI rendition. A mode where via a language model, we can control other parameters too, of the whole piece at a time.

This is beyond the notion of just AI instruments. It applies to the Whole performance.

Z

Here are some great examples of AI hugely improving over basic MIDI. The software is very primitive, but the direction is undeniable

2 Likes

I’m sorry… what? Since when can a saxophone not perform legato? Hell, even a trombone can do it. If, as you point out later, you even know who Chet Baker is, you should have a working knowledge of these instruments.

NotePerformer already does a decent job at what you’re asking, and all without adding to AI’s growing appetite for power, and water. If we keep up this drive for all things AI, soon it will claim a larger part of those than the rest of humanity combined, driving up the costs of those for everyone. Not to mention how the data centers and server farms are already gobbling up land.

No thanks.

12 Likes

Yes! And – here’s my favorite – it would understand that a “slur” marking for bowed string instruments is actually interpreted by players as a bowing instruction. Quite often these slurs are shorter than an entire phrase that needs to be played legato (connected). String players interpret the music to understand and execute the phrasing. Hello AI!

I don’t think Dorico needs another “mode” for this. This is putting the cart before the horse, so to speak. What is really probably needed is a standardized means of pre-communicating the “score” to the plugin that is handling the AI rendering by extending VST3, essentially looking at VST3 from the perspective of plugins that use AI generation and addressing any pain points where the standard is not currently ideal at supporting the way such plugins.

VSL is currently doing work with AI-generated instruments and they are planning to design their plugin so that you can switch back and forth between sampled and AI-generated passages because sometimes the sampled performance is better at doing certain things and sometimes the AI performance is better at doing other things and this way you can mix and match.

The type of “performance mode” you speak of, asking the AI to interpret different things, would probably be better suited as some kind of independent website where you could upload a score and it could spit out a recording, something like Udio or Suno but where you could upload a score and then guide the AI through the interpretation. This wouldn’t have to be Dorico specific and wouldn’t have to be associated with Steinberg in any way.

“This wouldn’t have to be Dorico specific and wouldn’t have to be associated with Steinberg in any way.” “independent website” So presumably associated with this purported other website? Dorico IMO, should develop the capability of interpeting MIDI in a more natural musical way. Just my opinion.

Z

Of course. But how is this related to your first post?

I’m not sure AI shaping of traditional MIDI is the best approach to begin with compared to a standard rules engine like NotePerformer uses. I would much rather see this first.

AI to me makes more sense when it is actually generating the final result, not just deciding how to drive the MIDI.

Integrating AI into Dorico itself would be a whole big thing because it would have to talk to a cloud infrastructure, which would have to be built, and then there would be privacy concerns and everything surrounding that. Look at the panic over AI features that Microsoft is adding where people think it is going to start spying on them. Don’t you think the same thing would happen with something like Dorico if they added AI into the software?

That’s why it makes more sense as I said to just facilitate standards for AI plugins to be able to be created, and then Dorico and other software can leverage those to allow people to install these plugins that can do AI rendering if they choose to.

1 Like

So, I stand corrected about the term for sax. Trouble with the term legato “smoothly connected” is that it is different for different instrument mechanics. For wind instruments it is a continuous breath (and valve or key work), for bowed instruments. To a piano for example it is entirely different to a trombone. Sometimes it is a matter of semantics. IMO.

FYI I played, in chronological order: trombone, trumpet, cornet, flute, guitars tenor sax, piano, Hammond and Rhodes.

1 Like