Offline AI Performance Rendering – a possible direction for Dorico

Dorico
1

Hello Dorico team and colleagues,

The recent progress in AI-assisted musical performance has been remarkable. Applications like ACE Studio demonstrate that expressive score rendering into a convincing musical performance is no longer a futuristic concept.

However, one limitation still stands out to me.

ACE Studio currently relies on cloud processing and requires an internet connection. While this is perfectly acceptable for many users, a significant number of composers, orchestrators and professional studios still prefer—or even require—an entirely offline workflow for reasons such as reliability, privacy, long-term accessibility and project archiving.

This made me think about a different possibility.

Instead of another playback engine, what if the future lies in a dedicated Offline AI Performance Renderer—one that works seamlessly inside Dorico while also being available as a VST plugin for Cubase, Nuendo and other DAWs?

The goal wouldn’t be to replace NotePerformer or existing playback solutions, but to introduce a new generation of expressive rendering based on locally running AI models.

Such a renderer could potentially offer:

  • fully offline rendering;
  • multiple interpretation models (classical, cinematic, jazz, ethnic, contemporary, etc.);
  • controllable musical phrasing instead of fixed playback rules;
  • deterministic, reproducible results;
  • compatibility with existing VST instrument libraries rather than requiring a proprietary ecosystem;
  • smooth integration into professional scoring workflows, including VE Pro.

I also wonder whether this could be an opportunity for collaboration rather than developing everything from scratch.

Wallander Instruments has decades of experience in expressive playback through NotePerformer. Cantai has already demonstrated AI-powered vocal rendering integrated with Dorico. ACE Studio has shown just how convincing AI performance rendering can become. If a collaboration with ACE Studio were ever considered, I think the most valuable outcome would be the development of a fully offline rendering solution suitable for professional production environments.

I believe this would be most successful as an open ecosystem rather than a closed one, allowing different interpretation models and future technologies to evolve over time while remaining compatible with existing VST instruments and professional workflows.

Dorico has already become one of the world’s finest notation applications.

Perhaps the next major step is not only writing beautiful scores, but also rendering them into genuinely expressive performances—with the same level of musical intelligence that Dorico already brings to notation.

Best regards,
Thurisaz

2

Interesting idea.

3

Arne (Wallander) brought up this scenario as what he sees as the future when the last version of NP dropped.

My assumption is the various library manufacturers have already started working on this. Or at least the smart ones are. VSL has already integrated AI into VE Pro. This is why I haven’t even looked at any libraries for the past couple years. They may be obsolete in a year or two.

4

Yes, this is the future of the Virtual Instruments, no doubt!
Would be great if we could have an AI Generative plugin that could be used in Dorico and Cubase/Nuendo.
ACE Studio 2.0 sounds amazing! Far more decent than any VI library. And the Lifetime price for the Artist Pro version is very affordable.
Just we need a plugin that will give us certain amount of control, in order to shape the sound to our preferences. That way we will avoid the risk of having equal sound to any piece of music that come out.
The AI driven libraries could give us, as composers, to act more as conductors, than as sound engineers. Actually this what I would expect as a result of potential collaboration between Steinberg and Wallander, Cantai, or Ace Studio.

5

Asking apps like Dorico (and associated VSTs) to incorporate AI to function locally (as opposed to online) makes me wonder whether my next home computer (or laptop) will have to be upgraded to a “supercomputer!” I wonder how much RAM will be needed, how many cores or simultaneous Silicon processors, and whether a 500 petabyte drive will be enough! Maybe I should get my home retrofitted with solar and a turbine, too. (How big should my backup generator be — I don’t think my current APC UPS will be sufficient!) :scream::rofl::wink:

Yet, all things considered, I would rather do it locally than rely on online services.

— Jim

6

Hi @jaskarbong,

Most probably you’ll need far less computing resources than any traditional hi-end VI. :upside_down_face:
I would give you Cantai as an example. It supports offline rendering and it’s very lightweight.
For Synthesizer V 2 you don’t need a supercomputer, too.

7

I’d be okay with this idea if it used only local resources, including any NPU or GPU you have on-board in your system. AIs biggest (outside the obvious copyright issues) problem as a tool right now is the amount and kind of resources it uses, especially water.

8

“Convincing”

Yup and if I put on a police uniform I can “convince” people I’m a police officer.
But that’s a crime. If I put on a suit and tie I can walk into a bank and pretend I’m an investor and pull some kind of fraud. But that’s a crime.

But apparently publishing fake music to the internet; stealing the space and reward away from real music, is perfectly fine now. No generation in human history would have accepted these falsehoods, except for this current one.

We need to bring back publishing standards. “Democratizing” artistry has only flooded the market with mediocrity and garbage, from people who are unqualified from producing the things they are producing. And they are unqualified because they don’t put in the work. They don’t put in the work because they only love the reward of the art; they don’t love the art.

Society never used to have these quality issues. Because they kept the land clean with the standards. Without the standard the goblins are free to trample on everything green and growing and so a beautiful world, led by merit, is converted into an industrial profit-fueled hell hole, led by who is most proficient at theft and falsehood.

9

IIRC Ace Studios products do not write music; they provide another approach to combining lyrics with sung tones along the lines of Synth V or Cantai, just using a different method.

10

Doctors should just create a medicine that cures all known diseases.

I’ll leave it to them to sort out the details.

11

First, let me say that I agree with your proposal – let’s keep our computer processing local. :grinning_face_with_smiling_eyes:

In my previous post, I was being a little bit facetious – well a lot really – but my point was that we keep asking apps to do more, then we grumble about how hard it is for an older computer to keep up with the needed resources. I have Cantai (and Synth V). My Dell desktop, which I made sure had plenty of RAM & storage, struggles to render a 4 voice composition (as demonstrated by the fan coming on and roaring)! It’s not enough any more and it’s not that old. My M1 Mac Mini, in contrast, doesn’t even blink, but I’m afraid I’ll have to significantly upgrade on the next go ‘round. I’m old enough to remember when 640k was a ton of RAM (I upgraded from 128k by pulling the chips — not memory boards — off the motherboard and replacing them) and having a 10 MB hard drive was more than I would ever need (or so I thought). In fact, it was more than enough storage and resources for me to design a desktop system to replace a mini mainframe system for the financial company I was working for at the time.

So my point remains; An application’s requirements will expand to use the resources available to it. While the marketplace will encourage us to buy “more.” At some point, the Return-on-Investment of our computing power will diminish and we will choose to “do without” rather than keep up. I expect that we are not there yet and may not be for quite a while.

Until we get to that point, I will agree that we need to have “smarter” offline code. Whether that means it is AI, fuzzy logic, or better algorithms is irrelevant to me. I just don’t want to spend all my retirement dollars trying to keep up with the latest technological advances (and the electric bill to run them :joy:)!

— Jim

12

In games, the plane has flattened significantly. You no longer need regular computer upgrades to play the latest AAA games. And with the massive price hikes on hardware driven first by crypto farming and now AI centres, there is a lot less talk about upgrading in gaming forums, most people thinking that it’s better not to do it while a few years back the trend was to chase the latest and greatest.

13

I guess I’m a product of the early days… I seem to remember playing a flight simulator game on an old MS-DOS 8088 unit. (Thinking about this now, the game was either some kind of wire frame rendering, or possibly on an early, underpowered 80386 with original windows!) It was so slow that, due to buffering, the plane would “crash” before it even rendered moving down the runway. :rofl:

But let’s set this aside as it is getting farther from the OP topic. :wink: