I would like to suggest the integration of an AI voice-cloning feature into Nuendo, specifically for audiobook and long-form narration workflows.
The idea is to create a voice model from an existing narrator’s recording and allow users to generate replacement lines simply by entering corrected text. This would eliminate the need to arrange additional recording sessions for minor corrections and pickups.
The generated audio should retain the original narrator’s vocal characteristics, accent and delivery, allowing it to blend naturally with the existing recording.
This could be an extremely useful addition to Nuendo’s audiobook post-production capabilities.
Similar functionality is already available in DaVinci Resolve, where AI voice generation and voice-related tools can be used in post-production. It would be wonderful to see Steinberg bring a similar capability into Nuendo, particularly with audiobook production and dialogue editing in mind.
I don’t agree with this. Might seem backwards, or “old thinking”, but I don’t think we should be cloning people’s voices effortlessly, whatever the benefit or convenience to us. I do NOT want ADR sessions to disappear, nor do you want audiobook recording sessions to disappear.
And where does the artists click, in Nuendo, to give his or her consent to this?
This. Please, no. And you would also (legally) have to flag the content as AI. Which could lead to exploitation issues in some territories. A recently-released spectral editing app has this advertised as a capability but - IMO - (1) it’s a toy, (2) it doesn’t really work properly and (3) it prints a mandatory AI flag to the output.
This is a huge legal quagmire - we deal with this in games. You need to get permission from the voice actor for starters and there’s massive room for abuse.
I get that this would help in those situations where you need to make a few last minute adjustments and can’t get the actor back into the studio in time (we often deal with this) but it’s a way bigger ask of Steinberg than you might think.
This train already left the station. Over a year ago.
If the budget is there, and if the talent is there, then ADR or re-recording in a studio for audiobooks and so on will continue. Not a problem. Cloning won’t change that.
However, I’m already receiving cloned voices in my AAFs from TV networks. They just clone the talent (or “talent”) when needed. It’s the editors or producers doing it. They’ve used 11Labs for a while and no doubt they’ll use Resolve for it as well as more move to use it for editing.
From my perspective as a re-recording engineer (not on scripted features) the question is simply if I move to Resolve to get the work done or if I do less good work or if I tell the client to do the cloning themselves - because the fact is they won’t spend money to do it the old fashioned way.
That’s the reality we’re facing. They simply won’t spend the money in a lot of cases. Either I do the job to the best of my ability (using cloning) or they’ll find a different engineer that does the job tothe best of their ability (using cloning).
Remember when drum machines were going to replace drummers? Ah…those were the days. The train has indeed left the station (and is heading toward a brick wall). People will demand these features. Big can of worms.
Some replies are not as airtight as they present itself; the OP’s request is for corrections & pickups on a finished recording the CREATOR ALREADY HAVE, not for replacing narrators or eliminating ADR; that’s a personal economic interest dressed-up as a universal moral principle. Those are different things.
The consent question is fair & the OP’s request is not dodging it; the feature would require explicit narrator authorization, full stop. But that’s a design requirement, not a reason the feature shouldn’t exist. Every DAW has licensing workflows for plugins, samples, stems. A consent gate is the same category of problem.
On the “legal quagmire” point, a bit dramatic but I agree it’s a bigger engineering & legal lift than “add a button.” The OP is not asking Steinberg to ship it next month. The OP is flagging that the use case is real, that it already exists in other tools (badly, in some cases), & that the market is going there regardless. If Steinberg doesn’t build it properly, someone else will, & Nuendo loses a workflow it should own.
The “exploitation in some territories” concern is real but it’s a compliance checkbox, not a reason to refuse the feature. The EU, US, thingy ect. requires transparency. Fine. The feature ships with metadata. That’s NOT a quagmire, that’s an implementation detail.
The existing spectral editing app being a toy doesn’t argue against the feature. It argues for Steinberg doing it RIGHT.
It seems like the OP is not asking for a voice actor replacement engine. The OP is asking - if the narrator flubbed one line on page 87 of a 14-hour book, can I fix it without booking a studio & flying them back? That’s the whole ask.
The point about networks already sending cloned voices in AAFs is the part that stands out to me. The choice framed by the re-recording engineer sounds less dramatic than the legal debate makes it seem: adapt or lose the work, since clients often won’t pay for the old process. That doesn’t erase the consent and AI-flagging concerns raised earlier, and I’d say those still matter. But if producers are already doing the cloning upstream, the studio discussion shifts a bit. Out of curiosity, are the cloned takes you receive usually labeled clearly, or does it vary?
Producers doing it doesn’t make it “more legal” (though I’ll admit that it is not illegal, until copyright infringement laws get in the way).
I worked on a big movie where the editor had, for screening convenience purposes, Elevenlabbed the voice of a AAA list US actor (SAG). That is, in fact, illegal and goes against the terms of the actors contract.
You know, some things are just UNDERSTOOD; producer, ect. intent doesn’t override a SAG-AFTRA contract, & “screening convenience” doesn’t make it less of a violation. That’s a fact, not a value / MORAL judgment.
The feature DOES NOT need to make it legal. It needs to make the USER(s) RESPONSIBLE FOR VERIFYING LEGALITY, which is EXACTLY HOW EVERY OTHER PART OF NUENDO WORKS.
Steinberg DOES NOT verify that your sample pack is licensed. It DOES NOT verify that your stem was legally recorded. It DOES NOT verify that the composer you’re mixing has actually assigned you the rights. The USER IS THE LEGAL ACTOR. Steinberg… ONLY… provides the tool.
A PROPER IMPLEMENTATION would open with a clear, non-dismissible disclosure - “You are responsible for confirming that the narrator’s contract permits AI-generated replacements. Use of this feature DOES NOT CONSTITUTE CONSENT on behalf of the narrator.”
That’s not a quagmire. That’s a ToU checkbox. Same category as “I confirm I have the rights to all audio in this project.”
The question isn’t “should this exist?” It’s “should it exist with a user-facing legal attestation, or not at all?” The answer to that is clearly the former.
Since the train already has left the station and there are literally dozens of apps and platforms that service this market (in all ways described): i’d suggest Steinberg to not go in this direction. It is a waste of energy and resources since they are simply too late. Focus on workflow, bugfixes, stability and open up for LLM / extensions / mcp to give users agentic control over Nuendo. Not to do illegal stuff, but to enhance and expand the featureset. So that an elevenlab-extension can be used or a sandboxes/harnessed local vocal cloner.
No, not always clearly labeled. Sometimes I can figure out by listening and then seeing that the filename is simply different. Either way, it’s not always clear.
From a practical standpoint, there’s also little I can do about it. I’m not the one that’s liable legally since I didn’t do the cloning, and I certainly wouldn’t flag it for a network every time I run into it because that would just mean a bunch more work (headaches) for them meaning next time they’ll just use someone else for the job.
You’re right of course. There are really three perspectives here; the producers / owners perspective, the talent, and then everyone working doing the actual editing and mixing. I fully agree that the talent’s rights should never be infringed upon, ever.
I think that in a lot of cases the talent will be 100% ok with this for several reasons: they don’t want to do more work on the project, they’re unable to do the work for scheduling reasons, they’re already paid and aren’t getting paid more for the work… and probably more reasons.
I think a lot of people these days get paid a fee to finish a task for a job, meaning that for example on-camera talent may have been paid to be on location to do work, then for one session of narration, and then they’re done. Production might then simply stipulate in their contract that they have the right to clone the voice for minor changes and that might be to the benefit of the talent who wouldn’t make any more money anyway.
It’s also worth pointing out that when I say “talent” I’m not just talking about actors or voice actors, I’m also talking about a sports documentary that follows a team and the coach flubs a line on-location and the odds of getting them to ADR is literally zero%. In fact, this is where a very large amount of cloning would be used I bet.
To the extent that we can have the cake and eat it too that’s what I’d prefer. If I don’t need to roundtrip to get the job done I won’t. It’s obviously better.
But sure, I agree that what you mention would likely be better “value” on R&D investment.
I understand everyone has a different workflow and schedules/timeframes to deliver stuff. But I think the Pro Tools Extensions are a good example of staying in the ‘daw’ and still process with other tools. Smart move by Avid, ARA on steroids. So SB needs to catch up…