I wonder if anybody else has noticed certain anomalies in NotePerformer3…
For those not familiar w/NP3, it’s an ostensibly artificially intelligent VST plug-in that “humanizes” Dorico’s MIDI playback by looking ahead in a score and making decisions about, presumably, articulation, timing, dynamics, micro-dynamics, etc. of each note.
In general, I’m pretty impressed. Real-time playback and audio recordings of my ensemble arrangements, made with Dorico+NP3, do sound a heckuva lot more authentic than do most step-entered MIDI compositions.
However, I recently found that NP3 does not always produce the same output from the same notes. In one case, I compared a Dorico audio export of an entire 15-minute piece with that of a truncated version that included only the last few minutes. I heard clear differences between the two, such as a piano overwhelming other instruments in one version, but integrating properly into the mix in the other. If I hadn’t accidentally found myself comparing the two audio files, I never would have noticed this.
Spurred on, I later found that playing 1-2 minute passages in real time produced different articulations and mixes, depending on where I began playback. This was not nearly as dramatic an effect, but once I had identified it, it was easy to hear each time.
I don’t have any examples handy at the moment, since my current compositions have advanced quite a bit since I started paying attention to all this a few months ago. And I’m now careful to always export audio recordings of an entire piece. But just in general: Has anybody else noticed anything similar in their own projects?
I really hope that someone who knows what he’s talking about will answer this, as I am very interested in Note Performer and would be very thankful to know the answer to this question.
In general, NP is fantastic, almost magical. Few would mistake its output for professional human playing, but even so, I sometimes can’t believe how authentic it does sound. Support sucks, though. Documentation is meager & good luck trying to get the company to respond to user queries. But despite all that – including the issue I raise here – I would kick and scream if somebody tried to take away my copy!
Arne has always been very responsive to my emails.
When I first purchased, I asked 3 or 4 questions, I think via email, and was always greeted with dead silence. This was like a year ago, during the waning stages of the pandemic, so maybe our experences differ due to extrinsic factors.
I have also noticed the differences between one playback and another. Rightly or wrongly, I have understood this to be another aspect of the “human-like” features of NP’s playback—two takes with real musicians will never be identical, either. However, it can be troublesome if we particularly like one interpretation only to find it disappear on the next play through.
…or if you’re trying to create a final audio version for mixdown, mastering, & distribution.
In my example, this 18-minute piece is written for (gulp!) piano/electric bass/percussion, two celli & arco contrabass, bass trombone & tuba, vibes, and church organ. I have no illusions of ever hearing something like this played live, so not being able to rely on Dorico/NP’s audio export being deterministic means I have to be extra careful.
But thanks, Mike, for confirming what I hear. Nice to know that I’m not the only one.
If Dorico humanisation is turned on, I would expect every playback to be different. I have no idea how that might impact the NP algorithms.
Arne replies faster than the speed of light and implements fixes instantaneously. It’s simply unbelievable that we have such a tool at a price that could be 10 times bigger
Dorico’s humanisation is deterministic, so it will be the same on every playback. If I remember correctly the (pseudo-)random numbers used are seeded by internal IDs related to the staff or instrument.
Richard, that’s what I thought. But it’s not what I hear. The differences don’t happen often, but when they do, they’re pretty dramatic.
The only reason I posted teh question here, and not directly to the company is because, as I’ve mentioned – despite some other people’s experience – I’ve never convinced the developers to respond to queries.
If you have a score which plays back differently on different occasions as a result of the humanisation then I’ll be happy to take a look.
Richard, next time I notice this, I’ll be glad to send to you. But for now, to identify the maybe 4 or 5 audio files that exhibited this anomaly – from the hundreds of archived recordings of a multi-hundred-measure composition for 10-piece ensemble (generated daily over the last 11 months) and then zoom in on the offending notes in the offending measures in those anomalous files – well, despite my best intentions, it’s not gonna happen.
My question was a general one, and it seems as though at least one person noticed the same thing I did. I’d ask people to simply pay attention to the issue b/c I’ve found that, once I became aware of the problem, it became pretty obvious. But if you’re not looking for it, & not exporting multiple audio excerpts from the same score, few people would notice. But it is there, and it is real.
I note that @Richard_Lanyon’s response is focussed on Dorico’s humanization and he confirms that this will not change from one playback to another. However, he doesn’t comment on NotePerformer’s humanization algorithms that I believe work independently of those incorporated into Dorico (or Sibelius, or Finale) and distinguish it from other virtual instruments.
Good point, Mike. I believe you’re correct in distinguishing between Dorico & NP functionality. (Actually, I hadn’t been aware that Dorico even had “humanization” features when I started this topic.) Maybe there’s something going on related to some non-obvious interaction between the two. But we’d have to understand the topology of Dorico’s plug-in workflow in order to say anything intelligent about that.
Not to digress, but I work in patent law, specializing in AI and machine-learning. So I have some familiarity with the technology. Although every copy of an artificially intelligent program is capable of producing diffferent results – depending on training – after being fully trained, a particular copy, given a specific input, should always produce the same result. This, of course, assumes that both Dorico and NP3, to the extent that they incorporate any AI-based components, ship fully trained and do not undergo further training during normal usage by end-users. Which seems like a pretty good assumption.
I am confident of changes in playback from one time to another when using NP but it is possible that NP’s playback is deterministic, too. That is, if I am listening to playback multiple times this usually means I am working on fine details in my Dorico file and making small changes between playbacks and (possibly incorrectly) assuming that any corresponding changes in musical “interpretation” will be limited to the small number of notes involved. It will be interesting to pay more attention when playing back files in the future.
A little more info: In my examples, I generated multiple audio files from excerpts of the same composition, editing the piece between exports only to delete material that I did not want to export. The score, other than trimming different measures each time, was identical. I could have eliminated one possible variable – deleting measures – if Dorico did not inexplicably lack the ability to export audio from a user-selectable subset of a flow (you can do this if you export MIDI). Maybe in version 5…
I guess, the meaning of “determinism” here, ironically enough, depends on the boundary conditions you impose on the term. If, for example, NP produces the same audio every time one exports the last 10 measures of a piece; and always produces the same output every time one exports the last 20 measures; yet the last 10 measures in the first case sound different than they do in the second case – is NP3 “deterministic”?
Well, the terminology is less important than the behavior. I can live with NP’s behavior in cases like this, regardless, so long as my expectations have been set correctly. But the meager NP documentation doesn’t go near that topic. Here’s hoping that “Arne” sees this topic and replies with an authoritative comment.
Ironically, since posting my last message, I just once again encountered this same issue. Playing the last three measures of a piece produced a pronounced vibrato from an arco contrabass. But the vibrato disappeared when I began playback 5 measures earlier. This was an easily audible difference and was repeatable during real-time Dorico playback. If I can find the time, I’ll export audio renditions of the two overlapping passages – hoping that the same discrepancy appears in the exported audio files.
Then we really would have something to talk about.
I think that this exact scenario shows what NP does: It takes the stream of input and produces its output based on what it “hears”. How one specefic note will be played totally depends on what notes NP “knows of” before.
In a nutshell, a long note will be played in one way when it is the very first note and in another way if it is at the end after some fast/short/accentuated notes. So if you start playback at this long note, NP treats it as a “first note” because that’s all it “knows”, but if you start one measure earlier including those fast/short/accentuated notes, the long note will be treated differently, producing different output.
So, yes, the spot where you start playback directly influences how each note will sound.
Estigy: Yes, that conforms with my understanding, although the only documentation I’ve found states that the content after a note determines how that note will be played, not the content before. Still, even if your assumptions are correct (and I bet they are!), that’s not good news. Producing unpredictable output – which may or may not match the final exported audio – is a bug, not a feature. And a big one. How many renderings have I forwarded to collaborators that did not sound like what I heard on my machine? This explains some of the inexplicable comments I’ve heard re: a recipient’s need to apply EQ & compression to what sounds to me like perfectly balanced content.
I cringe thinking of the hours I’ve wasted trying to get a large ensemble to fade out selectively, without realizing that changing one instrument’s decrescendo had without warning altered the articulation of other instruments in a different measure. And the fact that this is not spelled out in capital letters in the documentation makes the problem so much worse.
I believe, however, that this issue is fixable. In most scores, the influence of adjacent content is generally local. That is, a note’s articulation, etc., are most likely to be influenced by content that is relatively near in time to that note. Given that, an obvious solution is to extend the lookforward/lookback range such that the influence of notes falling at the extremes of that range become negligable. The sweet spot between performance and consistency chosen by the developer is apparently set too far toward performance. Perhaps the best solution is to make the default range user-selectable. A Ligeti choral work, e.g., would benefit from a larger range than would the Well-Tempered Clavier.
In my use case, where I’m meticulously step-entering large-ensemble pieces in score notation, the audio export is the final piece. If my painstakingly tweaked mixes change in unpredictable ways without warning, as a function of which notes I include in a playback or export, I need to be aware of that & to figure out what can be done to compensate. A kludgey solution for now, assuming that this issue is not addressed by the developer, would be to always export an entire piece and manually perform any excerpting on the exported file in an audio editor. Arg. No idea how to ensure consistent real-time playback, though. Sigh.
I have a feeling that you might have done much more research about NP to find the cause of your problem than I have 
I don’t find NP’s output to be undeterministic - given the same “environment” (like the starting position, maybe the selected players etc.)
I agree with you that describing NP’s documentation as “lacking” would hardly be an understatement, and having at least some insight in what it is doing and why would be a big improvement.
You could draw their attention to this very thread - the team there is very responsive. (I contacted them about some ambiguity on their website, and they changed it the very next day.)