Some thoughts on real-time MIDI recording

Forgive the cheesy intro. Bear with me.

Last night I had a dream that I was improvising on the piano. It was certainly a dream, because I was playing at a level that could only happen in a dream. :smiley: But what was so striking was that, even as I was dreaming, I was thinking, ā€œI wish there was a way to capture this.ā€ By that, I meant ā€œon paper.ā€ Not a new thought, Iā€™m sure.

To me, here are the things that make real-time MIDI unusable at present:

  1. Smart Split-point. A ā€œsmartā€ split point would be incredibly, phenomenally helpful: one that interpreted music played with two hands and made decisions about which hand was playing what. Current split point functionality is basically never correct. This is the least crucial because it can be fixed afterwards with moderate ease, but itā€™s still a big time commitment.

  2. Latency. Even a 30 ms latency makes it incredibly difficult to record (and current latency for me is much higher). I would dearly love the ability to hear the click but mute the VST, and instead use the local keyboard audio, which would have no latency.

  3. Legato notation. The recording should fill rests by extending the notes to make them legato, except perhaps when the playing is distinctly staccato. Far easier to add rests later, which I assume is a much less common scenario (at least it is for me). 99% of the time, I want the notes to be notated legato.

  4. Voice management. This is a tough one to parse what the performer wants, I imagine, but here again, 99% of the time I want one voice per hand. At present, Dorico (understandably) fills my recording with ties that have to be fixed individually (and Iā€™m unable to edit the duration of a single note within a chord).

  5. Editing. Dorico is already far superior in this regard, but a couple things would help. Particularly the ability to edit the duration of one note in a chord! Also, the ability to quantize (post-recording) a selection while in Write mode.

EDIT: I thought of another one, similar to voices. An option to limit the use of tiesā€¦ if a chord or a note is being held and a new note is detected, the previous note is stopped. In Studio One I think this is called ā€œavoid collisions.ā€ It essentially removes the overlap of voices in the piano roll.

I didnā€™t include flexible tempo on the list, which is a whole ā€˜nother consideration. The list above all presupposes the player is striking the notes exactly with the click, but how many of us can do that consistently? Thereā€™s other software out there (albeit unusable in other ways) that listens to a performance and adapts to it, even making evaluations of anacrusis. The possibilities are exciting.

Forgive the rehashing, as Iā€™m sure thereā€™s nothing in here that Paul and the team havenā€™t thought of. I just want to reiterate how stunningly game-changing this functionality would be. The ability for Dorico to more smartly encode music would set it apart even further. This may be blowing smoke, but itā€™s no exaggeration to say that I would literally pay 10x the cost for these features, as the amount of time and effort they would save me would be incalculable.

The computing power is doubtless available, but Iā€™m sure the time required to code such algorithms is daunting. Still, I do hope this will be a front-burner consideration for future versions. It would be revolutionary. Thanks guys.

1 Like
  1. I would think the easiest way to do the Smart Split-point would be to apply a ā€œRe-splitā€ on a different note to the current selection. An ā€˜intelligentā€™ algorithm to guess which hand is doing what would be amazing, but probably impossible.

  2. As for latency: have you got a big or small audio buffer? I found that a large buffer creates big latency, and you can reduce latency by using a small value.

  3. Dorico does have a setting to Fill Rests in the Quantisation settings, though Iā€™ve had some trouble with the results.

  4. Can you give an example of not being able to edit the length one note in a chord? I can do this.

  5. You can already re-quantize a selection after the recording.

  1. Sure, but I might have to do this literally every other measure. And if the split point is incorrect, the voice management will have been re-distributed as well, requiring double the edits.

  2. I understand, but the easiest and most fool-proof way to work around latency is to be able to mute the input. Also, different VSTā€™s will add to the latency.

  3. Iā€™m not in front of a computer right now, but in the past, if there are any tie chains involved, it makes it impossible. Of course, smarter voice management might eliminate most tie chains anyways.

  4. We might be talking about different things. What I mean is quantizing in the same way a DAW would snap notes to a rhythmic grid. Iā€™m not aware of this functionality at present.

It might be quicker, then, to play each hand separately! :wink:

Edit > Requantise lets you change the quantisation values for a selection. So, for example, if I have a piece that uses sixteenth notes very rarely, I can quantise to eighth notes, and then requantise the bars with the sixteenths afterwards.

I do appreciate the input, but Iā€™m afraid thatā€™s entirely missing the point.

I think real-time Midi recording/transcription comes in two flavoursā€¦ 1) quick note entry alternative to step-time and 2) capturing live keyboard performances. Iā€™m not sure Dorico has (or should haveā€¦) ambitions regarding point 2.

The pinnacle was reached over 20 years ago with MOTUā€™s FreeStyle sequencer. Long discontinued/abandoned unfortunately. But even today software exists (windows only afaik) that feature floating split point, per hand voice analysis, beat alignment editing AND very good MusicXML export. It even gets most things right even on a dated laptop without ASIO. Had Dorico had ambitions in this direction Iā€™m sure they would have studied these role models carefully before coming up with their own solution. :question:

  1. Smart split point works almost flawlessly in MainStage, so there is an algorithm that guesses right.

DP and Logic allow you to play rubato, then specific the bars, beats or even subdivisions. I prefer to play to a click, but it works well.

ScoreCloud parses real-time rubato pretty well too.

This would be amazing in Dorico!

  1. I donā€™t want to have to use a different program.
  2. DAWs are usually based on piano roll, not the printed score. I prefer the latter.

For the record, Iā€™m happy to continue playing to the click, I just want a reasonably-accurate performance to be more easily transcribed than it is at present.

I think smart split point makes more sense in a notation app than anywhere.

Donā€™t forget Cubase :slight_smile: if you keep the score window open the notation updates in real-timeā€¦

Iā€™ve only ever used Cubase for editing. Iā€™ll try out playing into it.

Since pianists have different size ā€œreach,ā€ I imagine that is one factor that complicates smart split-point. And even if one could select a ā€œreachā€ of, say, a tenth (and under what tempi that would be practicable), I am in awe of the amount of programming it would take to factor that into account.

While I agree that such capabilities would be useful, Iā€™m not sure how the cost/benefit works out at this stage of Dorico development or even whether a separate program would, in fact, be advisable.

As far as dealing with latency, I have the benefit of a keyboard that, although ancient, can produce its own sounds, and I salvaged an old pair of external computer speakers that I plug in. My current desktop has its own set of external speakers; and both sets have separate on/off and volume controls. So I can turn off MIDI Thru or turn off the computer speakers altogether and monitor through the speakers connected to my DX7. I know thatā€™s not a solution or even practical for everyone, but it works well for me.

You have a DX7? :flushed:

1 Like

Derek, MainStageā€™s smart split point works well enough to play overlapping string and brass lines without it getting confused. I see your DX7 and I raise you a Juno106 :slight_smile:

Yes and no. If youā€™re playing continuous lines then Mainstage generally gets it right. If you play a note that is on or around the shifting split point, Mainstage has to make a guess and often gets it wrong.

Isolated notes yes, it has no way of guessing. But if a line moves up or down to the split point, or chords in each part, it guesses right. Playing piano into Dorico, it would be fine if it gets it wrong sometimes.

Yup. For the record, I donā€™t expect a perfect recording with no cleanup!

Lots of excellent food for thought. I just wanted to add specifically with respect to the topic of latency when inputting notes in real time that I would strongly encourage Dorico to add the equivalent of Logicā€™s ā€œLow Latency Modeā€ button - when pressed before recording MIDI, it temporarily turns off unnecessary plug-ins affecting processing speed (like fx, compression, etc.) and does other things behind the scenes to temporarily reduce the perceived latency to 0. I use it literally every time I record in Logic (in part because my current CPU is slightly ā€œlong in the toothā€) and it works extremely well. As far as split points, Logic also has a little-used function (at least by me!) that allows you to record two-handed keyboard parts, and then afterwards use a pencil tool in their score editor to draw a horizontal line to separate what should go into the right hand (treble clef) and what should go in the left hand. It was a pretty slick concept (and possibly food for thought?) Having said that, Iā€™ve actually found it easier in Logic to simply duplicate the piano track and then mute/delete the left hand in the piano roll for one copy and do the reverse for the other (something it would be nice to more easily also do the equivalent of in Dorico - perhaps recording initially into one voice, or with an initial ā€œbest guessā€ split point, and then allowing us afterwards to ā€œlassoā€/indicate the left hand notes in the Piano Roll and have them automatically be moved to the bass clef?) My two cents.

  • D.D.