Slow Note Input - why does D do unnecessary stuff?

There have been multiple threads about Dorico and slow note input.
I have an animation:
entering 1/16 notes to harp and at the same time dynamics below CB stave jump up and down. This unnecessary action needs some computing :slight_smile:
Would there be a way to cut unnecessary activities during note input in Galley View and allow users to enter notes without a 0.5 sec delay between MIDI note and pressed duration on computer keyboard (pitch before duration)?

Also the real-time note spacing: please make an option to turn this off while inputting notes. Also wastes time.

slowni

BTW. for some awkward reason the forum states that I am not autorised to upload videos so I had to take the extra ffpmeg step from command line to fast convert screen recording .mov to animated gif. Is this restriction really necessary?

Did the forum actually say that you were unauthorised to upload videos, or did it just say that .mov is not an authorised file type for upload??

As Jack Reacher would say, “in investigations, details matter”. :grin:

You should find that simply renaming the .mov file to .mp4 will allow you to upload it here (file size permitting).

2 Likes

Yep. You are right.
I fell into a trap I have numerous times warned others about:
“READ THEM DARN ERROR MESSAGES” - do not just discard them.
Thank you for pointing out this. At least I learnt something today.
A bit embarassing but I will get over it.

I don’t think it’s the moving dynamics causing the slowness, per se. If you make this same input in Page view, it might give some insight into the layout (re-)calculations running in the background. If you have found the other threads about this, you may know that many different things can cause a slowdown: having many layouts open simultaneously, having Condensing turned on, projects with many flows and/or players, other software hogging the RAM or CPU. Is there something in this list that may apply to your situation?

Ultimately, I’m afraid that the continuous recalculation is very fundamental to the way Dorico is built, and there is no way to completely, meaningfully turn that off. But there are several steps you could take to streamline things:

  • if you haven’t quit and restarted the software in a while, do that
  • turn off other apps running in the background
  • if you don’t need playback: apply the Silence template
  • turn off Condensing until note input is done
  • close any additional layout tabs
  • make a ‘scratch’ layout containing only the flow you’re currently working on.

If that were the case, I would be very happy.
It isn’t.

I am working on a single tab, entering notes in galley view.
My computer is overpowered for notation work. The processor is basically idling.
The issue is not in lack of RAM or CPU used to the limit. Far from that.

There is something not right with how Dorico handles user input in this case. I would buy the need for calculations if the insert mode was on and I would enter notes somewhere in the beginning of the large flow. All further music would need to shift.
That is also not the case. Condensing has also sadly nothing to do with it. We are talking about entering notes in Galley view.

The good example of interaction is actually iphone. People were wondering why Android phones with much higher specs seemed relatively slower to use than iPhones. The reason was simple: when user touched the iPhone- at that time the resources were prioritised. Interaction with a user was primary.

And now Dorico:
I am in Galley view. Note input is active, pitch before duration, no insert mode.

I press a key on midi keyboard:

2024-12-18 20:27:47.883 [info] Executing command: NoteInput.MIDINoteInput?MIDIPitches=84
2024-12-18 20:27:48.051 [info] notifyPostCommandExecute: NoteInput.MIDINoteInput?MIDIPitches=84 (168 ms)

Just the shadow note to move is between 150-250 ms.
No need to calculate layouts etc. nothing has been entered.

To add a duration:

2024-12-18 20:31:48.813 [info] notifyPostCommandExecute: NoteInput.MIDINoteInput?MIDIPitches=72 (170 ms)
2024-12-18 20:31:49.562 [info] Executing command: NoteInput.NoteValue?LogDuration=kSemiQuaver
2024-12-18 20:31:49.819 [info] addEventsForAudition() adding 6 events to buffer #:0
2024-12-18 20:31:49.836 [info] processAuditionEvents() - processed 6 events
2024-12-18 20:31:49.965 [info] notifyPostCommandExecute: NoteInput.NoteValue?LogDuration=kSemiQuaver (402 ms)

That is close to 0.6 seconds and that is really seriously slow.

I am not the only one that needs to slow down the workflow because otherwise the notes do not get entered correctly. I have talked to users that are also working with big files. The larger the file gets the slower note entry will be.

“Don’t work with large files” is also not a choice for people who need to work with large files.

There are a few possibilities:

  1. better buffering for “pitch befor duration” workflow. If pitch before duration is not selected then MIDI note values are buffered and notes appear correctly. Why not buffer MIDI key > computer key alternating sequences? It can be done!

  2. postponing other calculations until active user interaction is done. Somehow Dorico can manage even real-time note input. The calculations are not messing that up. So it can be done!

At this point there should be a disclaimer that I am not a software engineer, let alone a Dorico developer. Those are just some steps that have helped me and others in the past.

Insert mode is certainly an additional complication but even without that on, inputting notes may change the casting-off somewhere, causing many details now needing to be re-calculated in a new context. I suppose that many of those are also dependent on each other limiting the possibility of multithreading.

Switching to Galley view does not automatically turn off Condensing (though I can see how that may be inferred) so it is still liable to slow you down even when its effects aren’t directly visible.

What I know is that updates to the software frequently include ‘performance improvements’ and in my experience, those are noticeable, too. If you suspect there may be something specific in your file or setup causing issues, I’m sure the developers are happy to take a look at the file or the diagnostic logs to maybe remedy that in a future version. But sadly that help can’t come from me.

A member from the Dorico team has told me that while working in Galley View turning condensing on/off does not affect the speed. My observation and also timing values from Dorico_application.log confirm that.
That is just for info.

Dorico does of course try to prioritise input from the user. (What else is it going to do?!)

But it is the case that in large projects, the impact of inputting even a single note can be significant, and indeed that as projects get larger and larger, the likelihood grows that inputting a single note will have a significant impact.

Every edit you make in Dorico causes it to run its entire layout processing stack. This is split into three phases (pre-spacing, spacing, and post-spacing). Each phase contains dozens of distinct steps. Many of these steps are carried out in parallel (typically either allocating a single staff, or a single instrument to its own thread), using multiple cores, but the process as a whole cannot be completely parallel, because at the end of each phase, all of the answers from each thread need to be reconciled before the next phase begins.

Dorico’s architecture is heavyweight, without question. Performance relies on Dorico processing the smallest possible span of music in response to each edit.

However, at a bare minimum, even in galley view, Dorico needs to recalculate the accidental state, which typically affects a minimum of two bars (the current bar, and the previous bar; if you’re unlucky, it will also affect future bars, if for example the note you are inputting is longer than the current bar). It may add or remove accidentals at multiple positions. The music in that region at a minimum will be respaced.

The span that needs to be processed can often be much longer than just two bars, however, if there are long slurs, or long groups of dynamics, or long cues, or other types of regions (bar number regions, slash regions, bar repeats, etc.). If you end up with a pathological case where many such long items overlap a little, the range can be extended significantly.

Even when you are inputting into a single instrument, because the music has to be respaced, every instrument has to be reprocessed. Every instrument therefore has the opportunity to contribute to the overall range that needs to be reprocessed.

There is significant complexity, therefore, in the way Dorico works out what music to reprocess in response to an edit. Certainly we have had bugs in this area of the program that result in much larger ranges being reprocessed than they should. We would need to see your specific score and have some specific steps (e.g. which layout, which flow, which bar, which instrument, what duration of note are you adding) to be able to say with certainty what is actually happening in your case.

It is certainly true that Dorico is much slower with large projects than we want it to be. With our small team, it is a constant challenge to determine what we should be working on at any given moment. There are no quick wins, no simple things we can do to radically change the performance profile of the software. It is likely that it would require multiple months of work from the majority of the team to make real headway in this area. There is significant risk that we could expend all that time and not manage to deliver significant improvement, and in the process introduce new bugs.

And during the time we spend working on performance, we cannot be working on other things, so the opportunity cost is significant. Users are impatiently waiting for improvements of all kinds. Performance is a feature, absolutely, but it has to compete for priority with every other functional improvement we want to deliver.

I know this is not what anybody working on large-scale projects in Dorico wants to hear, and believe me, I don’t particularly enjoy writing it. But this is the reality.

As a practical next step, give me the information I need to look into the specifics of your current project, and we can at least try to identify whether there is something unusual going on.

14 Likes

Thanks @dspreadbury for thorough clarification.

What I can do now is to train my brain for a new habit: alternate different methods wirh pressing [K]. When entering fast runs there is usually one duration involved so unchecking “pitch befor duration” would relieve me from some frustration. I will try!

Nevertheless- I will send a personal message with the file location.
Around 120 thousand notes- says statistics :slight_smile:

I can also understand why Dorico cannot buffer midi-computer keyboard sequences in one buffer. That would be possible if the shadow note wasn’t sticky. But is is and this has its advantages. The disadvantage is the buffering because the user also might want to enter one pitch more than once in a row. So toggling between input modes seems to be the path to take.

Hi @composerkaumann, besides the suggestions above, another little trick to speed up note input, avoiding some calculations, is to temporarily lock the layout in Engrave mode, while writing into Galley view. This can given some improvement in the speed (in my personal experience and in the acknowledgment by other users in the Forum).

What I have understood (and sorry if this is not technical correct) is that locking the layout avoids at least casting-off calculations (because the current systems and music frames casting-off don’t change ).
When you finish entering your music, dynamics etc
 in Galley view, you can then reset the layout in Engrave mode.
Try and see if you can see some improvement.

This was one of the first tricks I found in forum when I searched for a remedy. I tried that at once and the log showed me exactly the same times for note entry. No difference in speed.

THIS!
@dspreadbury also suggested the same.
This is the only thing that makes a difference!
This reduces the overall time of the note input for 5 times, being around 100+ms in total. I still get wrong notes when entering the runs though so I definitely will train myself to toggle “pitch before duration” on/off as necessary.

All that said I will also publicly say that while this is the most annoying thing for me in Dorico I am generally happy with the app and also glad about the switch I made in August 24. For a composer, Dorico provides so much more flexibility than Finale. Shifting musical material around is a dream. I am also a “keyboard person”, minimising the need for mouse (speed!) so customisable shortcuts, jump bar, popovers and scripts are a blessing. Thanks all for answering!

3 Likes

Daniel, thank you. This explanation is the best one I’ve seen this far detailing the inner mechanics of Dorico and makes it very clear to understand the challenges with working on large-scale projects.

May I ask, hypothetically, for the sake of argument:

from the programming standpoint, would it even be possible to “suspend” the layout processing stack from functioning altogether, make a bunch of inputs (notes, dynamics, slurs, playing techniques
) and/or edits, and then “run” the layout processing stack so that it would consider only the last state of entered/edited objects?

1 Like


 sort of like Lilypond.

I’d really need to consult the team about this to be sure, and they’re now mostly enjoying a well-earned Christmas break (as I should really be trying to as well), but my gut feeling is that the answer is “yes”, because if you think about it, when you open a project, what Dorico is effectively doing is running a whole series of edits in sequence, and then shows you the result at the end.

But this doesn’t get you very far. Let’s assume you could suspend Dorico’s response to your edits in order to make a bunch of discrete edits and then reconcile the state later on: you would not be able to get any feedback about the edits you had made until the end of the process. This would hardly be a satisfactory user experience.

2 Likes

My two cents : and if we muzzle any potential playback in the file (VST/Midi), wouldn’t this improve Dorico’s performance a bit ?
I say this because I don’t use audio playback (I have installed none of the audio player in Dorico because I don’t need them), and even if sometimes the input is a bit sleepy (1/4 seconde to react), I haven’t noticed such slowness, even in a large orchestral score by Ravel.

So my question is this : are they the layout calculations that are slowing Dorico down, or is it the audio correspondence particularly ?

So 
sort of like Lilypond. :wink:

@ObiwanKenobi 1/4 second = 250 ms, so that lag is definitely on the same scale as TĂ”nis reported. Indeed, playback can slow things down too (which is why switching it off was one of my suggestions, 13 posts ago) but in this case it’s definitely the layout processing stack causing trouble. And yes, Ravel writes some pretty big scores but TĂ”nis may well have been working on something much bigger. You can’t really compare them directly.

If I would be engraving then turning off playback would indeed be option although when I have engraved some music for my group(s) I usually still use audio to find mistakes easily. But for composing work- no way, audio feedback is essential.

My file stats are here, current state, some lines not present:

Flows:	6
Players:	33
Pages:	260
Instruments:	43

Staves:	27
Systems:	393
Bars:	2710
Frames:	44520
Empty frames:	21993
Non-empty frames:	22527
Duration:	116â€Č44″

Notes:	109722
Grace notes:	2
Rests:	18812
Bar rests:	21993
Tuplets:	4150

Clefs:	557
Octave lines:	16
Time signatures:	277
Holds and pauses:	38

Dynamics:	16667
Slurs:	8586
Playing techniques:	1081
Trills:	340
Glissandos:	654
Arpeggios:	349
Harp pedal changes:	149

Staff-attached text:	269
Immediate tempos:	227
Gradual tempos:	67
Lyrics:	8407


Hum
 I have not calculate the number of notes in Dahpnis et Chloe, but there are many
 :innocent: