Dorico performance benchmarks?

These sorts of comparisons are exactly how this very thread started. It’s a way up the thread but I certainly posted Dorico 3.5 comparisons for a top spec 2019 Intel MacBook Pro and an M1 machine (each machine with a base M1 processor will behave pretty similarly), and Daniel then posted similar comparisons for a dev native build of Dorico 4. Just scroll up!

2 Likes

Thank you very much, this was excactly what I was hoping for! I think I will buy the lowest tier iMac with 16 Gb RAM and 512 Gt of memory. It will be a secondary computer - at some point I will have to upgrade the MacBook Pro too.

1 Like

Testing my new M2 Pro Mini, I get:

1000 bars Condensing on/off: 2309ms / 2270ms.
100 bars: 396ms / 388ms.

This with HSSE (SE) template.

That’s surprisingly similar to Daniel’s original benchmarks for the first native build of Dorico 4 on his ‘base’ M1. 2529 / 2270; 300/376.

Comparing the logs against my M1 Pro, I’m getting pretty similar numbers there for various tasks on my own documents. CPU % usage for playback seems the same.
Even so, both are nearly twice as fast as my Intel Mini.

Assuming that the M2 is faster than the M1, and it’s not just the same chip with improved marketing, then perhaps “the CPU is not the limiting factor”, and there comes a point where Dorico just doesn’t get faster…? :thinking:

Ben, do you have the 10-core or 12-core CPU in your Mac Mini?

10-core. My M1 Pro is also 10-core, and they both have the same amount of RAM and storage.

The M2 has 4 Efficiency cores, instead of 2 for the M1, and they seem to do most of the work. I don’t know if that’s a factor.

Thanks, Ben. Yes, that’s right — four efficiency cores in both. The 12-core has two more performance cores for a total of 8 instead of 6 in the 10-core. Those two extra performance cores might be significant, but I’m not sure if it’s worth the extra $300.

I don’t know about Dorico, but Lilypond does not use multiple cores, as due to the nature of its layout engine and the nature of how you have to layout music in general, it’s a single threaded operation. It’s difficult to parallelize. Adding more cores does very little, and any performance you see by upgrading CPU’s comes from the the processor speed and other throughput optimisations. As said, I do not know if Dorico is single threaded in layout engine or not.

I am unfamiliar with the technical aspects of Dorico’s layout engine but recall the Dorico developer, Dr. Paul Walmsley, saying early on that the number of CPU processor cores was one of the more significant factors in determining Dorico’s speed.

With my Intel MacBook Pro 2019 this 1000 bars condensing test was 5179 ms. I wonder if this means that everything else is also more than twice as slow… :slight_smile: Perhaps I should look for a good deal on an M1 MacBook Pro!

Definitely. The difference between the Intel Macs and the M series is like night and day. To say nothing of the battery life.

2 Likes

I found this benchmark overview looking for experience with M2 Ultra Mac Studio. Although of less importance for speed and more relevant for projects with many instruments and extensive samples, it seems that multicore usage with certain DAW’s is a big issue. Cubase comes out very well as it uses both performance and efficiency cores but surprisingly Logic does not use the efficiency coores to the maximum. Dorico was not tested as test procedure bars on maximum playable tracks is not suited for it.

Here the benchmark figures for an M1Max 64 GB MBP.

3 flutes each bar filled with whole notes E flute 1, D flute 2, C flute 3 with Dorico 5.1:
1000 bars
Hallion Sonic Selection template
Condensing . enabled 777 ms, disabled 405 ms
+/- 20 ms

100 bars
Condensing enabled 244 ms, disabled 177 ms

XML export of this file as prescribed 6246 ms

The M1 Max has the same 10 cores as the M1 Pro, so it’s surprising that it should be faster at all than the 2309ms / 2270ms I was getting on my M1 Pro.

Testing again my M2 Pro (also 10 cores), I get 2282 ms for condensing on, 1648 ms off.

Are you sure you’re doing the same thing?

Attached an extract from the log. Here is again what I did for the 1000 bar test:

  1. I creates a new project with 999 bars (as 1000 is not possible) with 1 flute
  2. I duplicated the flute two times
  3. I entered an E. to the first staff, D to the second and C to the third
  4. I added 1 bar with shift+B to get 1000 bars
  5. I copied the first bar and after having highlighted the other 999 bars (with select to end of flow) pasted the notes into the selected bars
  6. I went to Edit end tagged Condensing to enable and once more to disable

I then saved and closed the file opened it again, waited for the the play button to get green and enabled and disabled a few times more resulting in the additional entries under 1000 bars in the file.

I then deleted 900 bars and disabled and enabled condensing a few times and did the export test for 100 bars. I also repeated the export after saving, closing anf opening again.
Dorcio Benchmark.pdf (22.9 KB)

The standard test is to use the Modern Orchestra ensemble template.

Why isn’t 1000 bars possible? Admittedly, you will already have 1 bar, so you only need to enter 999.

If you open a new project in Dorico you can enter the number of bars of the new project on the right side of the menu. The maximum is 999. So you have add one bar to get 1000.

I thought the idea to use this Modern Orchestra template was abandoned, Please look at your own post from December 2020 shown in the attached screenshot

I understood you yourself pulled back the idea of using the Modern Orchestra.

As far as I understood we were testing with three flutes and using the Silence (which I did not test) and the HSSE playback template (now renamed HAlion Sonic Selection) without using a project template to start with. That is what I tested.

I now did a second test creating a project from the Modern Orchestra Template with 999 bars, added 1 bar and only put notes E,D,C in 1000/100 bars of the three flute staffs. Subsequently I did the condensing and export test.

Here are the results close to the earlier figures you and others gave for the M1 (Pro).
Dorcio Benchmark MOT.pdf (22.0 KB)

It would be cool if we could also create a stress test benchmark similar to the ones used for DAW’s. Target would be to test core usage and effect of RAM size.

To make it independent of 3rd party libraries we could also use the MO template. We probably do not need a lot of bars but a lot of fast notes with all instruments playing at the same time. If that still works we could additional instruments till playback starts acting up. The default instruments are very light on the CPU so it might be challenging to get to the limit on 16GB+ machines.

Do we have an idea of exactly how much correlation there is between multi-core performance and editing speed (no playback)? I’m particularly interested in how much CPU performance would affect the speed of edits with condensing on.

I.e., if the M3 Max benches twice as fast as the M1 Max, does that correlate anywhere close to edits processing twice as fast? Or maybe a better comparison would be an MX Max vs MX Ultra. Has anyone been able to try this test on an M1 Ultra or M3 Max?

I could be wrong, but I’m pretty sure multi-core doesn’t really help editing. It’s more about single-core power.

1 Like

I agree and even think Daniel has said as much in the past, explaining that a lot of Dorico’s operations need to be done one after the other.

1 Like

In this thread, from almost 2 years ago, Daniel seems to be saying that Dorico does some things using multiple cores but in others it needs to be single-threaded. I tried limiting Dorico to a single core using Process Lasso and the performance hit was very substantial indeed.

He also acknowledged there were areas for improvement (hardware acceleration and, if I understand it correctly, general performance optimization), but they competed with multiple other things on the list. I would welcome and appreciate any updates or perspective on this.

I did a small (very un-scientific) experiment of adding and deleting a dynamic in a large score with condensing on over and over again, and I monitored core usage with Activity Monitor on MacOs 14.3.1, Dorico version 5.1.2. This is a 2021 MacBook Pro with M1Max. Here’s the CPU history graph:

Each of the visible spikes in CPU usage is one of these edits in Dorico. Although most usage seems to be allocated to cores 3-7, activity spikes are still present across all performance cores. A dev would probably need to clarify, but this seems to indicate that edit calculations with condensing on are indeed affected by multi-core performance; although, this of course does not show how much they’re affected.

1 Like