I’m getting ready to upgrade my AMD CPU. I was thinking it would be handy to work up a series of benchmarks to measure Dorico’s performance on any particular machine. I know there are plenty of benchmark tests available, but I mostly only care about Dorico in this case.
I know it would have to be a little informal, but I’d be really interested… both to see how the upgrade affects performance, and to see how other users’ machine perform.
If these tests were sufficiently taxing, they could be a helpful metric as users interact and look at upgrading to new systems.
Does anyone have suggestions for some specific, measurable tasks that could perhaps be timed? I had previously suggested something like adding 100 flows, then renaming one of them… or some sort of task related to condensing that could be performed with a stopwatch in hand. Thoughts?
This might be as much influenced by RAM as it is CPU, but definitely would measure one aspect of Dorico performance. Start with New from Template/Modern Orchestra, go to Play, then load the Silence playback template. After all the VST Instruments are gone, go back into Playback Templates, select the factory HSSE (SE) template, get ready to start your stopwatch, then hit “Apply and Close” and start your watch at the same time. After a couple of seconds the green play icon will turn to grey as Dorico begins to load VST sounds. Time how long it takes until the play icon is green again.
For something a bit faster, maybe try New from Template/Modern Orchestra, 4/4, add one thousand bars, enter whole notes E, D, C in Flutes 1, 2, 3, then turn on Condensing and time the amount of time it takes to show the condensed score. My system is pretty fast and it’s about 6 seconds for me.
This should be interesting. I tried your score setup and my MacBookPro also took about 6 seconds to show the condensed score. I’ll try it later on my MacPro, which is an older machine. Perhaps we could think up some more tests.
I seem to remember that the time it takes to perform certain operations can also change according to how long the program has been running. This would make these tests less reliable. Perhaps we should stipulate that these tests are done immediately after starting Dorico.
I don’t think there’s any meaningful tests you could do here. The easiest test would be time to load a large file, but that is just measuring the file loading algorithm. What matters to us I think is time spent entering the score, like flipping between modes and entering notes. On my 2013 MP that can get annoying laggy, but trying to measure the time by hand is too inaccurate.
…and if you’re trying out multiple machines then rather than fiddling around you could just do the task and then generate Dorico Diagnostics - you’ll find the logs in there.
Would it be best to load down Dorico—say, with a full orchestra, or 100 flows, or 1,000 bars—and then perform a series of tasks? Or would it also be helpful to check the log for timing of small tasks with a minimal score?
I guess that’s a question of how Dorico utilizes multiple cores, which is something I don’t entirely understand. I know it runs better on at least a 4-core machine, but I also know some folks with extremely powerful machines don’t seem to experience a proportionately higher performance improvement. And when I’ve tried large tasks (like adding tons of flows, or condensing 1,000 bars), I’ve been watching the CPU meters and don’t see much change.
I’m not sure my understanding goes much further than yours.
If I understand correctly, there are certain tasks - particularly ones involving Setup mode, but also condensing - that have to be done serially. You can’t work out how page 2 will look until you’ve worked out how page 1 looks, etc. It may be that there are certain sub-tasks within these tasks that can be spread across multiple threads, but on the whole I get the impression that these tasks are sent to a single core.
That’s pretty cool! Here are the entries for the 1000 bar condensing test on my desktop ( i9-9900K, 64 GB) vs laptop ( i7-7500U, 16 GB):
6952 ms, 9355 ms
When cut down to only 100 bars it’s:
1282 ms, 1600 ms
That’s really interesting. I’ll try that on my machines when I get back home. I would like to hear Paul or Daniel weigh in on what they think would be several of the most helpful tasks, then we can sort of standardize those into a benchmark test.
Now that’s intriguing. I’m pretty sure this 2019 Macbook Pro contains an i9-9880H, and has half as much RAM as Todd’s machine, yet I’m getting consistently lower numbers.
@FredGUnn are you running much other stuff in the background? I’ve got helpers for Dropbox, Google Drive, OneDrive, Stream Deck and a couple of backup widgets but that’s about it…
Did you bother to wait for HALion to finish loading patches in the background, and are you using a non-standard playback template?
Just closed out some of the background stuff and it’s about a half second faster at 6494 ms. I still have a lot of background apps that are set to automatically restart if I manually close them so there still is a bunch going on. Backblaze (online backup), NAS, antivirus, cooling management, UAD audio software, MIDI routing (so Dorico receives the MIDI signal), and some Adobe CC stuff that refuses to stay closed are all running.
Hmm. If I create a Diagnostics Report, it saves a ZIP file to the desktop. How is this different from the file(s) in Application Support which you mentioned?