My Windows 7 vs. Windows 8 DAWBench Results

Cubase
1

Before I replaced a couple 512GB hard drives on my DAW system with bigger ones, I thought it would be interesting to see how well Cubase would run on my system with Windows 8. I imaged my existing system onto the new drives via eSata, but before swapping the drives out I did DAWBench testing, first on my existing Windows 7 installation, and then after upgrading to Windows 8.

First off, my system/HW specs:

Interface: MOTU UltraLite-mk3 (pre-hybrid, FireWire 400 only)
Processor: Intel Core i7-950 @3.06 GHz (Bloomfield family, 4 cores, 8 threads). No overclocking, and I’m using the factory heat sink.
Motherboard: Gigabyte GA-EX58-UD5. (I have EIST and the C-states disabled.)
FireWire Chipset: On-board Texas Instruments 1394 OHCI Compliant Host Controller
FireWire Driver: 1394ohci.sys, Microsoft, 6/21/2006 (same driver on both OSes)
Case: Antec P183.
Hard drives: Three Western Digital Caviar Black drives, at 7200RPM @ 500 GB.
Power Supply: Corsair HX650.
RAM: Two Kingston DDR3 1333 sticks @ 6 GB each, for 12 GB total.
Wireless: Linksys WMP600N Wireless-N PCI Adapter with Dual-Band.
Optical: Plextor PX-B320SA Blu-ray Disc Combo.
Video: EVGA NVIDIA GeForce GTX 570.
OS Versions: Windows 7 Pro 64-bit with SP-1 / Windows 8 Pro 64-bit RTM
DAW Version: Cubase 6.5.3 32-bit (the DAWBench plugins were 32-bit only)

Before running the Windows 7 tests, I uninstalled Microsoft Security Essentials and Acronis True Image 2012, because I knew those apps aren’t compatible with Windows 8, and they both add a little overhead. I also made sure both systems had the “High Performance” power scheme selected in the Power Options control panel before testing to ensure top CPU availability.

***** DSP TESTS *****
In the DAWBench DSP tests, you load a project that has some audible audio tracks, plus 40 additional tracks with 8 of the same insert effect loaded (but disabled), for a total of 320 possible instances of that effect. You play the audio and enable the insert effects one by one until you start hearing pops or dropouts. The “score” for that test is the number of instances you can enable before the audio breaks up. The DAWBench DSP suite includes six projects for six effects, but I only tested two, because the public DAWBench database currently only lists reference results for three of them, and the latest version of one of those three (Elysia mpressor), is very broken and evil, compared to the one the DAWBench author used for his tests. It literally takes 5 seconds now to load a single instance of the plug-in, meaning it takes half an hour just to load the damn project.

I also conducted all of my tests at 256 samples, because that’s what I currently use in my work. At 256 samples, my interface’s reported latency numbers are 7.098 ms input, 12.018 output, 19.116 total.

DSP TEST 1: URS Classic Console Strip Pro (CSP)

Instances on Windows 7: 133
Instances on Windows 8: 93
Percent drop in performance: 30%

DSP TEST 2: Wave Arts MutiDynamics 5 (MD5)

Instances on Windows 7: 157
Instances on Windows 8: 102
Percent drop in performance: 35%

***** POLYPHONY TESTS *****
The DAWBench Polyphony projects include dozens of initially muted polyphonic tracks using Native Instruments Kontakt 4. While you listen to the handful of audible audio tracks, you unmute the polyphonic tracks until the audio breaks up. There are two flavors of this test: One with no effects applied, and one with a convolution reverb effect.

POLYPHONY TEST 1: No Convolution Reverb (NCV)

Notes of polyphony on Windows 7: 540
Notes of polyphony on Windows 8: 340
Percent drop in performance: 37%

POLYPHONY TEST 2: Convolution Reverb (CV)

Notes of polyphony on Windows 7: 520
Notes of polyphony on Windows 8: 260
Percent drop in performance: 50%

[u]* OBSERVATIONS *[/u]

  • Cubase and my plugins mostly survived the upgrade, although the first few times I launched Cubase I got errors that the soft eLicenser didn’t have sufficient permissions to run. I modified the Cubase shortcut to run as Administrator and that made the warnings go away.
  • My Waves plugins didn’t work after the upgrade. I didn’t investigate this, as they weren’t needed for the DAWBench tests.
  • The Cubase ASIO meter was a bit more erratic on Windows 8 than on Windows 7. In my previous experiences with DAWBench, I never saw the ASIO meter “redline” even when audio began breaking up. But on Windows 8 the little red emergency light blinked every second or so once the meter got above around 70%. This almost always accompanied pops or dropouts.
  • Normally on the DAWBench polyphony tests, I never see more than a 20-note difference between the “with convolution reverb” and “without convolution reverb” projects. But the gap between these two tests on Windows 8 was huge- 80 notes; a 24% difference.

[u]* SUMMARY *[/u]

While Cubase and my audio hardware functioned largely without error on Windows 8, the overall performance during testing was 30-50% worse than on Windows 7 on the same system. What accounts for the different results? Is it Cubase? Is it the OS? Is it the drivers? I don’t know why I got the results I got. I even re-ran most of the tests, just to make sure the numbers were consistent.

I am curious whether FireWire has anything to do with it, because I’ve heard mumblings about Microsoft wanting to deprecate FireWire support- although the driver appears to be the same on both OSes. (They have different version stamps- 6.1.7601.17514 on Win 7 and 6.2.9200.16384 on Win 8, but those are just generic stamps applied to every inbox driver that coincide with the OS build.) I don’t have any USB or PCIe interfaces to compare against though, so I wasn’t able to rule that out.

To be honest, despite the test results, Windows 8 would probably actually work okay for my day-to-day recording, considering that I never have more than a couple dozen plugins or notes of polyphony in my songs, whereas the DAWBench tests were breaking down in the hundreds. I’ve already swapped my drives out and returned to Windows 7, though. It’s fast and it works.

Please post your own before-and-afters if you have the time and interest! You can compare my results against the main DAWBench reference results available at the “DAWbench Universal Suite - Results Database : In Progress” thread at the DAWBench forums.

2

Wow, thorough test! I don’t have time to do a test like that - and since my system is very much like yours, also with firewire for main interface, it would probably have a similar result. But thanks for posting - another reassurance to not change OS in the near future…

3

That’s a good read, thank you for taking the time to do the testing. :slight_smile:

Since I was so slow to move to Win7 from XP, I am right now applauding my future laziness. Ha! But it could be that new drivers will make a difference, although in my case I am told that Mackie may not be replacing the current driver I am running. Firewire, yes.

Wow, all reports I have read say that 8 and 7 are very similar except for the touch screen option. Now I am thinking otherwise. Thanks again Ultimate Outsider.

4

Thanks for the thorough testing!
Now W8 is supposed to be faster than 7 on many ‘regular’ tasks. Noticed any of that?

5

It feels like they’ve done some more improvement on Windows 7’s already very good prefetching/app optimization. Launching (and shutting down) Cubase for the first time on Windows 8 actually took a bit longer than on Win 7, but subsequent runs of Cubase (even after restart) felt just as quick as on Win 7. Loading a couple of the DAWBench projects might even have been a little quicker on Win 8 as well.

System startup feels quicker, too, although I didn’t time it. However it didn’t load as quickly as it does on my Win 8 notebooks and desktop prototypes at work - perhaps because at work I don’t have any additional apps or devices installed but the bare essentials.

I am considering upgrading to a PCIe interface in the next couple of weeks. If I do that, I will re-test to see if FireWire was an ingredient in the difference between my test runs.

Also, for anyone who’s interested in Win 8 but put off by the interface changes, Stardock’s free Start8 is a pretty cool Start menu replacement. Lack of a Start menu is my #1 gripe with Win 8. It seems fine otherwise, despite the DAW performance gaps I encountered under heavy use. (Oh and the Waves plugin problem, which might have a simple solution. Don’t know about that.)

6

That’s an interesting test. The guys at cakewalk have done some testing too (using Sonar, ofcourse). They found nothing but improvements in performance and memory usage: Cakewalk - Announcement

I’ve also read from the WinRT developers blog (WinRT is Windows 8 for Arm-cpu’s) that several optimisations have been implemented in the kernel to reduce power consumption, memory footprint etc., and that the x86/x64 versions of Win 8 also should benefit from this as well.

Well, Win7 works fine for me, and I’m not really into this touch-UI-thingy on a desktop pc (and I hate greasy finger-marks on my monitor!)

Regards,
Kris.

7

I have seen the Sonar results, so it really makes me curious what’s different between my setup/experience and theirs. I can’t conclusively say it’s Microsoft’s fault yet. I can only say that Cubase doesn’t work as well on Win 8 as it does on Win 7 with my setup.

The thing that concerns me though is all the “reduce power consumption” work they’ve done in Windows 8… power savings are generally inversely proportional to performance. But we won’t know what the true story is until we see a broader array of results on different PCs and interfaces.

8

Thanks. I’ll see if I remember to check tomorrow but I do seem to recall something similar said on the SoS site from tests in “PC Notes” although one of the posters there, I think, did mention switching something around or off to get more or less the same as W7.

9

I have a program I wrote several years ago running on my PC that tracks changes in drivers, startup processes, and services, and after I uninstalled Microsoft Security Essentials on Windows 7, it notified me that Windows Defender was re-enabled. I’m curious about the differences between Windows Defender on Win 7 and Win 8… like what did MSE have that WD on Win 7 was missing? In any case, technically, both my Win 7 and Win 8 computers were running Windows Defender for the tests, but I understand that may not be 1-to-1.

I also used a local account for tests. I really do not like the MS account integration, and have never enabled that on a PC except as an experiment on a throwaway image at work.

I just happened to be curious and to have some spare time to experiment- but I also made sure I could revert back to Win 7 safely. I feel zero need to upgrade at this point, but if I ever make the move, I’ll wait until I hear solid word from my software and hardware vendors that Win 8’s officially supported.

10

Did you happen to compare any network activity at all?

Also, I have noticed on my laptop that the battery runs down faster when using W8. I am guessing the cause being the always-on graphics hardware acceleration in W8. If that’s the case, it could account for some increased interruption.

11

Hold your horses, here´s quick test on RT and x84 tablet… i think windows sucks on ARM…
http://www.mikropc.net/kaikki_uutiset/windows+8n+tablettiversio+on+hitaampi+kuin+tyopoytaversio/a836934
Sorry for the page being finnish, but check the video.

12

Doesn’t ARM processors sacrifice a some computing power in favor of energy efficiency. I read a number of articles on comparing the various mobile processors available. E.g. here, here and here. There are plenty more, just google or bing “ARM vs Atom”. This may account for some of the differences?

13

The Windows 7 vs. Windows 8 DAW benchmarks I did a week ago left me with more questions than answers. There was an obvious performance gap between the two operating systems in my tests, but none of my data gave me any hints as to why. In order to gain some more clarity on the issue (and satisfy my GAS), I picked up a few new interfaces of different kinds and ran some new tests.

The interfaces I tested for this round are:

  • MOTU PCIe-424 with 24I/O expander. (PCI Express)
  • MOTU UltraLite-mk3 pre-hybrid (FireWire 400)
  • Native Instruments Komplete Audio 6 (USB)
  • Virus TI Desktop interface mode (USB)

**** PREPARATION ****
In last week’s tests, I started out with my production DAW Windows 7 image on my original 500GB hard drive. After doing my Win 7 tests, I upgraded that image to Windows 8 for the second round.

I made several changes to my PC since then that necessitated re-testing my UltraLite results, so here’s what’s different between the tests I ran then and now:

  • I did not uninstall Microsoft Security Essentials or Acronis True Image before doing the Windows 7 tests, so Windows 7 had a little more of a resource burden this time around. (Turns out those programs result in around a 6% performance penalty for Windows 7. More on that later.)
  • This time I was running Windows 8 as a fresh install on a separate partition with only the minimum drivers and apps required for the tests.
  • These tests were all done with a new 2TB hard drive as the system drive.
  • Because of some equipment shuffling I did last week, I had to switch from using a 6-foot FireWire cable to a 15 foot one for the UltraLite tests. The 24I/O also came with a 15-foot cable.
  • Due to some new hardware I purchased there were more drivers and services loaded on both operating systems this time.

So, I expected (and observed) both some slightly reduced performance results versus last week’s on the UltraLite, and a slightly smaller gap between the Win 7 and Win 8 rounds.

**** THE TESTS ****
I performed each test at least once for each interface, on each operating system. If I encountered results that were better or worse than expected, I rebooted the computer and re-tested to confirm the results were accurate. Doing this never significantly changed the results. Just like last week the tests were the following.

  • CSP - Instances of URS Classic Console Strip Pro
  • MD5 - Instances of Wave Arts MultiDynamice 5
  • NCV - Notes of polyphony without effects on Kontakt 4
  • CV - Notes of polyphony with convolution reverb on Kontakt 4

RAW DATA

PER-INTERFACE RESULTS

24I/O

KOMPLETE AUDIO 6

ULTRALITE

VIRUS

**** OBSERVATIONS ****

  • The FireWire and PCIe interfaces saw a similar performance drop on Win 8 as on my previous tests. As I expected, the gap wasn’t as big because my Win 7 partition had more apps and services loaded than the Win 8 partition. The difference was still significant- a 21%-44% drop-off in performance on Windows 8.
  • The two USB interfaces fared significantly better on Windows 8, in a sense. The Komplete Audio 6 only suffered a 4-12% drop, and the Virus lost only 0-4%- and even scored 2% higher on the CSP test!
  • The goal of every test is to find the highest number of plugin instances or notes of polyphony without any audible pops or dropouts in the audio tracks of the project. With the PCIe and FireWire interfaces, if you were at that maximum, enabling one additional plugin or one more row of polyphony introduced maybe one pop every 3-4 seconds or so. Enabling a couple more would result in pops maybe every 2-3 seconds perhaps, and so on- but it took a while before you ended up with a simply unplayable result. The “no pops” threshold with these interfaces generally happened well before Cubase’s ASIO meter “redlined.” But with the USB interfaces, I could drive the ASIO meter all the way up into the red without hearing any pops- however, adding just a single plugin or polyphony track beyond that threshold would result in completely unplayable results; harsh buzzing, silent gaps, etc.

  • While USB seemed to have a higher playable threshold than FireWire and USB in terms of ability to perform under severe ASIO strain on Windows 8, the environment was much more fragile. For example, while playing at the “no pops” threshold on the PCIe/FireWire, you could tab around, launch programs, etc without interrupting the audio, but with the USB interfaces, even something as trivial as launching Notepad.exe- sometimes even just moving the mouse- resulted in severe disruption of the audio.
  • I used a program called RTL Utility, which measures actual round-trip latency on audio interfaces, giving you more accurate numbers than what you would see by looking in your DAW’s audio interface panel. I found that the latency numbers (both reported and real) between the two operating systems were virtually identical. The difference between Win 7 and Win 8 seems to be more a matter of load/bandwidth than raw speed.
  • In an effort to see if PCI/PCIe saturation had anything to do with the poor showing for the MOTU interfaces on Windows 8, I replaced my high-end NVIDIA GeForce GT 570 card with a much more modest GeForce GT 430, and also removed my PCI WiFi card. This improved the PCIe interface tests by around 3.5 percent overall, but had no effect on the FireWire results. Also, the PCIe performance boost occurred on both operating systems, so it didn’t help explain the gap.
  • Due to the differences in my Windows 7 setup between last week’s tests and this week’s tests, my Windows 7 tests in this round were around 6% worse. In other words, the extra software I normally have running for antivirus and backups incur a 6% performance hit. My Windows 8 test results between the two weeks for that interface were basically the same. So keep in mind that my Windows 7 results this week sort of have a 6% handicap. The Virus may have scored the same on both operating systems this week, but if I had uninstalled Microsoft Security Essentials and Acronis True Image, it very likely would be 6% worse on Windows 8. (Which is still much better than the other interfaces.)

**** CONCLUSIONS ****
It’s still way too early to declare that there’s a serious issue in Windows 8 (and if there is, what exactly is the root cause), but I think we can at least rule some potential culprits out.

  • It’s not Cubase. The TI Virus just about the same on both operating systems (with the caveat that the Windows 7 results have around a 6% handicap). If Cubase were the problem, I’d have expected the performance gap to be consistent across interfaces.
  • It’s not JUST FireWire. Both my PCIe and FireWire interfaces underperformed similarly on Windows 8. Efforts to reduce PCI saturation only slightly improved results for the PCIe interface, and didn’t affect the FireWire interface at all.
  • It’s not necessarily drivers. Both my PCIe and FireWire interfaces are MOTU products, but the two devices have different features, different hardware, and different drivers. It COULD be a case of unoptimized drivers in both cases… but if that’s the case, what’s different about Windows 8 that it would cause these logo-compliant Windows 7 drivers to underperform?
14

I forgot to mention in my previous benchmark post that as before, all tests were performed at 256 samples, with 44.1kHz sample rate, at 32-bits. I recorded both the reported and measured latency of each interface on both operating systems. The reported numbers (as shown in the Cubase interface panel) were identical on both OSes, and the measured numbers, as computer by Oblique Audio’s RTL Utility, were very nearly identical.

For reasons I haven’t been able to figure out yet, I wasn’t able to complete the RTL (round-trip latency) tests on the Virus TI Desktop. That’s why there’s no data for it on the “actual” test results.

RAW DATA (lower numbers are better)

CHART (shorter bars are better)

That’s it for now. Excuse me while I go off to ponder whether a 5.041ms latency advantage is worth the $1200 USD price difference between my PCIe-424 core system and the Komplete Audio 6.

15

Just a quick note to say thanks for this thread - it’s a gripping read! Hope you’re able to keep updating us on your results.

Regards,

Steve.

16

That’s it for now. Excuse me while I go off to ponder whether a 5.041ms latency advantage is worth the $1200 USD price difference between my PCIe-424 core system and the Komplete Audio 6.

For anyone reading…I don’t know how much of a joke that was…but, you didn’t pay a premium for lower latency (at least alone)…show me the benches of Komplete sending and receiving 24x24 IO simultaneously.

It’s really a different piece of kit for a different situation/studio. Not better or worse based on latency. Some of us have racks of gear to integrate–which means the Komplete would be an unusable fail.

Just a side note for new engineers.

17

I’m not clear as of what impact the changes may or not have on the ASIO interfase, or the overal Cubase audio framework, but as mentioned above and reported some time ago on the Sonar camp, there seem to be considerable changes made by Microsoft to the audio engine in Win8:

http://msdn.microsoft.com/en-us/library/windows/hardware/hh439707(v=vs.85).aspx

There is a good amount of technical reading there that I have myself not attempted to complete, not sure if you may have consider these as part of your consideration of possible causes for the changes in performance you’ve been seeing. I can’t help but wonder if some of these may actually have some impact in the Cubase performance, one way or another.

18

Yes, it was a joke. :slight_smile: I bought it because I have a whole rack full of external effects I want to wire up.

The SONAR tests might be misleading. DAWBench pushes the system to its absolute limits- to the point where audio begins to break down. The SONAR tests are just meter readings. Microsoft did improve performance in certain areas, but they’ve done a lot of power management work in Win 8 too, which almost always means reduced and/or inconsistent performance.

The guy who created the DAWBench suite is going to be doing his own Win 8 tests on SONAR very soon to see if we can get more of an apples-to-apples comparison.

19

Thanks, mate. These are all really interesting observations.

20

yes thanks for the testing and results. as i said in the gearslutz thread my initial impressions were good with windows 8 x64 and my current projects.

I’m looking forward to vin’s testing, I actually did the reaper template for him for the last round of tests :slight_smile:


mc