The Variability of XML Importing

This is perhaps a bit of a wider topic as it concerns other notation software.

I was wondering if anyone more in the know could shed any general light on the factors that govern the success/failure of importing XML files into Dorico. I’ve imported into Dorico from both Finale and Sibelius, with wildly varying success rates, from almost perfect, to a little messy, to downright unreadable, to not importing at all. It would seem that the complexity of the score might have something to do with it, but then again, I imported a complex orchestral score with a lot of string divisi etc and it wasn’t too bad, while I also imported a piece for 4 instruments, which was entirely unusable: all notes in the second half missing, rhythms incorrect, etc (shame, as this is the file I actually needed).

Two particular questions I’m wondering about:

i. what might cause the rhythms to end up different, i.e. actually different location in the bar? I know that tuplets sometimes mistranslate, but it’s not that.
ii. What might cause a whole section of the notation to disappear, i.e. the staves are there, along with some dynamics but no notes.

Without seeing (samples of ) the original XML and resulting Dorico import, I don’t see how one can give a useful answer to your questions.
Part may depend on the XML Input settings in preferences, and part may depend on what program generated the XML and how many elements of the import are supported (appropriately tagged) by the exporting program or Dorico.

Perhaps. I guess I’m asking if there’s anyone who knows XML or knows Dorico’s handling of them well enough to have a sense of whether there are particular things more likely to throw it, or things that it handles better.

Anyway, since you asked, the first attached pic (‘TD_1Finale2Sibelius3Dorico.png’) is (from l to r) the Finale original, the Sibelius XML import and then the Dorico import. The Sibelius would need a lot of cleaning up but isn’t too bad. The Dorico one seems to be missing the music from about 20 or so bars in. A markedly different response to the same file.

Out of interest I tried importing into Sibelius, saving as sibelius then exporting again. Same result. Also same result with uncompressed or compressed XMLs.

The second attached pic (‘WTF’) is an interesting thing Dorico decided to do with some slurs on an earlier page of the same file. (To be fair, Finale has done way weirder things, and that’s without importing an XML.)

In general, when you end up with lots of missing notes, it’s because there’s an invalid tuplet or otherwise non-notatable duration, which prevents Dorico from importing any further music in that voice. In other applications like Finale and Sibelius these kinds of things are sometimes a bit more recoverable because those applications think only in terms of discrete bars, but because Dorico has to try to interpret an entire voice in one go (later adding the barlines) it’s more likely that a tuplet with wonky durations will cause problems downstream. In our experience these kinds of problems tend to be more likely to occur in MusicXML files exported from Finale, perhaps because Finale allows users to create tuplets that look one way but have different underlying durations.

I dunno, WTF looks pretty good to me. Positively relaxing.

Do you have any hidden notes in the original file? A similar situation gave me results like the screenshot you posted.

That’s the old “copy and paste things in Sibelius and get multiple slurs stacked on top of each other” feature, before Sibelius did collision avoidance (or if you fixed the problem simply by switching collision avoidance off). If Sibelius stacks them all at the same position, you can’t see them.

Dorico does collision avoidance by default.

I did wonder about that. There are hidden notes in Finale’s layer 4, some created with TGtools for tremolo playback, and others for keyswitching. Additionally, those layer 4 notes aren’t always “complete” bars, i.e. when I wrote it I didn’t always add the remaining rests to make up the bar duration since they were hidden notes anyway, and that was just more to hide. sO I guess that could be contributing.

Thanks, that gives me a bit of insight as to what might be going on. I don’t recall there being any “impossible” tuplets, but it’s certainly possible there are. I do recall that ocassionally towards the end of a bar sometimes, Finale would for some reason not be able to fit the complete tuplet in the bar (e.g. the last note instead of being an 8th would be a double dotted sixteenth), and I’d have to force it to display correctly.

I really think hidden notes are the culprit here.

Here’s a pic that ended up being that.

Aren’t the bars indicated in the XML? Seems to me like if there’s a hitch it should be recoverable after the next bar marker rather than wipe the voice for the rest of the piece.

From durations set in MusicXML files I’ve looked at, I think it would have been a lot better to define them as ratios rather than numbers. You will always get some weird rounding errors with some kinds of tuplets otherwise;

Adrien, it’s messier than you might think, because MusicXML files can jump forwards and backwards in time from the middle of one bar to the middle of another. If you use multiple voices, jumping backwards to define each voice separately is a common way for MusicXML to define the music.

Also, the internal time unit in the MusicXML file can change within tuplets, so keeping track of exactly what “go back 1000 ticks” means isn’t trivial if there are multiple voices, some with tuplets and some without.

The MusicXML spec says you are not supposed to do sneaky things like define some notes, then jump back over them, and then change the time signature that applies to those notes, but I don’t think anybody actually enforces such things.

For example some versions of Sibelius “backspace” by a weird fraction of a beat when there is a clef change, presumably because that is how Sibelius positioned the clef symbol. When that gets combined with multiple voices and tuplets (including clef changes part way through tuplets), especially if a tuplet starts with a rest, the result is sometimes chaos.

The problem is that if you want to import MusicXML from 20 different notation apps, you have 20 different sets of idiosyncrasies to handle.

Thanks for the explanation Rob. Yeah I always thought it was dodgy to have a time unit, which hopefully is divisible by the most common divisions (1/4, 1/3, 1/5) into whole numbers, but not any others. If your notes and tuplets were defined as ratios, e.g. 1:4, 7:2 etc then there would be no rounding errors baked into the data file. Seems to me that for a program importing XML to display correct notation, they have to occasionally guess at what the notation should be based on the absolute duration in ticks.