Artificial intelligence for fingerings and bowings

The development team could have a collection of digital sheet music revised by human experts by which an artificial intelligence could learn when to indicate fingerings and bowings and when to omit those.

Whenever needed, this library of Dorico files could be extended to improve the algorithm.

Whether or not to make this digital sheet music collection publicly available is beyond the scope of my feature request.

Interesting idea, and theoretically possible, but it also seems an awful lot of work for little payoff, especially considering each instrument that takes fingering would need its own algorithm and reference library. Most highly-trained musicians I know aren’t interested in fingerings anyway. Unless it’s how Beethoven or Chopin did it himself, I’ll do my own fingerings, thanks.

4 Likes

On problem is that a skilled musician would only notate those that are counter-intuitive or unexpected in a certain context, where a beginner might need many markings and maybe simplified workarounds. And besides that, there is hardly ever just one ‘correct’ option. It’s very much a matter of taste, (lack of) virtuosity, being educated in a certain tradition, or simply what the concert master, in their unfathomable wisdom, imposes upon the orchestra, whether you like it or not.

2 Likes

If they are anything like breath marks for singers, every conductor I’ve ever sung for marks them in different places.

2 Likes

It might prove interesting to take a look at a performance part from a professional orchestra.
https://archives.nyphil.org/index.php/artifact/bd17408f-0055-420a-991e-022d2afc9389-0.1/fullview#page/1/mode/2up

Are the printed up/down bows Copeland’s own, or were the parts printed after annotating by the Concertmaster?

I don’t know how successful AI could be. There is an awful lot of context to consider, especially with string bowings.

I understand the excitement that some folks have about an iPad version of Dorico. I have no intention of using Apple technology. Moreover, I would have hoped that a pad-based product would be more like StaffPad, allowing for drawing bowings and fingerings quickly with a stylus. With such a tool, I could sit down with the Concertmaster and enter the bowings very quickly.

Rather than AI, I wonder if there might be merit to the idea of creating a service clearinghouse where people could upload their Dorico scores and contract with experts to do things like string bowings. Perhaps there could be all sorts of special expertise that could be brought to bear on compositions.

For example, maybe I’d like to have a 32-bar Sax soli section voiced like Supersax. Or maybe somebody is particularly good at creating drumset parts. I always find that to be a pain. I’d pay good money for some of these things.

I have contracted with people in past to write piano parts like Frank Mantooth would have played them, for example. A clearinghouse to find this expertise would be cool.

A lot of the previous replies refer to the different opinions that conductors, instrumentalists and singers don’t share. This means, if there are too many partially unpredictable rules to be taken into account, that it would decrease the efficiency. Like nobody would agree to anybody’s preferences. As if there were too many opinions to be taken into account.

And this is the point of artificial intelligence.

The other way round would be like describing someones face to a computer by defining rules by hand by writing them down, which is time-consuming. What an AI does, is defining the rules itself.

The real challenge is having high quality sheet music digitized and revised by human beings. The easiest part is the technical realization.

There already are enough tendencies that suggest that this kind of software will exist:

  • Google released TensorFlow a popular open-source library which accelerated and simplified AI for developers by years with less lines to code, compared to 20 years ago.
  • scanning applications are getting better
  • MusicXML supports fingering, which means we have a widely accepted data structure to store this information.
  • https://openscore.cc/ is an evolving platform founded in collaboration with IMSLP and Musescore for people to share digitized sheet music which gets supervised by people.

I think that would be an absolute fools errand. Every orchestra has their preferred markings and they archive photocopies so they can remark their rented scores with their preferred interpretation. It’s not even worth the effort.

1 Like

Indeed, and as with many other bits, hopefully this will be supported in the future.

This is the point of artificial intelligence. It doesn’t matter how much context there is to consider.

Whew, that’s a relief to know… I was worried there for a minute.

1 Like

Now that seven of you are sharing your feedback, I would appreciate if you could also share your thoughts on my previous feature request. As you all know the Steinberg server was set to read-only shortly after I posted it so that only Daniel Spreadbury got the time to respond. Explosion Presets I hope to see you there.

It is not a question of ‘agreement’. In any orchestra you will find a variety of fingerings being used within each string section.

Fingerings are (mostly) the personal preferences of each player - depending on their physique. Less frequently a conductor may require a particular string to be used, but that still leaves a lot of flexibility for each individual player.

Bowings are different. They are mandated/negotiated for two reasons: 1) Musical effect, 2) Visual effect. Bowing is one of the biggest weapons in the conductor’s armoury to create a ‘personal’ interpretation. I doubt they would welcome AI removing that element of their artistry!!

3 Likes

Daniel said all that was needed.

Like self-driving cars, most of the important context is simply not available to the AI computers. Some of it would be available, such as tempo and stylistic markings. Some would be available if the algorithm were able to look at the entire score and deduce how the fiddles should be playing a particular passage. But some is historical, knowing the history of this particular work, the tendencies of this particular composer, is peculiarities of the era.

I doubt it would ever be anything close to what an expert would determine. But certainly some simple decisions could be possible without all that context.

@ cparmerlee What you just said does not contradict with what I explained above: Artificial intelligence for fingerings and bowings - #8 by AaronVin

The printed bowings were either Copland’s or those of an editor as they were done by the copyist. It’s interesting that some of these are written over in pencil indicating the opposite bowing.

No, as long as the objective is to at least get something sensible quickly. As Romanos suggested, some people take these markings very personally, but I am in favor of something that quickly gets one into the ballpark.
For awhile, Finale supported a plug-in that used Band-in-a-box technology to do a surprisingly good job of voicing harmonies. I don’t think I ever used the plug-in’s voicings 100%, but it was great for creating a first draft of an arrangement quickly. I was very sad to see Finale drop that plug-in. I believe things like that are great opportunities for making the programs more useful and productive.

That’s not uncommon.