A case for Plug-ins / API

I deeply believe in the open-source community and Dorico has such a loyal community.

I am, therefore, often bewildered when Dorico’s develops work on features that can easily be achieved by community-generated plug-ins (for instance for respelling notes to a key, or retrograding a selection). Why isn’t the API completely exposed for us to write those features ourselves?

Dorico’s core engine is the SO powerful. I would much rather the expensive develops on staff focus their time working on the core engine, while letting users develop small features as plug-ins as they go. In the last update, so much functionality was added, but only a small portion of it was core functionality, while a good chunk of it was… useful small tools and features. Why not expose the API and be done with it?

In my work I often require many niche features. Features that I know will not make the top of the list of your development priorities any time soon, and perhaps never. This puts me in a difficult position. There are Some other software, that enables users to write plug-ins. In those environments, whenever I need my isosteric feature, I just make it. No need to wait for an update. No need to make feature requests. I just make it. And if anyone else finds the plug-in I wrote useful… well, they’re welcome to download it as well!

Is this something that we can expect to see soon?
I promise, if you’d expose a fully functional API, I will not make another feature requests again (and why would I? I’ll have any feature I can dream of).

1 Like

I actually see the open ability in the other software to create plug-ins as desperate compensation in an attempt to rectify the absence of important features.
That’s not to take away from the brilliant plug-in developers’ achievements.

2 Likes

I see your point, but vehemently disagree.
There’s a lot I have in terms of criticism for other apps (which is why I’m here), but facilitating plug-ins is certainly not one of them.

Successful plug-ins become part of the plug-ins that ship with the product out of the box. Niche plug-ins are available for weirdos (like me) to download.

This frees the developers to work on core features rather than on surface stuff. The kinds of stuff plug-ins allow is generally simple stuff, and in that sense they are fancy macros. There’s no need for a 6-figure/year developer to spend company’s money on creating fancy macros when a**holes like me can do them just as well.

I’d prefer for develops to make core improvements (like Dorico for iPad, or making the engine faster/smarter), but definitely not add simple features like “retrograde” or “transpose to scale” or whatever they call it. It’s both niche and too trivial to be worth an expensive developer’s time.

If and when I’ll want to use retrograde, I’ll see if another user wrote a plug-in to do that, and if not, I’ll write the plug-in myself. AND IF NOT, I’ll post here on the forum asking if someone can write the plug-in for me. I’ll bet you a million dollars, someone will say yes and deliver that plugin that day. It’s because this community is so great. The Dorico team should capitalize on their community.

1 Like

For the benefit of the incredibly talented and diligent developer who implemented the new and comprehensive set of musical transformation tools, I hope they don’t mind me taking the opportunity to highlight how extensive and detailed their work on these features was.

I understand the wider point you’re making, though, Michael. You’re right though that it has been discussed before and the team have shared their position – e.g. here from January:

5 Likes

Simple doesn’t mean lacking of detail or intention.
It just means that people who aren’t TOP developers can do it with a relatively small learning curve. At the same time developing the engine is something only top developers can do, and also only after a very steep and long learning curve.

Trust me, by any measure, the transformation tools are objectively simpler than developing the engine. Your team will agree, and they will not take offense.

As to Daniel’s remarks, I’d argue that plug-ins are by far the single most important functionality you could add, and that perhaps Dan is right. Everything else can wait. In my opinion, Dorico is stable enough and flexible enough to not need more “transformation tools” and the like. Plug-ins will allow everyone to get whatever they want whenever they want it, no matter how niche the need.

With plug-ins, the distance between now and being able to put a little heart above every note is a 20 minute away.
The distance between now and being able to export a list of every technique used in a certain part is 40 minutes away.
The distance between now and being able to color note-heads based on how low or high they are is 60 minutes away.

This has been proven to be a viable path with Sibelius and with MuseScore many times over. You community contains many technically savvy people, who may not want to wait until the next update is released, or perhaps the next one after that, just in order to discover that the tiny feature they want didn’t make the cut yet again.

I’d urge Daniel to reconsider his position.

1 Like

I’d like to politely suggest that if these features were “only 40 minutes away” they would have been implemented years ago.

There is also something to be said for programs that require so many add ons to be useful that they are hardly more than platform hosts. I can think of one such program… where most of the useful and talked about features come from said plugins. Hardly a healthy development model. That said, I get what youre saying, and I want to see such support going forward, but the team has also proven time and again that they have a grander vision in mind that is worth waiting for.

2 Likes

If you let me write plugins, I am willing to COMMIT to deliver the plug-ins I mentioned within 40 minutes. That’s how long it’d take me to do it in Finale/Sibelius.

As mentioned in the post that Lille quoted, even a super-well-established and developed piece of software like Adobe Acrobat relies on user-made plug-ins to supplement what is essentially a very strong core engine, without too much bloatware.

I wrote this message originally because I was forced, yet again, to open Sibelius where I can just have a higher level of control.

The Dorico teams has reached a point where they are trying to cater to different crowds with different needs. I don’t care about chord changes, I don’t use them at all, but the team spent a long time working on this for the people who do. I get it. But in the mean time, “my side” of the crowd gets little to no attention. plug-ins solve it completely. For everyone!

I am tired of having to rely on priorities that often have to do with cross-sections of their user-base that are very different than me.
I’m not saying my needs should be prioritized, quite the opposite! I’m saying let me write my own plug-ins and develop my own features, while you develop the feature you want, I am not deprived out of the features I need.

To the best of my knowledge, neither of the big competitors have rebuilt their equivalent of Play mode in the past year, or renamed their equivalent of Playback Techniques in the past two or three years (or whatever they’re now called), just to give two examples. Once particular plugins become established and widely used, bits of the host code become near enough untouchable, as altering them runs the risk that existing plugins get broken.

While some bits of Dorico’s core infrastructure may be “finished”, other bits clearly aren’t.

2 Likes

Well… if we’re honest, the need to re-vamp a core functionality is nothing to be proud of

The two big competitors clearly have bits of core functionality they need to overhaul. Unfortunately they can’t, and plugins are one of the reasons!

3 Likes

Plugins are certainly NOT the reason why certain core-functionality in competitors is not being re-hauled. You can be absolutely sure of that. Plug-ins are not at all part of that consideration.

And yet, I keep needing to go back to the competitors, because while Dorico has really cool system for auto-calculating harmonics, the competitors just let me do more.
The competitors do less out of the box, but they allow me to stretch the box!
Dorico ships with a lot out of the box. (90% of Dorico’s features are bloatware as far as I’m considered), but whenever I need anything that’s not 100% developed, I have no choice but go back to Sibelius (which I hate).

And the craziest thing is… the API is already there. That’s how they’re developing the app. They just need to expose the API to the end user.

And if some core-functionality changes and the plug-in that I wrote doesn’t work any more and I need to change a few lines of code to get it working again… BIG DEAL.

I understand your frustration that our own development priorities don’t match entirely with your own personal priorities, Michael, but I’m afraid you’re mistaken in your assertion that an API already exists and “just” needs to be exposed. (The word “just” is a really insidious one, and I try to avoid it when I’m speaking with my colleagues in the development team in these kinds of contexts, since it is a casual way to sweep aside all considerations about the true complexity of an issue.)

No API beyond the simple commands-based system used by both the existing Lua bindings and the new Remote Control API added in Dorico 4 exists at the moment.

I don’t have anything further to add above and beyond what I last wrote about this back in January. It is still the case that we do not expect to devote our limited development time to building out the plug-in API this year. But it is definitely something that remains in our plans and we will work on it when we judge the time to be right. Please understand that although we do our best to be open and transparent with our user community, we cannot always provide chapter and verse on all of the decisions we are making and the reasons behind them.

6 Likes

Daniel, I hear you.

It may shock you, but I didn’t migrate to Dorico because the program can calculate the pitch of harmonics for me out of the box. In fact, I wouldn’t mind using diamond shaped noteheads to indicate harmonics altogether.

I moved to Dorico because it got a lot of the CORE ideas right in my opinion. That is thanks to the meticulous planning spent, and in that respect it’s unparalleled. (And I’ve been here since V1, and even before, I was reading your blog and following the development like a suspense novel)

But one CORE idea that I think you got dead-wrong is that features must be perfect to be released. Features are features, they are not the core product. They can evolve and develop over time. They don’t need to be perfect. They can have bugs. They can be incomplete.

The development philosophy that made Dorico amazing, is also it’s undoing in my opinion. I don’t believe that only perfect well-polished features should be released, on the contrary, I believe that features very quickly reach the point of diminishing returns, where getting them from 90% to ~100% takes 80% of the time. I think it’s a waste of developer time, and a waste of the user-time who awaits updates.

Give us a dirty-clunky-half-broken API and fix it incrementally. There’s really no need for a shiny golden API with everything perfectly documented and provided with working examples. Start somewhere. Let people play with it. Then slowly improve it. I for one will be over the moon.

Of course, all of this said with 0 knowledge of how product-management actually works, so I’m talking out of my a**, but I’m deeply frustrated as you can tell.

Anyway, I’m very disappointed with how this went.

Unhappy customer.

Insert mode.
Force duration.
Play mode.
Percussion rests.
Condensing.
Font/paragraph Styles.
Local/global properties (and propagating).

These are just a few features, off the cuff, that disprove the notion that the development team only release something when it’s 100% perfect, and have later been tweaked.

I’m sure I could come up with more if I could be bothered to read the version histories at this time of night :wink:

1 Like

I think you wholly misunderstood my point. But pieroschlochauer is making the same point better than me.

I’m sorry that you’re unhappy, and of course I understand where you’re coming from. As I said in my previous reply, when your own priorities and the priorities of the product you’re using don’t line up, it’s frustrating. So I empathise with how you feel.

I wish we had more developers on the team, and could work on more things at once. We are currently one person down due to our colleague András leaving in October and the wheels of bureaucracy are turning sufficiently slowly that we have not yet advertised for the open position. Two of our most senior developers have been seconded to the Steinberg Licensing project for more than a year, meaning that they have been unable to make their usual sterling contributions to Dorico over that time. Another of our developers is spending the majority of his time working on a project that is Dorico-adjacent, but does not directly drive Dorico itself forward. You can read the names of the developers of the software in our About dialog, and you can count them. If you take three-and-a-half or four people away from that list, you’ll see as a proportion what kind of impact that departure and those other projects have on our development velocity.

To be clear, I am not complaining that our team is under-resourced. We are happy to make a major contribution to the Steinberg Licensing project because it benefits Dorico customers just as much as it does all other Steinberg customers. It is important that we contribute to other projects within the company that can make use of our skills and the technologies we have developed. But all of these things come at a cost to pushing the development of the core application forwards.

There will be another Dorico 4 update in a few weeks. The handful of us who are working on the software directly right now are working tirelessly to bring further useful improvements to the software as quickly as we can. You will find lots of useful improvements contained within. And my other colleagues in the team are itching to get back to working on Dorico just as soon as possible.

We certainly do strive to ensure that when we release a feature or set of functionality that it is fit for purpose, ideally more than merely fit for purpose, but it’s not the case that we are sitting on many half-finished features that we could ship but choose not to. We ship more or less everything that we work on just as soon as we can, barring any major problems that crop up in testing. Yes, we aim high, but we’re also pragmatic, and we certainly iterate on existing functionality. Just this week I have personally spent several days reworking some existing functionality in order to deliver useful, incremental improvement, and that improvement will (all being well) ship in our next update.

In the end, of course you must use the software that best suits your needs and that is available today. I hope it’s Dorico, but I understand that for some use cases (hopefully an increasingly narrow slice of use cases as we continue to drive the software forwards) that may not currently be the case. We will get there!

10 Likes

All I’m saying is that if you give me scripting, my priorities won’t need to line up. I’ll take care of my self and develop the features that I need.

I too wish you had more developers. But let’s remember, I’m a paying customer. I don’t need to empathize. Dorico is not longer he underdog it once was. I wish you had more developers, but the lack of them doesn’t make me go “awww ok, I’ll just wait”. I can’t wait. So at the end of the day, I’ll have to turn to software solutions that can provide me with my needs. (not making a threat, just a clear fact).

I love Dorico as a philosophy, and I love how tight-knit the community is with the development team.

But, it’s becoming glaringly obvious that Dorico is not for me. At least, not for a certain portion of my projects.

You know it’s funny, already in v.1 you shipped with compound meters and complex tuning systems. All this did a fantastic job of grabbing the attention of contemporary classical composers like myself. But for the most part, Dorico is not catered for a certain kind of contemporary music. I get it. I do. I promise. Vastly more users need chord symbols than they need aleatoric boxes. Knowing that my contemporary composerly priorities are currently at the bottom of the priority list, my hanging hope was the plug-ins! I thought that while you cater to your bigger audience base, I could take care of myself with some plug-in support. Understanding that that’s not imminently arriving is a no-go for me.

Let me put it this way: adding good, native support for features like aleatory/frame notation, cut-away scores, changing the number of staff lines, etc. are currently much higher up the priority list than building a scripting API.

However niche they may be, they are absolutely in our plans, and we see this area as a golden opportunity for us to solve those problems in notation software really for the first time – just as we have done with other things like open meter, condensing, and so on, and so on.

However, there’s no kind of plug-in API we could provide you based on Dorico’s current functionality that would make any of these kind of “contemporary composerly priorities” practical for you to build without us building those capabilities into the core of the software.

But I know that future plans don’t help much when you’re struggling to complete your projects today. I can only reiterate that I don’t think your vision for Dorico and ours are that different.

5 Likes

I agree that there are some features that I won’t be able to solve via API. And I’m happy to see part of your list of upcoming features. All of which would be very useful to me, and I agree that an API would be no substitute to that.

I’m quoting Kenneth Gaw from a parallel discussion on FB:

There are many possible features that are specific to a very limited set of users and and simply not worth developing from a commercial point of view. Third part plugins can fill this gap.

There’s a big gap even a simple API can facilitate.

Anyway, for the current project, I have to work with a different software. For other projects, when I can, I choose Dorico. Hoping to never need anything but Dorico in the future.

Thanks Daniel for taking the time to discuss this