Project/Flows vs Edition/Pieces and batch handling

Dorico
1

I’ve read the concept of Project and Flows and think I understand it (to some extent).

I assume it could be used, in a simple example, as

Project: “Piano Concerto No. 21 by Mozart”
Flow(s): Movements 1, 2, and 3

all contained in one Dorico file. This is all fine and clever thinking. It is a more refined version of “New Section” in some other notation software.

But, in the same way, one could generalize it to more than a Concerto or a Sonata etc. to

Project: “Romantic Piano Pieces”
Flow(s): Piece 1, Piece 2, …, Piece n – Each with separate title page etc.

This would aid in generating editions/collections etc. but it’s not the ‘traditional’ way of publishing collections and would pose quite some editorial and publishing problems “down the road”.

Myself I would assign each Piece a separate Dorico file (i.e. Project, with one Flow only). That said, and here’s the main question, can one batch print/process several open Dorico files/projects like Save All, Print All, Search/Find All etc.?

As a continuation, will it be possible in future versions, not in Version 1, to manage many projects from a “book”-like file (FrameMaker terminology, sorry for those that do not know this term), to compile and manage scores efficiently?

N.B. A book-file in FrameMaker is simply a container that holds chapters that let’s you make general settings for all chapters, re-arrange chapters, select which chapters (if not all) to print etc.

2

Both of your conceptions of how flows may be organised are perfectly possible in Dorico; you can use them either as distinct movements in the same piece, or totally separate pieces, or indeed both, within the same project.

At the moment there are no specific batch operations for e.g. printing all flows from multiple projects. Some of these operations might be most naturally handled by way of scripting, while it may well make sense for others to be integrated more directly into the software.

3

I think it makes sense to generalize Mats’ question a bit.

I feel it’s somewhat ironic that while software developers are very familiar with tools for version control of large projects, what they provide for end users who want to work on large projects is often on a scale between “nothing” and “so simplistic it’s almost useless”. That certainly applies to the two main notation apps.

The Dorico model of flows, layouts, etc seems like the beginning of a good way to handle this, at least for a “single user” projects. From Daniel’s presentations, it looks like a project could contain multiple “versions” of individual flows, etc, mixed and matched in any way that you want to work, and multiple layouts representing different “versions” of the complete project.

The missing piece of the jigsaw seems to be how to extend this to a “multi user” project where several people are working on different sections of the project simultaneously. Storing each user’s work in a separate “complete Dorico project,” and each group of users building their own ad-hoc scripts to stitch those “sub-projects” together using common formats/engraving options/etc, has got to be the wrong way to do it. Some sort of “check-in/check-out” options for individual pieces of a project (so a user can “check out” a particular part of the project which automatically creates a copy of all the relevant date, work on it independently, and then easily merge the updated version back into the full project) would match the basic concept of general-purpose version control systems.

I expect most notation software users would run away in horror if Dorico re-invented a wheel as powerful, complicated, and often incomprehensible as full-featured version control software like Git, but a small number of well-chosen operations would go a long way to handling this.

Something to think about for Dorico version n+1, perhaps?

4

I think Rob make sense, although I have not personally thought of multi-user collaboration on a project(s). “The more the merrier” certainly does not apply to music notation and version control, layers, password protection etc. would be valuable in future versions of Dorico.

My main concern is that since I don’t line up 50 or 60 pieces in a row in a (Dorico) project, but instead have 50 or 60 individual files it is with my current notation software (no name mentioned) a night-mare to make such projects efficient. Such a simple task as “Print/Save/Close all open files” (which exist in most software) is not available.* I hope it will appear in Dorico 1.x (x>0).

  • There is plug-in that can write all open files or files in a folder, but for some reason it’s not too stable on Windows 7, at least on my machine.

Find in Folder/Find in Open Files/Find in List of Project Files would be other such speed-boosting functions, although I understand that the Find function might not have top-priority in early versions of Dorico.

The improved notation engine we see in Dorico will certainly speed things up(!), but a few efficient commands like mention above would also be most valuable for anyone handling more than a single project.

On “compilation level”, combining pieces to editions, I can handle that with other software and do so efficiently today, compiling title page, preamble, table of contents, musical contents etc. in seconds. It might be “too much” for many users, but when ideas run dry in 2022 or so, perhaps we can see such functions added. :slight_smile:

5

I would have thought that efficient multi-user collaboration was a fairly obvious requirement for creating music for movies, live theatre productions, etc where a group of people may be working more or less independently on different parts of the project and there is always time pressure on assembling a “complete version” of the music to play and/or record live.

IIRC I saw a recent video about Pro Tools features for collaboration between multiple users at multiple locations working on a DAW based project, and the user interface looked very much like a lightweight software version control system! (Note, I don’t use either PT or Cubase).

Single-user DAW based projects have already headed in this direction, since they typically contain multiple takes of live recordings, multiple versions of mixdowns, etc. That sort of thing is “software version control” under a different name.

6

I too am very curious about how the multi-user aspect fits into the flow concept for projects. I’ve had to do it different ways when I have been the sub-contractor, but for large projects that I have overseen, I usually keep the “master” file myself, send out the templates or semi-completed files to my co-workers, and then when I receive them back, strip their work into the master, update the version number (my file naming system), and send to the proofreaders. After proofing, I or someone else will make the corrections, then create parts, proof parts, print, etc. I usually have a spreadsheet set up to keep track of all the steps in the process and who is doing what, and the file version number ensures I’m always on the most recent version of the file. (I did a job years ago for the NY Phil where the contractor, not me, accidentally sent out the wrong version for part extraction, so I’m kind of neurotic about this now.)

How does the flow concept fit into collaborative projects? Do you anticipate a collaborative workflow using flows would differ much from the above?

Also, what is sort of the top level of a flow that you would anticipate a user using? A single multi-movement work? A tour containing 14 separate charts for the same client using the same instrumentation? All the work for a client with the same instrumentation? I’m just trying to get a grasp on how I will actually use flows in actual work.

7

Some people are experimenting with “crowd-sourced” engraving:

http://forum.musicasacra.com/forum/discussion/12746/open-source-catholic-hymnal/p1

There are links to several blog pages discussing this here:
http://lilypondblog.org/category/using-lilypond/workflows/

8

I don’t think the concept of flows itself really changes the nature of collaborative work on its own. Really a flow is just an independent chunk of music, stored within a single project. You could even argue that absent features to easily update one flow, for example, after somebody else has worked on it, flows might even be less convenient for collaborative working than the old way of putting a single movement/cue/number in a separate file.

You’re not forced into any particular kind of workflow just because flows exist, of course. If you want to keep each chunk of music in its own project file, and deal with all of the issues of working with separate files that you are doing with your current scoring programs, then you can do that too.

Obviously in the fullness of time we do hope to have some support for collaborative workflows, with the ability to do the kinds of check-in/check-out operations that Rob describes earlier in this thread, but not in the first version.

It can really be whatever you want. You don’t even need to use the same instrumentation for each flow, as you can have a completely separate set of players in each flow if you need to. The advantage of using multiple flows in the same project is that you can easily combine the flows in different combinations in layouts, and the project benefits from having a single set of options that are consistently applied.

9

General questions for the idea of flows. Here’s an example of a current project I’m working on. A hymnal with 346 separate pieces. Some require different note spacing. All require different spacing on systems. Lyrics are adjusted manually in each to observe good note spacing. Each of the pieces are at a separate draft stage and will continue to be updated and edited for some time. So…

  • Will Dorico allow for such a large number of flows in a single document?


  • If so, will this tax performance in any way or are we talking about quite light metadata in the batch code?


  • Will it also allow for different “house styles” in the component flows?


  • Will it allow for different spacing defaults when it comes to note spacing, lyric spacing, system spacing? (or will it enforce a single set of defaults on the batch!


  • That might also be advantageous, so would I be able to enforce a single default on the batch and (if that’s the case) will it overwrite my manual adjustments?


  • Will it allow for versions/subversions in the individual component flows?


  • When the project is complete in all its parts, will Dorico allow me to add things like running headers and pagination?


  • Will export to XML be fully supported, meaning all the settings (individual and default) be preserved in situ

The ability for flows to be treated both entirely independently with regards the style, spacing and typography, but to be able to parse the batch for extra-musical things like pagination, “plate numbers”, copyright disclaimers, and running headers with left and right facing pages bearing different text when required would be beneficial I think.

Apologies if you have covered any of these questions already.

10

Will Dorico allow for such a large number of flows in a single document?

Yes, there’s no limit on the number of flows that a Dorico project can contain.

If so, will this tax performance in any way or are we talking about quite light metadata in the batch code?

That is an unanswered question at this point. We’ve not been testing with anywhere near that many flows.

Will it also allow for different “house styles” in the component flows?

Layout options can be changed per-layout (things like staff size, page size, margins, etc., though it’s possible to change staff size midway through a layout if need be), notation options can be changed per-flow but are common to all layouts (things like how beats are grouped together, accidental strategies, etc.), and engraving options (line thicknesses, gaps between things, font settings, etc.) are common both to all layouts and all flows.

Will it allow for different spacing defaults when it comes to note spacing, lyric spacing, system spacing? (or will it enforce a single set of defaults on the batch!

Note spacing settings are specified on a per-layout basis, so it will be common to all flows within a layout, though it can also be changed by way of a “note spacing change” event that can be inserted at any rhythmic position within a flow.

That might also be advantageous, so would I be able to enforce a single default on the batch and (if that’s the case) will it overwrite my manual adjustments?

You would have to delete any note spacing changes you had created in order to reset to the defaults specified in Layout Options.

Will it allow for versions/subversions in the individual component flows?

No.

When the project is complete in all its parts, will Dorico allow me to add things like running headers and pagination?

Yes.

Will export to XML be fully supported, meaning all the settings (individual and default) be preserved in situ

Dorico’s MusicXML export does not attempt to export presentational data like note spacing settings, bar widths, page layout, headers and footers, etc. It is focused on the pure semantics of the musical content. At the moment, only the first flow in a project will be exported to MusicXML, and although compressed MusicXML’s container format does allow the export of multiple individual pieces within the same container, I don’t believe any other application can import such a file successfully, so this is currently of limited use.

11

I think I would like to see how Dorico can handle MusicXML exports. If I have a project with x number of flows in the batch, I would need to be able to export all of them (even if they were output as separate files) for import into a collaborator’s music notation program - be they Sibelius, Finale or otherwise. I am assuming it is going to be an uphill struggle to break into the upper end of the market where the product would be used in larger organisations (publishers) and schools with pre-existing flavours of music notation apps and so the usability and extensibility (or otherwise) of the MusicXML support is something I would need as a freelancer when sharing my source files either during the proofing process or as final files.