Enjoyed the recent Scoring Notes Podcast with @pianoleo .
It got me thinking - what is the perfect use for the Apple Pencil in a scoring app?
My two cents :-
Annotation and Review in reading mode:
as it sounds, but annotations remain persistent as a mutable layer in write/engrave mode
it would be cool if, when you circle something, the thing that is circled become tagged for review, and you can click through these tags one by one in write and engrave mode, “resolving” them.
perhaps you could use hand-writing recognition to autogenerate comments to accompany the review tags…
Selection in write and engrave mode
improving the marquee by making it more like a lasso drawn by the pencil
Freehand/Pen tool in Engrave mode
For those times when it’s just easier to design a symbol than waste time with svg imports and wrestling with playing techniques… Could be like your average vector pen tool with handles etc.
Sketchpad - perhaps a specially templated layout with blank manuscript in it, so you can doodle prior to engraving? Perhaps some OMR could be used here to help transfer onto a flow…
Note input, or not?
the trouble with note input is that nobody really wants to deal with the time it takes and imprecision involved in hand-writing. Why write a note when you can play it instantly?
the competition’s app is reasonably quick for the initial input, but requires an awkward cross of hands to use the notepad in conjunction with the pencil.
-The core idea of not making as many mistakes behind Dorico’s design is at odds with freehand writing.
Without having thought about it a great deal as yet:
I’m happy to use forScore for PDF management and annotation for the time-being. Leads me to think that “mission creep” could get Steinberg started on a path to incorporating all the things that another app like forScore does very well right now.
I’d like to push notes around the staff with the pencil: up, down, extend, reduce length gestures?
Dorico to be Scribble-enabled to allow writing of comments (which are of course annotations) with the pencil.
I don’t think note input via handwriting would be at all desirable but I could certainly go for some tap to place style note input. As handy as the on screen keyboard is, it might be more streamlined to have a single view rather than having both the score and the keys open.
It’s been suggested on the forum previously but some form of pencil gestures could be useful here as well (not like the fiddly tilt interface that Product A went for, but something like swipe up and down for accidentals and left/right for duration).
I have found that I use the Apple Pencil when I want to choose (or lengthen) slurs, since my finger cannot choose a slur without choosing the starting note as well.
Agreed. After watching the sibelius demo video, I cannot fathom trying to do their method; it’s terribly slow. I could foresee an entry method where the pen plops down whatever note value you have selected (and will continue to do so until you exit or change values) but having to draw or do those weird tilting actions on the little sib keypad? no thank you.
Also, there’s been plenty of chatter about handwriting recognition… we need to let sleeping dogs lay here… there is a very prominent program that has dedicated itself wholly to developing this concept and is still doing so with only limited success after multiple years of exclusive development. The amount of resources it would take to develop this one feature for a subset of users would be insane. (This is assuming they don’t license someone else’s technology, but I’d wager that any company that would license such a feature doesn’t have one worth licensing… and the only one that might be worth licensing wouldn’t let their exclusive edge go to a main competitor…) Just use that program and then import xml when you’re done.
The problem I see with using the pencil for note input is that it is by nature a freehand tool - when you put it to the ipad you expect to use it just like a pencil, or a stylus, whereas music engraving applications are a strictly controlled input environment based on rigid procedural input. Product X’s approach feels to me to be a bad implementation of a good tool -an attempt to use that input for the sake of it instead of making a positive idiomatic use of the type of tool it is. I feel the same way about their finger gestures by the way, although it does work fine and probably quicker than any other alternative, without a keyboard attached.
I wouldn’t like to use it as a primary input, but it would be handy to have some sort of doodlepad - composing into Dorico does feel a bit like committing something before it’s ready. I haven’t yet found a good enough program for doodling music on the iPad (at least to my satisfaction), but it would be handy if it was part of the same app that I engrave with.
I think the takeaway for me is that the fastest input methods rely less on very precise movement and more on grid based actions. i.e. treating the pencil on the staff like you would a pencil tool on a piano roll.
Here’s an idea: in note input mode, drawing a small circle with the pencil inserts that pitch at the selected note value at the cursor. Not as sophisticated as true handwriting, but perhaps a useful alternative to the on-screen keyboard. I could see myself selecting note-values with my left hand (using the vertical extended toolbar) and drawing pitches with my right.
Maybe a similar approach could even be used for tabs, too, by writing digits instead of circles…
You have fundamentally two interaction choices as I see it -
MIDI+Computer keyboard
I think this is the quickest and easiest interface. With the CME keyboard + Apple they fit together nicely (I like Apple above MIDI so you can quickly access the arrow keys for navigation since they are at the bottom). This is similar to the desktop for familiarity and I can’t see any other being as quick, though you do need to carry around two extra keyboards
Touch
Switching pencil/finger is slow and annoying for me, so to work it would need to be left hand finger/thumb and right hand pencil. But then I’m tapping out notes on the piano keyboard with a pencil, clumsy! I don’t see this working.
The one use case that makes sense is for marking up scores for performance (personal notes. So just supporting graphical frames, where you can draw in the frame would be sufficient, along with the idea ideally that they would be different. Not part of the music canonically but performance notes.
FWIW I took the iPad + Keyboards to the library yesterday to get out of the house, and had a good time doing some composing. It works really well, nicer than being hooked up to yet another full computer. It convinced me that the iPad is now a viable device for this kind of work, and with the keyboards really doesn’t take up much room.
I’m finding similarly it’s nice to sit and play with the iPad version - it does feel freeing not to be at the laptop and “plugged in” as it were. Not sure if I’d do engraving work like this but composing certainly…
It’s a bit of a conundrum, I have an older 9" iPad pro which works OK with Dorico, I’d like a larger screen but the way they price it is that the MB Air is cheaper than the 12" pro. The iPad though triggers the ‘it’s a piece of paper’ part of my brain so is confortable and easy to work with. It’d go iPad but Apple typically is gimping the OS so all that capability doesn’t get fully utilized.
Yeah, handwriting recognition is not so easy – I’ve had StaffPad for years and I still can’t make it work for me, the handwriting recognition is a hit or miss. Nothing fancy, just basic stuff like writing in a quarter note or a half note. If I have to try that six times and the program still doesn’t get it, I’m kind of done. I’d rather have a program where I can enter it manually even if it takes two or three steps. The philosophy of the program is that you as user have to adapt to the program rather than the other way around (the program getting better at recognizing your scribbling).
I could see the existing grid expanded to accommodate the general idea of pencil input, although I’d hope we could just tap rather than draw (perhaps a grey floating note could appear and move up and down at the proper position in the grid before we actually tap the pen to screen). It would probably make the handwriting people happy to be able to input notes this way but also streamline the process and not require quite so much drawing, (I propose this separately from any annotation features.)
Have you noticed that you can already input notes with the apple pen to the pianoroll at the bottom panel?
It is not very fast, but it feels very pleasing to draw the notes to the piano roll by hand and see them appearing in the score.
At the moment it adds every chromatic note with a sharp (disregarding any key signature or context).
You have to click to add a note and then click and drag the end to lengthen or shorten it. It would be great if (with apple pencil) there was a mode where this would be just one smooth motion. (One drawn line sets the pitch and duration)
I did notice that however I didn’t seem to be able to actually lengthen the note reliably, more often than not I could only lengthen the playback of the note but the visual representation didn’t change.
Now that I’ve had a bit of a go at writing on the piano roll there’s a couple of things that would be useful.
Firstly, the ability to drag to set the length on the same tap as the note input. Currently it’s a tap to input and then a second tap and drag to set note length. Seems like an unnecessary step.
Secondly, some visual indication of adjacent notes, at the moment a series of the same note just looks like one long note.
Edit: I get that the tap to place is to not interfere with the ability to drag the interface, but perhaps with Apple Pencil input you can distinguish between finger to drag the view and pencil to input and edit notes.
I think where the note input to the piano roll with apple pencil would really shine is inputting microtonal music (for example just 24 EDO). You could actually draw the correct note in without the need for any further alterations after the note input. But it seems that dorico for ipad doesn’t yet support custom tonalities?