Conceptual work on MIDI editing

None of the pretty pictures everyone prefers yet :wink:

Feedback welcome.

@thorwil: I wasnā€™t referring 100% as an absolute value, more as the percentage of the editor window which will resemble the velocity.

Iā€™d reckon that the piano roll will show more than 1 octave and will be variable in height.

midi_editor_02_d.png shows the start of the velocity bars in the bottom of the window, which by coincidence is also C4. What if I scroll down? Or resize the window? Or what if there are some notes in the C3-B4 range?

-H-

i have been reading through your document with great interest. whilst i honestly believe audio-wise ardour is far beyond anything else, proprietary or opensource, midi editing is something that i still havenā€™t found an acceptable replacement for my old logic (on windows, before apple bought it and decided that 48% of their market werenā€™t worth supporting). gripes aside, i think this new midi functionality is a brilliant opportunity to claim the new crown. i have a couple of suggestions for midi implementation, and although they are going to be obviously biased towards my experiences in logic, a straight clone of any program would of course be bad.

note representation:
velocity represented as small bars inside the notes, left to right. 100% of the width means 100% (255) velocity etc.
in logic this is used as well as note colouring to see at a glimpse an accurate representation.

note input:
whilst i found it difficult to get the hang of at first, the way logic handled mouse inputs i find incredibly efficient. i never once needed to choose a tool-mode (pen, eraser, select etc), as the interface used the two (we could use three) mouse buttons effectively.
here is an idea:

left mouse button: used primarily for selecting and modifying existing events.
dragging the end of a note in a selection left and right changes the duration of all selected notes.
dragging around notes selects them.
dragging a note from the middle moves it left/right or up/down.
double clicking a note should bring up a mini editor window (edit start/end, duration, velocity properties directly)

middle mouse button: creates or deletes notes.
when clicking a blank spot in the matrix, a note of the last used duration/velocity is created. this was the single most useful feature for fast editing i found in logic.
clicking the middle button on a current note deletes it. maybe some people wouldnā€™t like this so make it optional, but again, iā€™m after speed here.

right mouse button: brings up context menu as always.
in logic this was the create events button, but i think since we can generally assume there is a middle button present for most people that it should remain a context-menu button. also useful to list some less common functions such as split note, join notes etc.

mouse wheel: used with modifiers.
some examples are: adjust duration or velocity, horizontal/vertical zoom.

i believe this sort of functionality as opposed to the ā€˜choose a tool, choose where you want to use itā€™ interface is far better for repetitive, serious work. maybe if people donā€™t like the learning curve, you can keep the tool-pallet way and make one ā€˜multi-toolā€™ that functions like this.

i just read the end of your document, and this seems similar to the ā€˜quasimodesā€™ idea. i agree that it is a steeper learning curve, but with the ā€˜multi-toolā€™ feature i think this is overcome.

sorry if this is a bit rambling, i have a headache :slight_smile:

Thank you porl, thatā€™s some great input.

For whatever reason I didnā€™t think of representing velocity with horizontal bars inside notes, I only thought about doing it vertically, requiring rather high rows to be readable.

The ability to change the duration of selected notes is something I overlooked. Doing it by dragging the end of one of the selected notes should be the most straightforward way.

Double clicking to bring up note (or selection) properties is a good idea.

Regarding the middle mouse button, it will likely be used for dragging the view like in GIMP or Inkscape.

Lots of pictures this time :wink:

very nice. my only problem with the split on/off velocity display is that it could be confused with two note events when you have a very busy midi passage. is it really used all that much? maybe just have the option to see the off velocity when you are displaying the actual values (top row). iā€™m not that much of a maestro with midi composition to use off velocities all that often though, maybe someone who does will kill me for suggesting that :slight_smile:

i definately like the idea of the numbers in the background of the first image. if they can be toggled on and off then i think everyone would be happy.

for the appearence of the velocity ā€˜barsā€™, i think what i would like to see would be a combination of the 2nd last and 3rd last rows. the bar from left to right helps to get a more precise setting of the velocity, but the colouring of the velocity helps see the overall change in a busy passage (this part here looks louder (more red) than this one). in logic, the whole note was coloured the same, and the ā€˜barā€™ (just a line in the middle of the note, but yours looks much nicer) would move inside it. i think your idea would look really nice if the bar was the only part that changed colour, with the underlying colour staying the same.
if you take the bottom left note image in your last picture, and go diagonally up and to the right, the next two notes in that line show what i mean. the note starts off blank and pale (low velocity), a bar comes out getting more and more red until it reaches the end of the note (100%) and is fully red (or whatever colour it goes to).

the last idea i had is more a ā€˜nicetyā€™ type idea. if the canvas moves to cairo in time, maybe make the notes that are highlighted not only show their handles (great idea by the way), but the middle go a tiny bit transparent to see the barlines through them. obviously this could be overdone, but i think a subtle indication of the underlying bar lines will help in editing durations, especially, once again, in crowded passages.

edit: i just realised a good example of this effect is the progress bar in ubuntu. in fact i think this may look better than using transparency, as it wonā€™t lose any of itā€™s colour and ā€˜fade outā€™, and wonā€™t need any fancy effects of the canvas.

keep it up!

i havenā€™t been this excited about a projectā€™s development in a long time :slight_smile:

porl: Showing off velocity should be an option (I think I mentioned that :wink:
Regarding bar-lines, good suggestion. I just created lines on the note that fade out to the middle to avoid a look too similar to several notes in a row. To be shown with the next set.

Looking good already!

Iā€™ve got a few questions, though.

  • If you embed the velocity in the note view, what is 100% velocity?
  • Will Ardour be able to record/display/edit sysex commands? (example: hihat foot controller, pitch bend, etc)

Considering the above 2 questions, Iā€™d reckon that a separate row with a drop-down-selection is the easiest way to be able to all sorts of information. (velocity, pitch bend, sysex, etc)

-H-

Hi

First off, Thorwil, your work is great. Your site is in my bookmarks so I can see what youā€™re up to!

The mock-ups look really nice. The numbers and colouring of beats seems really clear.

I like the look of the target areas at the starts and ends of notes, but I wonder what they would look like with lots of notes in a small space. Would there be any use in the handles being hidden when zoomed out? Some software uses a change in mouse icon when hovering over the start and end to show that the length can change; would the handles be as well as, or instead of? Iā€™m not sure weā€™d need both, and I prefer the handles.

As far as velocity goes, I would really like both coloured notes and a velocity bar. It would be great if you could use a modifier key and click and drag up or down on the centre of a note; when doing so, the colour would change and the relevant bar would move. Stacked notes would be fairly easy to deal with in this way.

Iā€™m with Porl; very exciting stuff!

Matt

mattb: Why, thank you!
Like I wrote, itā€™s likely better to show the handles on mouse-over only, as it could look confusing with many small notes otherwise.
The job of the handles is to show the target area (hover over note, see where you need to pull to move start/note/end), something a changing mouse cursor canā€™t do. The cursor is good for indicating the possible action. One could use a 4 way arrow here, but that isnā€™t terrible informative :slight_smile:

Not allowing moving the note vertically via dragging on the start or end handle to instead alter velocity this way might be worth considering.

Not allowing moving the note vertically via dragging on the start or end handle to instead alter velocity this way might be worth considering.

not sure about this one. it isnā€™t an intuitive thought firstly, and more importantly it would actually slow the velocity and movement functions down, since if you are trying to change just one you have to be extremely accurate in going horizontal or vertical, not diagonal at all.

look forward to your next mockup though! :slight_smile:

porl

porl: good points.

Underlays and smart dragging:

harold: MIDI velocity goes from 0 to 127. As old hardware might interpret a velocity 0 note on as an off, we should perhaps limit to 1 to 127. 100% = 127, but I wouldnā€™t advocate using percents at all. It just doesnā€™t match well.
Regarding sysex ā€¦ I donā€™t know how well Jack-MIDI does or will support it. I wouldnā€™t expect any support in Ardour above record/playback and a very simple visualization that just indicates thereā€™s something (and even that would need to be written by someone)

regarding sysex, once again i can only say what logic did. all it provided was an ā€˜event listā€™ view of midi data which was literally just a list of events eg

Event time value data
Noteon 0:1:2.345 c4 127
noteoff 1:2:3.45 c4 127
and so on (in this case data is showing velocity). sysex and controller parameters were also listed, and the list was filterable (show only note events, sysex etc).

it was also useful for finetuning things like tempo changes (which iā€™m not sure on the status of jack for this yet, i think there were problems).

just looked at your new mockups too.
nice idea with the gridlines showing through :stuck_out_tongue: i really like the way you implement the underlay, this was something that used to piss me off in logic. i would prefer the selection logic to be reversed (select all touching; modifier to select only fully enclosed) but thats just me i suppose. i think it would make things easier for people using long notes (for pad sounds or other weird samples) that may be 8 bars long for instance.

your last idea is great. i think i prefer the 2nd option (the slight red tint) as a screen full of the ones with the black lines might look a bit messy. there isnā€™t really a worry about whether or not the red is obvious enough, as it is very rare that you need to check that sort of thing - it is only if you want to use quantised notes usually, and if you are changing a note you are looking directly at it, so there is less need to see at a glance if 50 notes on screen all are exact. this is the job of quantising.

hope this helps
porl

harold: Ah! Velocity bars in the background wouldnā€™t be tied to the note grid. The range would be mapped to the full height.

porl: Showing sysex in a list view/editor might be straightforward, but I donā€™t think such a thing is high on Daveā€™s list.

If rubber-banding would select on touch, it would be harder to draw notes between other notes (and impossible to directly draw notes with partial overlap on the same row). If smart dragging is implemented, that is. That might be acceptable, though.

trueā€¦ didnā€™t think of that :slight_smile: what about using the middle mouse to place notes though? i still like the idea of a single middle click making a note with the same length/velocity as the previous inputted note, while a middle click and drag would make a note of a different length (changing all future notes). i think this is faster than having to constantly drag out the same length over and over again.

Thorwill, from one graphic artist to another: great work! Keep it up, Ardour is looking better and better.

Showing sysex in a list view/editor might be straightforward, but I donā€™t think such a thing is high on Daveā€™s list.

Iā€™ve been using sequencers since Notator on the Atari 1040ST (ancestor to Logic) and I must say that a list event editor is crucial to many MIDI editing tasks and not including one in ardour would be a big mistake, I really hope thatā€™s not what you meant.

Itā€™s going to be niiiiice :slight_smile:
Iā€™m looking forward to slaving ZynAddSubFX into Ardourā€¦

Whoa this is going to seriously rule, Ardour is amazing!
~bennyp

porl: left click on empty space would add notes (last used length or snap value). With middle-drag for scrolling, middle click could work as ā€œpageā€ forward or backward depending on which half it is in. Have to think about making that discoverable. Otherwise middle click could be used for zoom in (zoom out with modifier) ā€¦ or unix style pasting :slight_smile: