Thoughts on midi tracks

No replies
audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

Here are some thoughts on midi editing and view presentation, especially in the "midi track" domain. I am fully aware that the current implementation is far from finished, so I consider this input to the discussion on how things should work in the end.

1. The current midi track draws note separators across the entire session from start to end. Note separators only makes sense inside a region. Outside a region they only provides noise to the overall editor "picture", especially if they are close together.

2. A user might not want the same view range for every region in the track, even if stuff inside one track often corresponds to one instrument, which normally stays inside one consistent range of the frequency spectrum throughout a musical piece. Thus we may consider to attach the piano to the region itself, to allow different view ranges per region in a track. This again will result in problems when the start of the region (where such a piano would be attached) is beyond visibility. The remaining options are then to not allow different view ranges per track, and the other option is to not use a piano header at all, but only highlight the note fields (between the note separators) according to whether it represents a white note or a black note.

3. View range selection. The current model features two choices: View the entire note range (0-127) or view the range from the lowest note present to the highest note present. The problem here is when you want to add one note by using the mouse above or below the currently visible range. Assuming a uniform view range for all regions in a given track, i propose adding a vertical scrollbar to every midi track header. Some scrollbars have "zoom handles" at
each side of the box that represent the visible range, allowing to change the size of the visible range (Some other DAWs make use of the horizontal version of this type of scrollbar for the view selection of the session (often also with a miniature view of the session drawn behind it)). The ASCII mockup of the horizontal version:

|<|XXXXXXXXXXX|<|_________________|>|XXXXXXXXXXX|>|
 1             2        3          4             5

In order to extend the visible view, one grabs handle 2 or 4, given the direction one would want to extend in. To move the range, grab 3. Or optionally press 1 or 5, which strictly speaking are not needed.

4. Maximum track height. I consider the current limitation of track height too small to do efficient midi editing in all cases. If the view range includes many notes, the notes become too narrow to hit with the mouse pointer.

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

Right now I'm starting to do an implementation of the said "scrollbar" in GTK+/C, and I would surely appreciate comments on it's appearance and an appropriate name for it, or if there are simpler solutions to the range size/range movement problem.

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

I also propose extraction and modularization of "track background": A piano roll track background (with note separators or black and white fields) are naturally used for midi track background, but can also be used for automation track background, where the parameter controlled describes frequency or other note related data.

As illustrated in Thorwil's blog, it should be possible to provide a "ghost background"/"underlay" to every track with a piano roll background. The ghost background mirrors other tracks or regions below what is actually part of the track, and helps musically arranging different tracks in accordance with one another.

porl
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 2006-07-20
Posts:

nothing really interesting to say, other than the fact that i'm really interested to see what comes of this. i definitely like the piano roll black/white background (obviously more grey/darker grey) idea :)

porl

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

An early screenshot of some of the things I'm working on:
ardour_piano_roll.png

I would surely appreciate any comments:)

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

Then the first test shot of a scrollbar for selecting note range. This shot features the standard gtk scrollbar, which does not meet the needs of what I want. The special purpose scrollbar is not finished yet. Also it looks a bit cheesy with the gray color inside the red track header, but I hope people will at least understand the idea.
ardour_piano_roll2.png

thorwil
User offline. Last seen 3 years 45 weeks ago. Offline
Joined: 2007-11-05
Posts:

The piano bars should be more prominent inside the regions, not less. Empty track space could do without them, even. Then again, they help to identify midi tracks as such.

Moving the RMS buttons in the track headers out of line to fit in a scrollbar is not acceptable. I'd prefer to have dB rulers for audio tracks in one column with midi scrollers. dB rulers could work like in some sample editors, allowing to zoom into a dB range (nice on low level passages).

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

I totally agree with you. But there are several technical "difficulties" (I don't know color math that well, maybe others may have some ideas) when it comes to making the bars less prominent outside the regions. Unless it is possible to provide another alpha compositing function making the colors in the underlay more clear _in_ the region, we may have to draw rectangles _between_ the regions instead, to make the colors there less prominent. Currently (I think) it's impossible to increase the contrast between white and black bars by putting another rectangle above it, the colors would only tend towards the color of that rectangle, at best leaving contrast intact or reducing it.

Ruler/range selector: The plan is not to break the alignment of the buttons, but adding something into an audio track that I suspect might be a rarely used feature just to keep the alignment is not an optimal choice. I would prefer to alternatively keep that area in audio tracks empty.

We may also think of alternative ways to select the midi note range effectively, maybe something other than the proposed ruler/scrollbar. Any thoughts on that thorwil?

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

It appears that the range scrollbar is not the publicly wanted solution. There exists some other approaches though:
1. Use a modifier key combination and the mouse wheel inside the track to increase/decrease the view range. When in the upper part of the track, it increases/decreases the max note visible, and while in the lower track area the same applies to the lowest note visible.

porl
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 2006-07-20
Posts:

i think the modifier key+scrollwheel is the best. assuming there is some other 'full' view of the note matrix, changing the range in the arrange view is a convenience rather than a necessity, so it doesn't necessarily need to have screenspace dedicated to it. if there is no dedicated matrix window (i can't seem to get the trunk to compile so i can't check) then that is a different story though.

porl

skipkent
skipkent's picture
User offline. Last seen 3 years 9 weeks ago. Offline
Joined: 2007-07-06
Posts:

Has anyone brought up the idea of a dedicated 'sister' application to give midi sequencing support to Ardour, rather than adding to the primary codebase itself?

This could be much along the lines of SAWStudio's midi package:

http://www.sawstudio.com/products_midi_workshop.htm

This might be a way to keep the current audio application 'lean and clean' while still adding a dedicated sequencer for those who want it.

Just a thought. Have a great new year all!

thorwil
User offline. Last seen 3 years 45 weeks ago. Offline
Joined: 2007-11-05
Posts:

skipkent: That discussion is over. MIDI is _implemented_ in trunk. Nobody is going to remove it. Lean and clean? sheesh. You will hardly notice if you don't add MIDI tracks.

A trio of concept mockups on scrolling and zooming:
http://thorwil.wordpress.com/2007/12/28/ardour-scroomer/

porl
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 2006-07-20
Posts:

not a big fan of the third mockup, but i think the first two work well. maybe implement the second as the 'mouse over' version of the first? one thing though is i think the keyboard should look look a little more 'realistic', with the black keys shorter than the white. looks too much like a barcode otherwise :)

porl

thorwil
User offline. Last seen 3 years 45 weeks ago. Offline
Joined: 2007-11-05
Posts:

porl: the handles from the 2nd shouldn't appear on mouse-over only, because you need to either aim at them or avoid them, depending on what you want to do. Both much easier/faster if you can see them all the time.

Shorter black keys? Could do that.

porl
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 2006-07-20
Posts:

wouldn't the lighter/darker region show where to aim anyway? i pictured the handles more as a visual clue to highlight what that region does. when i say mouseover though, i mean when the mouse is over *any* part of the scroller, not individual mouse-over areas inside it.
ie: move the mouse over any part of the scroller area and the ends of the 'bright' section are highlighted to show they are 'hotspots'. i think having them on all the time will lead to a messy appearance, especially with many midi tracks on screen.

porl

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

I think the clue here is whether the range is what's _between_ the handles or the handles are also part of the range. In the case of the former, handles should not be transparent, or much less transparent than they appear in the mockup. In the case of the latter, the interesting information (where the range starts and ends) is made more difficult to see, because of the very prominence of the handlers... Anyway I think the latter is the way to go, because that way we can draw the key range across the entire track height, and not subtract the height of the handlers at the top or at the bottom.... eh, right! I hope this was clear enough.

thorwil
User offline. Last seen 3 years 45 weeks ago. Offline
Joined: 2007-11-05
Posts:

audun: clear enough ;)
The handles are drawn on top of the view range. But if the view range becomes too small, they would need to be drawn outside. Maybe they should default to half inside / half outside, but for this mockup I decided otherwise to not have it look more complicated.

Having the full area for drag-scrolling with no ifs or buts would be nice, so zooming via left-right movement should be considered even without on overlay (yellow lines). Another option would be using a modifier key for resizing the view range.

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

The handles could stay inside when the range is small, if the height of the handles is set to minimum-range/2. For instance we might decide that the smallest possible range in the midi editor is one octave.

We still have problems when the track height is small though. We cannot fit the scroomer into the smallest tracks.

thorwil
User offline. Last seen 3 years 45 weeks ago. Offline
Joined: 2007-11-05
Posts:

audun: right, could work out with a minimum range covering several notes.

Smallest track height isn't suitable for editing work, anyway ;)

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

midi scroomer implemented!
rendered using cairo, to get those nice antialiased keys.
midi-scroomer1.png

thorwil
User offline. Last seen 3 years 45 weeks ago. Offline
Joined: 2007-11-05
Posts:

Great, looking forward to try it!

porl
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 2006-07-20
Posts:

very nice. looks like it will fit right in with ardour's look too. nice job :)

porl

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

The latest screenshot showing both scroomer and piano roll header. The keys on the scroomer are likely to be removed. Also changed the midi region transparency to enhance the visibility of the piano roll lines beneath regions:
ardour-scroomer-pianoroll3.png

porl
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 2006-07-20
Posts:

i think the keys need to be on the 'scroomer', otherwise it is much harder to get a quick idea of the exact location of the zoom area. to tell you the truth, i'd prefer it without the larger keys next to it, as the light/dark areas on the region area show essentially the same information. i'm sure a lot of people would disagree with me though. maybe make it toggleable?

porl

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

The argument to make it stay there is to use it for testing notes, and to highlight notes when played. Initially I thought that it need not be there too, but now I'm convinced that it is a necessity.

porl
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 2006-07-20
Posts:

yeah, i didn't think of that. you are right. :)

keep it up!

porl

thorwil
User offline. Last seen 3 years 45 weeks ago. Offline
Joined: 2007-11-05
Posts:

New mockup based on audun's latest:
http://thorwil.wordpress.com/2008/01/09/ardour-scroomer-3/

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

I have now created a patch for people to test. It contains new scroomer widget/midi scroomer, new (unfinished) piano roll header widget, and a new Lineset canvas item that draws the midi track underlay.

midi_patch_ver1.patch

porl
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 2006-07-20
Posts:

i really like the mockup thorwil. the circles do make sense (in terms of ensuring they aren't confused for ranges). i also like the diagonal 'zoom' highlight (don't know how else to describe it... it is almost 2am here...). i still believe the circles (or whatever handles) should only appear when the mouse is over some part of the scroomer widgit. i don't mean the mouse has to be where the circles are for them to appear, but anywhere on the piano scroomer widget itself. i think many midi tracks on screen will look messy otherwise.

this is all looking really good though guys :)

porl

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

thorwil: love those retro looking piano keys with the little shadow:) Will try to do something similar in the code!

audun
audun's picture
User offline. Last seen 23 weeks 1 day ago. Offline
Joined: 2006-04-01
Posts:

I implemented something that looks like thorwil's shaded piano keys, and the shades are inverted on click (nothing happens on mouse over yet). It kind of looks like some 80's based drum machine or something, don't know if that is good or bad. At least I think it's kind of cool to look at:)