It’s easy to align the rhythms of rhythm instruments / tracks. There are clearly visible onsets / offsets in the drawn waveform. But the same is not true for less rhythmic instruments / vocal tracks. I’m personally concerned with vocals since I mostly deal with A Cappella music, but the same could be true for a variety of instruments. It’s not always evident where an “ooh” changes to an “aah,” and pitch changes on the same syllable are even harder to detect visibly (at least in a suitable zoom scale for quickly aligning multiple parts.)
Enter colors with a direct relationship to the frequency of the waveform at that location. The more I think about it the more complicated I realize it is; yet if it is pulled off it would be a huge convenience to many engineers / producers as a simple, elegant, unobtrusive addition the visual information being presented. The basic concept could be one of two. The first would be to only consider the fundamental frequency (a la pitch-tracking algorithms), though that wouldn’t help at all for making an “ooh”-“ah” transition visible. The second therefore could color the waveform according to the entire spectral content much like how a mixture of different frequencies of light produces a single color. I’m not entirely sure the latter would work since I’m not too familiar with principles of color perception for wide-band light signals, but if it were pulled off it would certainly be a more robust solution than only considering fundamental frequency (which is also prone to more errors than straight-up spectral analysis)
First thought for a mapping: red = 20Hz, Violet = 20KHz, logarithmic in between. This might, however, not yield a sufficient enough contrast between C3 and D3 to perceive. If this is the case, a potential solution is a modular map. 11Hz = Red, 100Hz = Violet, 101Hz = Red, 1KHz = Violet. 1.001KHz = Red, 10KHz = violet; you get the idea. How that would play into the spectral content color mixing I’m not really sure.
The goal is not necessarily to be able to read the pitch accurately from the color of the waveform, but to just be able to tell by inspection when each singer goes from A to G, so that all of the singers (or instruments) can be aligned by inspection even when onset / offset / change in intensity cues are not present or obvious.
Has anyone seen an idea like this in implementation? Any ideas on how the spectral content color-mixing could work?
With a decent spectrogram in logarithmic resolution instead of the waveform, this would be solved.
However the idea of detecting the pitch and displaying with a color is interesting. It should work for single-tone instruments like singers or bass when there’s not too much interference or noise. For complexer environments there will be problems.
OT: How can i upload a picture of such a spectrogram?
using color in this way is patented, and the patent still has many years to run.
if you don’t like that, please write to your elected representative. if you don’t have an elected representative, please organize a coup and overthrow your repressive regime.
Damn, that’s bad news… Really seemed a cool idea.
Btw patented doesn’t necessarily mean fees need to be paid, but you probably have already checked that
Email thread with comparisonics included below: it seems they might not be so keen on the idea of it going open-source. I guess the issue then becomes what does the patent cover exactly and what can be implemented in Ardour without violating it? Alternatively, how long do Patents last? Any legally-knowledgeable users on the forum?
Andrew,
Thanks for the link. I had not heard of this project. Our software is
proprietary, not open-source. Since it represents a major investment of
time and money, we must keep it proprietary.
Steve
----- Original Message -----
From: "Andrew Schwartz" <ozydingo@gmail.com>
- Hide quoted text -
To: "Steve Rice" <inquiry@comparisonics.com>
Sent: Monday, May 07, 2007 11:15 AM
Subject: Re: Regarding the patent on colored waveforms
> Hi Steve,
>
> Thank you for your prompt reply--I will most certainly be writing to
> Digidesign to request they reconsider this as a useful feature!
> However I am also interested in seeing a more widespread adoption; so
> I repose the question of what would have to be done to legally allow
> such a scheme to be used in an open-source project such as Ardour
> (www.ardour.org)?
>
> Many thanks,
> Andrew
>
> On 5/7/07, Steve Rice <inquiry@comparisonics.com> wrote:
>> Hi Andrew,
>>
>> Thank you for your inquiry. Yes, we have a patent on the technique of
>> coloring the waveform display to represent the frequency content of a
>> signal. We do have a free audio player that uses it (see the
>> Comparisonics
>> Audio Player at http://www.comparisonics.com/CSplayer.html), a free Web
>> search engine that uses it (see FindSounds.com at
>> http://www.findsounds.com), and an "almost free" audio
>> player/recorder/editor/search-engine that uses it (see FindSounds Palette
>> at
>> http://www.findsounds.com/palette.html). You will also find it in the
>> Sequoia and Samplitude DAWs from Magix (see
>> http://www.samplitude.com/eng/seq/).
>>
>> Approximately 300,000 users per month visit FindSounds.com, and see and
>> benefit from the colored waveform displays. Although Digidesign and
>> other
>> DAW manufacturers have been aware of our technologies since 1998, they
>> have
>> chosen to ignore us. We get frequent requests from ProTools users in
>> particular for the colored waveform, but we cannot put it into ProTools
>> without Digidesign's involvement. I encourage you to contact Digidesign
>> to
>> voice your support for this feature.
>>
>> Sincerely,
>> Steve Rice
>> Comparisonics Corp.
>>
>>
>> ----- Original Message -----
>> From: "Andrew Schwartz" <ozydingo@gmail.com>
>> To: <inquiry@comparisonics.com>
>> Sent: Monday, May 07, 2007 10:34 AM
>> Subject: [SPAM] Regarding the patent on colored waveforms
>>
>>
>> > Greetings! I have a simple inquiry:
>> >
>> > What does one have to do in order to use your patented
>> > waveform-coloring scheme? Is there a fee / royalty involved? I pose
>> > this question as someone interested in seeing this in implementation
>> > in DAWs in the hopefully near future.
>> >
>> > As a frequent editor of collegiate a cappella music; ie many many
>> > tracks in need of alignment which often lack easily distinguishable
>> > onset / envelope cues in the waveform, I have long held the idea that
>> > coloring waveforms by spectral content would be a great innovation for
>> > multitrack editors. I proposed the idea on the open source DAW
>> > Ardour.org forum, and a response came rather rapidly pointing me to
>> > comparisonics' web site. So I naturally am interested in what one
>> > would have to do to incorporate this idea into either open source or
>> > commercial software.
>> >
>> > Persaonlly I use Pro Tools for my work. But I proposed it first to
>> > Ardour out of fear that if it was an unexploited idea, Digidesign
>> > would attempt to patent it and prevent other major DAWs from adopting
>> > it. I would have attempted to program it myself, but my programming
>> > skills are not quite up to par enough to get noticed by the bigs. I'm
>> > sure you folks at comparisonics share my passion for music and the
>> > music editing community, and agree that this remarkable idea could
>> > seriously benefit the field of music production if not restricted and
>> > confined in its legal uses. You could also probably agree that
>> > neither open source for lack of budget nor big guys like Pro Tools for
>> > lack of need would pay fees to use this scheme. I say allows its use
>> > freely--under the condition, perhaps, that comparisonics be credited
>> > properly. As comparisonics grows, being officially endorsed by
>> > Digidesign, Steinberg, Cakewalk, and more could be huge!
>> >
>> > I eagerly await your response. Best wishes,
>> > Andrew Schwartz
>> >
>>
>>
>
I don’t quite see what “invention” there is patented. Getting pitch out of sound is quite some work but getting number=pitch to color is a no brainer.
As i can understand that we don’t want to consult any lawyers on this one, i think if we really need that feauture, there is a way to work around it. At least in countries where the patent system makes any sense.
Maybe painting the background in the color instead of the waveform is enough, maybe a seperate bar. But i don’t know nothing besides that patents on ideas quite suck.
The basic claim is basically a method for drawing a waveform wherein the “foreground pixels” are drawn with a color “that depends on frequency-dependent [sic] information”
Challengeability of the patent based on the obvious step to represent pitch with color aside, it seems the patent does not cover a system wherein the background pixels are drawn with a color that depends on frequency information of the foreground signal. A separate bar might be a little more elegant though, since it will never be obstructed by the waveform. I think it’s fair game, as long as it’s implemented before another patent comes out!
It’s astonishing that while the claims are clearly designed to cover all variations, they are so in love with the word “foreground”.
But I guess it comes down to: Paul Davis can not and should not afford a lawsuit over this silly patent.
Now, if there was a coder in a country with no software patents (extending to the foreseeable future - better outside the EU), and the colouring could be done by some kind of plugin …