FFT visualizing

8 replies [Last post]
rotoquezada
User offline. Last seen 6 years 52 weeks ago. Offline
Joined: 2006-07-02
Posts:

I've just got to finally compile Ardour2 on my gentoo (for Gentoo users a Hint: emerge dev-libs/glib-2.10.3, get also the pro-audio layman repository, easy as a cake) and tried the FFT view, nonethless is a work in progress it seems quite useful to me ... anyway, as the developer wanted ideas ... well, there's this guy who has wrote the greatest analysis program I've ever seen:

Sonic Visualizer: http://www.sonicvisualiser.org/index.html

This has the greatest FFT views I've ever seen and also a plugin architecture for Visualizing plugins (VAMP), and there are a bunch of ready-to-go visualizing plugins already, spectral views, note name views, etc. Why reinvent the wheel, do you think is there someway to embed this as the "Analize" function in ardour2? perhaps another Idea would be to add support for VAMP plugins for waveform views and analysis views, it just have to be seen how is their performance.

An idea I saw in this visualizer that shouldn'be that difficult to add in ardour is the possibility of annotate the waveforms ... I mean metadata as you're editing is not that bad, isn't it? doesn't the WAVE format support this in a kind of metadata chunk??

_________________________________________________________
It's not 'bout recording ... It's 'bout freezing time ...

sampo
User offline. Last seen 10 weeks 4 days ago. Offline
Joined: 2006-03-16
Posts:

THe visualizing plugins seem like a good idea. I must take a closer look at what they provide and how they are used.

cannam
User offline. Last seen 6 years 7 weeks ago. Offline
Joined: 2006-08-01
Posts:

Vamp is an API for non-realtime plugins in C or C++ that take audio as input and return something other than audio as output. The outputs can be moderately complex, e.g. two or three dimensional irregularly sampled floating point data.

Vamp is not a format for "visualisation plugins", in the sense that the plugins themselves don't get to draw anything -- they just return data that the host determines how to display.

Also, the format is aimed at analysis more than visualisation in the sense that it's far from optimal for extracting large quantities of data (e.g. a high-resolution spectrogram) from audio, but more appropriate to lower volume slightly structured data (e.g. note onset points, pitch tracking curves, tonal histograms). It is possible to write a spectrogram Vamp plugin (you can get one from the Mazurka project), but it isn't terribly efficient. (Sonic Visualiser's own spectrogram view does not use a plugin.)

Natural applications for Vamp plugins in Ardour would include things like beat and onset detection, audio-to-MIDI, alternatives to waveforms for visualisation (e.g. colours or curves based on timbral content), key estimation, etc.

Vamp has documentation and a C++ SDK, and I'd be happy to help out in any way if you're interested in using it.

Chris

sampo
User offline. Last seen 10 weeks 4 days ago. Offline
Joined: 2006-03-16
Posts:

Also thanks for pointing out that they are not "visualization" plugins. I did assume that they didn't actually present the data themselves (how silly would that be?). I'm quite sure one day we'll happen to be in irc (#ardour or #lad) at the same time and we can discuss this a bit further.

cannam
User offline. Last seen 6 years 7 weeks ago. Offline
Joined: 2006-08-01
Posts:

Well, not necessarily all that silly. Wavesurfer for example has "plugins" (in Tcl) that do actually handle display and user interaction.

Chris

rotoquezada
User offline. Last seen 6 years 52 weeks ago. Offline
Joined: 2006-07-02
Posts:

Sampo, Chris, if you get a way to embed sonic-analizer in ardour or at least use the visualizer ideas for audio data display in Ardour that would be a quantum leap for linux Audio, really. The waveform pattern as means for visualizing audio data is just not enough if you're getting deep into processing, and doesn't cover all needs.

Another Idea would be to link'em both, visualizer and ardour, using JACK transport, if the visualizer can handle a "not so big" window in real time it could be fed with ardour tracks just to have another point of view on what's happening. (I dream of a better world with dual headed xinerama, ardour on one monitor and sonic-visualizer or some other visualizer tool on the second monitor fed live with ardour sends ... with this in mind it's been a time that I'm thinkin' on hacking extace in order to make it work with JACK ... it's a real pity that baudline doesn't do JACK :( )

______________________________________________________
It's not about recording, it's about freezing time ...

sampo
User offline. Last seen 10 weeks 4 days ago. Offline
Joined: 2006-03-16
Posts:

We discussed this on #ardour and came to the conclusion that the VAMP plugins don't really fit into the composite analysis model of the FFT analysis window. We could write VAMP plugins which would work with the analysis window, but they wouldn't be usable in sonic visualizer or vice versa. :(

And what comes to visualizing tracks in Ardour, that might come one day but it's not really Ardours' function.

Piping the output of Ardours' tracks to sonic visualizer and syncing them via the transport could offer interesting possibilities though. Is this something that you, Chris, are interesed in?

cannam
User offline. Last seen 6 years 7 weeks ago. Offline
Joined: 2006-08-01
Posts:

Sonic Visualiser will probably get JACK transport support pretty soon -- it would be a handy feature generally.

What's much less likely to happen in the near future is real-time analysis or recording. Vamp plugins are almost by definition not real-time safe (strictly speaking there's no reason an actual plugin can't be, but it's seldom practical for significant plugins and in any case the C++ wrapper in the SDK is not RT safe) and SV expects to operate on audio file data.

You could load recorded files from your Ardour session into SV and sync the two with JACK transport, but only if the recordings are aligned with frame zero in Ardour (SV has no way to set an offset for the start of an audio clip, although that might not be hard to add, or else it could be made to apply an offset as part of the JACK transport sync support). Also, SV has no way to merge Ardour's one-per-channel audio files.

I'm definitely open to ideas for how to make SV more useful in conjunction with other tools -- the more generic (and easier to implement!) the better, of course.

Chris

sigblip
sigblip's picture
User offline. Last seen 7 years 24 weeks ago. Offline
Joined: 2006-11-03
Posts:

> it’s a real pity that baudline doesn’t do JACK

baudline 1.04 was released recently and it now has JACK support:

http://www.baudline.com/news.html#baudline_1.04