I am interested in hearing from the development team exactly what the philosophy of the development group as a whole is with respect to automated unit testing - going forward at least, but I’d be interested in hearing about any history behind the current testing strategy. I am very curious about this, as a musician, recent subscriber, and as a software developer with years of experience who is interested in possibly contributing some of my time
http://webchat.freenode.net/?channels=ardour
http://ardour.org/development.html
alot of your questions could probably be answered in the ardour irc if you can get in at the right time when the dev’s are about.
Ive popped in to ardour irc and had a few questions answered when was having problems and ive heard the dev’s discuss bugs and issues on there while working on the code
We have no automated testing strategy to speak of. The current “tests” in the source tree are very limited in scope.
For reference, a full automated test of ProTool apparently takes nearly 1 week to execute.
There are no plans to add more testing. I am a skeptic of testing strategies. When done well, they are invaluable and fantastic. When done poorly (which happens often), they are simply a source of more bugs to fix and more time wasted. Testing an application like Ardour is a very challenging task. Testing it in a cross-platform fashion simply adds to the fun.
oh well, at least you’re being honest about it. (i’m hoping to find a less public way to let you know what else I think…not that it’s all bad, but I’m afraid I can be pretty easily misunderstood, and my own experiences have not been in a development environment such as yours)…anyway, I haven’t really used the stuff enough yet to make any final judgement as to whether I can keep trying to use it)
See the links veda_sticks posted, IRC is generally the best place for development discussions, though I will warn you this particular one has happened at least twice that I know of, and probably far more that paul knows of:)
SeabladeI am a skeptic of testing strategies...
@Paul: It might be interesting to note - for amusement if nothing else - that (during some recent work on Mac / Windows) my first VST3 plugin passed all of the 19000 or so automated unit tests in the SDK’s validation tool…
and then failed to load in the first (non Steinberg) application I tried it with.
That particular issue wasn’t too complex as it turned out, and in no way reflects on the quality of the tests as such, but, I think it illustrates something about automated test generally even in a relatively simple context. It certainly wouldn’t have been practical to rely on that as the only test…
1 Like
first off before I go away forever, I have to say “thanks” for the effort; I am grateful that what’s here is here…that much said:
a. who was it who said anything about automatically testing down to every last feature? it shouldn’t be an all or nothing thing, should it?
b. it’s obvious that you guys have already made up your minds; so no point in me wasting any more of my time.
c. good luck(you’re going to need it)