The disk system on your computer was not able to keep up with Ardour.

Hey, dear Ardour community.

I’m encountering the following problem when I try to record with Ardour:
“The disk system on your computer was not able to keep up with Ardour.”
This happens after a few minutes while recording 8 tracks simultaneously via the Edirol FA-101 Firewire-Interface. (no plugins or effects added)

I’m using UbuntuStudio 9.10 with 2.6.31-9-rt Kernel and Ext-3 filesystem.
My Machine is a Lenovo N200 Notebook with 1.8Ghz Dualcore CPU, 2Gb RAM and 4200RPM SATA Harddrive.
hdparm says the following about the disk:
Timing cached reads: 1588 MB in 2.00 seconds = 794.00 MB/sec
Timing buffered disk reads: 160 MB in 3.02 seconds = 52.90 MB/sec

As I’ve read in other posts I’ve found via Google (the search engine in this forum doesn’t work), SATA drives don’t seem to have the necessity of running in DMA mode. (this was one of the suggestions I’ve already found)
Additionally, I already added myself to the “audio” group and edited the limits.conf -file (rtprio:99, memlock, nice:-19).
I tried using Freebob and FFADO with no success.

The issue occurs quite randomly. Sometimes after just a few minutes, sometimes after 20 minutes of recording. Once I “faked” the input signal on all 8 tracks (because I didn’t have 8 mics to plug into the FW-Interface) by connecting the outputs from the Vlc-player to each of the 8 Inputs via Jack and started recording with Ardour. This test-run went right through, 40 minutes long, until I manually stopped it.

Thank you very much in advance for any suggestions.

Greetings,
Steffen

My Machine is a Lenovo N200 Notebook with 1.8Ghz Dualcore CPU, 2Gb RAM and 4200RPM SATA Harddrive. hdparm says the following about the disk: Timing cached reads: 1588 MB in 2.00 seconds = 794.00 MB/sec Timing buffered disk reads: 160 MB in 3.02 seconds = 52.90 MB/sec

Your test results there are certainly false, that is faster than SSDs, and you are running your OS WITH writing to a very likely, very slow, laptop disk. I would bet your problem happens when either swapping occurs, or it needs to read information from other parts of the disk to load programs(cron etc.) which vastly lowers your performance.

There is a setting in your conf you can use to increase buffered reading and writing in Ardour to help prevent this, but it will only help you so much.(And I don’t remember it off hand but if you kick me here I can likely find it later).

Seablade

Make sure your filesystems are mounted with the “noatime” option. That can make a big difference. It prevents the OS from updating the access time on each file it reads.

4200RPM SATA Harddrive.
Your test results there are certainly false, that is faster than SSDs, and you are running your OS WITH writing to a very likely, very slow, laptop disk

I made a mistake there: It’s a 5400RPM Western Digital Scorpio (it’s not the disk that came along with the notebook) which seems to be not that bad regarding performance.
Anyway, I’ve seen people being troubled by the same issue on their desktop pcs even though they had faster harddisks. So, I thought that this issue is not mainly due to the performance of the harddisk but must have something to do with the configuration of Ardour and/or the OS.
Another point that speaks for a not-so-well-configured Ardour is that Audacity runs fine recording these 8 tracks simultaneously. (Since I only record these tracks without doing anything else at that time, both applications do the same job, don’t they?) Is Ardour using a different filetype than Audacity that needs more performance or something like that?

I am thankful for every suggestion I can get, so it would be a nice thing if you could find that option you’ve mentioned (buffered reading).

Make sure your filesystems are mounted with the "noatime" option. That can make a big difference. It prevents the OS from updating the access time on each file it reads.

I’ll try that one, thanks! Now that you’ve mentioned it I’ve found several websites that recommend this option to increase performance.

…damn this is so frustrating cause it’s like looking for a needle in a bundle of hay…

I made a mistake there: It's a 5400RPM Western Digital Scorpio (it's not the disk that came along with the notebook) which seems to be not that bad regarding performance. Anyway, I've seen people being troubled by the same issue on their desktop pcs even though they had faster harddisks. So, I thought that this issue is not mainly due to the performance of the harddisk but must have something to do with the configuration of Ardour and/or the OS.

Well one thing, as I mentioned above, is you are using the OS and SWAP on the same HD as you are writing to, which SEVERELY degrades performance any time somewhere else on the disk is accessed by the OS or swap, which is more often than you would think.

The second thing is no matter what spindle based HD you are using, those read numbers you posted are completely false and I suspect the write numbers are as well. You may be able to get that with a large RAID array, or off SLC based solid state disks(REALLY Expensive), but not off a single laptop based spindle HD. I am not saying that you are making them up mind you, what I am saying is what is being reported is wrong and pretty much impossible for a single spindle based 2.5" hard drive to accomplish.

Third thing, the setting I was thinking of is in ~/.ardour2/ardour.rc and is the one that reads…

Option name="track-buffer-seconds" value="XX"

Replace the XX with the desired value, NOTE this WILL affect seek times in a session as it buffers every time you seek.

Another point that speaks for a not-so-well-configured Ardour is that Audacity runs fine recording these 8 tracks simultaneously. (Since I only record these tracks without doing anything else at that time, both applications do the same job, don’t they?) Is Ardour using a different filetype than Audacity that needs more performance or something like that?

No Ardour is designed to write the audio to disk as fast as possible by default and expects you to have a fast enough disk system to do this. When running your OS, swap, and data off the same disk, this doesn’t work out so well.

  Seablade

seablade, are you sure his hdparm numbers are wrong?
I tested my Compaq nx9110 3.2GHz with an IBM Hitachi 60GB 4200RPM 8MB drive and get about half his speed; 367 and 21 (tried both with Ubuntu 9.10s hdparm and a freshly compiled one).

I also tested my AMD 2000+ with a Barracuda 160GB 7200 RPM 8MB and get 260/63.

Both disks are PATA.

MaxDamage, since the recording works when you generate the signals internally, from VLC, I’d look into the Edirol for the problem.
You shold be using FFADO since the FA-101 is fully supported by it.

Here are some pointers :
https://help.ubuntu.com/community/FireWire

The second thing is no matter what spindle based HD you are using, those read numbers you posted are completely false and I suspect the write numbers are as well.

I’m not so sure the figures are wrong. I get cached read speeds of around 1GB/sec and buffered disk reads of around 75 MB/sec on several of my 7200rpm drives according to hdparm. Performance of my system seems to bear this out. I also have another system with SSDs which has substantially faster buffered disk read speeds than these, so the figures look quite plausible to me. I suspect that the problem is somewhere else.

What firewire chipset is on the motherboard?

I get cached read speeds of around 1GB/sec and buffered disk reads of around 75 MB/sec on several of my 7200rpm drives according to hdparm.

Yet these numbers are effectively meaningless when it comes to actual testing useful scenarios for our purposes of multitrack audio. You need extended throughput testing(So uncached for one thing, and over a long period of time), and you need to look at the MINIMUM available at any given time to determine the numbers that apply to our purposes.

A better and more accurate test at the very least is going to be using dd and copying from /dev/random or similar to disk to test writing values. I can pretty well guarantee that on a 5400 RPM laptop HD with the os also running on the drive, you will be seeing nowhere close to those numbers. To give examples…

http://www.tomshardware.com/charts/2009-2.5-mobile-hard-drive-charts/h2benchw-3.12-Min-Write-Throughput,1114.html

That is the MIN Write performance of some of the top HDs out. Note that even now they cap at 50MB/s. Likewise on the reading side…

http://www.tomshardware.com/charts/2009-2.5-mobile-hard-drive-charts/h2benchw-3.12-Min-Read-Throughput,1111.html

Similar story. Certainly NOT the 700+ MB/s reported in that post, and likely be hdparm because you need to worry about uncached minimums, when dealing with a disk system error like was asked about.

Heck just for grins and giggles, look at the minimum write speeds for a velociraptor desktop drive here…

http://www.tomshardware.com/charts/3.5-hard-drive-charts/Minimum-WriteTransfer-Performance,669.html

These are the numbers that matter to the error in question. NOT Cached or Buffered Read or Write.

Add on top of this that running the OS on the same drive will mean your best performance for this drive will be destroyed in most cases(Unless your OS runs completely from memory and you never use SWAP, which requires a significant amount of custom tuning even on Linux) you are in actuality looking at MUCH lower numbers that matter for the purpose of this test.

Hopefully that explains it a bit better.

  Seablade

@peder

The Edirol would not cause a disk system error like is being reported. Only the disk not being able to keep up with Ardour’s requests for reading/writing would cause this. It is much more likely that there are random processes causing the spindle to move to other areas of the disk and it is not able to return and write out what Ardour is requesting at a fast enough time to keep up because of this. This is why it is a MUCH better idea to write to another disk all together than to read or write from the OS disk.

  Seablade

Hi Steffen,

The larger figure is meaningless, but 54MB/s is what I’d expect from a laptop disk. Fast SSDs are in the 200MB/s range. These are best case scenario read speeds. In the real world, any disk seeking will significantly reduce the performance for rotational disks.

I’ve fought this particular situation before, here is a list of things you might try, in no particular order:

  • Use an external drive, eSATA would be best, followed by FW800, FW400, USB 2.0
  • Use the realtime kernel and rtirq
  • Make sure the CPU governer has all CPUs set to maximum performance
  • If you have a big session try consolidating some of the tracks to avoid seeking (or get an SSD)
  • Delete or deactivate (not mute) any tracks you don’t need
  • Turn off plugins when recording
  • If you have formatted the disk with XFS, use xfs_fsr to defragment the disk (this is a good reason for using XFS or any filesystem that allows you to defragment it)

Some of the above are about maximising CPU, but I’ve seen ardour produce this error when the CPU gets so overloaded that it can’t service the disk. This is more likely if your external disk is USB 2.0 which needs more CPU that eSATA or Firewire.

Nick

Just for the sake of clarity, one more time, the read and write numbers are completely false in terms of measuring hard drive performance for this purpose.

I realized I didn’t clarify I meant for this purpose initially. As I mentioned above, you ened to look at the minimum read and write numbers, and given that there are many other processes reading and writing from disk at the same time, you will get substantially lower numbers for the minimums the moment that another area of the disk gets accessed.

  Seablade

And Nick got all the standard advice dead on:)

 Seablade

I have to disagree that the speed figures are completely meaningless. The cached read speed is the speed of a contiguous data read from the cache on the hard disk, not the disk itself, so it’s not really useful here. The buffered disk read speed is for contiguous data reads from the disk and is at least indicative of the performance that can be expected in a recording system, even if it isn’t an exact measurement.

In practice a recording system will be unable to achieve those speeds continuously because the heads spend a lot of time moving back and forth to different areas of the disk (unless you’re using an SSD) which will slow it down considerably, but I still believe that a higher reading from hdparm will indicate a higher obtainable read/write speed in a recording situation.

Back to the OP’s problem, I’d be a little surprised if a disk giving the hdparm figures quoted is the bottleneck here, unless the head positioning servo is faulty.

Another thing: how full is the file system? Use df to check. If there isn’t a lot of free space it can get badly fragmented, which will really slow things down.

The buffered disk read speed is for contiguous data reads from the disk and is at least indicative of the performance that can be expected in a recording system.

And has nothing to do with write speed either which is what is needed for recording data to the disk, which is where the OP is having problems supposedly.

In practice a recording system will be unable to achieve those speeds continuously because the heads spend a lot of time moving back and forth to different areas of the disk (unless you're using an SSD) which will slow it down considerably, but I still believe that a higher reading from hdparm will indicate a higher read/write speed in a recording situation.

It will indicate a higher performance in some regard but it will not be linear in any way, so it is effectively meaningless.

Back to the OP's problem, I'd be a little surprised if a disk giving the hdparm figures quoted is the bottleneck here, unless the head positioning servo is faulty.

You already mentioned what is likely the problem, if the heads are going back and forth over different areas of the drive, which happens MUCH more frequently when running the OS and Swap off the same drive, then performance is going to take a nosedive very quickly, and not linerally either. You will be getting a fraction of the performance as a result. This is why a single drive for the OS and data is not usually preferred. Can it be done? Certainly, in fact I did it for a long time, but I wouldn’t depend on it without some custom tweaking as was already mentioned above. It will be unreliable, sometimes working great and sometimes seemingly not working at all, and all it takes is that one time in a recording session to screw with things.

As I mentioned above, increase that setting to an appropriate amount that provides a balance between performance of not getting that error, and seek time in the session. Follow Nick’s suggestions is preferable in most cases.

    Seablade
Another thing: how full is the file system? Use df to check. If there isn't a lot of free space it can get badly fragmented, which will really slow things down.

This I do agree with however;)

 Seablade
if the heads are going back and forth over different areas of the drive, which happens MUCH more frequently when running the OS and Swap off the same drive, then performance is going to take a nosedive very quickly, and not linerally either. You will be getting a fraction of the performance as a result. This is why a single drive for the OS and data is not usually preferred.
Agreed :-)

To add to Nick’s comments, it can be beneficial to copy all of the data to another drive and then back again every once in a while if you’re using ext3. I do this whenever I notice fsck reporting more than a few percent of discontiguous data on my audio file system (which is on a different disk than the OS). Heavy editing in particular can cause a lot of fragmentation even if there’s plenty of space left on the disk.

Heavy editing in particular can cause a lot of fragmentation even if there’s plenty of space left on the disk Editing in Ardour doesn’t cause any disk fragmentation whatsoever. Nothing gets repeatedly written to disk except the session file, the history file and (probably) some backup session files.