Cannot seek to / Butler write behind

Installation & Configuration Linux
#1

Hello all, I hope this is something with a solution that I have just been unable to find…

After recording for a certian (long) amount of time, Ardour stops recording with errors like this:

[ERROR]: /media/Ichor/Ardour/9-15-14_jack2_44.1/interchange/9-15-14_jack2_44.1/audiofiles/Audio 1-2.w64: cannot seek to -2147483648 (libsndfile error: No Error.)
[ERROR]: AudioDiskstream 121: cannot write to disk
[ERROR]: Butler write-behind failure on dstream Audio 1

[ERROR]: AudioDiskstream 121: cannot write to disk
[ERROR]: AudioDiskstream “Audio 1”: cannot flush captured data to disk!
[ERROR]: /media/Ichor/Ardour/9-15-14_jack2_44.1/interchange/9-15-14_jack2_44.1/audiofiles/Audio 2-2.w64: cannot seek to -2147483648 (libsndfile error: No Error.)

I see this has come up back in 2004, and there was something mentioned about a patch:

http://lists.ardour.org/pipermail/ardour-users-ardour.org/2004-March/013601.html

Then it came up again in 2005 with no responses:

http://lists.ardour.org/pipermail/ardour-users-ardour.org/2005-February/014859.html

Looking at the info from multiple tests I have run; recording at 96Khz, I was able to record tracks that were 6:12:50:09 long. That matches up pretty well with the following math:

$ expr 2147483648 / 96000
22369
$ var/devel/utils/s2h/s2h 22369 # utility to convert seconds to hours
0y 0d 6h 12m 49s

Given the math at the CLI, I would guess that Ardour or libsndfile or jack2 or something is marking off timeslices with a long int and is just running out of numbers to keep track of time slices. It seems that this something that switching to a long long int would fix.

Is this anything people have ran into recently? Is there a “quick fix”, or at least a patch that I could apply and recompile Ardour with?

I get the same behavior writing in wav or w64 format, I am just now running a test in CAF format. I expect the same behavior. Those tests were all with 32-bit floating point sample format. I have not yet seen if it makes a difference in 24 or 16 bit integer format.

Background: Sometimes I’ll start recording stuff throughout the day and just leave all the mics on as people come and go and various music or jams happen. At 44.1Khz I am able to record for 13:35:05:25. I guess that is long enough for casual jam stuff and I could just restart things @96Khz when I am recording studio stuff, but it would be nice to be able to record longer at 96K.

In case any of this is important:

Hardware:
Intel® Core™2 Duo CPU E6550 @ 2.33GHz
8 GB RAM @800MHz

Presonus Firestudio Project

OS: Arch linux
$ uname -a
Linux av-arch-home 3.14.12-rt9-1-rt #1 SMP PREEMPT RT Sun Sep 7 11:13:32 EDT 2014 x86_64 GNU/Linux

Ardour3.5.378
jackdmp 1.9.10
libsndfile 1.0.25-3

#2

where did you get ardour from?

#3

just for clarification, ardour uses 64 bit file sizes everywhere it can, and libsndfile does the same. .wav file format cannot exceed the 2GB file size limit inherent in the format. .w64 should work and has been tested in the past. it is possible for packagers to mess this up by building ardour and/or libsndfile incorrectly.

#4

Paul,

Thanks for the quick response; I installed ardour from the arch repos, IE:
sudo pacman -S ardour

I have tried recording things in w64 format as well as CAF and both ran into the same problem with the write-behind errors and such.

Here are what Ardour and libsndfile look like as they are installed currently:

$ sudo pacman -Qi ardour
Name : ardour
Version : 3.5.380-1
Description : Professional-grade digital audio workstation
Architecture : x86_64
URL : http://ardour.org/
Licenses : GPL
Groups : None
Provides : None
Depends On : liblrdf liblo>=0.28 libsmf lilv aubio libgnomecanvasmm suil
Optional Deps : xjadeo: video monitoring
Required By : None
Optional For : None
Conflicts With : None
Replaces : None
Installed Size : 60342.00 KiB
Packager : Rashif Rahman (Ray) schiv@archlinux.org
Build Date : Wed May 14 14:31:30 2014
Install Date : Sun Sep 7 13:37:56 2014
Install Reason : Explicitly installed
Install Script : Yes
Validated By : Signature

$ sudo pacman -Qi libsndfile
Name : libsndfile
Version : 1.0.25-3
Description : A C library for reading and writing files containing sampled sound
Architecture : x86_64
URL : http://www.mega-nerd.com/libsndfile
Licenses : LGPL
Groups : None
Provides : None
Depends On : alsa-lib flac libvorbis
Optional Deps : None
Required By : gstreamer0.10-bad-plugins libpulse libsamplerate vamp-plugin-sdk zita-resampler
Optional For : lv2
Conflicts With : None
Replaces : None
Installed Size : 858.00 KiB
Packager : Eric Belanger eric@archlinux.org
Build Date : Thu Oct 24 16:20:24 2013
Install Date : Sun Sep 7 11:46:42 2014
Install Reason : Installed as a dependency for another package
Install Script : No
Validated By : Signature

It would seem that both of them are 64bit, but I may get the source and poke through the default config to see if anything is amiss.

I’ll update if I find anything interesting.

#5

It would be more productive to try out the version from ardour.org (free demo version is fine).

#6

I just downloaded the free version and installed it; I am running a test now to see if I have the same problem there. Should know in about 6 hours and 12 minutes; I’ll update when I get home from work.

#7

I popped into my system via SSH and it seems like it has stopped recording at the same data size as before:

Last test with Ardour3 free installed from ardour.org:

$ pwd
/media/Ichor/Ardour/9-17-14_Ardour3_free_test/interchange/9-17-14_Ardour3_free_test/audiofiles

$ ls -ltr | sed -r ‘s/ w.+t//g’
total 50331720
-rw-r–r-- 1 8589934736 Sep 17 14:34 Audio 1-1.w64
-rw-r–r-- 1 8589934736 Sep 17 14:34 Audio 2-1.w64
-rw-r–r-- 1 8589934736 Sep 17 14:34 Audio 3-1.w64
-rw-r–r-- 1 8589934736 Sep 17 14:34 Audio 4-1.w64
-rw-r–r-- 1 8589934736 Sep 17 14:34 Audio 5-1.w64
-rw-r–r-- 1 8589934736 Sep 17 14:34 Audio 6-1.w64

$ sndfile-info Audio\ 1-1.w64

Version : libsndfile-1.0.25

========================================
File : Audio 1-1.w64
Length : 8589934736
riff : 8589934736
wave
fmt : 48
Format : 0x3 => WAVE_FORMAT_IEEE_FLOAT
Channels : 1
Sample Rate : 96000
Block Align : 4
Bit Width : 32
Bytes/sec : 384000
fact : 32
frames : 2147483648
data : 8589934616

Sample Rate : 96000
Frames : 2147483648
Channels : 1
Format : 0x000B0006
Sections : 1
Seekable : TRUE
Duration : 06:12:49.621

previous test, having reinstalled ardour and coreutils (libsndfile):

$ pwd
/media/Ichor/Ardour/9-16-15_reinstalled_ardour_and_coreutils/interchange/9-16-15_reinstalled_ardour_and_coreutils/audiofiles

[whofferbert@av-arch-home audiofiles]$ ls -ltr | sed -r ‘s/ w.+t//g’
total 50331720
-rw-r–r-- 1 8589934736 Sep 17 08:18 Audio 4-1.w64
-rw-r–r-- 1 8589934736 Sep 17 08:18 Audio 1-1.w64
-rw-r–r-- 1 8589934736 Sep 17 08:18 Audio 2-1.w64
-rw-r–r-- 1 8589934736 Sep 17 08:18 Audio 3-1.w64
-rw-r–r-- 1 8589934736 Sep 17 08:18 Audio 6-1.w64
-rw-r–r-- 1 8589934736 Sep 17 08:18 Audio 5-1.w64

$ sndfile-info Audio\ 1-1.w64

Version : libsndfile-1.0.25

========================================
File : Audio 1-1.w64
Length : 8589934736
riff : 8589934736
wave
fmt : 48
Format : 0x3 => WAVE_FORMAT_IEEE_FLOAT
Channels : 1
Sample Rate : 96000
Block Align : 4
Bit Width : 32
Bytes/sec : 384000
fact : 32
frames : 2147483648
data : 8589934616

Sample Rate : 96000
Frames : 2147483648
Channels : 1
Format : 0x000B0006
Sections : 1
Seekable : TRUE
Duration : 06:12:49.621

I cannot say for ceratin without forwarding x over ssh, and I do not want to spend too much time on this while I am still at work, but I would bet that when I get home, I will find the same set of errors as I found before using Ardour from the Arch repositories. According to sndfile-info, both files ‘frames’ value are at the signed long int limit, 2147483648.

My suspicion is that something in Ardour or libsndfile is using long int instead of perhaps a long long int to count frames, and hitting that cap just causes things to fail. I understand that my use-case is not necessarily standard, but I would guess that there are probable people out there who ran into things like this before when recording for long amounts of time.

Thanks

#8

Your suspicion is not based on reading the code or knowing that going beyond 32 bit file sizes was something we focused on more than a decade ago.

However, your bugs are real and look like a subtle regression.

#9

Paul,

You are correct, I have not looked into the code yet. I do not mean to offend or cause problems; I called my thought a suspicion for a reason, namely the persistent recurrence of the 2147483648 number, leading me to what I guessed at.

If there is anything constructive I can do to help here, like setting some verbose or debug logging for Ardour, or anything else, please let me know. In the mean time, I will continue poking at things on my side and update if I find anything worth mentioning.

#10

What filesystem type are you recording to?

#11

ext4. I could format a disk to ext3 or some other FS and run a test recording to that, if it would give you any additional insight

$ pwd
/media/Ichor/Ardour
$ mount | grep Ichor
/dev/sde1 on /media/Ichor type ext4 (rw,noatime,data=ordered)

#12

no, ext4 should be OK. 32 or 64 bit Linux system?

#13

64 bit…

Kernel: 64 bit PREEMPT RT kernel.
$ uname -a
Linux av-arch-home 3.14.12-rt9-1-rt #1 SMP PREEMPT RT Sun Sep 7 11:13:32 EDT 2014 x86_64 GNU/Linux

8GB RAM
$ free -m
total used free shared buffers cached
Mem: 7985 6310 1675 84 222 3531
-/+ buffers/cache: 2556 5429
Swap: 4094 0 4094

Core2 Duo E6550
$ cat /proc/cpuinfo | grep “name”
model name : Intel® Core™2 Duo CPU E6550 @ 2.33GHz
model name : Intel® Core™2 Duo CPU E6550 @ 2.33GHz

Are there any other specifics you might want with regards to how the system or harware is configured?

#14

I suspect that the problem is that libsndfile is not being compiled with -D_FILE_OFFSET_BITS=64 even in the binary bundle we distribute. The Ardour code itself is built this way, so the error has to be in some other layer, and I think libsndfile is almost certainly it. I will attempt to put together a bundle soon-ish to test this.

#15

actually, this seems unlikely since libsndfile explicitly checks that sizeof (off_t) is the same as sizeof (sf_count_t). the latter is libsndfile’s own type for counting samples and is always a 64 bit signed integer. and on my 64 bit system, off_t is a 64 bit value also. so i am still puzzled about where the error can come from. Ardour 3 uses 64 bit integer types exclusively for counting.

#16

Thank you for the insight. I can test this again on a fresh/different system; perhaps there is something wrong with the upstream repo that libsndfile had been pulled from in arch? I’d be willing to set that up and see what happens. Think I should give this a shot on a regular Ubuntu distro, or try building coreutils/libsndfile from a source other than the repos?

#17

no, if it fails with the demo from ardour.org then it has nothing to do with your system. the bundles from ardour.org are entirely self-contained except for the C library and X Window libraries.

#18

And here is a link explaining why you don’t need 96kHz anyway…
https://xiph.org/~xiphmont/demo/neil-young.html

#19

Andreas,

Thank you for the interesting link to read. Informative, and while I may have questions and opposing viewpoints, here is not the place to ask or state them.

96KHz recording or not, the problem remains that at any sampling rate; there is a bug where too many frames or time chunks causes recording to fail. The same happens at 44.1KHz and 48KHz.

#20

billshire: just for the record, there has been no convincing refutation of Monty’s paper since he first published it. The Ardour forums would be a perfectly fine place to discuss it, though it probably warrants a new thread. It has been discussed elsewhere, including Hydrogen Audio.

I regret that I do not have time right now to look into the issue with the sample count. I hope I can find time soon.