8

I'm working on a software that builds Pacman packages (which basically are tarballs with some special metadata files). The test suite builds some packages, then compares the resulting package to a recorded expected result.

One of the fields in the metadata recorded in the package is the installed size of the package, determined by running du -s --apparent-size on the root directory before tar'ing it.

All of this works perfectly fine on my local Arch Linux boxes where I develop. The packages, including their installed size in bytes (not even kilobytes, bytes!) is reproduced exactly every time I run the test.

Now I've also enabled this test on Travis, where it runs (as far as I understand from the Travis docs) on an Ubuntu-12.04-based container. There, the test passes most of the times. Most of the times. Sometimes, it calculates installed sizes that are off by 80-99%.

Here is an example of a test that fails: https://travis-ci.org/holocm/holo/builds/89326780 (The test just before that succeeded.) One of the relevant diffs is

@@ -37,7 +37,7 @@
             pkgdesc = my foo bar package
             url = 
             packager = Unknown Packager
-            size = 37728
+            size = 1464
             arch = any
             license = custom:none
             replaces = foo-bar<2.1

The puzzling thing about this is that it only happens some of the time, with no apparent pattern. The test arranges the same files as it always does, runs du -s --apparent-size on the resulting tree, and arrives at a completely wrong result. I have tried to reproduce this on a Ubuntu 12.04 VM, and while I have seen it appear there once or twice, I could not see any patterns emerge there either that would help me reproduce the problem.

Maybe someone here has an idea what could cause this issue?

EDIT: Oh, there is one pattern that I observed, actually. du runs once for each testcase. When it fails for the first testcase, it will fail for all testcases in this run.

6
  • 1
    To clarify, all entries in the filesystem tree in question are plain contiguous files, symlinks and directories. There are no sparse files. There are no device files, FIFOs, or any other funky business. Nov 4 '15 at 23:04
  • What are the filesystems? Nov 5 '15 at 1:34
  • @Mark Wagner: It's AUFS, according to docs.travis-ci.com/user/workers/container-based-infrastructure Nov 5 '15 at 8:33
  • Some Possibilities: (1) size reported are correct, but in some cases there are some files stale remaining from other operations (2) Not sure of AUFS, but in NFS, deleted stale files will be renamed .nfsNNNNNNNN and these might count for the size inconsistency. How are you sure that the sizes reported are incorrect ? Can you try du on the individual SubDirectories and Files, so that the exact location of inconsistency can be checked ?
    – Prem
    Jan 13 '16 at 15:31
  • 1
    The problem you have is AUFS.... check the problems associated with it, check the reasons it's not in the latest kernels, check it's "stability", check it's "POSIX completeness".
    – Hvisage
    Jan 24 '16 at 20:55
1

Well, I've been prompted to put this as an answer by @derobert

The problem you have is AUFS.... check the problems associated with it, check the reasons it's not in the latest kernels, check it's "stability", check it's "POSIX completeness". – Hvisage Jan 24 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.