consider busybox-static for nest

[jrollins]

busybox-static is a single binary with no library dependencies. if we
could get the busybox cpio to not fail to unpack hardlinks, we could
use this instead and save quite a bit of clutter in the nest.

[jrollins]

I've looked into this some more. First of all, it appears that the ​cpio issue has been fixed upstream. However, busybox-static is debian is more than a year out of date (1.1 vs 1.9). On top of that, the busybox-static package in debian conflicts with busybox, but does not provide busybox, and is therefore uninstallable on any system where you need to have initrd-tools. The busybox maintainer is really asleep at the wheel. we need to file a bug report on this stuff.

[dkg]

i dunno if you saw ​the exchange about this in the debian bug tracker.

It lead me to ​the minutes from the debian-installer team meeting on 2007-09-26, which in turn lead me to examine Bastian Blank (waldi)'s corner of the d-i svn repo:

[0 dkg@squeak ~]$ mkdir -p ~/src/busybox/d-i
[0 dkg@squeak ~]$ cd ~/src/busybox/d-i
[0 dkg@squeak d-i]$ svn co -q svn://svn.debian.org/d-i/people/waldi/packages/busybox
[0 dkg@squeak d-i]$ head busybox/debian/changelog 
busybox (1:1.9.1-1) experimental; urgency=low

  * New upstream release.
  * Drop unused busybox-floppy-udeb.

 -- Bastian Blank <waldi@debian.org>  Fri, 15 Feb 2008 16:25:31 +0000

busybox (1:1.1.3-5) unstable; urgency=low

  * debian/config*:
[0 dkg@squeak d-i]$ 

Seems relevant here.

[dkg]

fwiw, i was able to build new busybox packages as a non-privileged user like this:

wget 'http://busybox.net/downloads/busybox-1.9.1.tar.gz'
ln -s busybox-1.9.1.tar.gz busybox_1.9.1.orig.tar.gz
tar xzf busybox-1.9.1.tar.gz
cd busybox-1.9.1
svn export -q svn://svn.debian.org/d-i/people/waldi/packages/busybox/debian
debuild -uc -us

[dkg]

Sadly, the new version of busybox doesn't seem to handle hardlinks either:

[0 dkg@squeak cdtemp.Ex1403]$ mkdir a
[0 dkg@squeak cdtemp.Ex1403]$ touch a/b
[0 dkg@squeak cdtemp.Ex1403]$ ln a/b a/c
[0 dkg@squeak cdtemp.Ex1403]$ mkdir z
[0 dkg@squeak cdtemp.Ex1403]$ find a | cpio -H newc --create  | (cd z && ./busybox-1.9.1/debian/busybox/bin/busybox cpio -i)
1 block
1 blocks
cpio: a/b not created: cannot resolve hardlink
cpio: a/c not created: cannot resolve hardlink
[0 dkg@squeak cdtemp.Ex1403]$ 

[dkg]

whoops, broken paste above:

(cd z && ./busybox-1.9.1/debian/busybox/bin/busybox cpio -i)

should have been:

(cd z && ../busybox-1.9.1/debian/busybox/bin/busybox cpio -i)

[jrollins]

So busybox 1.9.1-3 is in unstable now. I haven't tried it yet, but I really hope the cpio issue is fixed. I guess I never filed a bug about that issue because I thought it was fixed upstream. Why is it not fixed in the Debian package if it was fixed upstream? Is it not really fixed upstream?

However, we still need ​#375568 fixed before we can use it. I don't understand why that can't fix that issue. It seems so small.

[dkg]

You can use ​375568 to link to debian bugs here.

You can still fetch the package and extract the static executable for testing, though.

It looks like the hardlinks are still busted. Here's my test:

[130 dkg@squeak cdtemp.Z17720]$ aptitude download busybox-static
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Reading extended state information      
Initializing package states... Done
Building tag database... Done      
Get:1 http://ftp.us.debian.org testing/main busybox-static 1:1.1.3-5 [723kB]
Fetched 723kB in 1s (475kB/s)          
[0 dkg@squeak cdtemp.Z17720]$ dpkg -x busybox-static_1%3a1.1.3-5_i386.deb  ./
[0 dkg@squeak cdtemp.Z17720]$ mkdir a
[0 dkg@squeak cdtemp.Z17720]$ touch a/b
[0 dkg@squeak cdtemp.Z17720]$ ln a/b a/c
[0 dkg@squeak cdtemp.Z17720]$ mkdir z
[0 dkg@squeak cdtemp.Z17720]$ find a | cpio -H newc --create | (cd z && ../bin/busybox cpio -i)
Using fallback suid method
1 block
1 blocks
cpio: TRAILER!!! not created: cannot resolve hardlink
cpio: (null) not created: cannot resolve hardlink
*** glibc detected *** ../bin/busybox: double free or corruption (fasttop): 0x083c66e8 ***
======= Backtrace: =========
[0x80cd33c]
[0x80d062b]
[0x805177f]
======= Memory map: ========
08048000-08187000 r-xp 00000000 00:12 1425507    /tmp/cdtemp.Z17720/bin/busybox
08187000-0818a000 rw-p 0013e000 00:12 1425507    /tmp/cdtemp.Z17720/bin/busybox
0818a000-083e7000 rw-p 0818a000 00:00 0          [heap]
b7e00000-b7e21000 rw-p b7e00000 00:00 0 
b7e21000-b7f00000 ---p b7e21000 00:00 0 
b7f0d000-b7f0e000 rw-p b7f0d000 00:00 0 
bfb12000-bfb27000 rw-p bfb12000 00:00 0          [stack]
ffffe000-fffff000 r-xp 00000000 00:00 0          [vdso]
[134 dkg@squeak cdtemp.Z17720]$ 

[dkg]

Whoops! that was still with the old version of busybox-static.

Here's the new version:

[0 dkg@squeak cdtemp.q17932]$ aptitude download busybox-static/unstable
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Reading extended state information      
Initializing package states... Done
Building tag database... Done      
Get:1 http://ftp.us.debian.org unstable/main busybox-static 1:1.9.1-3 [729kB]
Fetched 729kB in 1s (593kB/s)          
[0 dkg@squeak cdtemp.q17932]$ dpkg -x busybox-static_1%3a1.9.1-3_i386.deb .
[0 dkg@squeak cdtemp.q17932]$ mkdir a
[0 dkg@squeak cdtemp.q17932]$ touch a/b
[0 dkg@squeak cdtemp.q17932]$ ln a/b a/c
[0 dkg@squeak cdtemp.q17932]$ mkdir z
[0 dkg@squeak cdtemp.q17932]$ find a | cpio -H newc --create | (cd z && ../bin/busybox cpio -i)
1 block
Using fallback suid method
1 blocks
cpio: a/b not created: cannot resolve hardlink
cpio: a/c not created: cannot resolve hardlink
[0 dkg@squeak cdtemp.q17932]$ 

So it looks like it still doesn't work :(

[dkg]

Still fails with busybox-static version 1:1.9.2-1, unfortunately.

[jrollins]

So we were wrong on a couple of counts:

busybox-static *is* installable on lenny without trashing initramfs-tools. I'm not sure why I had thought that it wasn't. initramfs-tools does not depend on busybox.

busybox-static cpio works to create hard links as long as the target is non-empty:

servo:/tmp/cdtemp.h19624 0$ mkdir foo
servo:/tmp/cdtemp.h19624 0$ echo test > foo/a
servo:/tmp/cdtemp.h19624 0$ ln foo/a foo/b
servo:/tmp/cdtemp.h19624 0$ mkdir bar
servo:/tmp/cdtemp.h19624 0$ find foo | cpio -H newc --create | (cd bar && busybox cpio -i)
1 block
1 blocks
servo:/tmp/cdtemp.h19624 0$ busybox | head -1
BusyBox v1.9.2 (2008-03-26 11:21:16 UTC) multi-call binary
servo:/tmp/cdtemp.h19624 0$ 

This is still a ​466771, but it should work for us since we aren't hard linking to empty files.

The question I have is: should we now depend on busybox-static instead of busybox? This would greatly simplify the create_initrd_nested.

[dkg]

I think we should Depend: busybox-static | busybox (possibly as a versioned dependency, so that people don't try to use debirf with the old version that crashes on unpack.

should work for us since we aren't hard linking to empty files.

What makes you say that we aren't hard linking empty files anywhere in the filesystem? Are there guarantees you ca provide about that which cover anything in the entire filesystem?

[jrollins]

Replying to dkg:

I think we should Depend: busybox-static | busybox (possibly as a versioned dependency, so that people don't try to use debirf with the old version that crashes on unpack.

I think we can go ahead and do this, but...

What makes you say that we aren't hard linking empty files anywhere in the filesystem? Are there guarantees you ca provide about that which cover anything in the entire filesystem?

I also think you're right with this and we should wait until we use the busybox cpio until this bug has been fixed upstream.

[jrollins]

cpio finally got fixed in busybox and works fine in 1.14.2, which is in squeeze. I changed the nest init to just use the busybox cpio in r1295, and it works fine. Consequently, with busybox-static the nest is now nice and tight!