autopkgtest¶
The autopkgtest package runs DEP-8 tests.
This page mostly aims at running these tests manually within a (E)LTS VM, for efficiently debugging, fixing and designing tests. This is useful when salsa-ci/ci.debian.net/ci.freexian.com fails at the end of a lengthy process with no debug access.
We focus on tooling available in stable releases, to ensure the commands won’t break following a regression testing/unstable.
See also https://wiki.debian.org/ContinuousIntegration/autopkgtest
Local run¶
This runs tests directly in the current host. This is enough for most tests and easiest to debug.
This complements working within a LTS VM for fine-grained patch-by-patch testing.
In your test VM, examples/cheat-sheet with stretch+ syntax:
cd package-version/
apt-get build-dep .
debuild
apt install autopkgtest
# additional automatic/ecosystem tests:
# https://manpages.debian.org/unstable/autodep8/autodep8.1.en.html
apt install autodep8
autopkgtest ../..._amd64.changes -- null
# save the test logs (but no more colors):
autopkgtest -o ../test.outdir ../..._amd64.changes -- null
# test local debian/tests/ changes, without auto-rebuilding
autopkgtest -B . -- null
# run only selected test
autopkgtest --test-name=TEST ...
# run only selected testS (>=bookworm)
autopkgtest --test-name=TEST1 --test-name=TEST2 ...
# ensure you run test with a non-root user
autopkgtest -u dla ...
# run a reverse dependency's tests, simply:
autopkgtest rdep_source_package_name -- null
TODO: can we pass DEB_BUILD_OPTIONS=xxx?
Full LXC environment (isolation-container)¶
This is for tests that would break the VM configuration (such as
/etc). It is easy to create LXC containers from within a VM,
here’s how.
In your test VM, example with apache2, new (stretch+) syntax.
LXC setup (buster and above):
apt install autopkgtest
apt install lxc lxc-templates debootstrap rsync distro-info
systemctl status lxc-net
# If necessary:
cat >> /etc/lxc/default.conf <<'EOF'
lxc.apparmor.profile = unconfined
EOF
# Create base container
# Add security updates to better match target environment
DIST=bullseye
cat <<EOF > ~/apt-security.sh
#!/bin/bash
echo 'deb http://security.debian.org/debian-security $DIST-security main' >> /etc/apt/sources.list
echo 'deb-src http://security.debian.org/debian-security $DIST-security main' >> /etc/apt/sources.list
apt update
apt -y upgrade
EOF
autopkgtest-build-lxc debian $DIST amd64 ~/apt-security.sh
TODO: adjust sources for buster/ELTS.
LXC setup (stretch):
apt install autopkgtest
apt install lxc debootstrap rsync ebtables
# Create network
apt install libvirt-daemon-system
# edit /etc/libvirt/qemu/networks/default.xml if needed (IP range...)
virsh net-autostart default
service libvirtd restart
cat > /etc/lxc/default.conf <<'EOF'
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = virbr0
EOF
# Create base container
# Add security updates to better match target environment
cat <<'EOF' > ~/apt-security.sh
#!/bin/bash
echo 'deb http://security.debian.org/debian-security stretch/updates main' >> /etc/apt/sources.list
echo 'deb-src http://security.debian.org/debian-security stretch/updates main' >> /etc/apt/sources.list
apt update
apt -y upgrade
EOF
autopkgtest-build-lxc debian stretch amd64 ~/apt-security.sh
Testing the package (buster and above):
DIST=bullseye
cd apache2-2.4.38/
apt-get build-dep .
debuild
# Test the package you're working on
autopkgtest --shell-fail ../apache2_2.4.38-3+deb10u9_amd64.changes \
-- lxc autopkgtest-$DIST-amd64
# Caveat: tests packages *from the archive* (not your new .deb):
#autopkgtest -B . -- lxc autopkgtest-$DIST-amd64
# use this instead:
autopkgtest -B . ../*.deb -- lxc autopkgtest-$DIST-amd64
Testing the package (stretch):
cd apache2-2.4.25/
apt-get build-dep apache2
debuild
# Test the package you're working on
autopkgtest ../apache2_2.4.25-3+deb9u11_amd64.changes \
-- lxc autopkgtest-stretch-amd64
# Caveat: tests packages *from the archive* (not your new .deb):
#autopkgtest -B . -- lxc autopkgtest-stretch-amd64
# use this instead:
autopkgtest -B . ../*.deb -- lxc autopkgtest-stretch-amd64
Full VM environment (isolation-machine)¶
If you really need isolation-machine (which can break the VM,
reboot it, etc.), or if you’d rather work from your host rather than
within a LTS VM, you’ll need to proceed differently.
First, create an autopkgtest VM image:
RELEASE=bullseye # or stretch, buster...
sudo autopkgtest-build-qemu --size=25G \
$RELEASE ./autopkgtest-$RELEASE.img
If you have an apt-cacher (note: 127.0.0.1 special-cased and works within qemu, through 10.0.2.2):
--apt-proxy=http://127.0.0.1:3142 \
For non-Debian (e.g. ELTS) dists, also pass its mirror and keyring (requires autopkgtest >= 5.29, cf. bookworm-backports; additionally trixie/5.49 has a bug when building buster so use >= 5.50, cf. trixie-backports):
--keyring=./freexian-archive-extended-lts.gpg \
--mirror=http://deb.freexian.com/extended-lts/ \
Sometimes the resulting image is truncated (< 1MB), try re-running the command.
Then, run the tests using the qemu backend (probably from outside
your LTS VM, unless you have support for nested virtualization):
autopkgtest --shell-fail --apt-upgrade [...] \
-- qemu /path/to/autopkgtest-$RELEASE.img
e.g.:
cd python2.7/
autopkgtest -sU \
. ../previous-run/binaries/*.deb -o ../out \
-- qemu ../../autopkgtest-buster.img
- mmdebstrap: starting with trixie,
mmdebstrapprovides mmdebstrap-autopkgtest-build-qemuas an alternative/replacement.
Snippet for ELTS build:
mmdebstrap-autopkgtest-build-qemu \
--boot=efi --size=25G \
--apt-proxy=http://127.0.0.1:3142 \
--keyring=/world-writable/freexian-archive-extended-lts.gpg \
--mirror=http://deb.freexian.com/extended-lts/ \
$RELEASE ./autopkgtest-$RELEASE.img
Snippet for SSH setup:
mmdebstrap-autopkgtest-build-qemu --sshkey=/pass/to/key.pub ...
Troubleshooting
As usual, if working from a source directory, clean-up it up first (especially big files), as autopkgtest will copy it to its VM.
If the VM status becomes [paused] (io-error) for no reason, check if /tmp is filled, especially if it’s a tmpfs (default in trixie).
When facing a bug, the maintainers are pretty responsive. In the team, jochensp and paride among others contributed to the toolchain.
To investigate an issue in the tests, remember the -s|--shell-fail
option, which here will just pause for you to access the VM through
serial and SSH (see setup below), actually available on boot.
Serial access:
minicom -D unix#/tmp/autopkgtest-qemu.xxx/ttyS0
minicom -D unix#/tmp/autopkgtest-qemu.xxx/ttyS1
minicom -D unix#/tmp/autopkgtest-qemu.xxx/monitor # QEMU monitor
# also try removing -nographics in /usr/share/autopkgtest/lib/autopkgtest_qemu.py
SSH access: autopkgtest redirects port 10022 to local SSH, but you
need it installed through autopkgtest-build-qemu --script=...:
cat <<'EOF' > ./install-openssh-server.sh
#!/bin/sh
# Executed in the host, remember to use chroot
chroot "$1" apt-get -o "Acquire::http::Proxy=$AUTOPKGTEST_SETUP_APT_PROXY" \
install -y openssh-server < /dev/null
chroot "$1" mkdir -m 700 /root/.ssh/
cp -a /home/you/.ssh/yourkey.pub "$1/root/.ssh/authorized_keys"
EOF
chmod 755 ./install-openssh-server.sh
autopkgtest-build-qemu ... --script=./install-openssh-server.sh
autopkgtest ... -- qemu autopkgtest-$RELEASE.img
ssh -p 10022 -o StrictHostKeyChecking=no root@127.0.0.1
To investigate an issue in a disk image:
qemu-img convert autopkgtest-$RELEASE.img autopkgtest-$RELEASE.raw
sudo kpartx -av autopkgtest-$RELEASE.raw
sudo mount /dev/mapper/loop0p1 /mnt/inspection/
...
sudo umount /mnt/inspection
sudo kpartx -dv autopkgtest-$RELEASE.raw
If for some reason things don’t work, an alternative is the ssh
backend to connect to an existing VM, see also
/usr/share/autopkgtest/ssh-setup/ to handle reboot & reset.
Last resort, make a snapshot of your VM and try running with the
null autopkgtest backend.
Local APT archive¶
The null driver can be confused by e.g. conflicting binary packages from the same source package, in which case you need to setup a local APT archive for your packages:
cd /usr/src/mysourcepackage/
apt-ftparchive packages . > Packages
apt-ftparchive release . > Release
echo 'deb [trusted=yes] file:///usr/src/mysourcepackage ./' >> /etc/apt/sources.list
apt update
Alternatively, for a single machine:
apt install local-apt-repository
mkdir /srv/local-apt-repository
cp -a *.deb /srv/local-apt-repository/
apt update