autopkgtest

The autopkgtest package runs DEP-8 tests.

This page mostly aims at running these tests manually within a (E)LTS VM, for efficiently debugging, fixing and designing tests. This is useful when salsa-ci/ci.debian.net/ci.freexian.com fails at the end of a lengthy process with no debug access.

As for 2025-02 bookworm is Debian Stable, and we still support jessie in ELTS, so we focus on stable tooling and document jessie. When trixie is released and jessie EOL, we can upgrade the tooling and drop jessie documentation.

See also https://wiki.debian.org/ContinuousIntegration/autopkgtest

Local run

This runs tests directly in the current host. This is enough for most tests and easiest to debug.

This complements working within a LTS VM for fine-grained patch-by-patch testing.

In your test VM, examples/cheat-sheet with stretch+ syntax:

cd package-version/
apt-get build-dep .
debuild

apt install autopkgtest
# additional automatic/ecosystem tests:
# https://manpages.debian.org/unstable/autodep8/autodep8.1.en.html
apt install autodep8

autopkgtest ../..._amd64.changes -- null
# save the test logs (but no more colors):
autopkgtest -o ../test.outdir ../..._amd64.changes -- null
# test local debian/tests/ changes, without auto-rebuilding
autopkgtest -B . -- null
# run only selected test
autopkgtest --test-name=TEST ...
# run only selected testS (>=bookworm)
autopkgtest --test-name=TEST1 --test-name=TEST2 ...
# ensure you run test with a non-root user
autopkgtest -u dla ...

# run a reverse dependency's tests, simply:
autopkgtest rdep_source_package_name -- null

In your test VM, example with rails, old (jessie) syntax:

cd rails-4.1.8/
apt-get build-dep rails
debuild

apt install autopkgtest
apt install ruby-spring ruby-jquery-rails ruby-sqlite3 ruby-sass-rails ruby-uglifier ruby-coffee-rails ruby-turbolinks ruby-jbuilder ruby-sdoc
apt install ruby-byebug ruby-web-console # stretch
# TODO: autopkgtest should install dependencies automatically, possibly those weren't due to APT::Install-Recommends "false", or it's a bug somewhere

adt-run -B ./ --- null
adt-run ../rails_4.1.8-1+deb8u7_amd64.changes --- null

TODO: can we pass DEB_BUILD_OPTIONS=xxx?

Full LXC environment (isolation-container)

This is for tests that would break the VM configuration (such as /etc). It is easy to create LXC containers from within a VM, here’s how.

In your test VM, example with apache2, new (stretch+) syntax.

LXC setup (buster and above):

apt install autopkgtest
apt install lxc lxc-templates debootstrap rsync ebtables
# Create network
apt install libvirt-daemon-system dnsmasq-base
virsh net-edit default  # edit if needed (IP range...)
virsh net-autostart default
service libvirtd restart
cat > /etc/lxc/default.conf <<'EOF'
lxc.net.0.type = veth
lxc.net.0.flags = up
lxc.net.0.link = virbr0
lxc.apparmor.profile = unconfined
EOF

# Create base container
# Add security updates to better match target environment
DIST=bullseye
cat <<EOF > ~/apt-security.sh
#!/bin/bash
echo 'deb http://security.debian.org/debian-security $DIST/updates main' >> /etc/apt/sources.list
echo 'deb-src http://security.debian.org/debian-security $DIST/updates main' >> /etc/apt/sources.list
apt update
apt -y upgrade
EOF
autopkgtest-build-lxc debian $DIST amd64 ~/apt-security.sh

TODO: use lxcbr0 which is now created by LXC.

LXC setup (stretch):

apt install autopkgtest
apt install lxc debootstrap rsync ebtables
# Create network
apt install libvirt-daemon-system
# edit /etc/libvirt/qemu/networks/default.xml if needed (IP range...)
virsh net-autostart default
service libvirtd restart
cat > /etc/lxc/default.conf <<'EOF'
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = virbr0
EOF

# Create base container
# Add security updates to better match target environment
cat <<'EOF' > ~/apt-security.sh
#!/bin/bash
echo 'deb http://security.debian.org/debian-security stretch/updates main' >> /etc/apt/sources.list
echo 'deb-src http://security.debian.org/debian-security stretch/updates main' >> /etc/apt/sources.list
apt update
apt -y upgrade
EOF
autopkgtest-build-lxc debian stretch amd64 ~/apt-security.sh

Testing the package (buster and above):

DIST=bullseye
cd apache2-2.4.38/
apt-get build-dep .
debuild

# Test the package you're working on
autopkgtest --shell-fail ../apache2_2.4.38-3+deb10u9_amd64.changes \
  -- lxc autopkgtest-$DIST-amd64
# Caveat: tests packages *from the archive* (not your new .deb):
#autopkgtest -B . -- lxc autopkgtest-$DIST-amd64
# use this instead:
autopkgtest -B . ../*.deb -- lxc autopkgtest-$DIST-amd64

Testing the package (stretch):

cd apache2-2.4.25/
apt-get build-dep apache2
debuild

# Test the package you're working on
autopkgtest ../apache2_2.4.25-3+deb9u11_amd64.changes \
  -- lxc autopkgtest-stretch-amd64
# Caveat: tests packages *from the archive* (not your new .deb):
#autopkgtest -B . -- lxc autopkgtest-stretch-amd64
# use this instead:
autopkgtest -B . ../*.deb -- lxc autopkgtest-stretch-amd64

Testing the package (jessie syntax):

adt-build-lxc debian jessie amd64
# TODO: apply security updates in adt-jessie-amd64
adt-run ../apache2_2.4.10-10+deb8u19_amd64.changes \
  --- lxc adt-jessie-amd64

Full VM environment (isolation-machine)

If you really need isolation-machine (which can break the VM, reboot it, etc.), or if you’d rather work from your host rather than within a LTS VM, you’ll need to proceed differently.

First, create an autopkgtest VM image:

RELEASE=bullseye  # or jessie, stretch, buster...
sudo autopkgtest-build-qemu --size=25G \
  $RELEASE ./autopkgtest-$RELEASE.img

If you have an apt-cacher (note: 127.0.0.1 special-cased and works within qemu, through 10.0.2.2):

--apt-proxy=http://127.0.0.1:3142 \

For non-Debian (e.g. ELTS) dists, also pass its mirror and keyring (requires autopkgtest >= 5.29, cf. bookworm-backports):

--keyring=./freexian-archive-extended-lts.gpg \
--mirror=http://deb.freexian.com/extended-lts/ \

Note: this was tested on bookworm and trixie/202502. Sometimes the resulting image is truncated (< 1MB), try re-running the command.

TODO: these autopkgtest Debian VMs don’t have security sources.

Then, run the tests using the qemu backend (probably from outside your LTS VM, unless you have support for nested virtualization):

autopkgtest --shell-fail --apt-upgrade [...] \
  -- qemu /path/to/autopkgtest-$RELEASE.img

e.g.:

cd python2.7/
autopkgtest -sU \
  . ../previous-run/binaries/*.deb -o ../out \
  -- qemu ../../autopkgtest-buster.img

Troubleshooting

As usual, if working from a source directory, clean-up it up first (especially big files), as autopkgtest will copy it to its VM.

If the VM status becomes [paused] (io-error) for no reason, check if /tmp is filled, especially if it’s a tmpfs (default in trixie).

When facing a bug, the maintainers are pretty responsive. In the team, jochensp and paride among others contributed to the toolchain.

To investigate an issue in the tests, remember the -s|--shell-fail option, which here will just pause for you to access the VM through serial and SSH (see setup below), actually available on boot.

Serial access:

minicom -D unix#/tmp/autopkgtest-qemu.xxx/ttyS0
minicom -D unix#/tmp/autopkgtest-qemu.xxx/ttyS1
minicom -D unix#/tmp/autopkgtest-qemu.xxx/monitor # QEMU monitor
# also try removing -nographics in /usr/share/autopkgtest/lib/autopkgtest_qemu.py

SSH access: autopkgtest redirects port 10022 to local SSH, but you need it installed through autopkgtest-build-qemu --script=...:

cat <<'EOF' > ./install-openssh-server.sh
#!/bin/sh
# Executed in the host, remember to use chroot
chroot "$1" apt-get -o "Acquire::http::Proxy=$AUTOPKGTEST_SETUP_APT_PROXY" \
  install -y openssh-server < /dev/null
chroot "$1" mkdir -m 700 /root/.ssh/
cp -a /home/you/.ssh/yourkey.pub "$1/root/.ssh/authorized_keys"
EOF
chmod 755 ./install-openssh-server.sh

autopkgtest-build-qemu ... --script=./install-openssh-server.sh

autopkgtest ... -- qemu autopkgtest-$RELEASE.img
ssh -p 10022 -o StrictHostKeyChecking=no root@127.0.0.1

To investigate an issue in a disk image:

qemu-img convert autopkgtest-$RELEASE.img autopkgtest-$RELEASE.raw
sudo kpartx -av autopkgtest-$RELEASE.raw
sudo mount /dev/mapper/loop0p1 /mnt/inspection/
...
sudo umount /mnt/inspection
sudo kpartx -dv autopkgtest-$RELEASE.raw

If for some reason things don’t work, an alternative is the ssh backend to connect to an existing VM, see also /usr/share/autopkgtest/ssh-setup/ to handle reboot & reset.

Last resort, make a snapshot of your VM and try running with the null autopkgtest backend.

mmdebstrap: starting with trixie, mmdebstrap provides

mmdebstrap-autopkgtest-build-qemu as an alternative/replacement.

Snippet for SSH setup:

mmdebstrap-autopkgtest-build-qemu ... -- \
  --include=openssh-server \
  --customize-hook='mkdir -m700 -p "$1/root/.ssh"' \
  --customize-hook="upload /pass/to/key.pub /root/.ssh/authorized_keys"

Local APT archive

adt-run (jessie) can be confused by e.g. conflicting binary packages from the same source package, in which case you need to setup a local APT archive for your packages:

cd /usr/src/mysourcepackage/
apt-ftparchive packages . > Packages
apt-ftparchive release . > Release
echo 'deb [trusted=yes] file:///usr/src/mysourcepackage ./' >> /etc/apt/sources.list
apt update

See for instance nginx.

Backporting

stretch -> jessie:

• $AUTOPKGTEST_TMP -> $ADTTMP

• build-essential (e.g. gcc) not installed by default, re-add it if necessary

Copyright (C) 2021, 2022, 2023, 2024, 2025 Sylvain Beucler