===========
autopkgtest
===========

The *autopkgtest* package runs `DEP-8 <https://dep-team.pages.debian.net/deps/dep8/>`_ tests.

This page mostly aims at running these tests manually within a (E)LTS
VM, for efficiently debugging, fixing and designing tests. This is
useful when salsa-ci/ci.debian.net/ci.freexian.com fails at the end of
a lengthy process with no debug access.

*As for 2025-07 bookworm is Debian Stable, so we focus on stable
tooling. When trixie is released, we can upgrade the tooling.*

See also https://wiki.debian.org/ContinuousIntegration/autopkgtest

.. contents:: :depth: 2

---------
Local run
---------

This runs tests directly in the current host. This is enough for most
tests and easiest to debug.

This complements working within a LTS VM for fine-grained
patch-by-patch testing.

In your test VM, examples/cheat-sheet with stretch+ syntax:

.. code-block:: bash

 cd package-version/
 apt-get build-dep .
 debuild
 
 apt install autopkgtest
 # additional automatic/ecosystem tests:
 # https://manpages.debian.org/unstable/autodep8/autodep8.1.en.html
 apt install autodep8
 
 autopkgtest ../..._amd64.changes -- null
 # save the test logs (but no more colors):
 autopkgtest -o ../test.outdir ../..._amd64.changes -- null
 # test local debian/tests/ changes, without auto-rebuilding
 autopkgtest -B . -- null
 # run only selected test
 autopkgtest --test-name=TEST ...
 # run only selected testS (>=bookworm)
 autopkgtest --test-name=TEST1 --test-name=TEST2 ...
 # ensure you run test with a non-root user
 autopkgtest -u dla ...

 # run a reverse dependency's tests, simply:
 autopkgtest rdep_source_package_name -- null

TODO: can we pass DEB_BUILD_OPTIONS=xxx?

------------------------------------------
Full LXC environment (isolation-container)
------------------------------------------

This is for tests that would break the VM configuration (such as
``/etc``). It is easy to create LXC containers from within a VM,
here's how.

In your test VM, example with *apache2*, new (stretch+) syntax.

LXC setup (buster and above):

.. code-block:: bash

 apt install autopkgtest
 apt install lxc lxc-templates debootstrap rsync distro-info
 systemctl status lxc-net
 # If necessary:
 cat >> /etc/lxc/default.conf <<'EOF'
 lxc.apparmor.profile = unconfined
 EOF

 # Create base container
 # Add security updates to better match target environment
 DIST=bullseye
 cat <<EOF > ~/apt-security.sh
 #!/bin/bash
 echo 'deb http://security.debian.org/debian-security $DIST-security main' >> /etc/apt/sources.list
 echo 'deb-src http://security.debian.org/debian-security $DIST-security main' >> /etc/apt/sources.list
 apt update
 apt -y upgrade
 EOF
 autopkgtest-build-lxc debian $DIST amd64 ~/apt-security.sh

TODO: adjust sources for buster/ELTS.

LXC setup (stretch):

.. code-block:: bash

 apt install autopkgtest
 apt install lxc debootstrap rsync ebtables
 # Create network
 apt install libvirt-daemon-system
 # edit /etc/libvirt/qemu/networks/default.xml if needed (IP range...)
 virsh net-autostart default
 service libvirtd restart
 cat > /etc/lxc/default.conf <<'EOF'
 lxc.network.type = veth
 lxc.network.flags = up
 lxc.network.link = virbr0
 EOF

 # Create base container
 # Add security updates to better match target environment
 cat <<'EOF' > ~/apt-security.sh
 #!/bin/bash
 echo 'deb http://security.debian.org/debian-security stretch/updates main' >> /etc/apt/sources.list
 echo 'deb-src http://security.debian.org/debian-security stretch/updates main' >> /etc/apt/sources.list
 apt update
 apt -y upgrade
 EOF
 autopkgtest-build-lxc debian stretch amd64 ~/apt-security.sh

Testing the package (buster and above):

.. code-block:: bash

 DIST=bullseye
 cd apache2-2.4.38/
 apt-get build-dep .
 debuild

 # Test the package you're working on
 autopkgtest --shell-fail ../apache2_2.4.38-3+deb10u9_amd64.changes \
   -- lxc autopkgtest-$DIST-amd64
 # Caveat: tests packages *from the archive* (not your new .deb):
 #autopkgtest -B . -- lxc autopkgtest-$DIST-amd64
 # use this instead:
 autopkgtest -B . ../*.deb -- lxc autopkgtest-$DIST-amd64

Testing the package (stretch):

.. code-block:: bash

 cd apache2-2.4.25/
 apt-get build-dep apache2
 debuild

 # Test the package you're working on
 autopkgtest ../apache2_2.4.25-3+deb9u11_amd64.changes \
   -- lxc autopkgtest-stretch-amd64
 # Caveat: tests packages *from the archive* (not your new .deb):
 #autopkgtest -B . -- lxc autopkgtest-stretch-amd64
 # use this instead:
 autopkgtest -B . ../*.deb -- lxc autopkgtest-stretch-amd64

---------------------------------------
Full VM environment (isolation-machine)
---------------------------------------

If you really need ``isolation-machine`` (which can break the VM,
reboot it, etc.), or if you'd rather work from your host rather than
within a LTS VM, you'll need to proceed differently.

First, create an autopkgtest VM image:

.. code-block:: bash

  RELEASE=bullseye  # or stretch, buster...
  sudo autopkgtest-build-qemu --size=25G \
    $RELEASE ./autopkgtest-$RELEASE.img

If you have an apt-cacher (note: 127.0.0.1 special-cased and works
within qemu, through 10.0.2.2):

.. code-block:: bash

    --apt-proxy=http://127.0.0.1:3142 \

For non-Debian (e.g. ELTS) dists, also pass its mirror and `keyring
<https://www.freexian.com/lts/extended/docs/how-to-use-extended-lts/>`_
(requires autopkgtest >= 5.29, cf. bookworm-backports):

.. code-block:: bash

    --keyring=./freexian-archive-extended-lts.gpg \
    --mirror=http://deb.freexian.com/extended-lts/ \

Note: this was tested on bookworm and trixie/202502. Sometimes the
resulting image is truncated (< 1MB), try re-running the command.

TODO: these autopkgtest Debian VMs don't have security sources.

Then, run the tests using the ``qemu`` backend (probably from outside
your LTS VM, unless you have support for nested virtualization):

.. code-block:: bash

  autopkgtest --shell-fail --apt-upgrade [...] \
    -- qemu /path/to/autopkgtest-$RELEASE.img

e.g.:

.. code-block:: bash

  cd python2.7/
  autopkgtest -sU \
    . ../previous-run/binaries/*.deb -o ../out \
    -- qemu ../../autopkgtest-buster.img

**Troubleshooting**

As usual, if working from a source directory, clean-up it up first
(especially big files), as autopkgtest will copy it to its VM.

If the VM status becomes *[paused] (io-error)* for no reason, check
if /tmp is filled, especially if it's a tmpfs (default in trixie).

When facing a bug, the maintainers are pretty responsive. In the team,
jochensp and paride among others contributed to the toolchain.

To investigate an issue in the tests, remember the ``-s|--shell-fail``
option, which here will just pause for you to access the VM through
serial and SSH (see setup below), actually available on boot.

Serial access:

.. code-block:: bash

  minicom -D unix#/tmp/autopkgtest-qemu.xxx/ttyS0
  minicom -D unix#/tmp/autopkgtest-qemu.xxx/ttyS1
  minicom -D unix#/tmp/autopkgtest-qemu.xxx/monitor # QEMU monitor
  # also try removing -nographics in /usr/share/autopkgtest/lib/autopkgtest_qemu.py

SSH access: autopkgtest redirects port 10022 to local SSH, but you
need it installed through ``autopkgtest-build-qemu --script=...``:

.. code-block:: bash

  cat <<'EOF' > ./install-openssh-server.sh
  #!/bin/sh
  # Executed in the host, remember to use chroot
  chroot "$1" apt-get -o "Acquire::http::Proxy=$AUTOPKGTEST_SETUP_APT_PROXY" \
    install -y openssh-server < /dev/null
  chroot "$1" mkdir -m 700 /root/.ssh/
  cp -a /home/you/.ssh/yourkey.pub "$1/root/.ssh/authorized_keys"
  EOF
  chmod 755 ./install-openssh-server.sh

  autopkgtest-build-qemu ... --script=./install-openssh-server.sh

  autopkgtest ... -- qemu autopkgtest-$RELEASE.img
  ssh -p 10022 -o StrictHostKeyChecking=no root@127.0.0.1

To investigate an issue in a disk image:

.. code-block:: bash

  qemu-img convert autopkgtest-$RELEASE.img autopkgtest-$RELEASE.raw
  sudo kpartx -av autopkgtest-$RELEASE.raw
  sudo mount /dev/mapper/loop0p1 /mnt/inspection/
  ...
  sudo umount /mnt/inspection
  sudo kpartx -dv autopkgtest-$RELEASE.raw

If for some reason things don't work, an alternative is the ``ssh``
backend to connect to an existing VM, see also
``/usr/share/autopkgtest/ssh-setup/`` to handle reboot & reset.

Last resort, make a snapshot of your VM and try running with the
``null`` autopkgtest backend.

**mmdebstrap**: starting with trixie, ``mmdebstrap`` provides
 ``mmdebstrap-autopkgtest-build-qemu`` as an alternative/replacement.

Snippet for SSH setup:

.. code-block:: bash

 mmdebstrap-autopkgtest-build-qemu --sshkey=/pass/to/key.pub

-----------------
Local APT archive
-----------------

The null driver can be confused by e.g. conflicting binary packages
from the same source package, in which case you need to setup a local
APT archive for your packages:

.. code-block:: bash

 cd /usr/src/mysourcepackage/
 apt-ftparchive packages . > Packages
 apt-ftparchive release . > Release
 echo 'deb [trusted=yes] file:///usr/src/mysourcepackage ./' >> /etc/apt/sources.list
 apt update

Alternatively, for a single machine:

.. code-block:: bash

 apt install local-apt-repository
 mkdir /srv/local-apt-repository
 cp -a *.deb /srv/local-apt-repository/
 apt update


| Copyright (C) 2021, 2022, 2023, 2024, 2025  Sylvain Beucler