apt update fails on ubuntu:jammy unless --security-opt seccomp=unconfined is specified

[tnir]

apt update (or apt-get update) fails on ubuntu:jammy (ubuntu@sha256:7599221e4e2e4eeb90ad282fc523139578f3dfea245b5049c4fee1b28776d2be) if --security-opt seccomp=unconfined is not specified.

$ docker run ubuntu:jammy apt-get update
Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [90.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [90.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [90.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Fetched 20.2 MB in 2s (10.8 MB/s)                         
Reading package lists... Done
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code

or

$ docker run ubuntu:jammy apt update

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Get:1 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [90.7 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [90.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [90.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:6 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Fetched 20.2 MB in 2s (11.0 MB/s)
Reading package lists...
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code

[woky]

@tnir, it looks like you hit this issue: https://bugs.launchpad.net/cloud-images/+bug/1943049 It should be fixed in latest docker.io package in archives. Can you try to update?

[tnir]

@woky Thanks. Nothing is changed even with https://hub.docker.com/layers/ubuntu/library/ubuntu/latest/images/sha256-c27987afd3fd8234bcf7a81e46cf86c2c4c10ef06e80f0869c22c6ff22b29f9d?context=explore (linux/amd64) 🤔 :

$ docker run --rm ubuntu@sha256:c27987afd3fd8234bcf7a81e46cf86c2c4c10ef06e80f0869c22c6ff22b29f9d apt update

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [90.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [90.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [90.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Fetched 20.2 MB in 3s (5897 kB/s)
Reading package lists...
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code

[tianon]

You need to update Docker and libseccomp on your host (not just a newer Docker image).

[andrewcorrigan]

What versions are required of Docker and libseccomp? I'm hitting similar issues to the above using:

Docker version 20.10.14, build a224086
libseccomp-2.3.1-4.el7.x86_64

[tianon]

Definitely at least libseccomp 2.4.2 or newer, but possibly even as high as 2.5.0.

[tnir]

Even with libseccomp 2.5.1-1ubuntu1~20.04.2, it does not work to me:

$ docker version
Client: Docker Engine - Community
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:45:33 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:43:42 2021
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.4.13
  GitCommit:        9cc61520f4cd876b86e77edfeb88fbcd536d1f9d
 gitpod:
  Version:          1.1.0
  GitCommit:        v1.1.0-0-g067aaf85
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
$ dpkg -l libseccomp2
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name              Version                Architecture Description
+++-=================-======================-============-============================================
ii  libseccomp2:amd64 2.5.1-1ubuntu1~20.04.2 amd64        high level interface to Linux seccomp filter

[woky]

@tnir It seems you're running focal host with docker-ce package from https://docs.docker.com/engine/install/ubuntu/, not docker.io from Ubuntu archive, but you're not using the latest version. Your docker-ce is at 5:20.10.12~3-0~ubuntu-focal but the Docker archive already contains 5:20.10.14~3-0~ubuntu-focal.

Anyway, I've tried to replicate your setup with

V='5:20.10.12~3-0~ubuntu-focal'
apt-get install docker-ce=$V docker-ce-cli=$V docker-ce-rootless-extras=$V containerd.io

so my versions are

root@localhost:~# dpkg -l docker-ce libseccomp2
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name              Version                     Architecture Description
+++-=================-===========================-============-====================================================
ii  docker-ce         5:20.10.12~3-0~ubuntu-focal amd64        Docker: the open-source application container engine
ii  libseccomp2:amd64 2.5.1-1ubuntu1~20.04.2      amd64        high level interface to Linux seccomp filter
root@localhost:~# docker version
Client: Docker Engine - Community
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:45:33 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:43:42 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.5.11
  GitCommit:        3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc:
  Version:          1.0.3
  GitCommit:        v1.0.3-0-gf46b6ba
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
root@localhost:~# 

And I still can't reproduce your bug:

root@localhost:~# docker run ubuntu:jammy apt-get update
Get:1 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [90.7 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [90.7 kB]
Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [90.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [693 B]
Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [621 B]
Fetched 20.2 MB in 1s (15.8 MB/s)
Reading package lists...
root@localhost:~# 

Any suggestions to reproduce this?

[errietta]

We can reproduce this. Neither 21.10 nor 20.04 have this issue, but latest and 22.04 tags do.
This fails on ci (buddy ci, no idea what version of docker they use on the host) AND my local macbook: Docker version 20.10.8, build 3967b7d.

[woky]

Can you try to run the following commands and post output all?

1. docker run -it ubuntu:jammy apt-get update
2. docker run -it --security-opt seccomp=unconfined ubuntu:jammy apt-get update
3. docker run -it --privileged ubuntu:jammy apt-get update

[errietta]

@woky
1:

errykostala in ~  > docker run -it ubuntu:jammy apt-get update
Unable to find image 'ubuntu:jammy' locally
jammy: Pulling from library/ubuntu
Digest: sha256:2a7dffab37165e8b4f206f61cfd984f8bb279843b070217f6ad310c9c31c9c7c
Status: Downloaded newer image for ubuntu:jammy
Get:1 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [90.7 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [90.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [90.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:6 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [621 B]
Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [693 B]
Fetched 20.2 MB in 3s (7590 kB/s)               
Reading package lists... Done
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code

(doesn't work)
2:

errykostala in ~  > docker run -it --security-opt seccomp=unconfined ubuntu:jammy apt-get update

Get:1 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [90.7 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [90.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [90.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:6 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [693 B]
Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [621 B]
Fetched 20.2 MB in 3s (7716 kB/s)                        
Reading package lists... Done

(seems to work)

3:

errykostala in ~  > docker run -it --privileged ubuntu:jammy apt-get update
Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [90.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu jammy InRelease [270 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [90.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [90.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
Get:8 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [621 B]
Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [693 B]
Fetched 20.2 MB in 3s (7406 kB/s)                        
Reading package lists... Done

(also works)

That's not really an acceptable fix though, because running with --privileged is a great backdoor into escaping the container and privescing into the host

[tnir]

--security-opt seccomp=unconfined is required to reproduce the problem to me as well.

[errietta]

@tnir it works fine with, doesn't work without, unless that's what you meant