DNS issue

[aleks223]

Hi,

I'm having issue with my jail when it comes to dns. During the boot of the jail, the following error message appears
systemd-nspawn[2596114]: Failed to mount /etc/resolv.conf (type n/a) on /mnt/ZPool_SSD/jailmaker/jails/docker/rootfs/run/systemd/resolve/resolv.conf (MS_BIND ""): No such file or directory

My jail is configured as follows:

startup=1
gpu_passthrough_intel=1
gpu_passthrough_nvidia=0
# Turning off seccomp filtering improves performance at the expense of security
seccomp=1

# Use bridge networking to provide an isolated network namespace,
# so docker can manage firewall rules
# Alternatively use --network-macvlan=eno1 instead of --network-bridge
# Ensure to change eno1/br1 to the interface name you want to use
# You may want to add additional options here, e.g. bind mounts
systemd_nspawn_user_args=--network-macvlan=enp2s0
	--resolv-conf=bind-host
#	--resolv-conf=replace-stub
#	--resolv-conf=copy-host
	--system-call-filter='add_key keyctl bpf'
	--bind='/mnt/ZPool_SSD/docker/data:/mnt/data'
	--bind='/mnt/ZPool_SSD/docker/stacks:/opts/stacks'
	--bind='/mnt/ZPool1/Anime:/mnt/media/jellyfin/anime'
	--bind='/mnt/ZPool1/Movies:/mnt/media/jellyfin/movies'
	--bind='/mnt/ZPool1/Documentaries:/mnt/media/jellyfin/documentaries'
	--bind='/mnt/ZPool1/TVShows:/mnt/media/jellyfin/tvshows'
	--bind='/mnt/ZPool1/Music:/mnt/media/navidrome/music'
	--bind='/mnt/ZPool1/eBooks:/mnt/media/calibre/ebooks'
	--bind='/mnt/ZPool1/Comics:/mnt/media/komga/comics'
	--bind='/mnt/ZPool_SSD/docker/data/nextcloud/config:/mnt/docker-aio-config'
	--bind='/mnt/ZPool1/NextcloudDocker:/mnt/nextcloud/userData'
	--bind='/mnt/ZPool1/Photos:/mnt/nextcloud/mount/photos'

# Script to run on the HOST before starting the jail
# Load kernel module and config kernel settings required for docker
pre_start_hook=#!/usr/bin/bash
	set -euo pipefail
	echo 'PRE_START_HOOK'
	echo 1 > /proc/sys/net/ipv4/ip_forward
	modprobe br_netfilter
	echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
	echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables

# Only used while creating the jail
distro=debian
release=bookworm

# Install docker inside the jail:
# https://docs.docker.com/engine/install/debian/#install-using-the-repository
# Will also install the NVIDIA Container Toolkit if gpu_passthrough_nvidia=1 during initial setup
# https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
initial_setup=#!/usr/bin/bash
	set -euo pipefail

	apt-get update && apt-get -y install ca-certificates curl
	install -m 0755 -d /etc/apt/keyrings
	curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
	chmod a+r /etc/apt/keyrings/docker.asc

	echo \
	"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
	$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
	tee /etc/apt/sources.list.d/docker.list > /dev/null

	apt-get update
	apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

	# The /usr/bin/nvidia-smi will be present when gpu_passthrough_nvidia=1
	if [ -f /usr/bin/nvidia-smi ]; then
	curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey -o /etc/apt/keyrings/nvidia.asc
	chmod a+r /etc/apt/keyrings/nvidia.asc
	curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
	sed 's#deb https://#deb [signed-by=/etc/apt/keyrings/nvidia.asc] https://#g' | \
	tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

	apt-get update
	apt-get install -y nvidia-container-toolkit

	nvidia-ctk runtime configure --runtime=docker
	systemctl restart docker
	fi

	docker info

# You generally will not need to change the options below
systemd_run_default_args=--property=KillMode=mixed
	--property=Type=notify
	--property=RestartForceExitStatus=133
	--property=SuccessExitStatus=133
	--property=Delegate=yes
	--property=TasksMax=infinity
	--collect
	--setenv=SYSTEMD_NSPAWN_LOCK=0

systemd_nspawn_default_args=--keep-unit
	--quiet
	--boot
	--bind-ro=/sys/module
	--inaccessible=/sys/module/apparmor

Additionally

root@truenas[/var/empty]# systemctl status jlmkr-docker.service
● jlmkr-docker.service - My nspawn jail docker [created with jailmaker]
     Loaded: loaded (/run/systemd/transient/jlmkr-docker.service; transient)
  Transient: yes
     Active: active (running) since Wed 2024-09-25 08:55:33 CEST; 52min ago
    Process: 2682431 ExecStartPre=/mnt/ZPool_SSD/jailmaker/jails/docker/.ExecStartPre (code=exited, status=0/SUCCESS)
   Main PID: 2682433 (systemd-nspawn)
     Status: "Container running: Ready."
      Tasks: 47
     Memory: 110.3M
        CPU: 14.190s
     CGroup: /system.slice/jlmkr-docker.service
             ├─payload
             │ ├─init.scope
             │ │ └─2682435 /usr/lib/systemd/systemd
             │ └─system.slice
             │   ├─console-getty.service
             │   │ └─2682482 /sbin/agetty -o "-p -- \\u" --noclear --keep-baud - 115200,38400,9600 vt220
             │   ├─containerd.service
             │   │ └─2682477 /usr/bin/containerd
             │   ├─dbus.service
             │   │ └─2682474 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
             │   ├─docker.service
             │   │ └─2683081 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
             │   ├─systemd-journald.service
             │   │ └─2682454 /lib/systemd/systemd-journald
             │   ├─systemd-logind.service
             │   │ └─2682475 /lib/systemd/systemd-logind
             │   └─systemd-networkd.service
             │     └─2682464 /lib/systemd/systemd-networkd
             └─supervisor
               └─2682433 /bin/systemd-nspawn --keep-unit --quiet --boot --bind-ro=/sys/module --inaccessible=/sys/module/apparmor --machine=docker --directory=rootfs --bind=/dev/dri --network-macvlan=enp2s0 --resolv-conf=bind-host

Sep 25 08:55:33 truenas systemd-nspawn[2682433]:          Starting systemd-user-sessions.service - Permit User Sessions...
Sep 25 08:55:33 truenas systemd-nspawn[2682433]: [  OK  ] Started dbus.service - D-Bus System Message Bus.
Sep 25 08:55:33 truenas systemd-nspawn[2682433]: [  OK  ] Finished systemd-user-sessions.service - Permit User Sessions.
Sep 25 08:55:33 truenas systemd-nspawn[2682433]: [  OK  ] Started console-getty.service - Console Getty.
Sep 25 08:55:33 truenas systemd-nspawn[2682433]: [  OK  ] Reached target getty.target - Login Prompts.
Sep 25 08:55:33 truenas systemd-nspawn[2682433]: [  OK  ] Started containerd.service - containerd container runtime.
Sep 25 08:55:33 truenas systemd-nspawn[2682433]: [  OK  ] Started systemd-logind.service - User Login Management.
Sep 25 08:55:39 truenas systemd-nspawn[2682433]:
Sep 25 08:55:39 truenas systemd-nspawn[2682433]: Debian GNU/Linux 12 docker pts/0
Sep 25 08:55:39 truenas systemd-nspawn[2682433]:

Any clue what could help solve this issue? I'm on Dragonfish-24.04.2.2.

[Lockszmith-GH]

What --resolv-conf=bind-host does, is basically do a bind mount in read-only mode to the host's /etc/resolv.conf

The error is complaining 'no such file exists'.
Do you have an /etc/resolv.conf on your TrueNAS SCALE host?

I too, am on Dragonfish, and my /etc/resolv.conf contains the IP address I configured in the UI's: Network -> Global Configuration -> Settings -> DNS Nameserver 1

Do you have that configured? If not, try setting that up - that should create the /etc/resolv.conf which in turn will resolve the error you get when starting the jail.

[aleks223]

Same thing over here. I also have an /etc/resolv.conf file on the truenas host. Here is the result:

cat /etc/resolv.conf
domain local
nameserver 1.1.1.1
nameserver 1.0.0.1

which is why I don't grasp the error. is it a permission issue? should I create something in /mnt/ZPool_SSD/jailmaker/jails/docker/rootfs/?

[Lockszmith-GH]

Interesting... Does this happen if you create a new jail?

[aleks223]

It does not!

[Lockszmith-GH]

OK, a lead - I would compare the rootfs /etc of both, see if you can spot the difference.

[aleks223]

Nevermind re-created another instance and all is ok there