Strict Filtering of Docker Containers

Apr 3, 2024 • Eric Garver

Introduction

Docker supports publishing ports for a container. This allows external access to the container. When firewalld is running these published ports are honored and a hole is opened in firewalld. For most users, e.g. workstations, this a good thing as docker works transparently.

For some users, this is not expected. They want firewalld to be strict. They want to only allow traffic explicitly configured via firewalld. Fortunately this can be achieved with some configuration.

Docker Configuration

To have full control of docker containers via firewalld one must first disable iptables in docker.

This can be done by adding iptables: false to the daemon configuration.

# cat /etc/docker/daemon.json
{
    "iptables": false
}

Then the host must be rebooted. Restarting Docker is not enough to clean up all the pre-existing iptables rules.

# reboot

Now the containers won’t have any iptables firewall rules automatically created.

Verify Docker Does Not Install iptables rules

This is optional and just for illustration purposes.

After the reboot starting a docker container will cause containers to not have internet access. This means that docker is not setting up iptables rules.

# docker run -it --rm debian:stable

# apt update
Ign:1 http://deb.debian.org/debian stable InRelease
Ign:2 http://deb.debian.org/debian stable-updates InRelease
Ign:3 http://deb.debian.org/debian-security stable-security InRelease
0% [Connecting to deb.debian.org]

In the next step this will be fixed by doing the networking natively in firewalld. Restarting the container is not necessary.

Firewalld Configuration

Now setup firewalld to natively perform all networking for docker with the following configuration.

firewall-cmd --permanent --zone docker --add-source 172.17.0.1/16

firewall-cmd --permanent --new-policy dockerToWorld
firewall-cmd --permanent --policy dockerToWorld --add-ingress-zone docker
firewall-cmd --permanent --policy dockerToWorld --add-egress-zone ANY
firewall-cmd --permanent --policy dockerToWorld --set-target ACCEPT
firewall-cmd --permanent --policy dockerToWorld --add-masquerade

firewall-cmd --reload

This creates a policy, dockerToWorld, to give the container internet access. Note that the --add-source above assumes the default address range used by docker.

Verify Firewalld Rules

After the firewalld rules are created verify the container has internet access.

# apt update
Get:1 http://deb.debian.org/debian stable InRelease [151 kB]
Get:2 http://deb.debian.org/debian stable-updates InRelease [55.4 kB]
Get:3 http://deb.debian.org/debian-security stable-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian stable/main amd64 Packages [8786 kB]
Get:5 http://deb.debian.org/debian stable-updates/main amd64 Packages [12.7 kB]
Get:6 http://deb.debian.org/debian-security stable-security/main amd64 Packages [150 kB]
Fetched 9203 kB in 1s (7189 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
8 packages can be upgraded. Run 'apt list --upgradable' to see them.

Adding Firewalld Native Forward Ports

Since docker is no longer using iptables ports published with --publish will no longer work. It’s simply ignored. Ports must be exposed with firewalld.

The first step is to create another policy, dockerFwdPort, to allow external access to the container.

firewall-cmd --permanent --new-policy dockerFwdPort
firewall-cmd --permanent --policy dockerFwdPort --add-ingress-zone ANY
firewall-cmd --permanent --policy dockerFwdPort --add-egress-zone HOST

Note: Older firewalld versions (before v2.0.z) require using egress-zone=ANY.

To add port forwarding (equivalent of docker –publish) to a specific container use --add-forward-port in the dockerFwdPort policy. This example forwards port 8080 to to 80. Note that the containers IP address must be known.

firewall-cmd --permanent --policy dockerFwdPort --add-forward-port port=8080:proto=tcp:toport=80:toaddr=172.17.0.2

Lastly reload the firewall.

firewall-cmd --reload

Results

This small amount of configuration allows firewalld to strictly filter docker container network traffic by doing all the networking natively in firewalld.

A Note About Podman

A similar configuration can be done with Podman. Podman 4.x can use environment variable NETAVARK_FW=none to disable the network plugin. Podman 5.x will have a containers.conf for it.

The firewalld configuration is the same except that the podman zone is used instead of the docker zone.

Bugs Referencing This Topic

This topic has been discussed and referenced in numerous reports.

firewalld 2.1.0 release

Jan 5, 2024 • Eric Garver

A new release of firewalld, version 2.1.0, is available.

This is a feature release. It also includes all bug fixes since v2.0.0.

git shortlog --no-merges --grep "^feat" v2.0.0..v2.1.0

Thomas Haller (3):

  • feat(service): add DNS over QUIC (DoQ) Service
  • feat(icmp): add ICMPv6 Multicast Listener Discovery (MLD) types
  • feat(fw): add ReloadPolicy option in firewalld.conf

Vinícius Ferrão (1):

  • feat(service): add submission service (tcp 587)

Vixea (1):

  • feat(service): Add alvr

nser77 (1):

  • feat(service): add vrrp

Source available here:

firewalld 2.0.0 release

Jun 23, 2023 • Eric Garver

A new release of firewalld, version 2.0.0, is available.

This is a major release. The major version is being bumped symbolically to reflect significant changes done in commit f4d2b80adc84 ("fix(policy): disallow zone drifting"). It does not contain any deliberate breaking changes.

Feature Blogs:

Additionally the release contains the below new features.

Eric Garver (4):

  • feat(direct): avoid iptables flush if using nftables backend
  • feat(zone): add support for priority
  • feat(nftables): add support for flowtable (software fastpath)
  • feat(nftables): support counters

Juris Lambda (2):

  • feat(service): add Zabbix Java Gateway
  • feat(service): add Zabbix Web Service

Nikolas Koesling (14):

  • feat(service): add game 0AD
  • feat(service): add game anno 1602
  • feat(service): add game anno 1800
  • feat(service): add game Civilization IV
  • feat(service): add game Civilization V
  • feat(service): add game factorio
  • feat(service): add game Minecraft
  • feat(service): add game Need For Speed: Most Wanted
  • feat(service): add game Stellaris
  • feat(service): add game Stronghold Crusader
  • feat(service): add game Super Tux kart
  • feat(service): add game Terraria
  • feat(service): add game Zero K
  • feat(service): add game Settlers

Pat Riehecky (3):

  • feat(service): add OpenTelemetry (OTLP) service
  • feat(service): define statsrv from RFC 996
  • feat(service): Add syscomlan

Source available here:

Software fastpath with nftables flowtable

May 25, 2023 • Eric Garver

Introduction

Firewalld gained support for nftables flowtable. This is a software fastpath that may significantly improve forwarding performance.

Nftables flowtable makes use of the kernel’s connection tracking to bypass much of the network stack. This accelerates data packets of established connections.

What It Looks Like

This feature can be enabled by setting NftablesFlowtable in /etc/firewalld/firewalld.conf. This setting defaults to off. To enable flowtable support set this value to your list of interfaces for which you want flowtable to be enabled, e.g. NftablesFlowtable=eth0 eth1.

This can be done manually or with a sed expression.

Example to enable eth0 and eth1:

# sed -i 's/^NftablesFlowtable=.*/NftablesFlowtable=eth0 eth1/' /etc/firewalld/firewalld.conf
# firewall-cmd --reload

When this feature is enabled firewalld adds the below additional nftables rules. It’s one additional rule and one flowtable object.

table inet firewalld {
        flowtable fastpath {
                hook ingress priority filter + 10
                devices = { eth0, eth1 }
        }
[..]
	chain filter_FORWARD {
		type filter hook forward priority filter + 10; policy accept;
		ct state { established, related } meta l4proto { tcp, udp } flow add @fastpath  <--- new rule
		ct state { established, related } accept
[..]

Performance Tests

This is the test topology used for gather performance test results.

flowchart LR
    iperf3_client-->eth0
    eth1-->iperf3_server
    subgraph DUT
        eth0-->firewalld
        firewalld-->eth1
        subgraph firewalld
        end
    end
    subgraph traffgen
        subgraph net_namespace
            subgraph iperf3_server
            end
        end
        subgraph iperf3_client
        end
    end

The device under test was artificially limited to two CPU cores. This was done specifically to stress the forward path.

The traffic generation uses 16 iperf3 instances run in parallel with 128 parallel streams for 60 seconds. This simulates 2048 concurrent connections. The benchmark is run 10 times to normalize the results and produce a standard deviation.

Below is a graph of the results of NftablesFlowtable disabled vs enabled. The absolute numbers are less important. The important takeaway is the relative performance improvement.

performance graph

More information

Nftables flowtable can accelerate TCP and UDP flows. Control packets will still take the traditional network path, i.e. they will take the slow path.

Firewalld supports source based zones with --add-source. These can also be accelerated, but keep in mind that flowtable is enabled on the interface. So you must make sure that traffic from that source is received on the interface that was added to NftablesFlowtable. If in doubt, always use --add-interface.

Summary

Nftables flowtable brings a significant performance improvement for forwarded traffic. This is applies to use cases like: network firewall, home router, and even container/VM traffic.

Zone Priorities

Apr 10, 2023 • Eric Garver

Introduction

Firewalld gained a new feature called Zone Priorities. This allows the user to control the order in which packets are classified into zones.

What It Looks Like

The zone priority can be set using command line option --set-priority. Similar to policies and rich rules, a lower priority value has higher precedence. e.g. -10 occurs before 100

# firewall-cmd --permanent --zone internal --set-priority -10

# firewall-cmd --permanent --zone internal --get-priority
-10

# firewall-cmd --permanent --info-zone internal
internal
  target: default
  ingress-priority: -10    <--- new field
  egress-priority: -10     <--- new field
  icmp-block-inversion: no
  interfaces: 
  sources: 
  services: dhcpv6-client mdns samba-client ssh
  ports: 
  protocols: 
  forward: yes
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules:

This will guarantee that packets are consider for the internal zone before other zones.

We can see this is the generated nftables rules.

chain filter_INPUT_POLICIES {
        iifname "dummy0" jump filter_IN_policy_allow-host-ipv6
        iifname "dummy0" jump filter_IN_internal     <--------- before ipset source @block_country in "drop" zone
        iifname "dummy0" reject with icmpx admin-prohibited
        ip saddr @block_country jump filter_IN_policy_allow-host-ipv6
        ip saddr @block_country jump filter_IN_drop
        ip saddr @block_country drop
        jump filter_IN_policy_allow-host-ipv6
        jump filter_IN_public
        reject with icmpx admin-prohibited
}

Control Ingress and Egress Independently

Using --set-priority will set the priority for both ingress and egress classification. This is sufficient for most use cases. However, they may be set independently with --set-ingress-priority and --set-egress-priority.

# firewall-cmd --permanent --zone internal --set-ingress-priority -10
# firewall-cmd --permanent --zone internal --set-egress-priority 100

# firewall-cmd --permanent --info-zone internal
internal (active)
  target: default
  ingress-priority: -10
  egress-priority: 100
  icmp-block-inversion: no
  interfaces: dummy0
  sources: 
  services: dhcpv6-client mdns samba-client ssh
  ports: 
  protocols: 
  forward: yes
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules:

Previous Behavior

Prior to this feature users had to rely on firewalld’s undocumented behavior of classification. It was impossible to classify interfaces before sources.

That order is roughly:

  1. source based, i.e. --add-source
    • these are sorted by zone name
  2. interface based, i.e. --add-interface

When zone priorities are equal then classification uses this legacy behavior.

Summary

It’s now possible to customize packet classification in firewalld using zone option --set-priority.