Multiple Gateways with Systemd

A common deployment scenario is you want to host both public and internal facing services on the same machine, and separating those is a grave concern.

I suggest you forego allow listing, firewalls and their ilk. Instead, host your services using gateways bound to completely separate interfaces. If your internal services are accessible to only your VPN network device they become impossible to access them from the outside world.

If you're in need of a VPN first, I have a instructions for setting up Wireguard

Sockets

First, we create an instanced service socket unit. A socket unit simultaneously improves security posture by having systemd bind to priviliged ports on our behalf; let's us bind to the same port multiple times — a requirement for running our parallel gateways; and allows for zero downtime restarts.

[Socket]
ListenStream=80
ListenStream=443
ListenDatagram=443

[Install]
WantedBy=sockets.target

Binding to Interfaces

Because the socket unit is instanced (suffixed with @) we can create named sockets for both the public and private gateways from same the unit file, differentiating them with a name.

We choose to which network device to bind by creating systemd drop-ins¹ for both sockets, gateway@private.socket and gateway@public.socket

Private

We'll bind gateway@private.socket to the VPN network device, most likely wg0 (rename as appropriate):

[Socket]
BindToDevice=wg0

Public

And then we bind gateway@public.socket to an internet accessible device. If you don't know what device name to use, this snippet will create a drop-in using the device that reaches out to the internet.

You may need to install jq

systemctl edit --stdin is available since systemd v256

ip --json route get 1.1.1.1 | jq '.[0].dev' \
    | xargs printf "[Socket]\nBindToDevice=%s" \
    | systemctl edit --stdin gateway@public.socket

In the case the snippet doesn't work, have a look at the output of ip addr for your network device and create the following drop-in file:

[Socket]
BindToDevice=<device name>

Gateway

Next up is the gateway service itself. Also instanced, the matching gateway service is started when the socket with the same name receives a connection. This is called socket activation²

For this deployment Caddy is the gateway of choice because it's operationally simple. Take a look at the Caddy installation documentation.

Once installed, depending on your distribution, you may have to disable the provided systemd service because it hijacks the ports we are going to use

systemctl disable --now caddy

And here's the configuration. There are a lot of options set, but they're mostly security related, and influenced by the systemd unit provided by the Arch package.

[Unit]
Description=Gateway %i
Documentation=https://caddyserver.com/docs/
After=network-online.target
Wants=network-online.target
Requires=gateway@%i.socket
After=gateway@%i.socket
StartLimitIntervalSec=14400
StartLimitBurst=10

[Service]
Type=notify
StateDirectory=%p/%i/caddy
ExecPaths=/usr/bin/caddy
Restart=on-failure
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
DynamicUser=yes
DevicePolicy=closed
LockPersonality=true
MemoryAccounting=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true
PrivateDevices=true
PrivateTmp=true
ProcSubset=pid
ProtectClock=true
ProtectControlGroups=true
ProtectHome=true
ProtectHostname=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectProc=invisible
ProtectSystem=strict
RemoveIPC=true
RestrictNamespaces=true
RestrictRealtime=true
RestrictSUIDSGID=true
Environment=XDG_CONFIG_HOME=/var/lib/%p/%i
Environment=HOME=%T
ExecStartPre=/usr/bin/caddy validate --config=/etc/%p/%i/Caddyfile
ExecStart=/usr/bin/caddy run --config=/etc/%p/%i/Caddyfile
ExecReload=/usr/bin/caddy reload --config=/etc/%p/%i/Caddyfile

Gateway Configuration

Gateways need a configuration to do anything of value, so here are two, one for both gateway@private.service and gateway@public.service.

The private gateway is configured to use only local certificates. That is, certificates that will only be valid to you inside your local network. It's a an opinionated choice that you can omit but I think it's simpler; I prefer to use .internal for my local services and it's impossible to generate publiclly trustable certificates for those domains.

Private Gateway

{
    auto_https disable_redirects

    storage file_system {$STATE_DIRECTORY}

    local_certs
    skip_install_trust

    default_bind fd/4 {
        protocols h1 h2
    }

    default_bind fdgram/5 {
        protocols h3
    }
}

http:// {
    bind fd/3 {
        protocols h1
    }
    redir https://{host}{uri}
}

https://{$SITE} {
    respond "Internal Service"
}

The private gateway is configured to listen for $SITE, and will need either a domain name or an IP address.

If you have a domain you use for internal services you can go ahead and replace $SITE. Otherwise, if you prefer to blitz through this guide (I don't judge you), use the IP of your VPN (wg0) interface with this snippet

ip --json addr show wg0 | jq '.[0].addr_info[] | select(.family == "inet") | .local' \
    | xargs printf "[Service]\nEnvironment=SITE=%s" \
    | systemctl edit --stdin gateway@private.service

Public Gateway

Almost identical, but uses publicly trusted certificates

{
    auto_https disable_redirects

    storage file_system {$STATE_DIRECTORY}

    # https://github.com/caddyserver/caddy/issues/7089
    https_port 80443

    default_bind fd/4 {
        protocols h2 h1
    }

    default_bind fdgram/5 {
        protocols h3
    }
}

http:// {
    bind fd/3 {
        protocols h1
    }
    redir https://{host}{uri}
}

https://{$SITE} {
    respond "Public Service"
}

Again, $SITE needs to be a domain name or an IP address.

If you've already pointed, or are going to point, A/AAAA records to this server, run the snippet

read -rep "your domain: " && echo -n $REPLY \
    | xargs printf "[Service]\nEnvironment=SITE=%s" \
    | systemctl edit --stdin gateway@public.service

Or use this snippet set the IP address of your internet facing device

ip --json route get 1.1.1.1 | jq '.[0].prefsrc' \
    | xargs printf "[Service]\nEnvironment=SITE=%s" \
    | systemctl edit --stdin gateway@public.service

If the snippet's dont work, manually create a drop-in

[Service]
Environment=SITE=<domain OR public ip>

Time to Test

Okay — start your sockets. There's no need to start the services because they'll be started when the sockets receive a connection.

systemctl enable --now gateway@{public,private}.socket

(enable means they'll automically start when your system boots)

Trying a connection to the public domain or public ip that you configured for $SITE should work from any device.

A connection to the private domain or private ip should only work on devices that are connected to your VPN.

I hope you don't, but if you need to debug, you can see the status of each service

systemctl status gateway@{public,private}.service

or check the logs of each individual service

journalctl -u gateway@private.service

...and even the sockets.

systemctl status gateway@{public,private}.socket