My home network and it's fuckery

Network design, incidents and other stuff that randomly breaks

I am doing weird network things in my free time. Notably all the dn42 stuff I’ve done. The obvious next step is to do more weird network things at home. So how is the home network set up?

                +---------------------------+     +--------------+
-- from ISP --> | DSL Modem providing PPPoE | --> | Raspberry Pi | -- to LAN -->
                +---------------------------+     +--------------+

The DSL Modem

That’s very straight forward, it’s a Modem that supports G.fast. There is a DSLAM in the basement which converts the fiber to DSL (G.fast).

             +-------+
-- fiber --> | DSLAM | -- DSL G.fast, to customers -->
             +-------+

The Modem is set to bridge mode so it just provides PPPoE to the local network behind it.

The Raspberry Pi

This is where things get more interesting because I actually control all the stuff running there. So what software is running there?

So, how to configure all that stuff?

Firstly: the layout of interfaces on the Pi.

   +------+      +-----+      +-------+
-- | eth0 | <--> | br0 | <--> | wlan0 | --
   +------+      +-----+      +-------+

pppd

/etc/ppp/peers/<isp>

mtu 1500                # Set the MTU of the interface
plugin pppoe.so         # Load the PPPoE plugin, on older versions of pppd this is called rp-pppoe.so
nic-br0                 # Over which interface pppd should try to communicate

noipdefault             # Don't try to determine the local IP address from the hostname
defaultroute            # Add the default route over the peers IP address
replacedefaultroute     # Replace existing default routes on the system

+ipv6                   # Enable IPv6
#defaultroute6          # Set an IPv6 default route, commented out, managed over IPv6 prefix delegation
ipv6cp-accept-local     # Accept the local interface identifier send by the peer
ipv6cp-accept-remote    # Accept the remote interface identifier send by the peer

persist                 # Do not exit when the connection drops, try to reconnect
maxfail 0               # Try to reconnect indefinetly
hide-password           # Hide the password in logs
noauth                  # Do not require the peer to authenticate itself
user "<ISP username>"   # The username for the PPP authentication

/etc/ppp/pap-secrets

"<ISP username"> * "<ISP password>"     # username server password (hostname)

For more information on the options see pppd(8).

dhcpcd

/etc/dhcpcd.conf

hostname famfo                                  # Tell my ISP to set famfo.<domain> as the rDNS (they don't :c)

option rapid_commit, classless_static_routes    # Request the rapid commit and classless static routers options from the server
waitip 6                                        # Wait for an IPv6 to be assigned before forking to the background

ipv6only                                        # Only configure IPv6
ipv6ra_noautoconf                               # Do not generate a SLAAC address for all interfaces
noipv6rs                                        # Do not do IPv6 router solicidation

nohook resolv.conf                              # Do not set the advertised DNS servers
nohook hostname                                 # Do not set the advertised hostname

interface ppp0
    ipv6rs                                      # Enable IPv6 router solicidation on the ppp0 interface (setting the IPv6 default route)
    iaid 1                                      # Set the Interface Association Identifier for ppp0
    ia_pd 1/::/48 br0/0/64                      # Delegate a /48 IPv6 prefix for the iaid 1 and assign a /64 to the br0 interface

For more information and options see dhcpcd.conf(5).

udhcp

/etc/udhcpd.conf

start 		192.168.42.10                       # Start of DHCP range
end 		192.168.42.254                      # End of DHCP range
opt router 	192.168.42.1                        # Set the default route via 192.168.42.1
opt	dns 	192.168.42.1	46.182.19.48        # Set the default DNS sever as 192.168.42.1 and fallback to 46.182.19.48 (digitalcourage)
opt	subnet 	255.255.255.0                       # This is a /24 subnet
opt	lease	864000                              # Leases are 10 days long
interface 	br0                                 # Bind to the br0 interface

For more information and options see https://udhcp.busybox.net/udhcpd.conf.

radvd

/etc/radvd.conf

interface br0 {
    AdvSendAdvert on;                                   # Send periodic router advertisements
    prefix ::/64 {};                                    # Advertise every non link-local prefix found on the interface
    RDNSS fd42:deca:fbad:f7::1 2a02:2970:1002::18 {};   # Advertise fd42:deca:fbad:f7::1 and 2a02:2970:1002::18 (digitalcourage) as DNS servers
};

For more information and options see radvd.conf(5).

nftables

#!/usr/sbin/nft -f
flush ruleset

# Masquerade all outgoing packets to the public IPv4 the interface got over ppp
table ip nat {
	chain postrouting {
		type nat hook postrouting priority srcnat; policy accept;
		oifname "ppp0" masquerade;
	}
}

# TCP MSS clamping, I'm just not even gonna try
table ip mangle {
    	chain forward {
		type filter hook forward priority mangle; policy accept;
		oifname "ppp0" tcp flags syn / syn,rst tcp option maxseg size 1400-65495 counter packets 2 bytes 120 tcp option maxseg size set rt mtu;
	}
}

# Drop all incomming packets that are not related, icmp, icmpv6 or link-local
table inet firewall {
	chain inbound {
		type filter hook input priority filter; policy drop;
		iifname != "ppp0" accept;
		ct state vmap { established:accept, related:accept, invalid:drop };
		icmp type echo-request accept;
		icmpv6 type {*} accept;
		ip6 saddr fe80::/10 accept;
	}
}

include "/var/lib/nftables/*.nft"
include "/etc/nftables.d/*.nft"

Read https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks and https://wiki.nftables.org/wiki-nftables/index.php/Quick_reference-nftables_in_10_minutes, it does a much better job than I could even try to do here myself.

hostapd

/etc/hostapd/hostapd.conf

interface=wlan0                     # wifi interface to use
bridge=br0                          # bridge wlan0 to br0

ctrl_interface=/var/run/hostapd     # path to the control interface
ctrl_interface_group=wheel          # group who can interact with the control interface

country_code=<wherever you are>     # ISO/IEC 3166-1 for your regulatory domain
ieee80211d=1                        # Respect and advertise the country code
ieee80211h=1                        # Enable DFS detection

hw_mode=any                         # Use all frequency bands supported by the driver
channel=0                           # Automatically set the wifi channel
preamble=1                          # Improves network perforamnce if the hardware supports it
macaddr_acl=0                       # Accept all clients unless they are in the mac deny list
auth_algs=3                         # Set the authentication algorithm

ieee80211n=1                        # Enable wifi4 support
ieee80211ac=1                       # Enable wifi5 support
ieee80211ax=1                       # Enable wifi6 support

wpa=2                               # Enable wpa
wpa_key_mgmt=WPA-PSK                # Enable WPA-Personal / WPA2-Personal
rsn_pairwise=CCMP                   # Set the pairwise cipher

ssid=<ssid>
wpa_passphrase=<password>

For more information and options see https://w1.fi/cgit/hostap/plain/hostapd/hostapd.conf.

This config file is spread across multiple Pis for maximum coverage and even handover between access points works quite well.

named and bird

I’ll leave these two as an exercise for the reader. Both of these configs are huge, complex and even more specific to my network than the rest. I am using bind as my own recursive DNS server.

Here are some good pointers for bind and bird: https://bind9.readthedocs.io, https://dn42.cc/wiki/howto/routing-daemons/bird2/

Incidents

Surprisingly enough, there is only one major incident that happened: IPv6 disconnects every time when the PPP session gets forcefully restarted by the ISP and dhcpcd needs a manual nudge. The dDNS script used to fail if it didn’t get an IPv6 so I couldn’t ssh in from the outside to fix that. Tldr; nobody was home and we couldn’t access internal stuff for a few days until someone restarted dhcpcd. I’ve since fixed the script to not fail on a missing v6 address (and am hosting a Ripe Atlas probe which also sets DNS things).