Milner is running OPNSense, which makes it a basically a router. The admin panel is accessible on https://milner.internal.uwcs.co.uk:55443 while on the LAN.
The main functions of OPNSense that we use (besides it acting as a router for the LAN) are:
*.internal.uwcs.co.uk (UnboundDNS)There's an ACME client under services that will fetch new certificates from Let's Encrypt. If you want to add a wildcard certificate on a subdomain (ie *.foo.uwcs.co.uk), you'll need a new certificate as *.uwcs.co.uk won't work. Certificates need to be explicitly added to the HTTPS frontend under virtual services if you want SSL termination to work for that cert.
Have a HTTP/TCP port internally that needs proxying out? HAProxy does that. Select Layer 7 for HTTP, and Layer 4 for TCP. You don't need to worry about HTTPS on the LAN side as HAProxy handles SSL termination.
Crawlers exist. And we're hosting warwick.wiki. So we've inevitably become a crawler honeypot. Awesome.
[Enter RATE LIMITING]
Because some moron decided that it was a good idea to set up HAProxy using OPNsense's GUI, the culprit not supporting rate limiting directly, rate limiting is being done through an "option pass-through". This means that it's buried somewhere you'll never find it.
Virtual Services > Public Services > [HTTP or HTTPS frontend] > [Enable Advanced Mode] > Advanced Settings > Option pass-through
stick-table type ip size 200k expire 30s store http_req_rate(10s)
http-request track-sc0 src
http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
I don't know if this has improved the scraping situation, and I have no measurable way of finding out if it has.
beryllium-but-virtual.internal.uwcs.co.uk, 8124. Note that the two go in separate fields. Give it a name service.service_backend and select the real server that you just created under servers (it should tab-complete).subdomain service_backend. This directs all traffic from subdomain.uwcs.co.uk to the service.Note that for wildcard redirects, you need both *.foo and foo for *.foo.uwcs.co.uk to work.
uwcs.co.uk domainThere's two things that need to happen:
uwcs.co.ukwarwickspeed.run sites_backendThe DHCP server hands out IPs between 192.168.0.10 and 192.168.0.254. Below 10 is for static leases.
Services -> DHCPv4 -> Leases. This is useful for seeing what hosts are currently on the network
Setting a static IP for something is usually a reasonable troubleshooting step. Services -> DHCPv4 -> [LAN], then scroll down to the list at the bottom, there's a plus button in the top right of the table.
The internal.uwcs.co.uk domain is the system one, set in System -> Settings -> General.
Flushing the DNS cache is another good troubleshooting step, on both milner and other machines.