readme docs
This commit is contained in:
goeranh
parent
6e0d407b1c
commit
9466ab3656
6 changed files with 1872 additions and 53 deletions
|
|
@ -1,7 +1,365 @@
|
|||
# Zentraler Reverse Proxy Für ehemals öffentliche IP-Adressen
|
||||
# Proxy Host - Central Reverse Proxy
|
||||
|
||||
Central reverse proxy at 141.56.51.1 running as a full VM (not LXC container).
|
||||
|
||||
## Overview
|
||||
|
||||
- **Hostname**: proxy
|
||||
- **IP Address**: 141.56.51.1
|
||||
- **Type**: Full VM (not LXC)
|
||||
- **Services**: HAProxy, OpenSSH (ports 1005, 2142)
|
||||
- **Role**: Central traffic router for all StuRa HTW Dresden services
|
||||
|
||||
## Architecture
|
||||
|
||||
The proxy is the central entry point for all HTTP/HTTPS traffic:
|
||||
- All public DNS records point to this IP (141.56.51.1)
|
||||
- HAProxy performs SNI-based routing for HTTPS traffic
|
||||
- HTTP Host header-based routing for unencrypted traffic
|
||||
- Automatic redirect from HTTP to HTTPS (except ACME challenges)
|
||||
- Backend services handle their own TLS certificates
|
||||
|
||||
## Services
|
||||
|
||||
### HAProxy
|
||||
|
||||
HAProxy routes traffic using two methods:
|
||||
|
||||
**1. HTTP Mode (Port 80)**
|
||||
- Inspects the HTTP Host header
|
||||
- Routes requests to appropriate backend
|
||||
- Forwards ACME challenges (/.well-known/acme-challenge/) to backends
|
||||
- Redirects all other HTTP traffic to HTTPS (301 redirect)
|
||||
- Default backend serves an index page listing all services
|
||||
|
||||
**2. TCP Mode (Port 443)**
|
||||
- SNI inspection during TLS handshake
|
||||
- Routes encrypted traffic without decryption
|
||||
- Backends terminate TLS and serve their own certificates
|
||||
- 1-second inspection delay to buffer packets
|
||||
|
||||
**Key features:**
|
||||
- Logging: All connections logged to systemd journal
|
||||
- Stats page: Available at http://127.0.0.1:8404/stats (localhost only)
|
||||
- Max connections: 50,000
|
||||
- Buffer size: 32,762 bytes
|
||||
- Timeouts: 5s connect, 30s client/server
|
||||
|
||||
### SSH Services
|
||||
|
||||
**Port 1005: Admin SSH Access**
|
||||
- Primary SSH port for administrative access
|
||||
- Configured in `services.openssh.listenAddresses`
|
||||
- Used for system management and deployments
|
||||
|
||||
**Port 2142: SSH Jump to srs2**
|
||||
- Forwards SSH connections to srs2 (141.56.51.2:80)
|
||||
- 30-minute session timeout
|
||||
- TCP keep-alive enabled
|
||||
- Used for accessing legacy systems
|
||||
|
||||
### Auto-Generated Forwarding
|
||||
|
||||
The proxy configuration is **partially auto-generated**:
|
||||
|
||||
1. **Static forwards**: Manually defined in the `forwards` attribute set in default.nix
|
||||
2. **Dynamic forwards**: Automatically generated from all NixOS configurations in this flake that have `services.nginx.enable = true`
|
||||
|
||||
When you deploy a new host with Nginx enabled, the proxy automatically:
|
||||
- Detects the nginx virtualHosts
|
||||
- Extracts the IP address from the host configuration
|
||||
- Generates HAProxy backend rules for ports 80 and 443
|
||||
|
||||
**No manual proxy configuration needed for nginx-enabled hosts!**
|
||||
|
||||
## Deployment Type
|
||||
|
||||
Unlike other hosts in this infrastructure, the proxy is a **full VM** (not an LXC container):
|
||||
- Has dedicated hardware-configuration.nix
|
||||
- Uses disko for declarative disk management (hetzner-disk.nix)
|
||||
- Network interface: `ens18` (not `eth0` like LXC containers)
|
||||
- Requires more resources and dedicated storage
|
||||
|
||||
## Disko Configuration
|
||||
|
||||
The proxy uses Btrfs with the following layout (hetzner-disk.nix):
|
||||
- **Filesystem**: Btrfs
|
||||
- **Compression**: zstd
|
||||
- **Subvolumes**:
|
||||
- `/` - Root filesystem
|
||||
- `/nix` - Nix store
|
||||
- `/var` - Variable data
|
||||
- `/home` - User home directories (if needed)
|
||||
|
||||
This provides better performance and snapshot capabilities compared to LXC containers.
|
||||
|
||||
## Deployment
|
||||
|
||||
See the [main README](../../README.md) for deployment methods.
|
||||
|
||||
### Initial Installation
|
||||
|
||||
**Using nixos-anywhere (recommended):**
|
||||
```bash
|
||||
nix run github:nix-community/nixos-anywhere -- --flake .#proxy --target-host root@141.56.51.1
|
||||
```
|
||||
|
||||
This handles disk partitioning via disko automatically.
|
||||
|
||||
### Updates
|
||||
|
||||
**Note**: SSH runs on port 1005, so you need an SSH config entry:
|
||||
|
||||
```bash
|
||||
# ~/.ssh/config
|
||||
Host 141.56.51.1
|
||||
Port 1005
|
||||
```
|
||||
|
||||
Then deploy:
|
||||
|
||||
```bash
|
||||
# From local machine
|
||||
nixos-rebuild switch --flake .#proxy --target-host root@141.56.51.1
|
||||
|
||||
# Or use auto-generated script
|
||||
nix run .#proxy-update
|
||||
```
|
||||
|
||||
## Network Configuration
|
||||
|
||||
- **Interface**: ens18 (VM interface)
|
||||
- **IP**: 141.56.51.1/24
|
||||
- **Gateway**: 141.56.51.254
|
||||
- **DNS**: 9.9.9.9, 1.1.1.1 (public DNS, not HTW internal)
|
||||
- **Firewall**: nftables enabled
|
||||
- **Open ports**: 22, 80, 443, 1005, 2142
|
||||
|
||||
## Adding New Services
|
||||
|
||||
### Method 1: Deploy a host with Nginx (Automatic)
|
||||
|
||||
The easiest method - just deploy a new host with nginx enabled:
|
||||
|
||||
```nix
|
||||
# hosts/newservice/default.nix
|
||||
services.nginx = {
|
||||
enable = true;
|
||||
virtualHosts."newservice.htw.stura-dresden.de" = {
|
||||
forceSSL = true;
|
||||
enableACME = true;
|
||||
locations."/" = {
|
||||
# your config
|
||||
};
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
The flake automatically:
|
||||
1. Discovers the nginx virtualHost
|
||||
2. Extracts the IP address from networking configuration
|
||||
3. Generates HAProxy forwarding rules
|
||||
4. No manual proxy changes needed!
|
||||
|
||||
**You still need to:**
|
||||
- Add DNS record pointing to 141.56.51.1
|
||||
- Deploy the proxy to pick up changes: `nix run .#proxy-update`
|
||||
|
||||
### Method 2: Manual forwarding (for non-Nginx services)
|
||||
|
||||
For services not using Nginx, manually add to the `forwards` attribute set in hosts/proxy/default.nix:
|
||||
|
||||
```nix
|
||||
forwards = {
|
||||
# ... existing forwards ...
|
||||
newservice = {
|
||||
dest = "141.56.51.XXX";
|
||||
domain = "newservice.htw.stura-dresden.de";
|
||||
httpPort = 80;
|
||||
httpsPort = 443;
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
Then deploy the proxy.
|
||||
|
||||
## SNI Routing Explained
|
||||
|
||||
**SNI (Server Name Indication)** is a TLS extension that includes the hostname in the ClientHello handshake.
|
||||
|
||||
How it works:
|
||||
1. Client initiates TLS connection to 141.56.51.1
|
||||
2. Client sends ClientHello with SNI field (e.g., "wiki.htw.stura-dresden.de")
|
||||
3. HAProxy inspects the SNI field (does not decrypt)
|
||||
4. HAProxy routes the entire encrypted connection to the backend (e.g., 141.56.51.13)
|
||||
5. Backend terminates TLS and serves the content
|
||||
|
||||
**Benefits:**
|
||||
- No TLS decryption at proxy (end-to-end encryption)
|
||||
- Each backend manages its own certificates
|
||||
- Simple certificate renewal via ACME/Let's Encrypt
|
||||
- No certificate syncing required
|
||||
|
||||
## ACME Challenge Forwarding
|
||||
|
||||
Let's Encrypt verification works seamlessly:
|
||||
|
||||
1. Backend requests certificate from Let's Encrypt
|
||||
2. Let's Encrypt queries DNS, finds 141.56.51.1
|
||||
3. Let's Encrypt requests `http://domain/.well-known/acme-challenge/<token>`
|
||||
4. HAProxy detects ACME challenge path
|
||||
5. HAProxy forwards to backend without HTTPS redirect
|
||||
6. Backend responds with challenge token
|
||||
7. Let's Encrypt verifies and issues certificate
|
||||
|
||||
This is why HTTP→HTTPS redirect has an exception for `/.well-known/acme-challenge/` paths.
|
||||
|
||||
## HAProxy Stats
|
||||
|
||||
Access HAProxy statistics page (localhost only):
|
||||
|
||||
```bash
|
||||
# SSH into proxy
|
||||
ssh -p 1005 root@141.56.51.1
|
||||
|
||||
# Access stats via curl
|
||||
curl http://127.0.0.1:8404/stats
|
||||
|
||||
# Or forward port to your local machine
|
||||
ssh -p 1005 -L 8404:127.0.0.1:8404 root@141.56.51.1
|
||||
# Then browse to http://localhost:8404/stats
|
||||
```
|
||||
|
||||
The stats page shows:
|
||||
- Current connections per backend
|
||||
- Health check status
|
||||
- Traffic statistics
|
||||
- Error counts
|
||||
|
||||
## Configuration Structure
|
||||
|
||||
The HAProxy configuration is generated from Nix using `lib.foldlAttrs`:
|
||||
|
||||
```nix
|
||||
forwards = {
|
||||
service_name = {
|
||||
dest = "141.56.51.XXX";
|
||||
domain = "service.htw.stura-dresden.de";
|
||||
httpPort = 80;
|
||||
httpsPort = 443;
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
This generates:
|
||||
- ACL rules for HTTP Host header matching
|
||||
- Backend definitions for ports 80 and 443
|
||||
- SNI routing rules for HTTPS
|
||||
- HTTP→HTTPS redirects (except ACME)
|
||||
|
||||
## Current Forwards
|
||||
|
||||
The proxy currently forwards traffic for:
|
||||
|
||||
**In this repository:**
|
||||
- git.adm.htw.stura-dresden.de → 141.56.51.7 (Forgejo)
|
||||
- wiki.htw.stura-dresden.de → 141.56.51.13 (MediaWiki)
|
||||
- pro.htw.stura-dresden.de → 141.56.51.15 (Redmine)
|
||||
- cloud.htw.stura-dresden.de → 141.56.51.16 (Nextcloud)
|
||||
|
||||
**External services (managed outside this repo):**
|
||||
- stura.htw-dresden.de → 141.56.51.3 (Plone)
|
||||
- tix.htw.stura-dresden.de → 141.56.51.220 (Pretix)
|
||||
- vot.htw.stura-dresden.de → 141.56.51.57 (OpenSlides)
|
||||
- mail.htw.stura-dresden.de → 141.56.51.14 (Mail server)
|
||||
- lists.htw.stura-dresden.de → 141.56.51.14 (Mailing lists)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### HAProxy not starting
|
||||
|
||||
```bash
|
||||
# Check HAProxy status
|
||||
systemctl status haproxy
|
||||
|
||||
# Check configuration syntax
|
||||
haproxy -c -f /etc/haproxy/haproxy.cfg
|
||||
|
||||
# View HAProxy logs
|
||||
journalctl -u haproxy -f
|
||||
```
|
||||
|
||||
### Backend not reachable
|
||||
|
||||
```bash
|
||||
# Check backend connectivity
|
||||
curl -v http://141.56.51.XXX:80
|
||||
curl -vk https://141.56.51.XXX:443
|
||||
|
||||
# Check HAProxy stats for backend status
|
||||
curl http://127.0.0.1:8404/stats | grep backend_name
|
||||
|
||||
# Test DNS resolution
|
||||
dig +short domain.htw.stura-dresden.de
|
||||
|
||||
# Check firewall rules
|
||||
nft list ruleset | grep 141.56.51.XXX
|
||||
```
|
||||
|
||||
### ACME challenges failing
|
||||
|
||||
```bash
|
||||
# Verify HTTP forwarding works
|
||||
curl -v http://domain.htw.stura-dresden.de/.well-known/acme-challenge/test
|
||||
|
||||
# Check HAProxy ACL for ACME
|
||||
grep -i acme /etc/haproxy/haproxy.cfg
|
||||
|
||||
# Verify no HTTPS redirect for ACME paths
|
||||
curl -I http://domain.htw.stura-dresden.de/.well-known/acme-challenge/test
|
||||
# Should return 200/404, not 301 redirect
|
||||
```
|
||||
|
||||
### SSH Jump not working
|
||||
|
||||
```bash
|
||||
# Check SSH jump frontend
|
||||
systemctl status haproxy
|
||||
journalctl -u haproxy | grep ssh_jump
|
||||
|
||||
# Test connection to srs2
|
||||
telnet 141.56.51.2 80
|
||||
|
||||
# Check HAProxy backend configuration
|
||||
grep -A 5 "ssh_srs2" /etc/haproxy/haproxy.cfg
|
||||
```
|
||||
|
||||
## Files and Directories
|
||||
|
||||
- **HAProxy config**: `/etc/haproxy/haproxy.cfg` (generated by Nix)
|
||||
- **HAProxy socket**: `/run/haproxy/admin.sock` (if enabled)
|
||||
- **Disk config**: `./hetzner-disk.nix` (disko configuration)
|
||||
- **Hardware config**: `./hardware-configuration.nix` (VM hardware)
|
||||
- **NixOS config**: `./default.nix` (proxy configuration)
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **No TLS decryption**: End-to-end encryption maintained
|
||||
- **No access logs**: HAProxy logs to journal (can be filtered)
|
||||
- **Stats page**: Localhost only (not exposed)
|
||||
- **Firewall**: Only necessary ports open
|
||||
- **SSH**: Custom port (1005) reduces automated attacks
|
||||
|
||||
---
|
||||
|
||||
## Zentraler Reverse Proxy Für ehemals öffentliche IP-Adressen
|
||||
|
||||
(Original German documentation preserved)
|
||||
|
||||
Die Instanzen können weitestgehend unverändert weiterlaufen, es müssen nur alle DNS-Einträge auf 141.56.51.1 geändert werden.
|
||||
|
||||
## HAProxy
|
||||
### HAProxy
|
||||
|
||||
Zum Weiterleiten der Connections wird HAProxy verwendet.
|
||||
HAProxy unterstützt verschiedene Modi.
|
||||
Für uns sind http und tcp relevant.
|
||||
|
|
@ -66,3 +424,12 @@ backend cloud_443
|
|||
...
|
||||
...
|
||||
```
|
||||
|
||||
## See Also
|
||||
|
||||
- [Main README](../../README.md) - Deployment methods and architecture
|
||||
- [Git README](../git/README.md) - Forgejo configuration
|
||||
- [Wiki README](../wiki/README.md) - MediaWiki configuration
|
||||
- [Redmine README](../redmine/README.md) - Redmine configuration
|
||||
- [Nextcloud README](../nextcloud/README.md) - Nextcloud configuration
|
||||
- [HAProxy Documentation](http://www.haproxy.org/#docs)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue