Server configuration for system core following version 1.2 guidelines
INTERNET USERS
│
┌─────────────────────┐
│ CLOUDFLARE │ TLS · WAF · HTTP/3 · Bot Filter · CDN
└─────────────────────┘
│ Trusted Edge Request (CF-Connecting-IP header)
┌─────────────────────┐
│ NGINX │ Real IP restore · Protocol sanity
│ (perimeter layer) │ Security headers · Rate limiting
└─────────────────────┘
│ Clean Request (port 6081)
┌─────────────────────┐
│ VARNISH │ HTML cache · TTL · stale-while-revalidate
│ (cache layer) │ Cookie allowlist · Bypass routing
└─────────────────────┘
│ Cache miss / pass (port 8080)
┌─────────────────────┐
│ APACHE + WORDPRESS │ Multisite · WooCommerce · Mercator
│ (app layer) │ GraphQL · Submission Core · PHP-FPM 8.4
└─────────────────────┘
Internal transport between layers uses loopback (127.0.0.1) only and is
never exposed publicly.
TUS audio upload path — large audio files bypass Varnish entirely:
Nginx terminates TUS connections and proxies directly to the TUS Node server
(port 1080), with no buffering, no body size limit, and 3600s timeouts.
Varnish also has a belt-and-suspenders req.backend_hint = tus_node rule
for any /files/ traffic that reaches it via a non-standard path.
| File | Purpose |
|---|---|
nginx/nginx.conf |
Global Nginx settings: real IP restore, geo/UA maps, rate-limit zones, upstream backends |
nginx/sites-available/system-core.conf |
Virtual host: TLS, security headers, UA/geo checks, SPARXSTAR header gate, per-route proxying |
varnish/default.vcl |
Cache policy: bypass rules, cookie allowlist, TTL/grace, image format negotiation |
apache/sites-available/system-core.conf |
WordPress multisite: mod_remoteip, HTTPS reconstruction, PHP-FPM 8.4, health endpoint |
| Service | Port | Binding |
|---|---|---|
| Nginx HTTPS | 443 | Public |
| Nginx HTTP (redirect) | 80 | Public |
| Varnish | 6081 | 127.0.0.1 |
| Apache + WordPress | 8080 | 127.0.0.1 |
| TUS Node server | 1080 | 127.0.0.1 |
# TLS certificates (Nginx)
/etc/ssl/certs/system-core.crt
/etc/ssl/private/system-core.key
# Worker-to-Origin shared secret (Nginx)
# Format — one line: "your-shared-secret-value" 1;
# This is referenced by: map $http_x_worker_origin_secret $is_trusted_worker
# Never commit the actual secret value to this repository.
/etc/nginx/secrets/worker-secret.conf
# PHP-FPM 8.4 socket (Apache)
/run/php/php8.4-fpm.sock
# Varnish / Apache health probe endpoint (plain text, no PHP)
echo "OK" > /var/www/html/health && chmod 644 /var/www/html/healthln -s /etc/nginx/sites-available/system-core.conf \
/etc/nginx/sites-enabled/system-core.confa2ensite system-core
a2enmod remoteip setenvif proxy_fcgi rewrite headers
systemctl reload apache2mod_remoteip— real visitor IP fromX-Forwarded-Formod_setenvif—HTTPS=onreconstruction before Mercator SUNRISEmod_proxy_fcgi— PHP-FPM 8.4 handlermod_rewrite— WordPress multisite rewritesmod_headers—Cache-Controlon admin routes
Nginx set_real_ip_from directives must be kept in sync with Cloudflare's
published ranges. Verify at each infrastructure review:
The X-Worker-Origin-Secret header is the secondary trust gate for
SPARXSTAR edge-auth headers (Section 14.4). Without a valid secret, Nginx
evaluates all SPARXSTAR headers to "" regardless of source IP, preventing
header spoofing even if a Cloudflare IP is somehow reachable directly.
Rotate the secret value by:
- Updating
/etc/nginx/secrets/worker-secret.conf - Updating the matching secret in the Cloudflare Worker (Worker Secrets)
- Reloading Nginx:
nginx -s reload
Cloudflare Worker
→ sets X-SPARXSTAR-* headers after JWT validation
→ sets X-Worker-Origin-Secret: <shared-secret>
→ forwards to Nginx origin
Nginx
→ verifies $is_trusted_worker via map on X-Worker-Origin-Secret
→ $pass_sparxstar_* maps: preserve headers if trusted, "" if not
→ forwards to Varnish → Apache
ssl_early_data off is set globally. 0-RTT must NOT be enabled on any
route that modifies state (auth, payments, form submissions) due to replay
attack risk (Section 4.1).
The default config ships with max-age=31536000 only (no includeSubDomains)
to prevent accidentally locking subdomains that are not yet HTTPS-only.
Before enabling includeSubDomains:
- Confirm every subdomain (
www.*,api.*,cdn.*,mail.*, etc.) has a valid TLS certificate and redirects HTTP → HTTPS. - Confirm no subdomain serves content over plain HTTP that must be reachable by end users.
- Once confirmed, update
nginx/sites-available/system-core.conf:add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
- Optionally add
; preloadand submit to https://hstspreload.org/ once theincludeSubDomainsversion has been live and stable for several weeks.
Authenticated sessions are identified by any of the following cookies:
wp_logged_inwordpress_logged_in_*wp-postpass_*woocommerce_cart_hashwoocommerce_items_in_cart*wp_woocommerce_session_*woocommerce_recently_viewedstore_notice*(e.g.store_notice[noticeid])SCF_*
Requests carrying these cookies bypass the cache entirely (Varnish pass).
All other cookies are stripped before the cache lookup so anonymous pages
are served from cache without cookie fragmentation.
Nginx logs include real_ip, country, and block_reason fields.
Fail2Ban rules must read the real_ip field, which holds $remote_addr —
the client IP reconstructed from CF-Connecting-IP by Nginx's
real_ip_header + set_real_ip_from processing. Do not write Fail2Ban
rules against the raw cf_connecting_ip field (the unverified request header)
or any leading IP in the common log field, which for Cloudflare-proxied traffic
would be a Cloudflare edge node address rather than the visitor's IP.
Large audio files are uploaded via the TUS resumable upload protocol
at the /files/ path. Every layer in the pipeline is explicitly configured to
give this traffic unobstructed, buffering-free passage.
Browser / Mobile Client
→ HTTPS POST/PATCH /files/ (TUS protocol)
Cloudflare
→ passes through; WAF rules must whitelist TUS methods (PATCH, HEAD, OPTIONS)
Nginx (perimeter)
→ location /files/ — proxy_request_buffering off
proxy_buffering off
client_max_body_size 0 (no body size limit)
proxy_read_timeout 3600s
proxy_send_timeout 3600s
→ proxies directly to TUS Node server (port 1080)
→ UA checks and rate limiting are EXEMPT for /files/ (TUS clients send
minimal headers per TUS spec §1.5)
Varnish (belt-and-suspenders)
→ req.backend_hint = tus_node (port 1080)
→ return(pipe) — Varnish does NOT buffer or cache any part of the upload
→ vcl_pipe sets Connection: close to prevent connection reuse after pipe ends
→ TUS Node backend has extended timeouts (first_byte=300s,
between_bytes=120s) to accommodate large files on slow mobile links
TUS Node server (port 1080)
→ stores chunks, manages upload state, assembles final audio file
→ Apache is never involved in the upload path
Varnish pass still buffers the full request body before forwarding.
pipe establishes a raw TCP tunnel between the client and the TUS Node,
forwarding bytes without buffering — essential for large audio files that
can exceed available Varnish memory and for TUS PATCH requests that must
not be interrupted.
| Layer | Setting | Value | Reason |
|---|---|---|---|
| Nginx | proxy_read_timeout |
3600s | Long-lived TUS sessions |
| Nginx | proxy_send_timeout |
3600s | Slow mobile uplinks |
| Nginx | proxy_connect_timeout |
75s | Loopback; 75s is generous |
Varnish tus_node |
first_byte_timeout |
300s | TUS Node may delay ACK for large chunks |
Varnish tus_node |
between_bytes_timeout |
120s | Bursty retry patterns on mobile |
Once the TUS Node has assembled the audio file it is moved to the served
assets path. Varnish caches processed audio and video assets with a
30-day TTL and Vary: Accept so clients that negotiate different
container formats receive the correct variant from cache.
| Asset type | Extensions |
|---|---|
| Audio | mp3, ogg, weba, wav, flac, aac, m4a, opus |
| Video | mp4, webm, mov, avi, mkv, m4v, mpeg, mpg, flv, wmv |
Static document assets are also served from the assets path and cached by
Varnish with the same 30-day TTL, but they do not use Vary: Accept
because there is no content-negotiated variant for these types.
| Asset type | Extensions |
|---|---|
| Documents | pdf, doc, docx, ppt, pptx, xls, xlsx |
This section documents key differences from the previous nginx.conf /
conf.d/spx-bot-mitigation-logic.conf setup to help ensure a smooth
cut-over without hard-to-diagnose regressions.
-
Disable
conf.d/spx-bot-mitigation-logic.confbefore activating the new config. The old file declaresgeoandmapblocks for UA/bot signals. The newnginx.confsubsumes all of these maps at http scope. Running both simultaneously will cause duplicate variable declaration errors and prevent Nginx from starting.mv /etc/nginx/conf.d/spx-bot-mitigation-logic.conf \ /etc/nginx/conf.d/spx-bot-mitigation-logic.conf.bak
-
Test config before reload:
nginx -t && nginx -s reload -
Update Fail2Ban filter regexes — the log format name is unchanged (
sparxstar_telemetry), but the field structure within log lines has changed. Verify your Fail2Ban filters match the new field names:real_ip="..."— use this field for ban decisions (TCP-verified IP); previously logged asCF-IP:...— filters must be updated to match the new quoted-field syntaxcf_connecting_ip="..."— raw CF header (informational only)trace_id="..."— new field; safe to ignore in Fail2Ban filters
| Area | Previous config | New config | Notes |
|---|---|---|---|
| Log format name | sparxstar_telemetry |
sparxstar_telemetry |
Unchanged — zero Fail2Ban disruption |
| Trace ID | TraceID:$request_id |
trace_id="$request_id" |
Same Nginx built-in variable; renamed field for log-parsing consistency; forwarded as X-Request-ID header to Varnish and Apache |
real_ip log field |
CF-IP:$http_cf_connecting_ip (raw header, spoofable) |
real_ip="$remote_addr" (TCP-restored, unspoofable) |
Fail2Ban rules must target real_ip not CF-IP |
| GeoIP ASN logging | ASN:$geoip2_asn ORG:$geoip2_asn_org (MaxMind) |
Removed — uses country="$http_cf_ipcountry" |
Eliminates MaxMind database dependency; Cloudflare provides country via header |
client_max_body_size |
500M (global) |
10m (global default); 0 for /files/, 1m for /graphql, 64m for /submission |
Per-route limits are safer; TUS path still has no limit |
| TUS rate limiting | limit_conn_zone tus_conn (concurrency) |
Exempt — no rate limit on /files/ |
/files/ is unconditionally exempt from both rate limiting and UA checks |
| Bot mitigation logic | conf.d/spx-bot-mitigation-logic.conf |
Inline in nginx.conf (map blocks) + sites-available/system-core.conf (set/if blocks) |
Must disable old conf.d file before activating new config |
| Cloudflare-only gate | Not present | geo $realip_remote_addr $from_cloudflare → return 403 at server scope |
New: blocks all non-Cloudflare direct-to-origin traffic using TCP connection IP |
| SPARXSTAR headers | Not gated | $pass_sparxstar_* maps gated on X-Worker-Origin-Secret |
New: header spoofing protection — headers only forwarded from verified CF Worker |
Previous format:
$remote_addr ... CF-IP:$http_cf_connecting_ip Geo:$http_cf_ipcountry TraceID:$request_id BlockReason:$block_reason ASN:$geoip2_asn ORG:$geoip2_asn_org
New format:
$remote_addr ... real_ip="$remote_addr" cf_connecting_ip="$http_cf_connecting_ip" country="$http_cf_ipcountry" trace_id="$request_id" host="$host" block_reason="$block_reason"
The $request_id trace ID is also forwarded as X-Request-ID to all
upstream services (Varnish, Apache, TUS Node), enabling end-to-end request
correlation across all log files using a single ID.