Operations & Reliability
Everything you need to run Gatekeeper in production with confidence.
The delivery pipeline#
report() ──> severity floor ──> redaction ──> per-sink floor
│
ShipEvent job (queue: siem)
│
GeoIP enrichment ──> sink->send()
│
retry ×3 (5s / 30s / 120s backoff)
│
final failure ──> local log (full event)
Two guarantees fall out of this design:
- No request latency. All network I/O (Sentinel, webhooks, GeoIP) happens on the worker.
- No silent loss. A sink that fails all retries writes the complete event document to the Laravel log, tagged with the sink name and error.
Queue worker#
php artisan queue:work --queue=siem --tries=3
Supervisor example:
[program:gatekeeper-worker]
command=php /var/www/app/artisan queue:work --queue=siem --sleep=1 --tries=3
autostart=true
autorestart=true
user=www-data
numprocs=1
Sizing: security events are low-volume compared to application jobs — one worker is typically plenty. If you share a worker across queues, list siem explicitly (--queue=default,siem) so events don't starve behind bulk jobs.
Synchronous mode (SIEM_QUEUE=false) is fine for local development and the file/log sinks, but puts Sentinel/webhook latency inside user requests — avoid in production.
Testing connectivity#
php artisan siem:test # every active sink
php artisan siem:test --sink=wazuh # one sink
Sends a synthetic siem_connectivity_test event synchronously (bypassing the queue and per-sink severity floors) and reports per-sink success or the exact exception. Run it after deploys and config changes; wire it into a health check if your SIEM connection is critical.
Noise management#
Layered controls keep the SIEM signal-rich and the alert channel quiet:
| Layer | Mechanism |
|---|---|
| Global floor | SIEM_MIN_SEVERITY drops events below a severity outright |
| Per-sink floor | minimum_severity per sink (Slack/Teams default to high) |
| Detection dedup | Abuse and enumeration detections fire once per IP per window |
| Blocked-IP throttle | blocked_ip_attempt at most once per IP per 5 minutes |
| Event toggles | Each auth event individually switchable |
A common production profile: everything ≥ info to Sentinel/Wazuh (full forensic trail), ≥ high to Slack (human attention), auth success events off if volume is a cost concern.
Multi-server deployments#
- Use a shared cache (
SIEM_CACHE_STORE=redis) so blocklists, abuse counters and enumeration windows are cluster-wide, not per-server. - Configure trusted proxies so
source.ipis the client, not your load balancer. - The Wazuh file sink writes per-server files — run an agent per server (normal Wazuh practice), or centralize with the syslog sink instead.
Local development#
SIEM_SINKS=log
SIEM_QUEUE=false
SIEM_HONEYPOT_AUTOBLOCK=false
Events appear in storage/logs/laravel.log immediately; nothing external is called; you can't lock yourself out by curling a honeypot.
To silence Gatekeeper entirely (e.g. in a test suite): SIEM_ENABLED=false.
Deployment checklist#
- [ ]
config/siem.phppublished and honeypot paths reviewed against real routes - [ ] Queue worker running and supervised for the
siemqueue - [ ]
php artisan siem:testgreen for every active sink - [ ] Trusted proxies configured (blocklist + per-IP detection depend on it)
- [ ]
SIEM_NEVER_BLOCKcovers health checks, monitors and office ranges - [ ] Shared cache store in multi-server setups
- [ ] GeoIP database update cron (MaxMind driver)
- [ ] Wazuh log rotation with
copytruncate(file sink) - [ ] SIEM-side: analytics rules / Wazuh rules created for
event_severity >= 7 - [ ] Slack/Teams webhook secrets stored in your secret manager, not committed
Troubleshooting#
| Symptom | Likely cause |
|---|---|
| No events anywhere | SIEM_ENABLED=false, no worker on the siem queue, or SIEM_MIN_SEVERITY too high |
| Events in log but not SIEM | Sink misconfigured — run siem:test --sink=<name> for the exact error |
Sink [x] failed in logs |
Delivery failed after retries; the logged entry contains the full event, nothing was lost |
| Everything from one IP | Trusted proxies not configured |
| Blocklist ineffective on some servers | Per-server cache store; switch to Redis |
| Slack/Teams silent | Event below the sink's minimum_severity — siem:test bypasses the floor, so use it to confirm the webhook itself |