Resource Limits Configuration

Configure resource constraints to ensure stable operation.

Configuration Options

max_memory_mb

Maximum memory usage in megabytes.

[limits]
max_memory_mb = 8192

When approached, the proxy may: - Reject new connections - Evict cached data - Trigger garbage collection

Sizing guidelines:

Connections	Recommended Memory
10,000	512 MB
50,000	2 GB
100,000	4 GB
500,000	16 GB

max_cpu_percent

Maximum CPU usage percentage.

[limits]
max_cpu_percent = 80

Valid range: 1-100

Leaving headroom (80%) allows for: - System processes - Burst handling - Monitoring agents

max_network_mbps

Maximum network throughput in megabits per second.

[limits]
max_network_mbps = 1000

Helps prevent network saturation on shared infrastructure.

max_file_descriptors

Maximum file descriptors to use.

[limits]
max_file_descriptors = 1000000

Each connection requires at least one file descriptor.

System configuration required:

# Check current limits
ulimit -n

# Set for current session
ulimit -n 1000000

# Permanent (add to /etc/security/limits.conf)
* soft nofile 1000000
* hard nofile 1000000

Resource Planning

Memory Calculation

Estimate memory usage:

Base memory:           ~50 MB
Per connection:        ~2 KB
Per cached JWT:        ~500 bytes
Per topic:             ~1 KB
Bloom filter:          capacity * 1.44 * ln(1/fpr) / 8 bytes
Message buffers:       thread_count * queue_size * avg_msg_size

Example: - 100,000 connections: 200 MB - 10,000 cached JWTs: 5 MB - 10,000 topics: 10 MB - Bloom filter (1M, 1%): 1.2 MB - 16 threads * 1000 queue * 1KB: 16 MB - Total: ~280 MB + 50 MB base = ~330 MB

CPU Considerations

CPU usage scales with: - Connection rate (handshakes) - Message throughput - JWT verification (if not cached) - Encryption overhead (TLS termination)

Benchmarks (per core): - Connection handling: ~10,000 conn/sec - Message delivery: ~100,000 msg/sec - JWT verification: ~1,000 verify/sec

File Descriptor Planning

Required FDs = connections + (peers * 2) + (listen_sockets) + overhead

Example:
- 100,000 connections
- 3 peers (6 FDs)
- 1 listen socket
- 100 overhead (logs, etc.)
= ~100,107 file descriptors

System Tuning

Linux Kernel Parameters

Add to /etc/sysctl.conf:

# Network stack
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.ip_local_port_range = 1024 65535

# TCP keepalive
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_keepalive_probes = 6

# Memory
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

# File descriptors
fs.file-max = 2000000
fs.nr_open = 2000000

Apply with:

sudo sysctl -p

Systemd Service Limits

In /etc/systemd/system/revenprox.service:

[Service]
LimitNOFILE=1000000
LimitNPROC=65535
LimitCORE=infinity

Container Limits

Docker:

docker run -d \
  --ulimit nofile=1000000:1000000 \
  --memory=8g \
  --cpus=4 \
  revenprox/sse-proxy

Kubernetes:

resources:
  limits:
    memory: 8Gi
    cpu: "4"
  requests:
    memory: 4Gi
    cpu: "2"

Example Configurations

Small Deployment

Up to 10,000 connections:

[limits]
max_memory_mb = 512
max_cpu_percent = 80
max_network_mbps = 100
max_file_descriptors = 16384

Medium Deployment

10,000 - 100,000 connections:

[limits]
max_memory_mb = 4096
max_cpu_percent = 80
max_network_mbps = 1000
max_file_descriptors = 131072

Large Deployment

100,000+ connections:

[limits]
max_memory_mb = 16384
max_cpu_percent = 90
max_network_mbps = 10000
max_file_descriptors = 1000000

Monitoring Limits

Track resource usage against limits:

Metric	Description	Alert When
`memory_used_mb`	Current memory usage	> 80% of max
`cpu_percent`	Current CPU usage	> 90% sustained
`fd_used`	File descriptors in use	> 80% of max
`connections_rejected`	Rejected due to limits	> 0

Graceful Degradation

When limits are approached:

80% threshold: Log warnings
90% threshold: Reject new connections
95% threshold: Start draining connections
100% threshold: Emergency measures (may drop connections)

Configure thresholds in your monitoring system:

# Prometheus alert example
- alert: RevenProxMemoryHigh
  expr: revenprox_memory_used_mb / revenprox_memory_limit_mb > 0.8
  for: 5m
  labels:
    severity: warning
  annotations:
    summary: "RevenProx memory usage above 80%"