mod_reqin_log

Apache HTTPD 2.4 module for logging all incoming HTTP requests as JSON lines to a Unix domain socket.

Features

• Non-blocking I/O: Logging never blocks worker processes
• Request-time logging: Logs at post_read_request phase, capturing request data before application processing
• Configurable headers: Select which HTTP headers to include in logs
• Header truncation: Limit header value length to protect against oversized logs
• Automatic reconnection: Reconnects to Unix socket on failure with configurable backoff
• Throttled error reporting: Prevents error_log flooding on persistent failures
• MPM compatible: Works with prefork, worker, and event MPMs

Requirements

• Apache HTTPD 2.4+
• GCC compiler
• APR development libraries
• Apache development headers (httpd-devel or apache2-dev)

Installation

Using Docker (recommended)

# Build all packages (DEB + RPMs for el7, el8, el9)
make package

# Test package installation
make test-package-deb       # Test DEB in Docker container
make test-package-rpm-el7   # Test el7 RPM (CentOS 7/RHEL 7)
make test-package-rpm-el8   # Test el8 RPM (Rocky 8/RHEL 8)
make test-package-rpm-el9   # Test el9 RPM (Rocky 9/RHEL 9)
make test-package           # Test all packages

Build from Source

# Clone or extract the source
cd mod_reqin_log

# Build the module
make

# Install (requires root privileges)
sudo make install

Configuration

Load the module and configure in your Apache configuration:

# Load the module
LoadModule reqin_log_module modules/mod_reqin_log.so

# Enable logging
JsonSockLogEnabled On

# Unix socket path
JsonSockLogSocket "/var/run/mod_reqin_log.sock"

# Headers to log (be careful not to log sensitive data)
JsonSockLogHeaders X-Request-Id X-Trace-Id User-Agent Referer

# Maximum headers to log
JsonSockLogMaxHeaders 10

# Maximum header value length
JsonSockLogMaxHeaderValueLen 256

# Reconnect interval (seconds)
JsonSockLogReconnectInterval 10

# Error report interval (seconds)
JsonSockLogErrorReportInterval 10

Important startup validation: if JsonSockLogEnabled On is set without a valid JsonSockLogSocket, Apache startup fails with a configuration error.

Configuration Directives

Directive	Type	Default	Description
JsonSockLogEnabled	On/Off	Off	Enable or disable logging
JsonSockLogSocket	String	-	Unix domain socket path
JsonSockLogHeaders	List	-	HTTP headers to include
JsonSockLogMaxHeaders	Integer	10	Max headers to log
JsonSockLogMaxHeaderValueLen	Integer	256	Max header value length
JsonSockLogReconnectInterval	Integer	10	Reconnect delay (seconds)
JsonSockLogErrorReportInterval	Integer	10	Error log throttle (seconds)

JSON Log Format

Each log entry is a single-line JSON object with a flat structure:

{
  "time": "2026-02-26T11:59:30Z",
  "timestamp": 1708948770000000000,
  "src_ip": "192.0.2.10",
  "src_port": 45678,
  "dst_ip": "198.51.100.5",
  "dst_port": 443,
  "method": "GET",
  "path": "/api/users",
  "host": "example.com",
  "http_version": "HTTP/1.1",
  "header_X-Request-Id": "abcd-1234",
  "header_User-Agent": "curl/7.70.0"
}

Fields

Field	Type	Description
time	String	ISO8601 timestamp with timezone
timestamp	Integer	Nanoseconds since epoch
src_ip	String	Client IP address
src_port	Integer	Client port
dst_ip	String	Server IP address
dst_port	Integer	Server port
method	String	HTTP method
path	String	Request path
host	String	Host header value
http_version	String	HTTP protocol version
header_<Name>	String	Flattened HTTP headers (e.g., header_X-Request-Id)

Note: Headers are logged as flat fields at the root level (not nested). Sensitive headers are automatically excluded.

Unix Socket Consumer

Create a Unix socket listener to receive log entries:

#!/usr/bin/env python3
import socket
import os
import json

SOCKET_PATH = os.environ.get("MOD_REQIN_LOG_SOCKET", "/var/run/mod_reqin_log.sock")

# Remove existing socket file
if os.path.exists(SOCKET_PATH):
    os.remove(SOCKET_PATH)

# Create Unix socket server
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
server.bind(SOCKET_PATH)
server.listen(5)
# Set secure permissions: owner and group only (not world-writable)
os.chmod(SOCKET_PATH, 0o660)

print(f"Listening on {SOCKET_PATH}")

while True:
    conn, addr = server.accept()
    data = b""
    while True:
        chunk = conn.recv(4096)
        if not chunk:
            break
        data += chunk
        if b"\n" in data:
            for line in data.decode().strip().split("\n"):
                if line:
                    log_entry = json.loads(line)
                    print(json.dumps(log_entry, indent=2))
            data = b""
    conn.close()

Note: Ensure the Apache user is in the socket file's group to allow connections.

Security Considerations

Built-in Sensitive Headers Blacklist

⚠️ The module automatically blocks logging of sensitive headers:

The following headers are always excluded from logs to prevent credential leakage:

• Authorization
• Cookie, Set-Cookie
• X-Api-Key, X-Auth-Token
• Proxy-Authorization
• WWW-Authenticate

These headers are silently skipped (logged at DEBUG level only).

Socket Security

• Socket permissions: Default to 0o660 (owner and group only)
• Recommended path: /var/run/mod_reqin_log.sock (not /tmp)
• Environment variable: Use MOD_REQIN_LOG_SOCKET to configure path
• Group membership: Ensure Apache user is in the socket's group

Additional Hardening

• Socket path length: Validated against system limit (108 bytes max)
• JSON size limit: 64KB max per log line (prevents memory DoS)
• NULL pointer checks: All connection/request fields validated
• Thread safety: Mutex protects socket FD in worker/event MPMs
• Error logging: Generic messages in error_log, details at DEBUG level

Troubleshooting

Module not loading

AH00534: mod_reqin_log: Unable to load module

Ensure the module path is correct and the file exists:

ls -la /usr/lib/apache2/modules/mod_reqin_log.so

Socket connection failures

[mod_reqin_log] Unix socket connect failed: /var/run/mod_reqin_log.sock

• Ensure the socket consumer is running
• Check socket file permissions
• Verify SELinux/AppArmor policies if applicable

No logs appearing

1. Verify JsonSockLogEnabled On is set
2. Verify JsonSockLogSocket path is configured
3. Check Apache error log for module errors
4. Ensure socket consumer is listening

Testing

Run Unit Tests

# Using Docker (recommended)
docker build -f Dockerfile.tests -t mod_reqin_log:tests .
docker run --rm mod_reqin_log:tests ctest --output-on-failure

# Or locally with cmocka
sudo dnf install cmocka-devel  # Rocky Linux
sudo apt install libcmocka-dev # Debian/Ubuntu
mkdir build && cd build
cmake ..
make test

Integration Testing

# Using GitLab CI (recommended)
# All integration tests run automatically in CI

# Or manually with the Python test suite
python3 tests/integration/test_integration.py --url http://localhost:8080

Build and Test Packages

# Build all packages (DEB + RPMs for el7, el8, el9)
make package

# Test package installation
make test-package-deb       # Test DEB in Docker
make test-package-rpm-el7   # Test el7 RPM in Docker
make test-package-rpm-el8   # Test el8 RPM in Docker
make test-package-rpm-el9   # Test el9 RPM in Docker
make test-package           # Test all packages

License

Apache License 2.0

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run tests
5. Submit a pull request