fix(ja4ebpf): split bpf2go generate into Ja4Tc + Ja4Ssl, fix RPM systemd-rpm-macros
- Use two separate //go:generate directives (Ja4Tc for tc_capture.c, Ja4Ssl
for uprobe_ssl.c) to avoid duplicate LICENSE symbol and multi-file clang issue
- Update loader.go to hold tcObjs/sslObjs separately with correct field names:
UprobeSslSetFd, UprobeSslReadEntry, UretprobeSslReadExit,
KprobeAccept4Entry, KretprobeAccept4Exit
- Add systemd-rpm-macros to all three RPM build stages (el8/el9/el10)
so that %{_unitdir} macro resolves correctly
- RPMs now build successfully for el8, el9, el10
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
toto
284
old/services/mod-reqin-log/README.md
Normal file
284
old/services/mod-reqin-log/README.md
Normal file
@ -0,0 +1,284 @@
|
||||
# mod_reqin_log
|
||||
|
||||
Apache HTTPD 2.4 module for logging all incoming HTTP requests as JSON lines to a Unix domain socket.
|
||||
|
||||
## Features
|
||||
|
||||
- **Non-blocking I/O**: Logging never blocks worker processes
|
||||
- **Request-time logging**: Logs at `post_read_request` phase, capturing request data before application processing
|
||||
- **Configurable headers**: Select which HTTP headers to include in logs
|
||||
- **Header truncation**: Limit header value length to protect against oversized logs
|
||||
- **Automatic reconnection**: Reconnects to Unix socket on failure with configurable backoff
|
||||
- **Throttled error reporting**: Prevents error_log flooding on persistent failures
|
||||
- **MPM compatible**: Works with prefork, worker, and event MPMs
|
||||
- **Built-in security**: Sensitive headers (Authorization, Cookie, etc.) are automatically excluded
|
||||
- **RPM packaging**: Standard RPM packages for Rocky Linux 8/9 and AlmaLinux 10
|
||||
|
||||
## Requirements
|
||||
|
||||
### Runtime
|
||||
- Apache HTTPD 2.4+
|
||||
- GCC compiler
|
||||
- APR development libraries
|
||||
- Apache development headers (`httpd-devel` or `apache2-dev`)
|
||||
|
||||
### Packaging (RPM)
|
||||
- Docker (for reproducible builds)
|
||||
- rpmbuild (inside Docker)
|
||||
|
||||
## Installation
|
||||
|
||||
### Using Docker (recommended)
|
||||
|
||||
```bash
|
||||
# Build all RPM packages (el8, el9, el10)
|
||||
make package
|
||||
|
||||
# Test RPM package installation
|
||||
make test-package-rpm-el8 # Test el8 RPM (Rocky 8/RHEL 8)
|
||||
make test-package-rpm-el9 # Test el9 RPM (Rocky 9/RHEL 9)
|
||||
make test-package-rpm-el10 # Test el10 RPM (AlmaLinux 10/RHEL 10)
|
||||
make test-package # Test all RPM packages
|
||||
```
|
||||
|
||||
### Build from Source
|
||||
|
||||
```bash
|
||||
# Clone or extract the source
|
||||
cd mod_reqin_log
|
||||
|
||||
# Build the module
|
||||
make
|
||||
|
||||
# Install (requires root privileges)
|
||||
sudo make install
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Load the module and configure in your Apache configuration:
|
||||
|
||||
```apache
|
||||
# Load the module
|
||||
LoadModule reqin_log_module modules/mod_reqin_log.so
|
||||
|
||||
# Enable logging
|
||||
JsonSockLogEnabled On
|
||||
|
||||
# Unix socket path
|
||||
JsonSockLogSocket "/var/run/logcorrelator/http.socket"
|
||||
|
||||
# Headers to log (be careful not to log sensitive data)
|
||||
JsonSockLogHeaders X-Request-Id X-Trace-Id User-Agent Referer
|
||||
|
||||
# Maximum headers to log
|
||||
JsonSockLogMaxHeaders 10
|
||||
|
||||
# Maximum header value length
|
||||
JsonSockLogMaxHeaderValueLen 256
|
||||
|
||||
# Reconnect interval (seconds)
|
||||
JsonSockLogReconnectInterval 10
|
||||
|
||||
# Error report interval (seconds)
|
||||
JsonSockLogErrorReportInterval 10
|
||||
```
|
||||
|
||||
> **Important startup validation:** if `JsonSockLogEnabled On` is set without a valid `JsonSockLogSocket`, Apache startup fails with a configuration error.
|
||||
|
||||
### Configuration Directives
|
||||
|
||||
| Directive | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `JsonSockLogEnabled` | On/Off | Off | Enable or disable logging |
|
||||
| `JsonSockLogSocket` | String | - | Unix domain socket path |
|
||||
| `JsonSockLogHeaders` | List | - | HTTP headers to include |
|
||||
| `JsonSockLogMaxHeaders` | Integer | 10 | Max headers to log |
|
||||
| `JsonSockLogMaxHeaderValueLen` | Integer | 256 | Max header value length |
|
||||
| `JsonSockLogReconnectInterval` | Integer | 10 | Reconnect delay (seconds) |
|
||||
| `JsonSockLogErrorReportInterval` | Integer | 10 | Error log throttle (seconds) |
|
||||
|
||||
## JSON Log Format
|
||||
|
||||
Each log entry is a single-line JSON object with a flat structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"time": "2026-02-26T11:59:30Z",
|
||||
"timestamp": 1708948770000000000,
|
||||
"src_ip": "192.0.2.10",
|
||||
"src_port": 45678,
|
||||
"dst_ip": "198.51.100.5",
|
||||
"dst_port": 443,
|
||||
"method": "GET",
|
||||
"path": "/api/users",
|
||||
"host": "example.com",
|
||||
"http_version": "HTTP/1.1",
|
||||
"header_X-Request-Id": "abcd-1234",
|
||||
"header_User-Agent": "curl/7.70.0"
|
||||
}
|
||||
```
|
||||
|
||||
### Fields
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `time` | String | ISO8601 timestamp with timezone |
|
||||
| `timestamp` | Integer | Microseconds since epoch (expressed as nanoseconds for compatibility) |
|
||||
| `src_ip` | String | Client IP address |
|
||||
| `src_port` | Integer | Client port |
|
||||
| `dst_ip` | String | Server IP address |
|
||||
| `dst_port` | Integer | Server port |
|
||||
| `method` | String | HTTP method |
|
||||
| `path` | String | Request path |
|
||||
| `host` | String | Host header value |
|
||||
| `http_version` | String | HTTP protocol version |
|
||||
| `header_<Name>` | String | Flattened HTTP headers (e.g., `header_X-Request-Id`) |
|
||||
|
||||
**Note:** Headers are logged as flat fields at the root level (not nested). Sensitive headers are automatically excluded. The `timestamp` field has microsecond precision (APR's `apr_time_now()` returns microseconds, multiplied by 1000 for nanosecond representation).
|
||||
|
||||
## Unix Socket Consumer
|
||||
|
||||
Create a Unix socket listener to receive log entries:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import socket
|
||||
import os
|
||||
import json
|
||||
|
||||
SOCKET_PATH = os.environ.get("MOD_REQIN_LOG_SOCKET", "/var/run/logcorrelator/http.socket")
|
||||
|
||||
# Remove existing socket file
|
||||
if os.path.exists(SOCKET_PATH):
|
||||
os.remove(SOCKET_PATH)
|
||||
|
||||
# Create Unix socket server
|
||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
server.bind(SOCKET_PATH)
|
||||
server.listen(5)
|
||||
# Set secure permissions: owner and group only (not world-writable)
|
||||
os.chmod(SOCKET_PATH, 0o660)
|
||||
|
||||
print(f"Listening on {SOCKET_PATH}")
|
||||
|
||||
while True:
|
||||
conn, addr = server.accept()
|
||||
data = b""
|
||||
while True:
|
||||
chunk = conn.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
data += chunk
|
||||
if b"\n" in data:
|
||||
for line in data.decode().strip().split("\n"):
|
||||
if line:
|
||||
log_entry = json.loads(line)
|
||||
print(json.dumps(log_entry, indent=2))
|
||||
data = b""
|
||||
conn.close()
|
||||
```
|
||||
|
||||
**Note:** Ensure the Apache user is in the socket file's group to allow connections.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Built-in Sensitive Headers Blacklist
|
||||
|
||||
⚠️ **The module automatically blocks logging of sensitive headers:**
|
||||
|
||||
The following headers are **always excluded** from logs to prevent credential leakage:
|
||||
- `Authorization`
|
||||
- `Cookie`, `Set-Cookie`
|
||||
- `X-Api-Key`, `X-Auth-Token`
|
||||
- `Proxy-Authorization`
|
||||
- `WWW-Authenticate`
|
||||
|
||||
These headers are silently skipped (logged at DEBUG level only).
|
||||
|
||||
### Socket Security
|
||||
|
||||
- **Socket permissions**: Default to `0o660` (owner and group only)
|
||||
- **Recommended path**: `/var/run/logcorrelator/http.socket` (not `/tmp`)
|
||||
- **Environment variable**: Use `MOD_REQIN_LOG_SOCKET` to configure path
|
||||
- **Group membership**: Ensure Apache user is in the socket's group
|
||||
|
||||
### Additional Hardening
|
||||
|
||||
- **Socket path length**: Validated against system limit (108 bytes max)
|
||||
- **JSON size limit**: 64KB max per log line (prevents memory DoS)
|
||||
- **NULL pointer checks**: All connection/request fields validated
|
||||
- **Thread safety**: Mutex protects socket FD in worker/event MPMs
|
||||
- **Error logging**: Generic messages in error_log, details at DEBUG level
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Module not loading
|
||||
|
||||
```
|
||||
AH00534: mod_reqin_log: Unable to load module
|
||||
```
|
||||
|
||||
Ensure the module path is correct and the file exists:
|
||||
```bash
|
||||
ls -la /usr/lib/apache2/modules/mod_reqin_log.so
|
||||
```
|
||||
|
||||
### Socket connection failures
|
||||
|
||||
```
|
||||
[mod_reqin_log] Unix socket connect failed: /var/run/logcorrelator/http.socket
|
||||
```
|
||||
|
||||
- Ensure the socket consumer is running
|
||||
- Check socket file permissions
|
||||
- Verify SELinux/AppArmor policies if applicable
|
||||
|
||||
### No logs appearing
|
||||
|
||||
1. Verify `JsonSockLogEnabled On` is set
|
||||
2. Verify `JsonSockLogSocket` path is configured
|
||||
3. Check Apache error log for module errors
|
||||
4. Ensure socket consumer is listening
|
||||
|
||||
## Testing
|
||||
|
||||
### Run Unit Tests
|
||||
|
||||
```bash
|
||||
# Using Docker (recommended)
|
||||
docker build -f Dockerfile.tests -t mod_reqin_log:tests .
|
||||
docker run --rm mod_reqin_log:tests ctest --output-on-failure
|
||||
|
||||
# Or locally with cmocka
|
||||
sudo dnf install cmocka-devel # Rocky Linux
|
||||
sudo apt install libcmocka-dev # Debian/Ubuntu
|
||||
mkdir build && cd build
|
||||
cmake ..
|
||||
make test
|
||||
```
|
||||
|
||||
### Build and Test Packages
|
||||
|
||||
```bash
|
||||
# Build all RPM packages (el8, el9, el10)
|
||||
make package
|
||||
|
||||
# Test RPM package installation
|
||||
make test-package-rpm-el8 # Test el8 RPM in Docker
|
||||
make test-package-rpm-el9 # Test el9 RPM in Docker
|
||||
make test-package-rpm-el10 # Test el10 RPM in Docker
|
||||
make test-package # Test all RPM packages
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Apache License 2.0
|
||||
|
||||
## Contributing
|
||||
|
||||
1. Fork the repository
|
||||
2. Create a feature branch
|
||||
3. Make your changes
|
||||
4. Run tests
|
||||
5. Submit a pull request
|
||||
Reference in New Issue
Block a user