code_public/logcorrelator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: remove obsolete config and update documentation

Browse Source 
- Remove config.example.conf (replaced by config.example.yml)
- Update Dockerfile to use YAML config
- Update README.md with YAML configuration examples
- Remove old directive-based config documentation
- Update package paths (DEB and RPM) in README

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
This commit is contained in:
Jacquin Antoine
2026-02-27 16:14:53 +01:00
parent f4d95eed41
commit 85f7af357c
3 changed files with 52 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
Dockerfile
View File

@ -54,7 +54,7 @@ FROM gcr.io/distroless/base-debian12 AS runtime
COPY --from=builder /usr/bin/logcorrelator /usr/bin/logcorrelator COPY --from=builder /usr/bin/logcorrelator /usr/bin/logcorrelator
# Copy example config # Copy example config
COPY --from=builder /build/config.example.conf /etc/logcorrelator/logcorrelator.conf COPY --from=builder /build/config.example.yml /etc/logcorrelator/logcorrelator.yml
# Create necessary directories in builder stage (distroless has no shell) # Create necessary directories in builder stage (distroless has no shell)
COPY --from=builder /tmp/runtime-root/var /var COPY --from=builder /tmp/runtime-root/var /var
@ -65,7 +65,7 @@ COPY --from=builder /tmp/runtime-root/etc /etc
# Set entrypoint # Set entrypoint
ENTRYPOINT ["/usr/bin/logcorrelator"] ENTRYPOINT ["/usr/bin/logcorrelator"]
CMD ["-config", "/etc/logcorrelator/logcorrelator.conf"] CMD ["-config", "/etc/logcorrelator/logcorrelator.yml"]
# ============================================================================= # =============================================================================
# RPM build stage - create .rpm package entirely in Docker # RPM build stage - create .rpm package entirely in Docker
@ -84,7 +84,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
COPY --from=builder /usr/bin/logcorrelator /tmp/pkgroot/usr/bin/logcorrelator COPY --from=builder /usr/bin/logcorrelator /tmp/pkgroot/usr/bin/logcorrelator
# Copy config and systemd unit # Copy config and systemd unit
COPY --from=builder /build/config.example.conf /tmp/pkgroot/etc/logcorrelator/logcorrelator.conf COPY --from=builder /build/config.example.yml /tmp/pkgroot/etc/logcorrelator/logcorrelator.yml
COPY logcorrelator.service /tmp/pkgroot/etc/systemd/system/logcorrelator.service COPY logcorrelator.service /tmp/pkgroot/etc/systemd/system/logcorrelator.service
# Create directory structure and set permissions # Create directory structure and set permissions
@ -106,7 +106,7 @@ RUN fpm -s dir -t rpm \
--vendor "logcorrelator" \ --vendor "logcorrelator" \
-p /tmp/logcorrelator-${VERSION}.rpm \ -p /tmp/logcorrelator-${VERSION}.rpm \
usr/bin/logcorrelator \ usr/bin/logcorrelator \
etc/logcorrelator/logcorrelator.conf \ etc/logcorrelator/logcorrelator.yml \
etc/systemd/system/logcorrelator.service \ etc/systemd/system/logcorrelator.service \
var/log/logcorrelator \ var/log/logcorrelator \
var/run/logcorrelator var/run/logcorrelator
@ -147,4 +147,4 @@ RUN go mod download || true
COPY . . COPY . .
# Default command: run with example config # Default command: run with example config
CMD ["go", "run", "./cmd/logcorrelator", "-config", "config.example.conf"] CMD ["go", "run", "./cmd/logcorrelator", "-config", "config.example.yml"]

101
README.md
View File

@ -60,18 +60,21 @@ docker run -d \
--name logcorrelator \ --name logcorrelator \
-v /var/run/logcorrelator:/var/run/logcorrelator \ -v /var/run/logcorrelator:/var/run/logcorrelator \
-v /var/log/logcorrelator:/var/log/logcorrelator \ -v /var/log/logcorrelator:/var/log/logcorrelator \
-v ./config.conf:/etc/logcorrelator/logcorrelator.conf \ -v ./config.example.yml:/etc/logcorrelator/logcorrelator.yml \
logcorrelator:latest logcorrelator:latest
``` ```
### Depuis le package RPM (Rocky Linux 8+) ### Depuis les packages (DEB/RPM)
```bash ```bash
# Générer le RPM # Générer les packages
./build.sh ./build.sh
# Installer le package # Installer le package DEB (Debian/Ubuntu)
sudo rpm -ivh dist/logcorrelator-1.0.0.rpm sudo dpkg -i dist/deb/logcorrelator_1.0.0_amd64.deb
# Installer le package RPM (Rocky Linux 8+)
sudo rpm -ivh dist/rpm/logcorrelator-1.0.0-1.x86_64.rpm
# Activer et démarrer le service # Activer et démarrer le service
sudo systemctl enable logcorrelator sudo systemctl enable logcorrelator
@ -88,66 +91,56 @@ sudo systemctl status logcorrelator
go build -o logcorrelator ./cmd/logcorrelator go build -o logcorrelator ./cmd/logcorrelator
# Exécuter # Exécuter
./logcorrelator -config config.example.conf ./logcorrelator -config config.example.yml
``` ```
## Configuration ## Configuration
La configuration utilise un fichier texte simple avec des directives : La configuration utilise un fichier YAML :
```bash ```yaml
# Format: directive value [value...] # Service configuration
# Lignes starting with # sont des commentaires service:
name: logcorrelator
language: go
service.name logcorrelator # Input sources (at least 2 required)
service.language go inputs:
unix_sockets:
- name: apache_source
path: /var/run/logcorrelator/apache.sock
format: json
- name: network_source
path: /var/run/logcorrelator/network.sock
format: json
# Inputs (au moins 2 requis) # File output
input.unix_socket apache_source /var/run/logcorrelator/apache.sock json outputs:
input.unix_socket network_source /var/run/logcorrelator/network.sock json file:
enabled: true
path: /var/log/logcorrelator/correlated.log
# Outputs # ClickHouse output
output.file.enabled true outputs:
output.file.path /var/log/logcorrelator/correlated.log clickhouse:
enabled: false
dsn: clickhouse://user:pass@localhost:9000/db
table: correlated_logs_http_network
output.clickhouse.enabled false # Correlation configuration
output.clickhouse.dsn clickhouse://user:pass@localhost:9000/db correlation:
output.clickhouse.table correlated_logs_http_network key:
output.clickhouse.batch_size 500 - src_ip
output.clickhouse.flush_interval_ms 200 - src_port
time_window:
# Corrélation value: 1
correlation.key src_ip,src_port unit: s
correlation.time_window.value 1 orphan_policy:
correlation.time_window.unit s apache_always_emit: true
network_emit: false
# Politique des orphelins
correlation.orphan_policy.apache_always_emit true
correlation.orphan_policy.network_emit false
``` ```
### Directives disponibles Exemple complet dans `config.example.yml`.
| Directive | Description | Défaut |
|-----------|-------------|--------|
| `service.name` | Nom du service | `logcorrelator` |
| `service.language` | Langage | `go` |
| `input.unix_socket` | Socket Unix (name path [format]) | Requis |
| `output.file.enabled` | Activer sortie fichier | `true` |
| `output.file.path` | Chemin fichier | `/var/log/logcorrelator/correlated.log` |
| `output.clickhouse.enabled` | Activer ClickHouse | `false` |
| `output.clickhouse.dsn` | DSN ClickHouse | - |
| `output.clickhouse.table` | Table ClickHouse | - |
| `output.clickhouse.batch_size` | Taille batch | `500` |
| `output.clickhouse.flush_interval_ms` | Intervalle flush | `200` |
| `output.clickhouse.max_buffer_size` | Buffer max | `5000` |
| `output.clickhouse.drop_on_overflow` | Drop si overflow | `true` |
| `output.stdout.enabled` | Sortie stdout (debug) | `false` |
| `correlation.key` | Clés de corrélation | `src_ip,src_port` |
| `correlation.time_window.value` | Valeur fenêtre | `1` |
| `correlation.time_window.unit` | Unité (ms/s/m) | `s` |
| `correlation.orphan_policy.apache_always_emit` | Émettre A seul | `true` |
| `correlation.orphan_policy.network_emit` | Émettre B seul | `false` |
## Format des logs ## Format des logs

41
config.example.conf
View File

@ -1,41 +0,0 @@
# logcorrelator configuration file
# Format: directive value [value...]
# Lines starting with # are comments
# Service configuration
service.name logcorrelator
service.language go
# Input sources (at least 2 required)
# Format: input.unix_socket <name> <path> [format]
input.unix_socket apache_source /var/run/logcorrelator/apache.sock json
input.unix_socket network_source /var/run/logcorrelator/network.sock json
# File output
output.file.enabled true
output.file.path /var/log/logcorrelator/correlated.log
# ClickHouse output
output.clickhouse.enabled false
output.clickhouse.dsn clickhouse://user:pass@localhost:9000/db
output.clickhouse.table correlated_logs_http_network
output.clickhouse.batch_size 500
output.clickhouse.flush_interval_ms 200
output.clickhouse.max_buffer_size 5000
output.clickhouse.drop_on_overflow true
output.clickhouse.async_insert true
output.clickhouse.timeout_ms 1000
# Stdout output (for debugging)
output.stdout.enabled false
# Correlation configuration
correlation.key src_ip,src_port
correlation.time_window.value 1
correlation.time_window.unit s
# Orphan policy
# apache_always_emit: always emit A events even without matching B
# network_emit: emit B events alone (usually false)
correlation.orphan_policy.apache_always_emit true
correlation.orphan_policy.network_emit false