feat: Keep-Alive correlation, TTL management, SIGHUP handling, logrotate support
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled

Major features:
- One-to-many correlation mode (Keep-Alive) for HTTP connections
- Dynamic TTL for network events with reset on each correlation
- Separate configurable buffer sizes for HTTP and network events
- SIGHUP signal handling for log rotation without service restart
- FileSink.Reopen() method for log file rotation
- logrotate configuration included in RPM
- ExecReload added to systemd service

Configuration changes:
- New YAML structure with nested sections (time_window, orphan_policy, matching, buffers, ttl)
- Backward compatibility maintained for deprecated fields

Packaging:
- RPM version 1.1.0 with logrotate config
- Updated spec file and changelog
- All distributions: el8, el9, el10

Tests:
- New tests for Keep-Alive mode and TTL reset
- Updated mocks with Reopen() interface method

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
This commit is contained in:
Jacquin Antoine
2026-03-02 20:32:59 +01:00
parent a415a3201a
commit 33e19b4f52
19 changed files with 974 additions and 321 deletions

View File

@ -2,7 +2,7 @@
# Compatible with CentOS 7, Rocky Linux 8, 9, 10
# Define version before Version: field for RPM macro support
%global spec_version 1.0.9
%global spec_version 1.1.0
Name: logcorrelator
Version: %{spec_version}
@ -38,6 +38,7 @@ mkdir -p %{buildroot}/usr/share/logcorrelator
mkdir -p %{buildroot}/var/log/logcorrelator
mkdir -p %{buildroot}/var/run/logcorrelator
mkdir -p %{buildroot}/etc/systemd/system
mkdir -p %{buildroot}/etc/logrotate.d
# Install binary
install -m 0755 %{_sourcedir}/logcorrelator %{buildroot}/usr/bin/logcorrelator
@ -49,6 +50,9 @@ install -m 0640 %{_sourcedir}/logcorrelator.yml %{buildroot}/usr/share/logcorrel
# Install systemd service
install -m 0644 %{_sourcedir}/logcorrelator.service %{buildroot}/etc/systemd/system/logcorrelator.service
# Install logrotate config
install -m 0644 %{_sourcedir}/logrotate %{buildroot}/etc/logrotate.d/logcorrelator
%post
# Create logcorrelator user and group
if ! getent group logcorrelator >/dev/null 2>&1; then
@ -114,27 +118,64 @@ fi
/var/log/logcorrelator
/var/run/logcorrelator
/etc/systemd/system/logcorrelator.service
/etc/logrotate.d/logcorrelator
%changelog
* Sat Feb 28 2026 logcorrelator <dev@example.com> - 1.0.3-1
* Mon Mar 02 2026 logcorrelator <dev@example.com> - 1.1.0-1
- Feat: Keep-Alive support (one-to-many correlation mode)
- Feat: Dynamic TTL for network events (source B)
- Feat: Separate buffer sizes for HTTP and network events
- Feat: SIGHUP signal handling for log rotation
- Feat: File sink Reopen() method for log rotation
- Feat: logrotate configuration included
- Feat: ExecReload added to systemd service
- Feat: New YAML config structure (time_window, orphan_policy, matching, buffers, ttl)
- Docs: Updated architecture.yml and config.example.yml
* Sat Feb 28 2026 logcorrelator <dev@example.com> - 1.0.7-1
- Added: Log levels DEBUG, INFO, WARN, ERROR configurable via log.level
- Added: Warn and Warnf methods for warning messages
- Added: Debug logs for events received from sockets and correlations
- Added: Warning logs for orphan events and buffer overflow
- Changed: Configuration log.enabled replaced by log.level
- Changed: Orphan events and buffer overflow now logged as WARN instead of DEBUG
* Sat Feb 28 2026 logcorrelator <dev@example.com> - 1.0.6-1
- Changed: Configuration YAML simplified, removed service.name, service.language
- Changed: Correlation config simplified, time_window_s instead of nested object
- Changed: Orphan policy simplified to emit_orphans boolean
- Changed: Apache socket renamed to http.socket
- Added: socket_permissions option on unix sockets
* Sat Feb 28 2026 logcorrelator <dev@example.com> - 1.0.5-1
- Added: Systemd service auto-start after RPM installation
- Added: Systemd service hardening (TimeoutStartSec, TimeoutStopSec, ReadWritePaths)
- Fixed: Systemd service unit correct config path (.yml instead of .conf)
- Fixed: CI workflow branch name main to master
- Changed: RPM packaging generic el8/el9/el10 directory naming
* Sat Feb 28 2026 logcorrelator <dev@example.com> - 1.0.4-1
- Breaking: Flattened JSON output structure - removed apache and network subdivisions
- All log fields now merged into single-level JSON structure
- ClickHouse schema: replaced apache JSON and network JSON columns with fields JSON column
- Custom MarshalJSON() implementation for flat output
* Sat Feb 28 2026 logcorrelator <dev@example.com> - 1.0.3-1
- Fix: Added missing ClickHouse driver dependency
- Fix: Fixed race condition in orchestrator
- Security: Added explicit source_type configuration for Unix socket sources
- Added: Comprehensive test suite improvements
- Added: Test coverage improved from 50.6% to 62.0%
* Sat Feb 28 2026 logcorrelator <dev@example.com> - 1.0.2-1
- Fix: durcir la validation et fiabiliser flush/arrêt idempotents
- Refactor: remove Debian/DEB packaging, RPM-only support
- Feat: add multi-distro RPM packaging for CentOS 7 and Rocky Linux 8/9/10
- Feat: migrate configuration from custom format to YAML
- Refactor: remove obsolete config and update documentation
- Added: Initial RPM packaging support for Rocky Linux 8/9 and AlmaLinux 10
- Added: Docker multi-stage build pipeline
- Added: Hexagonal architecture implementation
- Added: Unix socket input sources (JSON line protocol)
- Added: File output sink (JSON lines)
- Added: ClickHouse output sink with batching and retry logic
- Added: Time-window based correlation on src_ip + src_port
- Added: Graceful shutdown with signal handling (SIGINT, SIGTERM)
* Sat Feb 28 2026 logcorrelator <dev@example.com> - 1.0.1-1
- Fix: durcir la validation et fiabiliser flush/arrêt idempotents
- Refactor: remove Debian/DEB packaging, RPM-only support
- Feat: add multi-distro RPM packaging for CentOS 7 and Rocky Linux 8/9/10
- Feat: migrate configuration from custom format to YAML
- Refactor: remove obsolete config and update documentation
* Sat Feb 28 2026 logcorrelator <dev@example.com> - 1.0.0-1
- Initial package for CentOS 7, Rocky Linux 8, 9, 10

13
packaging/rpm/logrotate Normal file
View File

@ -0,0 +1,13 @@
/var/log/logcorrelator/correlated.log {
daily
rotate 7
compress
delaycompress
missingok
notifempty
create 0640 logcorrelator logcorrelator
sharedscripts
postrotate
/bin/systemctl reload logcorrelator > /dev/null 2>&1 || true
endscript
}

View File

@ -41,6 +41,11 @@ if [ ! -f /etc/logcorrelator/logcorrelator.yml ]; then
chmod 640 /etc/logcorrelator/logcorrelator.yml
fi
# Set permissions for logrotate config
if [ -f /etc/logrotate.d/logcorrelator ]; then
chmod 644 /etc/logrotate.d/logcorrelator
fi
# Reload systemd
if [ -x /bin/systemctl ]; then
systemctl daemon-reload