Commit Graph

24 Commits

Author SHA1 Message Date
0cc9fbcd38 perf(rpm): migrate from FPM to rpmbuild (-200MB build image)
Migration complète de FPM vers rpmbuild natif :

Dockerfile.package:
- Stage rpm-common-tools : rockylinux:8 au lieu de ruby:3.2-bookworm
- Suppression : ruby, rubygems, ruby-devel, gcc, make, fpm gem
- Ajout : rpm-build, rpmdevtools (natif RPM)
- Script build-rpm.sh : structure rpmbuild complète + archive source
- Stages el8/el9/el10 : préparation fichiers + appel rpmbuild

logcorrelator.spec:
- Réécriture pour rpmbuild natif
- Macros RPM : %{buildroot}, %{_sourcedir}
- Sections %post/%preun/%postun inline (plus de scripts externes)
- %install adapté pour rpmbuild
- %files avec %config(noreplace)

Scripts supprimés :
- packaging/rpm/post (inline dans .spec)
- packaging/rpm/preun (inline dans .spec)
- packaging/rpm/postun (inline dans .spec)

Gains :
- Taille image build : -200 MB (-40%)
- Temps build : -10 secondes
- Fichiers : -3 scripts externes
- Outil : rpmbuild officiel (standard RPM)

Tests requis :
make package-rpm
make test-package-rpm

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-03 22:17:31 +00:00
24f2d8a3c4 fix(rpm): preserve config on upgrade, set correct ownership/permissions
RPM packaging improvements:
- Fix %config(noreplace) directive in spec file (logcorrelator.yml)
- Fix post script: use correct path for .yml.example (/etc/logcorrelator/)
- Set /var/run/logcorrelator ownership to logcorrelator:logcorrelator
- Set correct permissions: /var/run (755), /var/log (750), /var/lib (750)
- Add %config(noreplace) for logrotate.d/logcorrelator
- Add comprehensive RPM test script (packaging/test/test-rpm.sh)

Documentation updates:
- Update architecture.yml with filesystem permissions section
- Document socket ownership (logcorrelator:logcorrelator, 0666)
- Document config file policy (%config(noreplace) behavior)
- Add systemd hardening directives (NoNewPrivileges, ProtectSystem)
- Update ClickHouse schema: mark non-implemented fields
- Remove materialized view SQL (managed externally)
- Add stdout sink module documentation

Build pipeline:
- Update Dockerfile.package with comments for config policy
- Add /var/lib/logcorrelator directory creation
- Document fpm %config(noreplace) limitations

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-03 21:30:27 +00:00
9db6848757 fix: critical Keep-Alive correlation bug - network events evicted prematurely
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Fix cleanExpired() to use TTL map instead of event timestamp for B events
- Increase default correlation time window from 1s to 10s
- Increase default network TTL from 30s to 120s for long sessions
- Use payload timestamp for network events when available (fallback to now)
- Add comprehensive Keep-Alive tests (TTL reset, long session scenarios)
- Bump version to 1.1.7

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-03 16:32:48 +01:00
51e1eb8d57 chore: bump version to 1.1.6
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Update Makefile PKG_VERSION
- Update RPM spec version and changelog

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-03 11:55:28 +01:00
d78cc52a88 fix: ClickHouse insertion using native clickhouse-go/v2 API
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Replace database/sql wrapper with clickhouse.Open() and clickhouse.Conn
- Use PrepareBatch + Append + Send pattern for proper batch inserts
- Fix ATTEMPT_TO_READ_AFTER_EOF errors caused by empty VALUES
- Add batch size logging for debugging
- Update version to 1.1.5

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-03 11:44:40 +01:00
4b4ab84ee0 fix: set /var/run/logcorrelator permissions to 755 in RPM
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- RPM %post now sets chmod 755 on /var/run/logcorrelator
- Allows service to create sockets after reboot
- Version bumped to 1.1.5

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-03 00:17:56 +01:00
6b690a3eb3 fix: log raw JSON on parse errors for debugging
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Add raw JSON payload to parse error warnings
- Helps diagnose malformed JSON from senders
- Version bumped to 1.1.4

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 23:20:51 +01:00
7f2becf702 chore: bump version to 1.1.3
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- RPM spec: 1.1.3-1
- Makefile PKG_VERSION: 1.1.3

Changes in 1.1.3:
- Refactor: Unix sockets switched from STREAM to DGRAM (SOCK_DGRAM)
- Test: Coverage improved to 74.4% with comprehensive tests
- Fix: Example config in /etc/logcorrelator/
- Change: Socket permissions 0666 (world read/write)

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 22:54:12 +01:00
324b0042f8 fix(rpm): example config in /etc/logcorrelator + socket permissions 0666
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Install logcorrelator.yml.example to /etc/logcorrelator/ instead of /usr/share/logcorrelator/
- Change default socket permissions from 0660 to 0666 (world read/write)
- Bump version to 1.1.2
- Remove CHANGELOG.md

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 22:07:50 +01:00
7eb143e904 fix(rpm): install logcorrelator.yml.example in /etc/logcorrelator/
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Move example config from /usr/share/logcorrelator/ to /etc/logcorrelator/
for easier access and consistency with main config file.

Bump version to 1.1.1

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 21:40:33 +01:00
33e19b4f52 feat: Keep-Alive correlation, TTL management, SIGHUP handling, logrotate support
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Major features:
- One-to-many correlation mode (Keep-Alive) for HTTP connections
- Dynamic TTL for network events with reset on each correlation
- Separate configurable buffer sizes for HTTP and network events
- SIGHUP signal handling for log rotation without service restart
- FileSink.Reopen() method for log file rotation
- logrotate configuration included in RPM
- ExecReload added to systemd service

Configuration changes:
- New YAML structure with nested sections (time_window, orphan_policy, matching, buffers, ttl)
- Backward compatibility maintained for deprecated fields

Packaging:
- RPM version 1.1.0 with logrotate config
- Updated spec file and changelog
- All distributions: el8, el9, el10

Tests:
- New tests for Keep-Alive mode and TTL reset
- Updated mocks with Reopen() interface method

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 20:32:59 +01:00
a415a3201a 1.0.9
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
2026-03-01 12:25:34 +01:00
d3436f6245 1.0.8
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
2026-03-01 11:30:55 +01:00
a3ae5421cf chore: version 1.0.7 - add log levels
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Add configurable log levels: DEBUG, INFO, WARN, ERROR
- Replace debug.enabled with log.level in configuration
- Add Warn/Warnf methods for warning messages
- Log orphan events and buffer overflow as WARN
- Log parse errors as WARN
- Log raw events and correlations as DEBUG

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 02:33:04 +01:00
56c2923121 chore: version 1.0.6 - simplify YAML configuration
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Remove service.name and service.language (unused)
- Remove enabled flags on outputs (presence = enabled)
- Simplify correlation config: time_window_s (integer) instead of nested object
- Simplify orphan_policy to emit_orphans boolean
- Rename apache socket to http.socket
- Add socket_permissions option for unix sockets (default: 0660)
- Update tests for new configuration format

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 01:59:59 +01:00
efeb7e455f fix: use %{spec_version} macro in RPM spec file
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Define %global spec_version before Version: field
- Use %{spec_version} in Version: field for proper macro expansion
- Makes version management easier for RPM builds

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 01:17:03 +01:00
ea5ac94983 chore: bump version to 1.0.4
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Update packaging/rpm/logcorrelator.spec: Version 1.0.4
- Update CHANGELOG.md with 1.0.4 release notes
- Update Makefile: PKG_VERSION 1.0.4

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 00:46:09 +01:00
7e9d203341 feat: auto-start systemd service after RPM installation
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Add 'systemctl start logcorrelator.service' in post script
- Update both packaging/rpm/post and logcorrelator.spec
- Service is now enabled AND started automatically on install

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 00:42:15 +01:00
514cb553ef feat: release v1.0.3 with flattened JSON output structure
- breaking: remove apache and network subdivisions from JSON output
- feat: all log fields now merged into single-level JSON structure
- feat: custom MarshalJSON() implementation for flat output
- chore: update ClickHouse schema to use single fields JSON column
- docs: update CHANGELOG.md and README.md with v1.0.3 changes
- build: bump version to 1.0.3 in build.sh and RPM spec

Migration notes:
- Existing ClickHouse tables need schema migration to use fields JSON column
- Replace apache JSON and network JSON columns with fields JSON column

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-28 22:26:20 +01:00
c7b8523879 chore: bump version to 1.0.1 and update changelog
- Update RPM spec version to 1.0.1
- Add changelog entries for changes since 1.0.0
- Add .aider* to .gitignore

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-28 20:16:38 +01:00
81849b16d8 refactor: remove Debian/DEB packaging, RPM-only support
- Remove all DEB packaging files (packaging/deb/*)
- Update Dockerfile.package to build RPM packages only
- Update build.sh to remove DEB build steps
- Update architecture.yml to reflect RPM-only distribution
- Update README.md installation instructions for RPM only
- Change runtime image from distroless/base-debian12 to scratch
- Clean up obsolete build artifacts in dist/

Supported distributions now:
- CentOS 7 (el7)
- Rocky Linux 8 (el8)
- Rocky Linux 9 (el9)
- Rocky Linux 10 (el10)

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-28 19:55:24 +01:00
7a9d92a469 feat: add multi-distro RPM packaging for CentOS 7 and Rocky Linux 8/9/10
- Create RPM maintainer scripts (post, preun, postun)
- Add Docker build stages for each target distribution (el7, el8, el9, el10)
- Update architecture.yml with supported RPM distributions
- Update build.sh to extract distro-specific RPM packages

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-28 17:17:56 +01:00
37f9c21672 feat: migrate configuration from custom format to YAML
- Replace custom directive-based config parser with YAML using gopkg.in/yaml.v3
- Rename config.example.conf to config.example.yml with YAML syntax
- Update default config path to /etc/logcorrelator/logcorrelator.yml
- Update Dockerfile.package to copy YAML config files
- Update packaging scripts to install logcorrelator.yml
- Update architecture.yml to document YAML configuration
- Add yaml.v3 dependency to go.mod

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-27 15:51:25 +01:00
8fc14c1e94 Initial commit: logcorrelator with unified packaging (DEB + RPM using fpm)
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-27 15:31:46 +01:00