code_public/logcorrelator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
47 Commits 1 Branch 0 Tags
c352e06b8841b7ca1eb83a97b9267850e6e974a1
Commit Graph

14 Commits

Author SHA1 Message Date
Jacquin Antoine
c352e06b88 fix: create socket parent directory on startup
Some checks failed
Build and Test / test (push) Has been cancelled
Details
Build and Test / build (push) Has been cancelled
Details
Build and Test / docker (push) Has been cancelled
Details 
- Create /var/run/logcorrelator/ if missing before binding sockets
- Fixes issue with tmpfs /var/run being cleared on reboot
- Add filepath import for directory handling

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-03-03 00:17:01 +01:00
Jacquin Antoine
6b690a3eb3 fix: log raw JSON on parse errors for debugging
Some checks failed
Build and Test / test (push) Has been cancelled
Details
Build and Test / build (push) Has been cancelled
Details
Build and Test / docker (push) Has been cancelled
Details 
- Add raw JSON payload to parse error warnings
- Helps diagnose malformed JSON from senders
- Version bumped to 1.1.4

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-03-02 23:20:51 +01:00
Jacquin Antoine
24aa84bd9c test: add comprehensive tests to improve coverage
Some checks failed
Build and Test / test (push) Has been cancelled
Details
Build and Test / build (push) Has been cancelled
Details
Build and Test / docker (push) Has been cancelled
Details 
- observability: added tests for LogLevel.String(), Warn(), Warnf(), Infof(),
  Debugf(), Error(), WithFields(), and concurrent access patterns
- file: added tests for Reopen(), Close(), empty/whitespace paths,
  validateFilePath allowed/rejected paths, concurrent writes, Flush(),
  and marshal errors
- config: added tests for TimeWindowConfig.GetDuration(),
  CorrelationConfig getters, validation scenarios (no inputs, no outputs,
  duplicate sockets, ClickHouse validation), and LogConfig.GetLevel()

Coverage improvements:
- observability: 57.7% → 79.5%
- file: 68.6% → 78.6%
- config: 69.8% → 97.7%
- total: 68.6% → 74.4%

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-03-02 22:52:09 +01:00
Jacquin Antoine
15ca33ee3a refactor: switch Unix sockets from STREAM to DGRAM mode
Some checks failed
Build and Test / test (push) Has been cancelled
Details
Build and Test / build (push) Has been cancelled
Details
Build and Test / docker (push) Has been cancelled
Details 
- Change net.Listen("unix") to net.ListenUnixgram("unixgram")
- Replace connection-based Accept() with ReadFromUnix() datagram reading
- Remove connection limiting semaphore (not needed for DGRAM)
- Update tests with datagram-specific tests
- Socket permissions default to 0666 (world read/write)

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-03-02 22:43:10 +01:00
Jacquin Antoine
324b0042f8 fix(rpm): example config in /etc/logcorrelator + socket permissions 0666
Some checks failed
Build and Test / test (push) Has been cancelled
Details
Build and Test / build (push) Has been cancelled
Details
Build and Test / docker (push) Has been cancelled
Details 
- Install logcorrelator.yml.example to /etc/logcorrelator/ instead of /usr/share/logcorrelator/
- Change default socket permissions from 0660 to 0666 (world read/write)
- Bump version to 1.1.2
- Remove CHANGELOG.md

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-03-02 22:07:50 +01:00
Jacquin Antoine
33e19b4f52 feat: Keep-Alive correlation, TTL management, SIGHUP handling, logrotate support
Some checks failed
Build and Test / test (push) Has been cancelled
Details
Build and Test / build (push) Has been cancelled
Details
Build and Test / docker (push) Has been cancelled
Details 
Major features:
- One-to-many correlation mode (Keep-Alive) for HTTP connections
- Dynamic TTL for network events with reset on each correlation
- Separate configurable buffer sizes for HTTP and network events
- SIGHUP signal handling for log rotation without service restart
- FileSink.Reopen() method for log file rotation
- logrotate configuration included in RPM
- ExecReload added to systemd service

Configuration changes:
- New YAML structure with nested sections (time_window, orphan_policy, matching, buffers, ttl)
- Backward compatibility maintained for deprecated fields

Packaging:
- RPM version 1.1.0 with logrotate config
- Updated spec file and changelog
- All distributions: el8, el9, el10

Tests:
- New tests for Keep-Alive mode and TTL reset
- Updated mocks with Reopen() interface method

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-03-02 20:32:59 +01:00
Jacquin Antoine
27c7659397 fix: renforcer corrélation A/B et sorties stdout/fichier
Some checks failed
Build and Test / test (push) Has been cancelled
Details
Build and Test / build (push) Has been cancelled
Details
Build and Test / docker (push) Has been cancelled
Details 
Co-authored-by: aider (openrouter/openai/gpt-5.3-codex) <aider@aider.chat>
 2026-03-01 12:10:17 +01:00
Jacquin Antoine
a3ae5421cf chore: version 1.0.7 - add log levels
Some checks failed
Build and Test / test (push) Has been cancelled
Details
Build and Test / build (push) Has been cancelled
Details
Build and Test / docker (push) Has been cancelled
Details 
- Add configurable log levels: DEBUG, INFO, WARN, ERROR
- Replace debug.enabled with log.level in configuration
- Add Warn/Warnf methods for warning messages
- Log orphan events and buffer overflow as WARN
- Log parse errors as WARN
- Log raw events and correlations as DEBUG

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-03-01 02:33:04 +01:00
Jacquin Antoine
56c2923121 chore: version 1.0.6 - simplify YAML configuration
Some checks failed
Build and Test / test (push) Has been cancelled
Details
Build and Test / build (push) Has been cancelled
Details
Build and Test / docker (push) Has been cancelled
Details 
- Remove service.name and service.language (unused)
- Remove enabled flags on outputs (presence = enabled)
- Simplify correlation config: time_window_s (integer) instead of nested object
- Simplify orphan_policy to emit_orphans boolean
- Rename apache socket to http.socket
- Add socket_permissions option for unix sockets (default: 0660)
- Update tests for new configuration format

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-03-01 01:59:59 +01:00
Jacquin Antoine
87b94f3c18 feat: add main entry point and stdout sink for Docker build 
- Create cmd/logcorrelator/main.go as the application entry point
  - Loads configuration from YAML file
  - Initializes Unix socket sources, file/ClickHouse/stdout sinks
  - Sets up correlation service and orchestrator
  - Handles graceful shutdown on SIGINT/SIGTERM
  - Supports -version flag to print version

- Add internal/adapters/outbound/stdout/sink.go
  - Implements CorrelatedLogSink interface for stdout output
  - Writes JSON lines to standard output

- Fix .gitignore to use /logcorrelator instead of logcorrelator
  - Prevents cmd/logcorrelator directory from being ignored

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-02-28 23:32:25 +01:00
Jacquin Antoine
514cb553ef feat: release v1.0.3 with flattened JSON output structure 
- breaking: remove apache and network subdivisions from JSON output
- feat: all log fields now merged into single-level JSON structure
- feat: custom MarshalJSON() implementation for flat output
- chore: update ClickHouse schema to use single fields JSON column
- docs: update CHANGELOG.md and README.md with v1.0.3 changes
- build: bump version to 1.0.3 in build.sh and RPM spec

Migration notes:
- Existing ClickHouse tables need schema migration to use fields JSON column
- Replace apache JSON and network JSON columns with fields JSON column

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-02-28 22:26:20 +01:00
Jacquin Antoine
180c57c35b chore: release v1.0.2 with critical fixes and test improvements 
- fix: add missing ClickHouse driver dependency
- fix: resolve race condition in orchestrator (single goroutine per source)
- feat: add explicit source_type config for Unix socket sources
- test: improve coverage from 50.6% to 62.0%
- docs: add CHANGELOG.md with release notes
- build: update version to 1.0.2 in build scripts and Dockerfiles

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-02-28 21:45:00 +01:00
Jacquin Antoine
7e9535122e fix: durcir la validation et fiabiliser flush/arrêt idempotents 
Co-authored-by: aider (openrouter/openai/gpt-5.3-codex) <aider@aider.chat>
 2026-02-28 20:10:28 +01:00
Jacquin Antoine
8fc14c1e94 Initial commit: logcorrelator with unified packaging (DEB + RPM using fpm) 
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
 2026-02-27 15:31:46 +01:00