- feat: new config directive include_dest_ports ([]int) in correlation section
- feat: if non-empty, only events with a matching dst_port are correlated
- feat: filtered events are silently ignored (not correlated, not emitted as orphan)
- feat: new metric failed_dest_port_filtered tracked in ProcessEvent
- feat: DEBUG log 'event excluded by dest port filter: source=A dst_port=22'
- test: TestCorrelationService_IncludeDestPorts_AllowedPort
- test: TestCorrelationService_IncludeDestPorts_FilteredPort
- test: TestCorrelationService_IncludeDestPorts_EmptyAllowsAll
- docs(readme): full rewrite to match current code (v1.1.12)
- docs(readme): add include_dest_ports section, fix version refs, clean outdated sections
- docs(arch): add dest_port_filtering section, failed_dest_port_filtered metric, debug log example
- fix(config.example): remove obsolete stdout.level field
- chore: bump version to 1.1.12
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- feat(observability): metrics server with /metrics and /health endpoints
- feat(observability): correlation metrics (events, success/failed, reasons, buffers)
- feat(correlation): IP exclusion filter (exact IPs and CIDR ranges)
- feat(correlation): pending orphan delay for late-arriving B events
- fix(stdout): sink is now a no-op for data; JSON must never appear on stdout
- fix(clickhouse): all flush errors were silently discarded, now properly logged
- fix(clickhouse): buffer overflow with DropOnOverflow now logged at WARN
- fix(clickhouse): retry attempts logged at WARN with attempt/delay/error context
- feat(clickhouse): connection success logged at INFO, batch sends at DEBUG
- feat(clickhouse): SetLogger() for external logger injection
- test(stdout): assert stdout remains empty for correlated and orphan logs
- chore(rpm): bump version to 1.1.11, update changelog
- docs: README and architecture.yml updated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Update RPM version numbers to 1.1.6
- Fix config file name (.yml not .conf)
- Add complete configuration example with current schema
- Add ClickHouse DSN format documentation
- Add Troubleshooting section (ClickHouse, MV, sockets, systemd)
- Update project structure with accurate file names
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- http_logs_raw: partition by toDate(ingest_time), order by ingest_time
- http_logs: explicit columns (no DEFAULT), extracted by MV
- mv_http_logs: full SELECT with JSONExtract* + coalesce for all fields
- Add 17 HTTP header fields (User-Agent, Accept, Sec-CH-UA, etc.)
- New ORDER BY: (time, src_ip, dst_ip, ja4)
- architecture.yml: match new schema with MV query details
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- README.md: use http.socket instead of apache.sock
- architecture.yml: use http.socket instead of apache.sock
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Remove CorrelationKeyFull() alias, use CorrelationKey() everywhere
- Remove duplicate TimeProvider interface from ports/source.go
- Remove unused time import from ports/source.go
- Update README.md: replace ./build.sh and ./test.sh with make commands
- Update RPM package names in README to match current version (1.0.3)
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- breaking: remove apache and network subdivisions from JSON output
- feat: all log fields now merged into single-level JSON structure
- feat: custom MarshalJSON() implementation for flat output
- chore: update ClickHouse schema to use single fields JSON column
- docs: update CHANGELOG.md and README.md with v1.0.3 changes
- build: bump version to 1.0.3 in build.sh and RPM spec
Migration notes:
- Existing ClickHouse tables need schema migration to use fields JSON column
- Replace apache JSON and network JSON columns with fields JSON column
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Remove all DEB packaging files (packaging/deb/*)
- Update Dockerfile.package to build RPM packages only
- Update build.sh to remove DEB build steps
- Update architecture.yml to reflect RPM-only distribution
- Update README.md installation instructions for RPM only
- Change runtime image from distroless/base-debian12 to scratch
- Clean up obsolete build artifacts in dist/
Supported distributions now:
- CentOS 7 (el7)
- Rocky Linux 8 (el8)
- Rocky Linux 9 (el9)
- Rocky Linux 10 (el10)
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
