Commit Graph

23 Commits

Author SHA1 Message Date
bd45344d19 v1.1.11: Fix exclude_source_ips config loading and debug logging
Major fixes:
- Add exclude_source_ips to mergeConfigs() - config file values now properly loaded
- Add validation for exclude_source_ips (IP/CIDR format validation)
- Remove JA4SENTINEL_LOG_LEVEL env var from systemd service
- Config file log_level now respected without env override

Debug logging improvements:
- Log IP filter entries at startup (debug mode)
- Track filtered packet count with atomic counter
- Display filter statistics at shutdown via GetFilterStats()
- New debug logs in tlsparse component

Testing:
- Add 6 new unit tests for exclude_source_ips and log_level config loading
- Test mergeConfigs() behavior with empty/override values
- Test validation of invalid IPs and CIDR ranges

Documentation:
- Update architecture.yml with ipfilter module
- Document config loading priority and notes
- Update api.Config fields (LocalIPs, ExcludeSourceIPs, LogLevel)

Files changed:
- internal/config/loader.go (merge, validation, helpers)
- internal/config/loader_test.go (6 new tests)
- internal/tlsparse/parser.go (GetFilterStats, counter)
- cmd/ja4sentinel/main.go (debug logging)
- packaging/systemd/ja4sentinel.service (remove env var)
- architecture.yml (ipfilter module, config_loading section)

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-04 15:55:00 +01:00
952701d4da release: version 1.1.9 - Source IP exclusion feature
FEATURE:
- Add exclude_source_ips configuration option
- Support single IPs and CIDR ranges (IPv4/IPv6)
- Filter packets before TLS processing
- New ipfilter package with unit tests
- Log exclusion configuration at startup

Packaging:
- Update RPM spec to version 1.1.9
- Update changelog with feature details

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-04 12:02:00 +01:00
432509f8f4 feature: add source IP exclusion with CIDR support
Features:
- Add exclude_source_ips configuration option
- Support single IPs (192.168.1.1) and CIDR ranges (10.0.0.0/8)
- Filter packets in parser before TLS processing
- Log exclusion configuration at startup
- New ipfilter package with IP/CIDR matching
- Unit tests for ipfilter package

Configuration example:
  exclude_source_ips:
    - "10.0.0.0/8"       # Exclude private network
    - "192.168.1.1"      # Exclude specific IP
    - "172.16.0.0/12"    # Exclude another range
    - "2001:db8::/32"    # IPv6 support

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-04 11:57:48 +01:00
bf93ee6c4a release: version 1.1.8 - Critical parser crash fix
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
CRITICAL FIX:
- Resolve crash in TLS parser with nil decode context
- Use gopacket.NewPacket with LinkTypeIPv4/IPv6
- Fixes panic: runtime error: invalid memory address or nil pointer dereference
- Properly handles raw IP packets after SLL header stripping

Packaging:
- Update RPM spec to version 1.1.8
- Update changelog with crash fix details

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-04 11:35:24 +01:00
4b072057e8 release: version 1.1.7 - Improve error logging
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
Fix:
- Enhance error messages with src_ip, src_port, dst_ip, dst_port
- Add conn_id and payload_len to fingerprint errors
- Easier debugging of truncated ClientHello payloads

Packaging:
- Update RPM spec to version 1.1.7
- Update changelog with logging improvements

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-04 11:09:54 +01:00
131b204edf fix: improve error logging with source/destination details
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
Logging improvements:
- Add src_ip, src_port, dst_ip, dst_port to tlsparse error logs
- Add connection details to fingerprint error logs (conn_id, payload_len)
- Include 'unknown' placeholders for packets that fail before parsing

This helps debug issues with truncated ClientHello payloads
and identify problematic connections more easily.

Example log output:
  WARN Failed to generate fingerprints
    src_ip=192.168.1.10 src_port=54321 dst_ip=10.0.0.1 dst_port=443
    conn_id=192.168.1.10:54321->10.0.0.1:443 payload_len=128
    error="failed to parse ClientHello: extension data truncated"

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-04 11:07:01 +01:00
027730b360 release: version 1.1.6 - Add local IP filtering and SLL support
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
Features:
- Add local_ips configuration option for filtering traffic to local machine
- Auto-detection of local IP addresses (excludes loopback 127.x.x.x, ::1)
- Support interface 'any' for capturing on all network interfaces
- Add Linux SLL (cooked capture) support for interface 'any'
- Generate BPF filter with 'dst host' for local IP filtering
- Add LinkType field to RawPacket for proper packet parsing

Testing:
- Add unit tests for local IP detection (detectLocalIPs, extractIP)
- Add unit tests for SLL packet parsing (IPv4 and IPv6)
- Update capture tests for new packetToRawPacket method

Configuration:
- Update config.yml.example with local_ips documentation
- Update RPM spec to version 1.1.6 with changelog

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-04 11:02:53 +01:00
34e6ef1d15 fix: add error callback for UNIX socket connection errors in main.go
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
- Configure ErrorCallback on output builder to log socket connection failures
- Errors now visible via appLogger.Error when socket connection fails
- Helps diagnose missing socket or permission issues

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 23:56:24 +01:00
23f3012fb1 release: version 1.1.2 - Add error callback mechanism and comprehensive test suite
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
Features:
- Add ErrorCallback type for UNIX socket connection error reporting
- Add WithErrorCallback option for UnixSocketWriter configuration
- Add BuilderImpl.WithErrorCallback() for propagating callbacks
- Add consecutive failure tracking in processQueue

Testing (50+ new tests):
- Add integration tests for full pipeline (capture → tlsparse → fingerprint → output)
- Add tests for FileWriter.rotate() and Reopen() log rotation
- Add tests for cleanupExpiredFlows() and cleanupLoop() in TLS parser
- Add tests for extractSNIFromPayload() and extractJA4Hash() helpers
- Add tests for config load error paths (invalid YAML, permission denied)
- Add tests for capture.Run() error conditions
- Add tests for signal handling documentation

Documentation:
- Update architecture.yml with new fields (LogLevel, TLSClientHello extensions)
- Update architecture.yml with Close() methods for Capture and Parser interfaces
- Update RPM spec changelog

Cleanup:
- Remove empty internal/api/ directory

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 23:24:56 +01:00
52c9f2f6f4 feat: add logrotate support with SIGHUP signal handling
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
- Add Reopenable interface in api/types.go for log rotation support
- Add FileWriter.Reopen() method to reopen log files after rotation
- Add MultiWriter.Reopen() method to propagate rotation to all writers
- Update main.go to handle SIGHUP signal for systemctl reload
- Add logrotate configuration file (packaging/logrotate/ja4sentinel)
- Update systemd service with ExecReload for graceful rotation
- Update architecture.yml with logrotate documentation
- Update RPM spec and Dockerfile.package to include logrotate files
- Bump version to 1.1.0

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 20:50:47 +01:00
965720a183 release: version 1.0.9 - Add SNI, ALPN, TLS version extraction and architecture.yml compliance
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
New features:
- Extract SNI (Server Name Indication) from TLS ClientHello
- Extract ALPN (Application-Layer Protocol Negotiation) protocols
- Detect TLS version from ClientHello using tlsfingerprint library
- Add ConnID field for TCP flow correlation
- Add SensorID field for multi-sensor deployments
- Add SynToCHMs timing field for behavioral detection
- Add AsyncBuffer configuration for output queue sizing

Architecture changes:
- Remove JA4Hash from LogRecord (JA4 format includes its own hash portions)
- Update api.TLSClientHello with new TLS metadata fields
- Update api.LogRecord with correlation, TLS, and timing fields
- Ensure 100% compliance with architecture.yml specification

Tests:
- Add unit tests for TLS extension extraction (SNI, ALPN, Version)
- Update tests for new LogRecord schema without JA4Hash
- Add tests for AsyncBuffer configuration

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 19:32:16 +01:00
fd162982d9 v1.0.8: Add configurable log level and immediate service stop
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
- Add log_level config option (debug, info, warn, error)
- Add JA4SENTINEL_LOG_LEVEL environment variable support
- Set TimeoutStopSec=2 for immediate stop on restart/stop
- Consolidate config files into single example (config.yml.example)
- Update RPM changelog

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 02:51:11 +01:00
d89c90dc03 release: version 1.0.7
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 02:28:00 +01:00
f0276b17ab release: version 1.0.6
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 01:49:43 +01:00
a69de782cb release: version 1.0.5 - fix TCP options detection (NOP/EOL/SACK)
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 01:27:27 +01:00
dcd6bd0a94 feat: add systemd sdnotify support (ready, watchdog, stopping)
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
- Add github.com/coreos/go-systemd/v22/daemon dependency
- Signal SdNotifyReady after configuration is loaded
- Start watchdog goroutine that pings systemd every WatchdogSec/2
- Signal SdNotifyStopping during graceful shutdown
- Update systemd unit file:
  - Type=notify (instead of simple)
  - WatchdogSec=30 (auto-restart if service hangs)
  - NotifyAccess=main (only main process can notify)

Benefits:
- systemd knows when service is truly ready
- Automatic detection of hung/frozen service
- Better integration with systemd supervision
- More accurate service status reporting

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 01:06:20 +01:00
c7e8fe874f fix: renforcer limites TLS, timeouts socket et validation config
Some checks failed
Build RPM Package / Build RPM Packages (CentOS 7, Rocky 8/9/10) (push) Has been cancelled
Co-authored-by: aider (openrouter/openai/gpt-5.3-codex) <aider@aider.chat>
2026-02-28 20:01:39 +01:00
dfd5e49dd9 feat(config): add configurable packet channel buffer size
Some checks failed
Build DEB Package / Build DEB Package (Debian/Ubuntu) (push) Has been cancelled
Build RPM Package / Build RPM Package (Rocky Linux) (push) Has been cancelled
- Add PacketBufferSize field to api.Config struct
- Add DefaultPacketBuffer constant (1000 packets)
- Add JA4SENTINEL_PACKET_BUFFER_SIZE environment variable support
- Update mergeConfigs to handle PacketBufferSize override
- Update main.go to use configurable buffer size with fallback
- Update config.yml.example with packet_buffer_size option

Allows tuning for high-traffic environments by increasing buffer size
via config file or environment variable

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-27 00:07:45 +01:00
e4b8f5ab86 fix(cmd): close packetChan after capture goroutine finishes
- Add close(packetChan) after captureEngine.Run() completes
- Ensures packet processor goroutine exits cleanly when channel is drained
- Prevents potential goroutine leak during shutdown

Fixes potential resource leak identified in architecture audit

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-27 00:05:16 +01:00
56d8164e7f test(cmd): add unit tests for main.go
- Add TestFormatPorts covering empty, single, and multiple ports
- Add TestMain_VersionFlag_VerifiesOutput checking version variables
- Add TestFlagParsing verifying CLI flag parsing behavior
- Fix .gitignore to only ignore root-level binary, not cmd/ja4sentinel/

Implements testing.policy.requirements.test_skeletons from architecture.yml

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-27 00:01:39 +01:00
9280cb545c feat: version 1.0.0 avec corrections critiques et nommage de packages
Ajout du point d'entrée principal :
- cmd/ja4sentinel/main.go : pipeline complet avec gestion des signaux
- Intégration des modules (capture, tlsparse, fingerprint, output)
- Shutdown propre avec context.Context

Corrections du parsing TLS :
- Flow key unidirectionnel (client → serveur uniquement)
- Timeout de flux configurable via FlowTimeoutSec
- Structure ConnectionFlow simplifiée

Améliorations de l'API :
- Champs TCPMSS et TCPWScale en pointeurs (omitempty correct)
- NewLogRecord mis à jour pour les champs optionnels

Mise à jour de l'architecture :
- architecture.yml : documentation des champs optionnels
- Règles de flux unidirectionnel documentées

Système de packages :
- Version par défaut : 1.0.0
- Nommage cohérent : ja4sentinel_1.0.0_amd64.deb
- Scripts build-deb.sh et build-rpm.sh simplifiés
- Extraction correcte des checksums

Tests :
- TestFlowKey mis à jour pour le format unidirectionnel
- Tous les tests passent (go test ./...)
- go vet clean

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-26 23:24:42 +01:00
6cd6c4c3b8 fix: sécuriser shutdown, config par défaut et reconnexion socket
Co-authored-by: aider (openrouter/openai/gpt-5.3-codex) <aider@aider.chat>
2026-02-25 21:44:40 +01:00
efd4481729 feat: implémentation complète du pipeline JA4 + Docker + tests
Nouveaux modules:
- cmd/ja4sentinel/main.go : point d'entrée avec pipeline capture→parse→fingerprint→output
- internal/config/loader.go : chargement YAML + env (JA4SENTINEL_*) + validation
- internal/tlsparse/parser.go : extraction ClientHello avec suivi d'état de flux (NEW/WAIT_CLIENT_HELLO/JA4_DONE)
- internal/fingerprint/engine.go : génération JA4/JA3 via psanford/tlsfingerprint
- internal/output/writers.go : StdoutWriter, FileWriter, UnixSocketWriter, MultiWriter

Infrastructure:
- Dockerfile (multi-stage), Dockerfile.dev, Dockerfile.test-server
- Makefile (build, test, lint, docker-build-*)
- docker-compose.test.yml pour tests d'intégration
- README.md (276 lignes) avec architecture, config, exemples

API (api/types.go):
- Ajout Close() aux interfaces Capture et Parser
- Ajout FlowTimeoutSec dans Config (défaut: 30s, env: JA4SENTINEL_FLOW_TIMEOUT)
- ServiceLog: +Timestamp, +TraceID, +ConnID
- LogRecord: champs flatten (ip_meta_*, tcp_meta_*, ja4*)
- Helper NewLogRecord() pour conversion TLSClientHello+Fingerprints→LogRecord

Architecture (architecture.yml):
- Documentation module logging + interfaces LoggerFactory/Logger
- Section service.systemd complète (unit, security, capabilities)
- Section logging.strategy (JSON lines, champs, règles)
- api.Config: +FlowTimeoutSec documenté

Fixes/cleanup:
- Suppression internal/api/types.go (consolidé dans api/types.go)
- Correction imports logging (ja4sentinel/api)
- .dockerignore / .gitignore
- config.yml.example

Tests:
- Tous les modules ont leurs tests (*_test.go)
- Tests unitaires : capture, config, fingerprint, output, tlsparse
- Tests d'intégration via docker-compose.test.yml

Build:
- Binaires dans dist/ (make build → dist/ja4sentinel)
- Docker runtime avec COPY --from=builder /app/dist/

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-25 20:02:52 +01:00