feat: ja4-platform monorepo — 5 services unified, tests & RPM builds standardized

Services:
- ja4sentinel: TLS/JA4 fingerprint capture daemon (Go, libpcap)
- logcorrelator: JA4 log correlation engine (Go, ClickHouse)
- mod_reqin_log: Apache module (C, JSON request logging)
- bot_detector: ML bot detection pipeline (Python)
- dashboard: FastAPI/Streamlit analytics UI (Python)

Shared libraries:
- shared/go/ja4common: logger, config, shutdown, ipfilter (Go module)
- shared/python/ja4_common: ClickHouseClient, ClickHouseSettings (Python package)
- shared/clickhouse/: canonical SQL migrations (10 files)

Build & packaging:
- Unified 3-stage Dockerfile.package for Go RPMs (el8/el9/el10)
- go.work workspace linking sentinel, correlator, ja4common
- Makefile with test-all, build-all, rpm-* targets

Fixes applied:
- go.work: 1.21 → 1.24.6 (required by sentinel)
- correlator Dockerfiles: golang:1.21 → golang:1.24
- replace directives in go.mod for ja4common local path
- pyproject.toml: setuptools.backends → setuptools.build_meta
- Removed static libpcap linking (unavailable on Rocky 9)
- Fixed data races in output/writers_test.go (sync.Mutex + atomic.Int32)
- Rewrote corrupted test files (logger_test.go × 2)

Test coverage:
- correlator: 67.1% total (unixsocket 80.5%, config 91.7%, app 83.3%, multi 87.7%, stdout 100%)
- sentinel: all 10 packages pass (api, capture, config, fingerprint, ipfilter, logging, output, tlsparse)

Documentation:
- README.md + docs/ (architecture, development, 5 services, shared libs, DB schema & migrations)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
toto
2026-04-07 16:42:59 +02:00
commit d469e39da7
278 changed files with 1621301 additions and 0 deletions

View File

@ -0,0 +1,36 @@
3215,human
12322,human
5410,human
15557,human
21502,human
9036,human
8218,human
39180,human
3303,human
6730,human
9044,human
15600,human
13030,human
25256,human
5432,human
6848,human
12392,human
49686,human
6714,human
49203,human
6661,human
8469,human
20676,human
3320,human
3209,human
8881,human
6805,human
29562,human
31334,human
8422,human
25255,human
8447,human
12635,human
6830,human
8412,human
35369,human
1 3215 human
2 12322 human
3 5410 human
4 15557 human
5 21502 human
6 9036 human
7 8218 human
8 39180 human
9 3303 human
10 6730 human
11 9044 human
12 15600 human
13 13030 human
14 25256 human
15 5432 human
16 6848 human
17 12392 human
18 49686 human
19 6714 human
20 49203 human
21 6661 human
22 8469 human
23 20676 human
24 3320 human
25 3209 human
26 8881 human
27 6805 human
28 29562 human
29 31334 human
30 8422 human
31 25255 human
32 8447 human
33 12635 human
34 6830 human
35 8412 human
36 35369 human

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff