feat(dashboard): rebuild SOC dashboard + fix ClickHouse SQL

Complete rewrite of the SOC dashboard using FastAPI + Jinja2 + htmx + Chart.js + Tailwind CSS.
Replaces the old React/Vite frontend with server-rendered templates.

Dashboard pages:
- Overview: KPIs, timeline chart, threat distribution, top IPs
- Detections: paginated/filterable anomaly table
- Scores: ml_all_scores with AE error & XGB prob columns
- Traffic: HTTP logs with method/host filters
- IP Investigation: full deep-dive (scores, features, HTTP logs, classify)
- Classification: SOC feedback form + history
- Features: AI + thesis feature stats
- Models: scoring stats + model metadata

API: 9 JSON endpoints with parameterized queries, sort whitelists

SQL fixes:
- 05_aggregation_tables: add deduplicate_merge_projection_mode
- 11_views: fix nested aggregate (argMax inside sum)
- 12_thesis_features: remove invalid 'let' bindings, fix groupArrayIf type

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
toto
2026-04-08 03:21:05 +02:00
parent 228ad7026a
commit b735bab5a5
120 changed files with 1444 additions and 24933 deletions

View File

@ -0,0 +1,43 @@
{% extends "base.html" %}
{% block title %}JA4 SOC — Features ML{% endblock %}
{% block content %}
<div class="space-y-6">
<h2 class="text-lg font-semibold text-white">Features ML — Statistiques agrégées</h2>
<!-- AI Features -->
<div class="bg-gray-900 rounded-xl p-5 border border-gray-800">
<h3 class="text-sm font-medium text-gray-400 mb-3">Features AI (view_ai_features_1h)</h3>
<div id="ai-stats" class="text-gray-500 text-sm">Chargement...</div>
</div>
<!-- Thesis Features -->
<div class="bg-gray-900 rounded-xl p-5 border border-gray-800">
<h3 class="text-sm font-medium text-gray-400 mb-3">Features Thèse §5 (view_thesis_features_1h)</h3>
<div id="thesis-stats" class="text-gray-500 text-sm">Chargement...</div>
</div>
<!-- Score distribution chart -->
<div class="bg-gray-900 rounded-xl p-5 border border-gray-800">
<h3 class="text-sm font-medium text-gray-400 mb-3">Distribution des scores d'anomalie</h3>
<canvas id="score-dist-chart" height="200"></canvas>
</div>
</div>
{% endblock %}
{% block scripts %}
<script>
function renderStats(data, containerId) {
const el = document.getElementById(containerId);
if (!data || Object.keys(data).length === 0) { el.textContent = 'Aucune donnée disponible'; return; }
el.innerHTML = '<div class="grid grid-cols-2 md:grid-cols-4 gap-3">' +
Object.entries(data).map(([k,v]) => {
let val = typeof v === 'number' ? v.toFixed(4) : v;
return `<div class="bg-gray-800 rounded-lg p-3"><div class="text-[10px] text-gray-500 truncate">${k}</div><div class="text-sm text-gray-200 font-mono">${val}</div></div>`;
}).join('') + '</div>';
}
async function loadFeatures() {
try {
const r = await fetch('/api/features'); const d = await r.json();
renderStats(d.ai_features, 'ai-stats');
renderStats(d.thesis_features, 'thesis-stats');
} catch(e) { console.error(e); }
}
loadFeatures();
</script>
{% endblock %}