feat(dashboard): add JA4 fingerprint and cluster investigation pages

- /ja4/{fingerprint} page: 8 KPIs, timeline, threat pie, IP scores
  table, ASN/geo charts, HTTP logs, AI features — full JA4 investigation
- /cluster/{cid} page: 8 KPIs, timeline, threat/JA4/ASN/host charts,
  member table with bulk classify — full campaign investigation
- /api/ja4/{fingerprint} and /api/cluster/{cid} API endpoints
- fmtJA4 links now navigate to /ja4/ investigation page
- campaigns.html: 'Ouvrir' button links to /cluster/{cid} full page
- Fix: double-brace {{param}} in non-f-string queries → single {param}
  (was causing HTTP 500 on all parameterized ClickHouse queries)
- 50 routes total, all tests pass, 0 JS console errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
toto
2026-04-09 14:05:52 +02:00
parent 70188b508c
commit 702c0d5edb
6 changed files with 778 additions and 5 deletions

View File

@ -0,0 +1,253 @@
{% extends "base.html" %}
{% block title %}JA4 SOC — JA4 {{ ja4[:24] }}…{% endblock %}
{% block page_title %}
Empreinte JA4
<span class="relative inline-block ml-1"><button onclick="docToggle(this)" class="doc-btn" aria-label="Aide"></button><div class="doc-panel">
<h4>Investigation JA4</h4>
<p>Analyse complète d'une empreinte JA4 : toutes les IPs l'utilisant, distribution des menaces, timeline, comportement réseau et logs HTTP.</p>
<p><strong>Workflow :</strong> Identifiez si l'empreinte est partagée par des bots (même JA4, IPs différentes) ou un navigateur légitime.</p>
<p class="doc-source">Sources : ml_all_scores, ml_detected_anomalies, http_logs</p>
</div></span>
{% endblock %}
{% block header_actions %}
<code class="text-xs text-purple-400 font-mono bg-gray-900 px-3 py-1.5 rounded-lg border border-gray-800 select-all max-w-[400px] truncate" title="{{ ja4 }}">{{ ja4 }}</code>
{% endblock %}
{% block content %}
<div class="space-y-4">
<!-- KPIs -->
<div class="grid grid-cols-2 md:grid-cols-4 xl:grid-cols-8 gap-3" id="kpi-row"></div>
<!-- Row 1: Timeline + Threats -->
<div class="grid grid-cols-1 lg:grid-cols-3 gap-4">
<div class="section-card lg:col-span-2">
<div class="section-header">
<span class="section-title">
Timeline activité
<span class="relative inline-block ml-1"><button onclick="docToggle(this)" class="doc-btn"></button><div class="doc-panel">
<h4>Timeline JA4</h4>
<p>Sessions et IPs actives par heure utilisant cette empreinte. Un pic indique une campagne coordonnée.</p>
<p class="doc-source">Source : ml_all_scores GROUP BY hour</p>
</div></span>
</span>
</div>
<div class="section-body"><div id="timeline-chart" style="height:220px"></div></div>
</div>
<div class="section-card">
<div class="section-header">
<span class="section-title">
Distribution menaces
<span class="relative inline-block ml-1"><button onclick="docToggle(this)" class="doc-btn"></button><div class="doc-panel">
<h4>Répartition des niveaux de menace</h4>
<p>Proportion des sessions classifiées par niveau. Une JA4 avec 100% HIGH est probablement un outil automatisé.</p>
<p class="doc-source">Source : ml_all_scores WHERE ja4=…</p>
</div></span>
</span>
</div>
<div class="section-body"><div id="threat-chart" style="height:220px"></div></div>
</div>
</div>
<!-- Row 2: IPs using this JA4 + ASN/Geo -->
<div class="grid grid-cols-1 lg:grid-cols-3 gap-4">
<div class="section-card lg:col-span-2">
<div class="section-header">
<span class="section-title">
IPs associées
<span class="relative inline-block ml-1"><button onclick="docToggle(this)" class="doc-btn"></button><div class="doc-panel">
<h4>IPs utilisant cette empreinte</h4>
<p>Liste des IP sources ayant présenté cette JA4. Cliquez pour investiguer une IP. Un grand nombre d'IPs avec la même JA4 suggère un botnet.</p>
<p class="doc-source">Source : ml_all_scores</p>
</div></span>
</span>
<span class="text-xs text-gray-500" id="ip-count"></span>
</div>
<div class="overflow-x-auto" style="max-height:400px;overflow-y:auto">
<table class="data-table"><thead><tr>
<th style="width:16px"></th>
<th>IP</th><th>Score</th><th>Threat</th><th>Hits</th>
<th>Host</th><th>ASN</th><th>Pays</th><th>Browser</th><th>Date</th>
</tr></thead><tbody id="scores-body"></tbody></table>
</div>
</div>
<div class="space-y-4">
<div class="section-card">
<div class="section-header"><span class="section-title">Infrastructure (ASN)</span></div>
<div class="section-body"><div id="asn-chart" style="height:180px"></div></div>
</div>
<div class="section-card">
<div class="section-header"><span class="section-title">Géographie</span></div>
<div class="section-body"><div id="geo-chart" style="height:180px"></div></div>
</div>
</div>
</div>
<!-- Row 3: HTTP Logs -->
<div class="section-card">
<div class="section-header">
<span class="section-title">
Logs HTTP récents
<span class="relative inline-block ml-1"><button onclick="docToggle(this)" class="doc-btn"></button><div class="doc-panel">
<h4>Requêtes HTTP avec cette JA4</h4>
<p>Échantillon du trafic brut (24h). Analysez les paths, User-Agents et méthodes pour confirmer l'automatisation.</p>
<p class="doc-source">Source : http_logs WHERE ja4=…</p>
</div></span>
</span>
</div>
<div class="overflow-x-auto" style="max-height:300px;overflow-y:auto">
<table class="data-table"><thead><tr>
<th>Time</th><th>IP</th><th>Method</th><th>Host</th><th>Path</th><th>HTTP</th><th>User-Agent</th>
</tr></thead><tbody id="http-body"></tbody></table>
</div>
</div>
<!-- Row 4: AI Features -->
<div class="section-card" id="features-section" style="display:none">
<div class="section-header"><span class="section-title">Features ML (par IP×Host)</span></div>
<div class="overflow-x-auto" style="max-height:300px;overflow-y:auto">
<table class="data-table"><thead><tr>
<th>IP</th><th>Host</th><th>Hits</th><th>Velocity</th><th>Fuzz</th>
<th>Post%</th><th>Browser</th><th>ASN</th><th>Label</th>
</tr></thead><tbody id="features-body"></tbody></table>
</div>
</div>
</div>
{% endblock %}
{% block scripts %}
<script>
const JA4 = '{{ ja4 }}';
const THREAT_COLORS = {CRITICAL:'#ef4444',HIGH:'#f97316',MEDIUM:'#eab308',NORMAL:'#6b7280',KNOWN_BOT:'#3b82f6',LEGITIMATE_BROWSER:'#22c55e',ANUBIS_DENY:'#dc2626'};
async function loadJA4Detail() {
try {
const r = await fetch('/api/ja4/' + encodeURIComponent(JA4));
const d = await r.json();
const p = d.profile || {};
// KPIs
document.getElementById('kpi-row').innerHTML = [
kpi('Sessions', fmtNum(p.total_sessions)),
kpi('IPs uniques', fmtNum(p.unique_ips)),
kpi('Hosts ciblés', fmtNum(p.unique_hosts)),
kpi('Hits totaux', fmtNum(p.total_hits)),
kpi('Score moyen', p.avg_score != null ? parseFloat(p.avg_score).toFixed(4) : '—'),
kpi('Score max', p.max_score != null ? parseFloat(p.max_score).toFixed(4) : '—'),
kpi('Menaces', fmtNum(p.threat_count)),
kpi('Navigateurs', (p.browser_list||[]).join(', ') || '—'),
].join('');
// Timeline
const tl = d.timeline || [];
if (tl.length) {
const ch = echarts.init(document.getElementById('timeline-chart'));
ch.setOption(ecBase({
tooltip: ecTooltip({trigger:'axis'}),
grid:{left:40,right:20,top:20,bottom:30},
xAxis:{type:'category',data:tl.map(r=>(r.hour||'').substring(11,16)),axisLabel:{color:EC_TEXT,fontSize:10}},
yAxis:[
{type:'value',name:'Sessions',nameTextStyle:{color:EC_TEXT,fontSize:10},axisLabel:{color:EC_TEXT,fontSize:10}},
{type:'value',name:'IPs',nameTextStyle:{color:EC_TEXT,fontSize:10},axisLabel:{color:EC_TEXT,fontSize:10},splitLine:{show:false}}
],
series:[
{name:'Sessions',type:'bar',data:tl.map(r=>r.sessions),itemStyle:{color:'#6366f1'},barWidth:'60%'},
{name:'IPs actives',type:'line',yAxisIndex:1,data:tl.map(r=>r.active_ips),lineStyle:{color:'#f97316'},itemStyle:{color:'#f97316'},smooth:true}
]
}));
}
// Threat pie
const threats = d.threats || [];
if (threats.length) {
const ch2 = echarts.init(document.getElementById('threat-chart'));
ch2.setOption(ecBase({
tooltip:ecTooltip({trigger:'item'}),
series:[{type:'pie',radius:['30%','70%'],
label:{color:EC_TEXT,fontSize:10},
data:threats.map(t=>({name:t.threat_level,value:t.cnt,itemStyle:{color:THREAT_COLORS[t.threat_level]||'#6b7280'}}))}]
}));
}
// Scores table (all IPs with this JA4)
const scores = d.scores || [];
document.getElementById('ip-count').textContent = scores.length + ' sessions';
document.getElementById('scores-body').innerHTML = scores.map(row => {
const ip = String(row.src_ip||'').replace('::ffff:','');
return `<tr onclick="window.location='/ip/${encodeURIComponent(ip)}'">
<td><span class="w-2 h-2 rounded-full inline-block" style="background:${THREAT_COLORS[row.threat_level]||'#6b7280'}"></span></td>
<td>${fmtIP(row.src_ip)}</td>
<td>${fmtScore(row.anomaly_score)}</td>
<td>${threatBadge(row.threat_level)}</td>
<td class="font-mono text-xs">${row.hits||0}</td>
<td class="text-xs max-w-[120px] truncate">${escapeHtml(row.host||'')}</td>
<td class="text-xs max-w-[120px] truncate">${fmtASN(row.asn_org)}</td>
<td>${fmtCountry(row.country_code)}</td>
<td class="text-xs">${escapeHtml(row.browser_family||'')}</td>
<td class="text-[11px] text-gray-400">${(row.detected_at||'').substring(0,16)}</td>
</tr>`;
}).join('') || '<tr><td colspan="10" class="text-center text-gray-500 py-6">Aucune session</td></tr>';
// ASN breakdown
const asnMap = {};
scores.forEach(r => { if(r.asn_org) asnMap[r.asn_org] = (asnMap[r.asn_org]||0)+1; });
const topASN = Object.entries(asnMap).sort((a,b)=>b[1]-a[1]).slice(0,8);
if (topASN.length) {
const ch3 = echarts.init(document.getElementById('asn-chart'));
ch3.setOption(ecBase({tooltip:ecTooltip({trigger:'axis'}),
grid:{left:10,right:30,top:5,bottom:5,containLabel:true},
yAxis:{type:'category',data:topASN.map(r=>r[0]).reverse(),axisLabel:{color:EC_TEXT,fontSize:9,width:100,overflow:'truncate'},axisLine:{show:false}},
xAxis:{type:'value',show:false},
series:[{type:'bar',data:topASN.map(r=>r[1]).reverse(),barWidth:'60%',itemStyle:{color:'#f97316'},
label:{show:true,position:'right',color:EC_TEXT,fontSize:10}}]}));
}
// Geo breakdown
const geoMap = {};
scores.forEach(r => { if(r.country_code) geoMap[r.country_code] = (geoMap[r.country_code]||0)+1; });
const topGeo = Object.entries(geoMap).sort((a,b)=>b[1]-a[1]).slice(0,8);
if (topGeo.length) {
const ch4 = echarts.init(document.getElementById('geo-chart'));
ch4.setOption(ecBase({tooltip:ecTooltip({trigger:'item'}),
series:[{type:'pie',radius:['25%','65%'],
label:{color:EC_TEXT,fontSize:10},
data:topGeo.map(([k,v])=>({name:k,value:v}))}]}));
}
// HTTP logs
document.getElementById('http-body').innerHTML = (d.http_logs||[]).map(row => `<tr>
<td class="text-[11px] whitespace-nowrap text-gray-400">${row.time||''}</td>
<td>${fmtIP(row.src_ip)}</td>
<td class="font-mono text-xs">${escapeHtml(row.method||'')}</td>
<td class="text-xs max-w-[100px] truncate">${escapeHtml(row.host||'')}</td>
<td class="text-xs max-w-[200px] truncate font-mono" title="${escapeHtml(row.path||'')}">${escapeHtml(row.path||'')}</td>
<td class="font-mono text-[11px] text-gray-400">${escapeHtml(row.http_version||'')}</td>
<td class="text-xs max-w-[200px] truncate text-gray-400">${escapeHtml(row.header_user_agent||'')}</td>
</tr>`).join('') || '<tr><td colspan="7" class="text-center text-gray-500 py-4">Aucun log</td></tr>';
// AI features
const feats = d.ai_features || [];
if (feats.length) {
document.getElementById('features-section').style.display = '';
document.getElementById('features-body').innerHTML = feats.map(row => {
const ip = String(row.src_ip||'').replace('::ffff:','');
return `<tr onclick="window.location='/ip/${encodeURIComponent(ip)}'">
<td>${fmtIP(row.src_ip)}</td>
<td class="text-xs">${escapeHtml(row.host||'')}</td>
<td class="font-mono text-xs">${row.hits||0}</td>
<td class="font-mono text-xs">${parseFloat(row.hit_velocity||0).toFixed(2)}</td>
<td class="font-mono text-xs">${parseFloat(row.fuzzing_index||0).toFixed(3)}</td>
<td class="font-mono text-xs">${parseFloat(row.post_ratio||0).toFixed(3)}</td>
<td class="text-xs">${escapeHtml(row.browser_family||'')}</td>
<td class="text-xs">${escapeHtml(row.asn_org||'')}</td>
<td class="text-xs">${escapeHtml(row.asn_label||'')}</td>
</tr>`;
}).join('');
}
} catch(e) { console.error('JA4 detail load error:', e); }
}
function kpi(label, value) {
return `<div class="kpi-card"><div class="text-[10px] text-gray-500 uppercase tracking-wider mb-1">${label}</div><div class="text-lg font-bold text-white">${value}</div></div>`;
}
loadJA4Detail();
</script>
{% endblock %}