Files
ja4-platform/services/dashboard/backend/templates/classify.html
toto 85d3b95b7b feat: HTTP/2 passive fingerprinting with individual SETTINGS fields
Complete implementation of HTTP/2 passive fingerprinting per thesis §2.5.3:

mod-reqin-log (C module):
- Replace connection-level filter with ap_hook_process_connection (APR_HOOK_FIRST)
  to capture H2 preface before mod_http2 takes over the connection
- AP_MODE_SPECULATIVE read of 512 bytes from c->input_filters
- Parse SETTINGS, WINDOW_UPDATE, PRIORITY flags, pseudo-header order
- Output individual SETTINGS params as separate JSON fields (IDs 1-6, 8)
- Read H2 notes from c1 (master connection) for mod_http2 secondary conns
- Fix header_order_signature JSON length bug (26→strlen)

ClickHouse schema:
- Add 8 new columns to http_logs: h2_has_priority, h2_header_table_size,
  h2_enable_push, h2_max_concurrent_streams, h2_initial_window_size,
  h2_max_frame_size, h2_max_header_list_size, h2_enable_connect_protocol
- Use Int32/Int64 with DEFAULT -1 to distinguish absent vs zero
- Update mv_http_logs to extract individual fields via JSONHas/JSONExtractInt
- Migration 04_http2_fields.sql updated for existing deployments

Correlator:
- Accept both timestamp_ns and timestamp field names (backward compat)

Integration:
- Enable HTTP/2 in Apache: Protocols h2 http/1.1 in httpd-integration.conf

Validated end-to-end via Playwright: H2 curl traffic → mod-reqin-log →
correlator → ClickHouse with all 12 H2 columns populated correctly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-11 02:33:45 +02:00

215 lines
14 KiB
HTML

{% extends "base.html" %}
{% block title %}JA4 SOC — Classifier{% endblock %}
{% block page_title %}
Classification SOC
<span class="relative inline-block ml-1"><button onclick="docToggle(this)" class="doc-btn"></button><div class="doc-panel">
<h4>Feedback analyste SOC</h4>
<p>Classifiez les IPs pour entraîner le modèle XGBoost supervisé. Les labels sont utilisés au prochain cycle ML.</p>
<p><strong>Workflow :</strong> 1. Consultez les IPs suggérées (non classifiées). 2. Classifiez-les. 3. Les labels alimentent XGBoost au prochain cycle.</p>
<p><strong>Vrai positif :</strong> Confirme un bot détecté. <strong>Faux positif :</strong> Trafic légitime mal détecté. <strong>Suspect :</strong> À surveiller.</p>
<p class="doc-source">Source : soc_feedback → XGBoost training</p>
</div></span>
{% endblock %}
{% block content %}
<div class="space-y-4">
<!-- KPIs -->
<div class="grid grid-cols-2 md:grid-cols-4 gap-3">
<div class="kpi-card"><div class="text-[11px] text-gray-500 mb-1">Total classifiées</div><div class="text-xl font-bold text-brand-500" id="kpi-total">0</div></div>
<div class="kpi-card"><div class="text-[11px] text-gray-500 mb-1">✅ Vrais positifs</div><div class="text-xl font-bold text-red-400" id="kpi-tp">0</div></div>
<div class="kpi-card"><div class="text-[11px] text-gray-500 mb-1">❌ Faux positifs</div><div class="text-xl font-bold text-green-400" id="kpi-fp">0</div></div>
<div class="kpi-card"><div class="text-[11px] text-gray-500 mb-1">⚠️ Suspects</div><div class="text-xl font-bold text-yellow-400" id="kpi-suspect">0</div></div>
</div>
<div class="grid grid-cols-1 lg:grid-cols-3 gap-4">
<!-- Classification form -->
<div class="section-card">
<div class="section-header"><span class="section-title">Nouvelle classification
<span class="relative inline-block"><button onclick="docToggle(this)" class="doc-btn"></button><div class="doc-panel">
<h4>Classifier une IP</h4>
<p>Saisissez une IP ou cliquez sur une suggestion. La classification est immédiatement enregistrée et sera utilisée par XGBoost au prochain cycle.</p>
<p class="doc-source">Table : soc_feedback</p>
</div></span>
</span></div>
<div class="section-body space-y-3">
<div>
<label class="block text-[11px] text-gray-500 mb-1">Adresse IP</label>
<input type="text" id="cls-ip" placeholder="ex: 192.168.1.100" class="w-full px-3 py-2 bg-gray-800 border border-gray-700 rounded-lg text-sm text-gray-300 focus:border-brand-500 focus:outline-none">
</div>
<div>
<label class="block text-[11px] text-gray-500 mb-1">Classification</label>
<div class="grid grid-cols-3 gap-2">
<button class="cls-type-btn px-3 py-2 rounded-lg text-sm font-medium transition-colors bg-red-500/20 text-red-400 border border-red-500/30 hover:bg-red-500/30" data-cls="true_positive">✅ Vrai positif</button>
<button class="cls-type-btn px-3 py-2 rounded-lg text-sm font-medium transition-colors bg-green-500/20 text-green-400 border border-green-500/30 hover:bg-green-500/30" data-cls="false_positive">❌ Faux positif</button>
<button class="cls-type-btn px-3 py-2 rounded-lg text-sm font-medium transition-colors bg-yellow-500/20 text-yellow-400 border border-yellow-500/30 hover:bg-yellow-500/30" data-cls="suspicious">⚠️ Suspect</button>
</div>
</div>
<div>
<label class="block text-[11px] text-gray-500 mb-1">Commentaire (optionnel)</label>
<textarea id="cls-comment" rows="2" placeholder="Raison de la classification..." class="w-full px-3 py-2 bg-gray-800 border border-gray-700 rounded-lg text-sm text-gray-300 focus:border-brand-500 focus:outline-none resize-none"></textarea>
</div>
<button id="cls-submit" class="w-full px-4 py-2 bg-brand-600 text-white rounded-lg text-sm font-medium hover:bg-brand-700 transition-colors disabled:opacity-30" disabled>Sélectionnez un type ci-dessus</button>
<div id="cls-result" class="text-sm"></div>
</div>
</div>
<!-- Suggested IPs + Distribution chart -->
<div class="lg:col-span-2 space-y-4">
<!-- Suggested IPs -->
<div class="section-card overflow-hidden">
<div class="section-header"><span class="section-title">IPs suggérées (non classifiées)
<span class="relative inline-block"><button onclick="docToggle(this)" class="doc-btn"></button><div class="doc-panel">
<h4>Suggestions de classification</h4>
<p>IPs détectées comme anomalies dans les 3 derniers jours qui n'ont pas encore de label SOC. Triées par score descendant.</p>
<p><strong>Action :</strong> Cliquez sur une IP pour la pré-remplir dans le formulaire, ou utilisez les boutons rapides.</p>
<p class="doc-source">Source : ml_detected_anomalies LEFT JOIN soc_feedback</p>
</div></span>
</span></div>
<div class="overflow-x-auto" style="max-height:35vh; overflow-y:auto">
<table class="data-table"><thead><tr>
<th>IP</th><th>Score max</th><th>Menace</th><th>Détections</th><th>JA4</th><th>ASN</th><th>Pays</th><th>Action</th>
</tr></thead><tbody id="suggested-body"></tbody></table>
</div>
</div>
<!-- Distribution chart -->
<div class="section-card">
<div class="section-header"><span class="section-title">Répartition des classifications
<span class="relative inline-block"><button onclick="docToggle(this)" class="doc-btn"></button><div class="doc-panel">
<h4>Distribution des labels</h4>
<p>Ratio bot/légitime/suspect dans les labels SOC. Un bon ratio aide XGBoost à apprendre. Visez ≥100 labels par catégorie.</p>
<p class="doc-source">Source : soc_feedback GROUP BY classification</p>
</div></span>
</span></div>
<div class="section-body"><div id="dist-chart" style="height:180px"></div></div>
</div>
</div>
</div>
<!-- Recent classifications -->
<div class="section-card overflow-hidden">
<div class="section-header"><span class="section-title">Historique des classifications
<span class="relative inline-block"><button onclick="docToggle(this)" class="doc-btn"></button><div class="doc-panel">
<h4>Classifications récentes</h4>
<p>Les 50 dernières classifications effectuées par les analystes SOC. Chaque label sera utilisé par XGBoost au prochain cycle ML.</p>
<p class="doc-source">Source : soc_feedback ORDER BY created_at DESC</p>
</div></span>
</span></div>
<div class="overflow-x-auto" style="max-height:40vh; overflow-y:auto">
<table class="data-table"><thead><tr>
<th>Date</th><th>IP</th><th>Classification</th><th>Commentaire</th>
</tr></thead><tbody id="cls-history"></tbody></table>
</div>
</div>
</div>
{% endblock %}
{% block scripts %}
<script>
let selectedCls = '';
// ── Classification type toggle ──
document.querySelectorAll('.cls-type-btn').forEach(btn => {
btn.onclick = () => {
document.querySelectorAll('.cls-type-btn').forEach(b => b.classList.remove('ring-2','ring-white'));
btn.classList.add('ring-2','ring-white');
selectedCls = btn.dataset.cls;
const sub = document.getElementById('cls-submit');
sub.disabled = false;
sub.textContent = {true_positive:'✅ Classifier comme Vrai positif',false_positive:'❌ Classifier comme Faux positif',suspicious:'⚠️ Classifier comme Suspect'}[selectedCls];
};
});
// ── Submit ──
document.getElementById('cls-submit').onclick = async () => {
const ip = document.getElementById('cls-ip').value.trim();
if (!ip) { alert('Veuillez saisir une IP'); return; }
if (!selectedCls) { alert('Sélectionnez un type'); return; }
try {
const r = await fetch('/api/classify', {method:'POST', headers:{'Content-Type':'application/json'},
body:JSON.stringify({src_ip:ip, classification:selectedCls, comment:document.getElementById('cls-comment').value})});
const d = await r.json();
document.getElementById('cls-result').innerHTML = r.ok
? `<span class="text-green-400">✓ ${escapeHtml(ip)}${d.classification}</span>`
: `<span class="text-red-400">✗ ${escapeHtml(d.detail||'erreur')}</span>`;
if (r.ok) { document.getElementById('cls-ip').value=''; document.getElementById('cls-comment').value=''; loadAll(); }
} catch(e) { document.getElementById('cls-result').innerHTML = `<span class="text-red-400">✗ ${e}</span>`; }
};
function prefillIP(ip) {
document.getElementById('cls-ip').value = ip;
document.getElementById('cls-ip').scrollIntoView({behavior:'smooth',block:'center'});
document.getElementById('cls-ip').focus();
}
function quickClassify(ip, cls) {
document.getElementById('cls-ip').value = ip;
selectedCls = cls;
document.getElementById('cls-submit').click();
}
async function loadAll() {
try {
const [stats, suggested, history] = await Promise.all([
fetch('/api/classify/stats').then(r=>r.json()),
fetch('/api/classify/suggested').then(r=>r.json()),
fetch('/api/classifications').then(r=>r.json()),
]);
// ── KPIs ──
const byType = {};
(stats.stats||[]).forEach(r => { byType[r.classification] = r.cnt; });
document.getElementById('kpi-total').textContent = fmtNum(stats.total||0);
document.getElementById('kpi-tp').textContent = fmtNum(byType.true_positive||0);
document.getElementById('kpi-fp').textContent = fmtNum(byType.false_positive||0);
document.getElementById('kpi-suspect').textContent = fmtNum(byType.suspicious||0);
// ── Distribution chart ──
const CLS_COLORS = {true_positive:'#ef4444',false_positive:'#22c55e',suspicious:'#eab308'};
const CLS_LABELS = {true_positive:'✅ Vrai positif',false_positive:'❌ Faux positif',suspicious:'⚠️ Suspect'};
if (stats.total > 0) {
const el = document.getElementById('dist-chart');
const ch = echarts.init(el);
ch.setOption(ecBase({
tooltip: ecTooltip({trigger:'item'}),
series: [{type:'pie', radius:['35%','65%'], center:['50%','50%'],
label:{color:EC_TEXT,fontSize:11,formatter:'{b}\n{d}%'},
data:(stats.stats||[]).map(r=>({name:CLS_LABELS[r.classification]||r.classification,value:r.cnt,itemStyle:{color:CLS_COLORS[r.classification]||'#6b7280'}}))}]
}));
} else {
document.getElementById('dist-chart').innerHTML = '<div class="flex items-center justify-center h-full text-gray-500 text-sm">Aucune classification — commencez à labéliser !</div>';
}
// ── Suggested IPs ──
document.getElementById('suggested-body').innerHTML = (suggested.suggested||[]).map(row => `<tr>
<td class="whitespace-nowrap cursor-pointer hover:text-brand-400" onclick="prefillIP('${escapeHtml(row.src_ip)}')">${fmtIP(row.src_ip)}</td>
<td>${fmtScore(row.worst_score)}</td>
<td>${threatBadge(row.threat_level)}</td>
<td class="font-mono text-xs">${row.detection_count||0}</td>
<td class="text-xs font-mono max-w-[80px] truncate">${fmtJA4(row.ja4)}</td>
<td class="text-xs max-w-[100px] truncate">${row.asn_org ? fmtASN(row.asn_org) : ''}</td>
<td>${fmtCountry(row.country_code)}</td>
<td class="whitespace-nowrap">
<button onclick="quickClassify('${escapeHtml(row.src_ip)}','true_positive')" class="px-1.5 py-0.5 text-[10px] bg-red-500/20 text-red-400 rounded hover:bg-red-500/30" title="Vrai positif">✅</button>
<button onclick="quickClassify('${escapeHtml(row.src_ip)}','false_positive')" class="px-1.5 py-0.5 text-[10px] bg-green-500/20 text-green-400 rounded hover:bg-green-500/30" title="Faux positif">❌</button>
<a href="/ip/${encodeURIComponent(row.src_ip)}" class="px-1.5 py-0.5 text-[10px] bg-gray-700 text-gray-300 rounded hover:bg-gray-600 inline-block" title="Détail">🔍</a>
</td>
</tr>`).join('') || '<tr><td colspan="8" class="text-center text-gray-500 py-4">Toutes les IPs ont été classifiées 🎉</td></tr>';
// ── History ──
document.getElementById('cls-history').innerHTML = (history.data||[]).map(row => `<tr onclick="window.location='/ip/${encodeURIComponent(row.src_ip)}'">
<td class="text-xs text-gray-400">${(row.created_at||'').substring(0,16)}</td>
<td class="whitespace-nowrap">${fmtIP(row.src_ip)}</td>
<td><span class="badge ${row.classification==='true_positive'?'badge-critical':row.classification==='false_positive'?'badge-low':'badge-medium'}">${escapeHtml(row.classification)}</span></td>
<td class="text-xs max-w-[300px] truncate text-gray-400">${escapeHtml(row.comment||'')}</td>
</tr>`).join('') || '<tr><td colspan="4" class="text-center text-gray-500 py-4">Aucune classification</td></tr>';
} catch(e) { console.error('classify load error:', e); }
}
// ── Pre-fill from URL ──
const urlIP = new URLSearchParams(window.location.search).get('ip');
if (urlIP) document.getElementById('cls-ip').value = urlIP;
loadAll();
</script>
{% endblock %}