Complete implementation of HTTP/2 passive fingerprinting per thesis §2.5.3: mod-reqin-log (C module): - Replace connection-level filter with ap_hook_process_connection (APR_HOOK_FIRST) to capture H2 preface before mod_http2 takes over the connection - AP_MODE_SPECULATIVE read of 512 bytes from c->input_filters - Parse SETTINGS, WINDOW_UPDATE, PRIORITY flags, pseudo-header order - Output individual SETTINGS params as separate JSON fields (IDs 1-6, 8) - Read H2 notes from c1 (master connection) for mod_http2 secondary conns - Fix header_order_signature JSON length bug (26→strlen) ClickHouse schema: - Add 8 new columns to http_logs: h2_has_priority, h2_header_table_size, h2_enable_push, h2_max_concurrent_streams, h2_initial_window_size, h2_max_frame_size, h2_max_header_list_size, h2_enable_connect_protocol - Use Int32/Int64 with DEFAULT -1 to distinguish absent vs zero - Update mv_http_logs to extract individual fields via JSONHas/JSONExtractInt - Migration 04_http2_fields.sql updated for existing deployments Correlator: - Accept both timestamp_ns and timestamp field names (backward compat) Integration: - Enable HTTP/2 in Apache: Protocols h2 http/1.1 in httpd-integration.conf Validated end-to-end via Playwright: H2 curl traffic → mod-reqin-log → correlator → ClickHouse with all 12 H2 columns populated correctly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
215 lines
14 KiB
HTML
215 lines
14 KiB
HTML
{% extends "base.html" %}
|
|
{% block title %}JA4 SOC — Classifier{% endblock %}
|
|
{% block page_title %}
|
|
Classification SOC
|
|
<span class="relative inline-block ml-1"><button onclick="docToggle(this)" class="doc-btn">ⓘ</button><div class="doc-panel">
|
|
<h4>Feedback analyste SOC</h4>
|
|
<p>Classifiez les IPs pour entraîner le modèle XGBoost supervisé. Les labels sont utilisés au prochain cycle ML.</p>
|
|
<p><strong>Workflow :</strong> 1. Consultez les IPs suggérées (non classifiées). 2. Classifiez-les. 3. Les labels alimentent XGBoost au prochain cycle.</p>
|
|
<p><strong>Vrai positif :</strong> Confirme un bot détecté. <strong>Faux positif :</strong> Trafic légitime mal détecté. <strong>Suspect :</strong> À surveiller.</p>
|
|
<p class="doc-source">Source : soc_feedback → XGBoost training</p>
|
|
</div></span>
|
|
{% endblock %}
|
|
{% block content %}
|
|
<div class="space-y-4">
|
|
<!-- KPIs -->
|
|
<div class="grid grid-cols-2 md:grid-cols-4 gap-3">
|
|
<div class="kpi-card"><div class="text-[11px] text-gray-500 mb-1">Total classifiées</div><div class="text-xl font-bold text-brand-500" id="kpi-total">0</div></div>
|
|
<div class="kpi-card"><div class="text-[11px] text-gray-500 mb-1">✅ Vrais positifs</div><div class="text-xl font-bold text-red-400" id="kpi-tp">0</div></div>
|
|
<div class="kpi-card"><div class="text-[11px] text-gray-500 mb-1">❌ Faux positifs</div><div class="text-xl font-bold text-green-400" id="kpi-fp">0</div></div>
|
|
<div class="kpi-card"><div class="text-[11px] text-gray-500 mb-1">⚠️ Suspects</div><div class="text-xl font-bold text-yellow-400" id="kpi-suspect">0</div></div>
|
|
</div>
|
|
|
|
<div class="grid grid-cols-1 lg:grid-cols-3 gap-4">
|
|
<!-- Classification form -->
|
|
<div class="section-card">
|
|
<div class="section-header"><span class="section-title">Nouvelle classification
|
|
<span class="relative inline-block"><button onclick="docToggle(this)" class="doc-btn">ⓘ</button><div class="doc-panel">
|
|
<h4>Classifier une IP</h4>
|
|
<p>Saisissez une IP ou cliquez sur une suggestion. La classification est immédiatement enregistrée et sera utilisée par XGBoost au prochain cycle.</p>
|
|
<p class="doc-source">Table : soc_feedback</p>
|
|
</div></span>
|
|
</span></div>
|
|
<div class="section-body space-y-3">
|
|
<div>
|
|
<label class="block text-[11px] text-gray-500 mb-1">Adresse IP</label>
|
|
<input type="text" id="cls-ip" placeholder="ex: 192.168.1.100" class="w-full px-3 py-2 bg-gray-800 border border-gray-700 rounded-lg text-sm text-gray-300 focus:border-brand-500 focus:outline-none">
|
|
</div>
|
|
<div>
|
|
<label class="block text-[11px] text-gray-500 mb-1">Classification</label>
|
|
<div class="grid grid-cols-3 gap-2">
|
|
<button class="cls-type-btn px-3 py-2 rounded-lg text-sm font-medium transition-colors bg-red-500/20 text-red-400 border border-red-500/30 hover:bg-red-500/30" data-cls="true_positive">✅ Vrai positif</button>
|
|
<button class="cls-type-btn px-3 py-2 rounded-lg text-sm font-medium transition-colors bg-green-500/20 text-green-400 border border-green-500/30 hover:bg-green-500/30" data-cls="false_positive">❌ Faux positif</button>
|
|
<button class="cls-type-btn px-3 py-2 rounded-lg text-sm font-medium transition-colors bg-yellow-500/20 text-yellow-400 border border-yellow-500/30 hover:bg-yellow-500/30" data-cls="suspicious">⚠️ Suspect</button>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<label class="block text-[11px] text-gray-500 mb-1">Commentaire (optionnel)</label>
|
|
<textarea id="cls-comment" rows="2" placeholder="Raison de la classification..." class="w-full px-3 py-2 bg-gray-800 border border-gray-700 rounded-lg text-sm text-gray-300 focus:border-brand-500 focus:outline-none resize-none"></textarea>
|
|
</div>
|
|
<button id="cls-submit" class="w-full px-4 py-2 bg-brand-600 text-white rounded-lg text-sm font-medium hover:bg-brand-700 transition-colors disabled:opacity-30" disabled>Sélectionnez un type ci-dessus</button>
|
|
<div id="cls-result" class="text-sm"></div>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Suggested IPs + Distribution chart -->
|
|
<div class="lg:col-span-2 space-y-4">
|
|
<!-- Suggested IPs -->
|
|
<div class="section-card overflow-hidden">
|
|
<div class="section-header"><span class="section-title">IPs suggérées (non classifiées)
|
|
<span class="relative inline-block"><button onclick="docToggle(this)" class="doc-btn">ⓘ</button><div class="doc-panel">
|
|
<h4>Suggestions de classification</h4>
|
|
<p>IPs détectées comme anomalies dans les 3 derniers jours qui n'ont pas encore de label SOC. Triées par score descendant.</p>
|
|
<p><strong>Action :</strong> Cliquez sur une IP pour la pré-remplir dans le formulaire, ou utilisez les boutons rapides.</p>
|
|
<p class="doc-source">Source : ml_detected_anomalies LEFT JOIN soc_feedback</p>
|
|
</div></span>
|
|
</span></div>
|
|
<div class="overflow-x-auto" style="max-height:35vh; overflow-y:auto">
|
|
<table class="data-table"><thead><tr>
|
|
<th>IP</th><th>Score max</th><th>Menace</th><th>Détections</th><th>JA4</th><th>ASN</th><th>Pays</th><th>Action</th>
|
|
</tr></thead><tbody id="suggested-body"></tbody></table>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Distribution chart -->
|
|
<div class="section-card">
|
|
<div class="section-header"><span class="section-title">Répartition des classifications
|
|
<span class="relative inline-block"><button onclick="docToggle(this)" class="doc-btn">ⓘ</button><div class="doc-panel">
|
|
<h4>Distribution des labels</h4>
|
|
<p>Ratio bot/légitime/suspect dans les labels SOC. Un bon ratio aide XGBoost à apprendre. Visez ≥100 labels par catégorie.</p>
|
|
<p class="doc-source">Source : soc_feedback GROUP BY classification</p>
|
|
</div></span>
|
|
</span></div>
|
|
<div class="section-body"><div id="dist-chart" style="height:180px"></div></div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Recent classifications -->
|
|
<div class="section-card overflow-hidden">
|
|
<div class="section-header"><span class="section-title">Historique des classifications
|
|
<span class="relative inline-block"><button onclick="docToggle(this)" class="doc-btn">ⓘ</button><div class="doc-panel">
|
|
<h4>Classifications récentes</h4>
|
|
<p>Les 50 dernières classifications effectuées par les analystes SOC. Chaque label sera utilisé par XGBoost au prochain cycle ML.</p>
|
|
<p class="doc-source">Source : soc_feedback ORDER BY created_at DESC</p>
|
|
</div></span>
|
|
</span></div>
|
|
<div class="overflow-x-auto" style="max-height:40vh; overflow-y:auto">
|
|
<table class="data-table"><thead><tr>
|
|
<th>Date</th><th>IP</th><th>Classification</th><th>Commentaire</th>
|
|
</tr></thead><tbody id="cls-history"></tbody></table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
{% endblock %}
|
|
{% block scripts %}
|
|
<script>
|
|
let selectedCls = '';
|
|
|
|
// ── Classification type toggle ──
|
|
document.querySelectorAll('.cls-type-btn').forEach(btn => {
|
|
btn.onclick = () => {
|
|
document.querySelectorAll('.cls-type-btn').forEach(b => b.classList.remove('ring-2','ring-white'));
|
|
btn.classList.add('ring-2','ring-white');
|
|
selectedCls = btn.dataset.cls;
|
|
const sub = document.getElementById('cls-submit');
|
|
sub.disabled = false;
|
|
sub.textContent = {true_positive:'✅ Classifier comme Vrai positif',false_positive:'❌ Classifier comme Faux positif',suspicious:'⚠️ Classifier comme Suspect'}[selectedCls];
|
|
};
|
|
});
|
|
|
|
// ── Submit ──
|
|
document.getElementById('cls-submit').onclick = async () => {
|
|
const ip = document.getElementById('cls-ip').value.trim();
|
|
if (!ip) { alert('Veuillez saisir une IP'); return; }
|
|
if (!selectedCls) { alert('Sélectionnez un type'); return; }
|
|
try {
|
|
const r = await fetch('/api/classify', {method:'POST', headers:{'Content-Type':'application/json'},
|
|
body:JSON.stringify({src_ip:ip, classification:selectedCls, comment:document.getElementById('cls-comment').value})});
|
|
const d = await r.json();
|
|
document.getElementById('cls-result').innerHTML = r.ok
|
|
? `<span class="text-green-400">✓ ${escapeHtml(ip)} → ${d.classification}</span>`
|
|
: `<span class="text-red-400">✗ ${escapeHtml(d.detail||'erreur')}</span>`;
|
|
if (r.ok) { document.getElementById('cls-ip').value=''; document.getElementById('cls-comment').value=''; loadAll(); }
|
|
} catch(e) { document.getElementById('cls-result').innerHTML = `<span class="text-red-400">✗ ${e}</span>`; }
|
|
};
|
|
|
|
function prefillIP(ip) {
|
|
document.getElementById('cls-ip').value = ip;
|
|
document.getElementById('cls-ip').scrollIntoView({behavior:'smooth',block:'center'});
|
|
document.getElementById('cls-ip').focus();
|
|
}
|
|
|
|
function quickClassify(ip, cls) {
|
|
document.getElementById('cls-ip').value = ip;
|
|
selectedCls = cls;
|
|
document.getElementById('cls-submit').click();
|
|
}
|
|
|
|
async function loadAll() {
|
|
try {
|
|
const [stats, suggested, history] = await Promise.all([
|
|
fetch('/api/classify/stats').then(r=>r.json()),
|
|
fetch('/api/classify/suggested').then(r=>r.json()),
|
|
fetch('/api/classifications').then(r=>r.json()),
|
|
]);
|
|
|
|
// ── KPIs ──
|
|
const byType = {};
|
|
(stats.stats||[]).forEach(r => { byType[r.classification] = r.cnt; });
|
|
document.getElementById('kpi-total').textContent = fmtNum(stats.total||0);
|
|
document.getElementById('kpi-tp').textContent = fmtNum(byType.true_positive||0);
|
|
document.getElementById('kpi-fp').textContent = fmtNum(byType.false_positive||0);
|
|
document.getElementById('kpi-suspect').textContent = fmtNum(byType.suspicious||0);
|
|
|
|
// ── Distribution chart ──
|
|
const CLS_COLORS = {true_positive:'#ef4444',false_positive:'#22c55e',suspicious:'#eab308'};
|
|
const CLS_LABELS = {true_positive:'✅ Vrai positif',false_positive:'❌ Faux positif',suspicious:'⚠️ Suspect'};
|
|
if (stats.total > 0) {
|
|
const el = document.getElementById('dist-chart');
|
|
const ch = echarts.init(el);
|
|
ch.setOption(ecBase({
|
|
tooltip: ecTooltip({trigger:'item'}),
|
|
series: [{type:'pie', radius:['35%','65%'], center:['50%','50%'],
|
|
label:{color:EC_TEXT,fontSize:11,formatter:'{b}\n{d}%'},
|
|
data:(stats.stats||[]).map(r=>({name:CLS_LABELS[r.classification]||r.classification,value:r.cnt,itemStyle:{color:CLS_COLORS[r.classification]||'#6b7280'}}))}]
|
|
}));
|
|
} else {
|
|
document.getElementById('dist-chart').innerHTML = '<div class="flex items-center justify-center h-full text-gray-500 text-sm">Aucune classification — commencez à labéliser !</div>';
|
|
}
|
|
|
|
// ── Suggested IPs ──
|
|
document.getElementById('suggested-body').innerHTML = (suggested.suggested||[]).map(row => `<tr>
|
|
<td class="whitespace-nowrap cursor-pointer hover:text-brand-400" onclick="prefillIP('${escapeHtml(row.src_ip)}')">${fmtIP(row.src_ip)}</td>
|
|
<td>${fmtScore(row.worst_score)}</td>
|
|
<td>${threatBadge(row.threat_level)}</td>
|
|
<td class="font-mono text-xs">${row.detection_count||0}</td>
|
|
<td class="text-xs font-mono max-w-[80px] truncate">${fmtJA4(row.ja4)}</td>
|
|
<td class="text-xs max-w-[100px] truncate">${row.asn_org ? fmtASN(row.asn_org) : ''}</td>
|
|
<td>${fmtCountry(row.country_code)}</td>
|
|
<td class="whitespace-nowrap">
|
|
<button onclick="quickClassify('${escapeHtml(row.src_ip)}','true_positive')" class="px-1.5 py-0.5 text-[10px] bg-red-500/20 text-red-400 rounded hover:bg-red-500/30" title="Vrai positif">✅</button>
|
|
<button onclick="quickClassify('${escapeHtml(row.src_ip)}','false_positive')" class="px-1.5 py-0.5 text-[10px] bg-green-500/20 text-green-400 rounded hover:bg-green-500/30" title="Faux positif">❌</button>
|
|
<a href="/ip/${encodeURIComponent(row.src_ip)}" class="px-1.5 py-0.5 text-[10px] bg-gray-700 text-gray-300 rounded hover:bg-gray-600 inline-block" title="Détail">🔍</a>
|
|
</td>
|
|
</tr>`).join('') || '<tr><td colspan="8" class="text-center text-gray-500 py-4">Toutes les IPs ont été classifiées 🎉</td></tr>';
|
|
|
|
// ── History ──
|
|
document.getElementById('cls-history').innerHTML = (history.data||[]).map(row => `<tr onclick="window.location='/ip/${encodeURIComponent(row.src_ip)}'">
|
|
<td class="text-xs text-gray-400">${(row.created_at||'').substring(0,16)}</td>
|
|
<td class="whitespace-nowrap">${fmtIP(row.src_ip)}</td>
|
|
<td><span class="badge ${row.classification==='true_positive'?'badge-critical':row.classification==='false_positive'?'badge-low':'badge-medium'}">${escapeHtml(row.classification)}</span></td>
|
|
<td class="text-xs max-w-[300px] truncate text-gray-400">${escapeHtml(row.comment||'')}</td>
|
|
</tr>`).join('') || '<tr><td colspan="4" class="text-center text-gray-500 py-4">Aucune classification</td></tr>';
|
|
|
|
} catch(e) { console.error('classify load error:', e); }
|
|
}
|
|
|
|
// ── Pre-fill from URL ──
|
|
const urlIP = new URLSearchParams(window.location.search).get('ip');
|
|
if (urlIP) document.getElementById('cls-ip').value = urlIP;
|
|
|
|
loadAll();
|
|
</script>
|
|
{% endblock %}
|