Le z-score par tuile et le vmax au percentile 98 par tuile donnaient une
échelle de couleur non jointive (écart-type variant de 0,05 a 0,58 m selon
la dalle). Références de normalisation figées (médianes mesurées sur 20
dalles réelles) et vmax fixe 3,8 (p98 médian) : même rugosité physique =
même couleur partout.
Remplace NearestNDInterpolator (cKDTree sur les 25 M de points valides,
plusieurs secondes par dalle trouée en mode IGN pur) par
distance_transform_edt : plus proche voisin en une passe O(n).
Voisins équidistants arbitres differemment, sans effet sur les rendus.
Replaces the pure-Python heapq implementation with a compiled binary
min-heap (~200x faster for large grids). Falls back to Python when
numba is unavailable. Uses a flat array view for heap elevation
comparisons to avoid 2D indexing issues in nopython mode.
La webapp (carte + vignettes) et la génération de tuiles se déploient sur
deux machines : image légère Dockerfile.webapp (FastAPI + Pillow AVIF
natif + pyproj) sur Raspberry Pi, pipeline complet sur la machine de
traitement. LIDAR_GENERATION_URL délègue /api/generate, /api/preview et
/api/status ; /api/sync ramène les tuiles par rsync puis régénère
vignettes et index localement. Token partagé optionnel
(LIDAR_API_TOKEN/LIDAR_REMOTE_TOKEN). Retire du dépôt les journaux
internes (.swival, audit-findings) et les données (data/, notebooks/).
Doc : docs/DEPLOY_WEBAPP.md.
- Web map: multi-select picker in the generation bar (aspect, slope,
positive openness, anisotropic openness, wavelet) passed to the API;
the layer panel is restricted to the same shortlist (PANEL_VIZ) and
a refresh button rebuilds the index when new layers appear on disk;
jobs started outside the UI are now adopted into the visible queue
- Uniform colors across tiles: openness/anisotropic/sailore now store
local z-scores, and all renderers use fixed ranges (0-3 sigma, SVF
0-1 physical, slope 0-30 deg) instead of per-tile percentile stretches
- Ray-tracing falls back to CPU when VRAM is exhausted so openness and
SVF no longer fail silently on shared GPUs
- build_index merges visualizations available at only one resolution
into the displayed tile so in-progress layers stay visible
- 11 new tests (142 passing)
Small archaeological features (ditches, walls, post-holes) were drowned out
by large-scale topography (100-200m). Fix: replace 100/200m with finer
scales (3, 8, 15m), weight 5-10m heaviest, clip absolute z-score to 3.0
before combination to prevent large-scale outliers from dominating.
GPU multi-processing fix:
- gpu.py: revert to CUDA_VISIBLE_DEVICES approach with lazy CuPy init
(Device.use() caused CUDA_ERROR_NO_BINARY_FOR_GPU on GPU 1)
- CuPy is imported lazily on first to_gpu() call, allowing
CUDA_VISIBLE_DEVICES to be set before CUDA context creation
- nvidia-smi used for GPU count detection (no CUDA import needed)
- pipeline.py: add tip message suggesting -w N when multiple GPUs detected
Rendering improvements:
- Title: split into bold title (14pt) + italic description (10pt)
- North arrow: moved inside data area (top-right) with transparent
background — no longer overlaps title
- Colorbar: full height (compass gap removed), ScalarFormatter with
useOffset=False to prevent scientific notation on small values
Performance:
- rendering.py: save matplotlib figure to BytesIO instead of temp PNG
file — eliminates disk I/O between matplotlib and PIL
- visualizations.py: cap max_dist at 300 for ray-tracing (SVF,
openness, aniso_open) — avoids 500+ iterations at 0.2m resolution
- pipeline.py: deduplicate n_gpus calculation in parallel path
- Replace RdBu_r with Crameri roma for all 6 relief-family colormaps
(curvature, mslrm, lrm, tpi, sailore, aniso_open): perceptually
uniform, CVD-friendly, dark center makes near-zero values visible
- Replace hsv with twilight for aspect: perceptually uniform cyclic
colormap with no hue discontinuity
- Add cmcrameri dependency (falls back to RdBu_r if unavailable)
- Fix IGN location map: pass min_zoom=8 to download_ign_tiles so zoom
10 context maps work (was hardcoded to min_zoom=15, blocking the
loop entirely for zoom_level=10)
- Expand location map context from 3x extent to 80km fixed radius for
better regional context at zoom 10
- Remove dead code: flow colormap entry, _nice_scale dead loop,
aspect='auto' in location map imshow, outdated PDF report order list
- Fix SVF docstring: 8 directions → 16 directions
- validate_laz: verify point data accessibility (not just headers) to detect
corrupted COPC files that pass header checks but fail on data reads
- classify_ground: fallback from CSF to SMRF when CSF produces no ground
points or PDAL errors (fixes 2/9 failing tiles)
- MSRM: preserve sign in weighted combination so RdBu_r colormap shows
both red (elevated) and blue (depressed) instead of red only
- Add Sky-View Factor (SVF) visualization: cos²(horizon angle) over 16
directions, excellent for archaeological earthwork detection
- Add Anisotropic Openness: directional weighting (NW-SE/NE-SW) enhances
linear feature detection aligned with common settlement patterns
- Remove anomalies and flow visualizations (replaced by SVF + aniso_open)
- Location inset: use IGN topographic map at zoom 10 instead of simplified
France outline, with red rectangle marker and fallback
- Remove flow (hydrological accumulation) from VIZ_STEPS
- MSRM/TPI/roughness/anomalies: revert z-score (x-mean)/std to std normalization x/std
to preserve contrast and visibility of linear features (paths, ditches, trenches)
- MSRM: adaptive scales based on resolution, archaeological weight combination
- TPI: extend from 2 to 4 scales (3m/15m/50m/200m) with weighted combination
- Hillshade: 8 directions instead of 4, altitude 35° instead of 30°
- LRM: adaptive sigma based on resolution
- Openness: doubled radius (100m instead of 50m)
- Roughness: multi-scale (3m fine + 15m broad) instead of single 5x5 window
- Anomalies: uses MSRM multi-scale relief instead of single LRM 15m
- Wavelet: 8 adaptive scales, std normalization, archaeological weights
- Remove svf (Sky-View Factor) and local_dominance visualizations
- Add AVIF format support (default), quality 98
- Add multi-resolution support (-r 0.5,0.2)
- Improve Ctrl+C handling for immediate process termination
- Update rendering.py descriptions for all modified visualizations
Two optimizations to avoid ~2min wasted per file on re-runs:
1. pipeline.py: Check which visualizations need regeneration before
computing SharedDEM. If all WebP outputs exist, skip SharedDEM
entirely. If only IGN overlays need updating, also skip SharedDEM.
2. visualizations.py: Make SharedDEM attributes lazy (filled, gradient,
lrm_15) so only the data actually needed is computed. For example,
if only hillshade is regenerated, LRM at 15m is never calculated.
- Remove generate_rrim, generate_multi_hillshade, _compute_openness_both
- Remove corresponding VIZ_STEPS entries, COLORMAPS, RGB_LEGENDS, and tests
- Fix DTM resolution mismatch: existing DTM at different resolution is now
regenerated instead of silently reused
- Propagate actual DTM resolution to visualizations and rendering
- Add --init to docker run commands for proper signal handling on Ctrl+C
- Add .playwright-mcp/ to .gitignore
Three new visualizations complementing existing SVF/openness/LRM/MSRM:
- RRIM (Red Relief Image Map): RGB composite combining positive openness
(R), inverted slope (G), negative openness (B). Uses ray-tracing
to compute both openness values in a single pass.
- Multi-Hillshade RGB: 3 azimuths (315°, 135°, 45°) mapped to R/G/B
channels with slope blending. Color reveals structure orientation.
- Local Dominance: (dem - local_min) / (local_max - local_min) using
min/max filters. Measures local height position — complements openness.
Also adds:
- _compute_openness_both() helper for shared ray-tracing (used by RRIM)
- xp_maximum_filter() in gpu.py (GPU/CPU abstraction)
- Entries in COLORMAPS, RGB_LEGENDS, VIZ_STEPS, and is_rgb detection
- All NaN handling follows existing patterns (nan_mask restoration)
- Remove PMF from ground classification options (PDAL recommends SMRF over PMF)
- Auto-detection now uses CSF for urban/complex terrain instead of PMF
- Add z_std > 30m heuristic to auto-select CSF for complex terrain
- Fix pos_open/neg_open lambda missing 'shared' parameter (NameError in workers)
- Fix NaN mask not restored in hillshade, slope, aspect, curvature
(gradient-based products computed on filled DEM lost NaN transparency)
- Add nan_mask parameter to _save_tif for centralized NaN restoration
- DTM TIF kept by default (no longer deleted after WebP conversion)
- Fix gpu_cleanup import missing in visualizations.py (NameError in workers)
- Fix t_pdf referenced before assignment when PDF is skipped
- Skip classification+DTM when DTM exists regardless of --force
- --force now only regenerates WebP/PDF, not classification/DTM
- --force-classification forces reclassification when needed
- Add laspy repair fallback for corrupt LAZ files (EVLR errors)
- Keep DTM TIF by default for reuse (--no-keep-tif to delete)
- Increase space between image and bottom cartouche (0.12→0.19)
If the DTM .tif exists and --force is not set, skip both ground
classification and DTM generation entirely. Previously, the pipeline
would spend 3+ minutes reclassifying ground even when the DTM was
already present and would be reused anyway.
Also includes: SharedDEM cache, enhanced WebP cartouche (compass rose,
adaptive scale bar, enriched info bar), removed COG/viewer, UTF-8
fix for parallel workers, skip logic for DTM and PDF.
- Positions d'axes fixes (data_left/bottom/width/height_frac) pour alignement
pixel-parfait entre terrain et ortho/topo
- aspect='equal' au lieu de 'auto' pour conserver les proportions géographiques
- Colorbar descriptive pour les visualisations RGB (ortho/topo)
- Comblage des petits trous DTM (< 1m) via rasterio.fill.fillnodata
- Suppression de la visualisation "dépressions"
- Hillshade composite: 0.7*hillshade + 0.3*cos(slope)
- D8 flow accumulation accéléré par numba JIT (fallback Python)
- Flag --keep-tif pour conserver les TIFF intermédiaires
- --force supprime aussi les TIF existants avant régénération
- ETA affiché pendant la génération des visualisations
- Répertoires temp dans temp/ pour traitement parallèle
- Suppression de generate_texture() de visualizations.py
- Suppression de l'entrée 'texture' de VIZ_STEPS et COLORMAPS
- Suppression du test TestTexture
- Mise à jour README (19 → 18 visualisations)
- Mise à jour AGENTS.md (17 → 16 fonctions generate_*)
- Convert MaskedArray to ndarray avant np.percentile() via
np.asarray(data.compressed()) et np.ma.filled(data, np.nan)
- Supprimer RuntimeWarning "Mean of empty slice" dans MSRM et
ondelette avec warnings.catch_warnings()
- Ajout import warnings dans visualizations.py
- Ajout classification automatique du sol (SMRF/PMF/CSF) avec détection
heuristique (ratio retours uniques > 0.6 → PMF urbain, sinon SMRF)
- Pré-traitement PDAL recommandé avant classification: ELM + outlier
removal (cell=5.0, threshold=2.0 adapté au calcaire rocailleux)
- Options CLI: --ground-classification {auto,smrf,pmf,csf} et
--force-classification pour forcer la reclassification
- Fix double logging (logger.propagate = False)
- Fix --force non transmis dans run.sh (réécriture parsing arguments)
- Fix warning numpy 'partition will ignore mask': conversion MaskedArray
en ndarray avant np.percentile()
- Ajout liblaszip8 + lazrs pour support LAZ dans Docker et laspy
- Tests unitaires pour PMF, CSF et auto-détection
- DTM: plus d'interpolation, les zones sans LiDAR restent NaN
- Ajout _fill_nans() et _filter_nanaware(): remplissent les NaN par
nearest-neighbor avant filtrage, puis restaurent le masque NaN
- Toutes les visualisations avec filtres (LRM, MSLRM, TPI, SAILORE,
roughness, anomalies, wavelet) utilisent _filter_nanaware pour
éviter l'érosion des bords de données
- _save_tif() écrit nodata=float('nan') quand le tableau contient des NaN
- Les zones sans données restent vides dans les visualisations
- Les calculs ne sont pas faussés par des valeurs interpolées
- Suppression de l'interpolation NearestNDInterpolator dans create_dtm_fast
- Les pixels sans données LiDAR restent NaN dans le DTM et les
visualisations — pas de valeurs fictives qui faussent les calculs
- nodata=float('nan') dans le GeoTIFF de sortie pour identifier les vides
- _save_tif() détecte automatiquement les NaN et écrit le flag nodata
- Suppression de generate_solar (éclairage solaire) des visualisations
- Accélération GPU de hillshade, slope, aspect, curvature, depressions,
anomalies, roughness, texture GLCM, flow (sink filling)
- Nettoyage mémoire GPU entre visualisations (gpu_cleanup)
- Correction OOM texture GLCM: calcul entropie bin par bin au lieu d'un
tableau 3D massif sur GPU
- Correction bug: xp_minimum_filter manquant dans imports visualizations
- Option --file accepte plusieurs noms complets sans extension
- run.sh affiche l'aide si appelé sans arguments
- Option --test pour exécuter les tests unitaires dans Docker
- Filtre ReturnNumber>=1 intégré dans le pipeline PDAL (plus d'erreur SMRF)
- 60 tests unitaires: GPU, visualisations, rendering, DTM, pipeline, CLI
- Ajout pytest au Dockerfile
- Découpage du monolithe process_lidar.py (~2750 lignes) en package
lidar_pipeline/ avec 9 modules (gpu, dtm, visualizations, ign,
rendering, pipeline, cli, __init__, __main__)
- Logging configurable: -v (verbose avec timestamps) et --debug
(détails internes fichier:ligne)
- Option --force pour régénérer tous les fichiers (par défaut skip
les WebP existants)
- Option --file NOM pour traiter un seul fichier LAZ (tests rapides)
- ProcessPoolExecutor avec répertoires temporaires uniques par worker
- Suppression du code mort (geomorphons, hillshade_ne, nodata_mask)
- Aucun fichier TIFF résiduel après conversion WebP
- setup.py pour installation pip, stub process_lidar.py compatible
