lidar_rendu/AGENTS.md

Workflow

• install: docker build -t lidar-lidar . (deps baked into the image)
• one-command local start (GPU optional): ./start.sh (map on 8973), ./start.sh process [options], ./start.sh logs|stop — creates input//output/ as the current uid, adds docker-compose.cpu.yml (gpus: !reset) when Docker cannot reach any GPU.
• build: docker build -t lidar-lidar .
• build the lightweight map (Raspberry Pi, two-machine deployment — see docs/DEPLOY_WEBAPP.md): docker compose -f docker-compose.maps.yml up -d --build (image Dockerfile.maps, no PDAL/GPU)
• build the tile generator (processing machine): docker compose -f docker-compose.worker.yml up -d --build (service worker = mapserve on the full image: API + tiles + inventory for the remote maps)
• test all: ./run.sh --test (rebuilds the image automatically before the tests; with a direct docker run, rebuild by hand first)
• test file: docker run --rm lidar-lidar python3 -m pytest -v --pyargs lidar_pipeline.tests.<module>
• test case: docker run --rm lidar-lidar python3 -m pytest -v --pyargs lidar_pipeline.tests.<module>::<TestClass>::<test_method>
• lint: not configured
• format: not configured
• after every edit: ./run.sh --test
• RULE 1 — always run through docker compose (never a direct docker run): local map/API → docker compose up -d --build serve (port 8973, mapserve on the full image); one-off processing → docker compose run --rm --build process [options]; logs → docker compose logs -f serve; stop → docker compose down.
• RULE 2 — ALWAYS --build: the code is baked into the image (never mounted). Without --build, up/run reuse the existing image and the OLD code runs. --build is almost instant thanks to the cache (.dockerignore keeps input/ and output/ out of the context). After an edit, docker compose up -d --build serve recreates the container on fresh code.
• quick test without rebuild (code mounted over the image): docker run --rm -e PYTHONPATH=/app -v $(pwd)/lidar_pipeline:/app/lidar_pipeline lidar-lidar python3 -m pytest --pyargs lidar_pipeline.tests -q (~3 min; prefix with timeout 600, and NO | tail pipe, which hides the progress)
• debug: ./run.sh --debug (file:line logging); container shell: docker run --rm -it -v $(pwd)/input:/data/input -v $(pwd)/output:/data/output --entrypoint bash lidar-lidar
• ./run.sh passes its own defaults to the CLI: -r 0.5 and -w 1 unless given (the CLI alone defaults to -r 0.2, -w auto); image quality, format and ground classification fall through to the CLI defaults (60, avif, ign).
• updating a remote lightweight map (git checkout + local maps override, see docker-compose.maps.override.yml.example): ssh <host> "cd <checkout> && git pull && docker compose -f docker-compose.maps.yml -f docker-compose.maps.override.yml up -d --build" (procedure in docs/DEPLOY_WEBAPP.md).
• backfilling the quality sidecars (tiles processed before the sidecar existed): the fixed command of process (compose) is replaced entirely as soon as any argument follows the service name, so --quality-backfill alone does not work — use docker compose run --rm --build process python3 -m lidar_pipeline /data/input -o /data/output --quality-backfill.

Conventions

• A single web server: mapserve.py (image lidar-maps, port 8975 lightweight / 8973 worker). The old webapp (webapp.py, export.py, index.html/_APP_JS) has been removed: tile generation (the scope of the historical webapp — /api/preview, /api/generate, /api/status, /api/stop, /api/queue/clear, /api/cell) lives in mapserve.py, the interface in web/map.{html,css,js} (read by mapui.py, _MAP_* constants kept, written as app.css/app.js by write_map_assets() and baked into the images). On the lightweight image without LIDAR_GENERATION_URL, /api/status answers available: false and the interface hides the Generation tab.
• Single-panel interface (web/map.{html,css,js}): one tabbed panel View / Tile / PDF export / Generation (hidden when the generator is unavailable or not allowed) / Share, replacing the old stack of blocks. Below 720 px wide, the panel becomes a bottom sheet with three heights (closed/half/full; handle #sheetGrip dragged or simply tapped, or the active tab tapped again). A click on a tile selects it (outline) and fills the Tile tab (extent, IGN, strip alignment) without switching the displayed tab; clicking the same tile again deselects it, clicking another moves the selection. Escape deselects the tile, then collapses the panel (icon strip on desktop, closed sheet on phone — the strip stays usable, a click on a tab expands the panel again). Keyboard shortcuts: 1–5 (tabs, no effect when the tab is hidden), P (next display mode), Escape. Relief intensity (&I=, contrast 0.5–2×, default 1×): a personal comfort setting stored in lidarMapView_v2.intensity and shared in the link, never frozen as a server default (★ Set as default ignores it). Two themes, light/dark (lidar-theme, auto by default, follows the system), and every shared setting set on :root as CSS variables (no hard-coded colour in the components). localStorage keys (through lsGet/lsSet, silent in private browsing): lidarMapView_v2 (view/layers), lidar-print (export settings), lidar-panel (tab, sheet height, collapse to the icon strip), lidar-theme.
• index.py = catalogue + shared registries (no interface any more): VIZ_LABELS/VIZ_LEGENDS, display defaults (DEFAULT_VIZ/PRECISION_VIZ/VIEW_MODES), PANEL_VIZ/KEYWORD_TO_STEP, scan_tiles/cells_with_all_viz, thumbnails + sub-tiles + the index_tiles.json inventory (build_index). The inventory is served by mapserve's /api/tiles to the lightweight machines (LIDAR_SOURCE_URL).
• Generation is 0.2 m only (policy): /api/generate (GENERATE_RESOLUTIONS in mapserve.py), the compose process command and the CLI -r default all produce 0.2 m exclusively; 0.5 m stays available via an explicit -r 0.5 (note: ./run.sh still passes -r 0.5 by default). Completeness detection (complete_cells) requires the viz at 0.2 m only.
• North-to-south generation: tiles are processed by decreasing row (row = northing in km), then increasing column — find_laz_files (pipeline.py) for batch passes and _resolve_request (mapserve.py) for runs started from the map. Workers take the files in submission order: the map fills from top to bottom during a run (an explicit --file on the CLI keeps the user's order). Generation parallelism: LIDAR_WORKERS (10 in docker-compose.yml, auto in docker-compose.worker.yml and when unset).
• Full sub-tiling: _CARTO_SUBTILED_VIZ (empty in index.py) splits ALL layers into 500 m quadrants at 0.2 m; all in AVIF 4:2:0 q75 (_SUBTILE_AVIF_QUALITY), encoded ONCE from the original raster: tif_to_crop calls index.write_subtiles (through rendering._write_subtiles_from) right after the tile (so they are newer than it, and build_index does not re-encode them; the old chain tile q60 → sub-tile q55 stacked two losses: 18.1 dB/SSIM 0.84 against 19.5 dB/0.93). 4:2:0 tops out around 19.5 dB on the relief (per-pixel hue averaged over 2 × 2): only 4:4:4 would go further (q75: 28 dB, ~1.6× heavier). Exception: flat level maps (_SUBTILE_LOSSLESS_VIZ, the precision layer) in lossless WebP (.webp, accepted by tiles.py). Pillow ignores lossless=True for AVIF (q75, lossy); in RGB no AVIF setting is exact. A layer that fails to split falls back to the whole tile (_fallback_full_dalle) without penalising the others.
• XYZ tiles reusable outside the project: /tiles/{layer}/{z}/{x}/{y}.png follows the OpenStreetMap scheme (256 px, EPSG:3857, RGBA PNG transparent outside coverage, CORS *) — JOSM, iD, QGIS, uMap and MapLibre consume it as is, with discovery through TileJSON / WMTS / josm.imagery.xml. The 512 px variant (@2x.avif) is reserved for the internal interface (half as many requests over HTTP/1.1). Any change to the URL template breaks client configurations: changing it requires an explicit decision.
• Naming: all code identifiers, comments, logs, help and user-facing strings are English; data keys inherited from the French era (relief_oriente, densite_sol, visualisations/, --ign-classes sol, stats key telechargees…) are stable contracts and must not be renamed.
• Adding a visualization requires 4 edits: (1) generate_X() in visualizations.py, (2) entry in VIZ_STEPS in pipeline.py, (3) entry in COLORMAPS in rendering.py (or RGB_LEGENDS for an RGB output), (4) entry in VIZ_LEGENDS in index.py (title/legend/description + sampled cmap gradient — single text source merged into COLORMAPS at import, also served in /api/map/meta and the TileJSON). Missing any one breaks the pipeline.
• generate_* signature is strict: (dem_file, basename, vis_dir, resolution, shared=None) returning Path on success, None on failure. IGN overlays (ortho, topo) omit shared.
• Return None on failure, never raise: dtm.py, visualizations.py, and ign.py all return None to let the pipeline continue. Raising aborts the entire file.
• Logger is always logging.getLogger("lidar"), never __name__. All modules route through this single logger so worker processes can configure it.
• Joint scan-line adjustment (3rd pass of strip alignment, STRIP_ALIGN_VERSION 3): each scan line (~150 lines/s, split by _scan_line_ids: flyback of the scan_angle sawtooth) can be offset AND tilted (roll) by 1 to 15 cm — isolated sunken lines, a whole pass tilted and visible as a band (measured on LHD_FXX_0999_6882). _joint_line_corrections (dtm.py) re-fits each line (a + b·u) against the consensus of the OTHER strips in its 1 m cell (local plane through the slope), least squares trimmed at 5 MAD via bincount, damped step 0.5, stop below 1 mm, re-centring (no global drift), estimated on 1 point in 3; CuPy when the GPU is active, numpy fallback. On top of that, a calibration profile per strip and per degree of scan angle (STRIP_ANGLE_BIN, shared by all lines of the strip, mean removed, slope kept because the per-line tilt is undetermined when the swath only partly crosses the tile; sparse bins = neighbouring value) removes the non-linear across-swath errors (±1.5 cm measured at the edge), a source of steps parallel to the flight line at the swath edge. Lines without overlap: _scan_line_corrections_beam (against the surface of their own strip, line-to-line component only, σ 3 lines). Validated on never-seen blocks (10 m checkerboard). Jitter by time windows (2nd pass) is no longer computed when scan_angle exists (redundant). CPU cost ~26 s per tile for the whole alignment (73 s before). Sidecar: lines, line_window, line_cell, line_model.
• Filling gaps between points, bounded by the envelope (GAP_FILL_VERSION 3, _fill_small_gaps in dtm.py): at 0.2 m ~80 % of the pixels contain no point. The old fillnodata at 1 m filled every pixel within 1 m of a point — each isolated point became a flat 2 m disc (hue atan2(0,0) = saturated pink in the oriented relief) and each hole received an extrapolated 1 m band (coloured fringe). Now: morphological closing of the measured pixels (nothing is extended outwards) whose radius follows the local point spacing (GAP_RADIUS_K 1.5 × spacing measured over 5 m, steps GAP_RADII_M 1/1.5/2/3 m, 1 m minimum to plug car-sized holes), then islands < GAP_MIN_ISLAND_M2 (1 m²) removed. Morphology with numpy slices (_morph_step, ~10× scipy); ~5–9 s per 5000² tile. Version stored in the GeoTIFF tag LIDAR_GAP_FILL (2 = envelope, 3 = + density sidecar file): absent/different ⇒ DTM regenerated (_gap_fill_matches in pipeline.py). rasterio.fill.fillnodata writes into its input: always pass it a copy.
• Fixed-scale openness: generate_openness normalises with frozen references OPENNESS_POS_REF / OPENNESS_NEG_REF (degrees, medians measured on 15 tiles spread over the territory) instead of a per-tile z-score — same openness = same colour, seamless mosaic.
• Filename special-cases in _expected_output_path(): pos_open → positive_openness, neg_open → negative_openness, hillshade → hillshade_multi.
• Imposed generation settings: the map no longer offers a classification or stitching setting — _build_command (mapserve.py) always passes --ground-classification ign --ign-classes sol --edge-buffer 100; any ground_class/ign_classes/reclassify/edge_buffer fields sent are ignored. CLI defaults aligned (ign, 100).
• IGN classification by direct extraction: _extract_ign_ground (dtm.py) filters the classes with laspy (returns ≥ 1) and writes the ground LAS without PDAL (4.9 s instead of 13.5 s per tile); the read done by the automatic detection is reused (_LAST_READ). PDAL remains the fallback.
• Pyramid updated during rendering: the pipeline rebuilds the inventory after each tile by default (incremental_index unless --no-index, whatever the launcher), with a deferred pass when the 3 s debounce skips a tile. On the map side, _bg_poller watches the inventory every BG_WATCH_S = 10 s (local file or upstream payload) and starts a scan as soon as it changes; tiles of modified source tiles go to the front of the queue (_bg_enqueue(front=True)), ahead of the backlog.
• Full pyramid generated ahead of time: by default tiles.zoom_cached stores ALL levels up to native (TILE_CACHE_MAX_Z = TILE_MAX_NATIVE_Z 19, TILE_EVEN_LEVELS off) and background maintenance — enabled with LIDAR_TILE_BACKGROUND=1, set in the Pi override — generates them ahead of time up to 18@2x (default of LIDAR_TILE_BACKGROUND_MAX_Z); on the Pi it DOWNLOADS them from the worker. Map capped at zoom 19 (maxZoom, 1 screen px = 1 LiDAR px, never an upscaled tile). Reason: on-the-fly rendering on the Pi (~170 ms/tile, 3 at a time) gave 1.5–6 s per screen at zooms 17–19. Bounded maintenance queue (queue_max): each source tile keeps in _bg["incomplete"] the index of its first tile rejected for lack of room and resumes from there on the next scans (before: the first scan filled the queue and the rest of the pyramid was never generated). ~110,000 @2x tiles for 3,240 source tiles (~5 GB). Reduced storage still possible (LIDAR_TILE_EVEN_LEVELS=1 → EvenLevelTileLayer, even levels only, native 19 requested as is).
• Bounded map memory (Pi, mem_limit: 1g): _open_source (tiles.py) caches a detached copy of each source (an open AVIF image keeps its decoder, ~18 MB more per 2500² quadrant) and counts 4 bytes/pixel (PIL stores RGB on 32 bits); Dockerfile.maps sets MALLOC_MMAP_THRESHOLD_=1048576 so that glibc returns large buffers to the system. Without these two points, browsing at high zoom (levels rendered on the fly) climbed to ~700 MB RSS and the container was killed in a loop by the OOM killer (502 on the Traefik side).
• Standalone map (Pi without worker): upstream (LIDAR_SOURCE_URL/LIDAR_MAPS_URL) down ⇒ the map is still served from disk. _remote_payload (tiles.py) also remembers the failure (TTL 60 s, timeout 10 s: otherwise every request paid the network timeout again and /api/map/meta stopped answering), then the local index (_build_index) takes over; circuit breakers on sources (_SOURCE_OFFLINE, 60 s) and tiles (_UPSTREAM, re-armed on expiry); a fetched source is dated to its upstream version (os.utime) so the local index keeps the same dates; no upstream request for a tile without a source tile when the inventory comes from upstream. In cache-only mode, a stale tile keeps being served (no-store, X-Tile-Pending) until the new one arrives.
• Pyramid downloaded in the background: with LIDAR_MAPS_URL, maintenance (_bg_process_one) DOWNLOADS the tiles of the stored levels from upstream (stat telechargees), without waiting for them to be displayed, and with no pause between two downloads (the worker copes); local rendering as a fallback when upstream does not answer, which alone is followed by LIDAR_TILE_BACKGROUND_PAUSE (Pi resources).
• First appearance of source tiles (_apply_seen, index_xyz/.sources_seen.json): effective date of a source = max(version, first entry in the inventory). A source tile written before but inventoried after an XYZ tile (upstream index TTL, inventory debounce) therefore invalidates that tile — otherwise a permanent hole at that level, on disk, in memory and in the browser (unchanged ?v= stamp). Persistent registry; absent with an existing cache (upgrade) ⇒ everything is invalidated once.
• Browser cost of the map (measured in Chrome, Intel Iris Xe): the darkening filter of the OSM base map (.base-dark) is set on the layer container, never on each tile — identical rendering (per-pixel operations), GPU process 92 % → 56 % while panning and 99 % → 53 % on wheel zoom. LiDAR layers use keepBuffer: 1 (512 px tiles = 1 MB decoded each). Measured with no notable effect: the cards' backdrop-filter, isolation/mix-blend-mode set to "normal", will-change. JS heap 2–5 MB, idle CPU ~1 %. AVIF decoding ~12 ms/tile against ~5 ms for WebP (accepted trade-off: AVIF storage).
• Fast AVIF encoding: AVIF_SPEED = 9 (rendering.py, tiles.py, _SUBTILE_AVIF_SPEED in index.py) — a 5000 × 5000 px tile encoded in 0.6 s instead of 4 s (+3 % size, −0.3 dB); encoding was the longest step of rendering a layer.
• Workers bounded by VRAM: each pool process takes a fixed slot when it is created (_init_worker_slot in pipeline.py, multiprocessing queue) computed by gpu_worker_slots (gpu.py): at most (free VRAM − LIDAR_GPU_RESERVE_MIB 512) / LIDAR_GPU_WORKER_MIB (2048, estimated peak: CUDA context + joint CuPy alignment in float64 + EDT of _fill_nans) workers per GPU, at least one, interleaved; the excess runs on the CPU (force_cpu). The old round-robin by file number put 6 workers on each 8 GB RTX 5060 with LIDAR_WORKERS=auto (12): OOM. Unreadable VRAM (nvidia-smi failing): unbounded round-robin.
• Preparation on the GPU: _fill_nans (cupyx distance transform) and the SharedDEM gradients run on the GPU when it is active (CPU fallback). NUMBA_CACHE_DIR=/tmp/numba-cache (Dockerfile) avoids recompiling the numba kernels in every worker.
• Default output is AVIF, not WebP. Use --format webp for WebP. Quality default is 60 (visually lossless on smooth colour ramps, ~÷3 vs q98).
• Vertical alignment of flight strips: each tile mixes several passes (1–2 PointSourceId per pass), sometimes vertically biased by a few cm (±2.5 cm measured on 1000_6882). create_dtm_fast measures the robust offset of each strip (ground points, 1 m cells, median surface iterated 3×) and subtracts offsets ≥ 0.5 cm (STRIP_ALIGN_THRESHOLD in dtm.py) before rasterization. Offsets computed per tile (they drift along a flight line: never a global table), memoised per ground LAS, recorded in DTM/*_dtm*_stripalign.json (cache sidecar: absent, or different version/threshold/parameters ⇒ DTM regenerated). Can be disabled: --no-strip-align.
• Intra-strip jitter (2nd pass of the alignment): successive scan lines of the SAME pass can be randomly offset vertically (sensor vibration / high-frequency trajectory noise) — a constant offset per strip is not enough. _strip_jitter_offsets splits each strip into GPS time windows (STRIP_JITTER_BIN = 0.1 s, time origin specific to each strip), measures the robust offset of each window against the median surface of the OTHER strips (shared 1 m cells, ≥ STRIP_JITTER_MIN_CELLS = 40 cells), smooths the series (rolling median STRIP_JITTER_SMOOTH = 5 windows), clamps it to ± STRIP_JITTER_MAX (10 cm), then interpolates it at each point's GPS time (_apply_strip_jitter); where two strips overlap, each gets a series (each absorbs its share). Requires the gps_time dimension (silently skipped otherwise). Sidecar version 2 (series in jitter), covered by --no-strip-align.
• Layers produced and served = PANEL_VIZ (index.py, currently ('relief_oriente', 'densite_sol')): pipeline without --only/--skip (panel_steps()), generation started from the map (_panel_viz_steps in mapserve.py) and served layers (tiles.available_layers filters: panel, /tiles/…, TileJSON, WMTS, JOSM). The other visualizations can still be computed with --only. Map tests lift the restriction via _setup(..., panel=None).
• Precision (densite_sol) and map display: create_dtm_fast writes the density of the kept ground points (stitching band included) to DTM/*_dtm*_density.tif (1 m cells, 3 × 3 mean, pts/m²); generate_densite_sol quantises it into 16 levels (density_levels: fixed log scale, level k from 0.25 × 2^(k/2) pts/m²) and keeps it at 1 m (1000² tile, ~200 KB in lossless WebP, rendering.LOSSLESS_GRAY_KEYWORDS); nearest-neighbour tiles (tiles.NEAREST_LAYERS) to keep 16 crisp greys at zooms 18–19. Interface (web/map.js, View tab): no more layer stack — one main layer (DEFAULT_VIZ) and three modes relief / precision / compare (VIEW_MODES, sliding bar, link &P=compare&C=left:right:%); both (old product) read as relief. "How to read" text: the reading field of VIZ_LEGENDS (map + PDF).
• Oriented relief (relief_oriente, the only relief layer of the map): a single RGB image (3-band uint8 GeoTIFF, rendered as is like ortho/topo via RGB_KEYWORDS in rendering.py) — CIELAB lightness = local positive openness (DTM − Gaussian RELIEF_DETREND_M = 10 m, radii RELIEF_RADII_M = 5/10/20 m, 16 directions) 65 % + shading 35 %; hue = aspect, fixed chroma (RELIEF_CHROMA). Fixed log scale RELIEF_OPEN_RANGE (no per-tile statistic) and a total support of ~60 m (20 m maximum ray radius + 4σ = 40 m of the 10 m Gaussian detrend), still < the 100 m stitching band: seamless tiles. Fast: detrend + radii on a decimated grid ~0.8 m (RELIEF_GRID_M), a dedicated kernel that only accumulates the mean of the angles (_mean_horizon_*: CuPy RawKernel on GPU, parallel numba on CPU, numpy fallback), fused colourisation (numba) or vectorised without trigonometry (CuPy) through an L* × hue table (_relief_lut). ~5 s of computation excluding preparation on a 12-core CPU. Any constant change changes the rendering: regenerate the tiles (--only relief_oriente --force).
• Downsampled openness: generate_openness computes the ray tracing (the most expensive step: 532 s/tile at 0.2 m on CPU) on a block-decimated grid (OPENNESS_DOWNSAMPLE = 2: block max for positive, min for negative — preserves the reliefs that bound the horizon) then resamples bilinearly. Cost ÷ factor³: 532 s → 40 s (×13). Archaeological signal preserved (corr. 0.93 after smoothing); the sub-metre noise texture disappears. --openness-downsample 1 = full resolution. SVF is NOT affected (anisotropic openness no longer exists).
• Edge stitching between tiles: large-kernel renderings (openness/SVF: 100 m radii; LRM: 15 m) truncate their window at the tile edge — bands of artefacts at every tile change. --edge-buffer N (default 100; always applied by generation from the map, EDGE_BUFFER_METERS = 100 m in mapserve.py, no checkbox any more; 0 = off on the command line) rasterizes the DTM over the nominal 1 km tile aligned on the grid plus an N m band filled with the ground points of the 8 neighbouring LAZ files (_neighbor_ground_points in dtm.py: streaming PDAL, crop + IGN class filter; missing neighbour = automatic download from the IGN catalogue before the run, kept apart in input/edge_neighbors/ so as not to swell the corpus of global passes (deduplicated over the whole batch, _fetch_edge_neighbors in pipeline.py); not found or failure = empty band). Visualizations compute on the extended extent, then rendering.py (_core_tile_window, via tif_to_crop/tif_to_png) crops the outputs to the exact 1 km tile read from the LHD name — the AVIF images stay aligned 1 km squares in the mosaic. Buffer recorded in the GeoTIFF tag LIDAR_EDGE_BUFFER of the DTM: changing --edge-buffer invalidates the DTM cache automatically (tag absent = 0). Neighbouring bands are not strip-aligned (context only, cropped out of the final image). Cost: ~7 s of reading per neighbour + ~44 % more pixels at 100 m/0.2 m. Name outside the LHD pattern: option ignored (header bounds, no cropping).
• Tests use lazy imports inside each test function, never at module top, to avoid importing CuPy/GDAL at import time.
• _-prefixed names are critical private: _create_ground_pipeline, _fallback_to_smrf, _fill_nans, _init_gpu, _process_file_standalone — do not call from outside their module.
• build_index() writes the inventory + the source levels: output/index_tiles.json (source tiles, layers, versioned URLs — served by /api/tiles), thumbnails index_thumbs/ (≈3.9 m/px + _mid 1.56 m/px) and quadrants index_subtiles/ — levels of the XYZ pyramid (tiles.py). Each tile of the current run carries its WGS84 corners for the progress frames. No HTML any more: the interface lives in mapui.py.
• PDF export (export_pdf.py): Pillow + pyproj + reportlab, no numpy — runs in the lightweight image alone. /api/export/frame (frame) and /api/export/pdf (sheet) in mapserve.py, never delegated to LIDAR_GENERATION_URL, single export lock (_export_lock, 429 when busy). Composed directly in L93 (tiles.sources_in_bbox/load_source, crop + Lanczos resampling), without going through the XYZ tiles. Sheet: L93 grid (step depends on the scale), WGS84 corners, north arrow accounting for meridian convergence, graphic and numeric scale, 8-point orientation rose, legend text taken from VIZ_LEGENDS, quality inset (white hatching = "not recorded", a "Missing data (no relief)" line for tiles outside coverage), 300 dpi in A4 / 250 dpi in A3 (bounds the Pi's memory). NODATA_RGB copies visualizations.RELIEF_NODATA_RGB (numpy is absent from the lightweight image); outside the tiles' extent = hatched white.
• Quality sidecar (quality.py): output/quality/{basename}.json (QUALITY_VERSION), ground density per 50 m cell and acquisition dates, computed after the DTM (ensure_quality, including on an already cached DTM — it then measures the input LAZ directly); batch backfill --quality-backfill (see Workflow); copied into the quality table of index_tiles.json (build_index) and onto the lightweight machines (tiles._persist_remote_quality, from the upstream inventory).

Architecture Notes (from code audit 2025-09)

Module structure & data flow

• cli.py → pipeline.py (LidarArchaeoPipeline) → per-file: dtm.py (classify + rasterize) → visualizations.py (17 products) → rendering.py (GeoTIFF→AVIF) → index.py (catalogue: thumbnails + inventory)
• gpu.py provides CuPy/NumPy proxy (xp), lazy init, OOM fallback. safe_gpu_call wraps all non-IGN viz calls.
• mapserve.py (FastAPI, image lidar-maps) serves the XYZ pyramid (tiles.py), the interface (mapui.py), the /api/tiles inventory + static source tiles for the lightweight machines, and the generation API: pipeline as a subprocess on the full image, delegation via LIDAR_GENERATION_URL on the lightweight image (Raspberry Pi). Single job + persistent queue (_job_lock is an RLock: /api/status calls _queue_summary() under the lock). LIDAR_API_TOKEN (worker) / LIDAR_REMOTE_TOKEN (lightweight) protect the calls; LIDAR_REGEN_CIDR restricts generation to the local network.
• progress.py writes JSONL events (O_APPEND, atomic) read by mapserve for live progress (frames on the map).

Key design decisions (intentional, do not "fix")

• _res_suffix hardcodes 0.5 as "no suffix": coupled to index.py parsing (_strip_res_suffix defaults to 0.5 when no suffix). Changing requires sidecar metadata.
• GPU scoring (major*1000 + minor*100 + mem_mi): compute capability priority is intentional — a newer GPU with less VRAM is preferred.
• mapserve.py reads env at import time: deployment-focused single-purpose server, env is set once in docker-compose.
• Repeated try/except in visualizations (same pattern in every generate_*): intentional convention for uniform return None behavior.
• _d8_accumulate_numba defines @njit inside the function: cache=True makes subsequent calls fast; the Python function object creation is negligible.
• pkill -9 -f "pdal pipeline <output>/temp/" in the cli.py signal handler: belt-and-suspenders alongside os.killpg. Scoped to this run's temp directory to avoid killing unrelated PDAL processes (e.g. a concurrent generation job).

Performance characteristics

• _priority_flood uses numba JIT binary heap (single int64 array, flat view for elevation). Python heapq fallback if numba unavailable.
• _d8_accumulate_numba uses numba with argsort top-down sweep. Python fallback exists.
• Ray-tracing (SVF, openness): processes one direction at a time to limit VRAM. Auto-falls back to CPU on OOM via _ray_trace_horizons.
• Multi-resolution: 0.5 m has no filename suffix whatever its position; every other resolution (including the 0.2 m default) uses a _r0p2 style suffix. Ground classification done once, shared across resolutions.
• ProcessPoolExecutor batch runs are unlimited by default; set LIDAR_BATCH_TIMEOUT (seconds) to cap a batch's wall clock and cancel the remaining workers past it.

Numba usage pattern

• Defined at function scope with @njit(cache=True) — first call compiles (~2-3s), subsequent calls hit disk cache.
• Must use flat 1D array views (arr.ravel()) for integer indexing — 2D arrays with a single int index return a row slice in nopython mode.
• Pattern: try numba → return None on ImportError → caller falls back to pure Python.

Commit & Pull Request Guidelines

Commits use imperative tense, short single-line subjects (~60–80 chars), no prefixes or scopes. Compound commits are common — multiple related changes joined by commas or "and". Examples: Fix multi-GPU with lazy CuPy init + rendering improvements, Add multi-resolution support and remove PDF generation, Fix corrupted COPC detection, add CSF→SMRF fallback, improve MSRM colormap, add SVF and anisotropic openness.

No PR template, no CI pipeline, no issue tracker. This is a standalone Docker project with no formal PR process.