lidar_rendu/README.md

lidar_rendu

See through the forest. Turn France's open LiDAR HD point clouds into seamless, archaeology-grade relief maps — served as a fast slippy map, reusable XYZ tiles and print-ready PDF field sheets.

_{Neuf-Brisach (Haut-Rhin), Vauban's star fortress (UNESCO World Heritage), rendered from one 1 km LiDAR HD tile. Every bastion, moat and ravelin reads at a glance; buildings are black (no ground points).}

---

Why

The French mapping agency (IGN) publishes LiDAR HD: a nationwide airborne laser scan, ~10 points/m², free under the Licence Ouverte 2.0. Hidden under forest canopy and fields are hollow ways, trenches, burial mounds, field systems and forgotten walls — but raw point clouds are hard to read, and naïve DEM renderings show seams at every tile edge, stripes from the flight lines and colour blotches that look like relief but aren't.

lidar_rendu is an end-to-end pipeline that solves exactly that:

• One command from an IGN tile ID to a browsable map.
• One carefully designed layer — oriented relief — instead of fifteen colormaps to flip through.
• Seamless at any scale: fixed scales, no per-tile statistics, 100 m overlap borrowed from the eight neighbouring tiles.
• Measurement-aware: a companion precision layer shows where the terrain is measured and where it is interpolated, so you never mistake a gap for a feature.

Gallery

Hartmannswillerkopf (tile 1011_6760) — the WWI battlefield of 1915 under dense forest: roads, trench lines and shell craters appear on the slope.	PDF field sheet exported from the map (A4 landscape, 1:5,000): Lambert 93 grid, true-north arrow, scale bar, legend and data-quality inset.

Highlights

Oriented relief	A single RGB image: CIELAB lightness carries local openness (bumps light, hollows dark), hue carries slope orientation at constant lightness — so colour never fakes relief.
Flight-strip calibration	Each tile mixes several passes, sometimes offset by a few cm. The pipeline estimates per-strip offsets, per-scan-line shift and roll, and a per-beam cross-track profile, then removes them before rasterising. No more stripes.
Honest gap filling	At 0.2 m, ~80 % of pixels contain no point. Gaps are closed morphologically within the point envelope, with a radius that follows local point spacing — nothing is extrapolated outward.
Seamless tiles	DTMs are built on the nominal 1 km tile plus a 100 m buffer taken from the neighbours (downloaded automatically), then cropped back exactly.
Fast	GPU (CuPy) when available, numba otherwise. Ground extraction straight from IGN classes with laspy (~5 s per tile), relief kernel ~5 s on a 12-core CPU, AVIF encoding in 0.6 s.
Slippy map	Built-in Leaflet UI: tile selection with IGN metadata, relief / precision / side-by-side compare, adjustable intensity, light & dark themes, mobile bottom sheet, shareable links.
Standard tiles	/tiles/{layer}/{z}/{x}/{y}.png in the OpenStreetMap scheme, plus TileJSON, WMTS and a JOSM imagery file — drop it into QGIS, JOSM, iD, uMap or MapLibre.
Generate from the map	Draw a rectangle: the tiles are downloaded from IGN, processed, and appear on the map row by row as they finish.
Print	Vector PDF sheets (A4/A3, 1:1,000 – 1:10,000) with grid, WGS84 corners, meridian convergence, legend and point-density inset.
Runs on a Raspberry Pi	A lightweight image (no PDAL, no GPU) serves the map and delegates generation to a worker machine; it keeps working offline.

Quick start

Requirements: Docker with Compose ≥ 2.24. An NVIDIA GPU is optional.

git clone https://github.com/<you>/lidar_rendu.git && cd lidar_rendu

./start.sh                                 # build the image, start the map on http://localhost:8973/
./start.sh process --fetch-tiles 1037,6779 # download one IGN tile (Neuf-Brisach) and process it
./start.sh logs                            # follow the server log
./start.sh stop                            # stop everything

start.sh creates input/ and output/ owned by you and checks whether Docker can reach a GPU; if not, it adds docker-compose.cpu.yml and everything runs on the CPU (slower, same output).

Tile IDs are the IGN grid coordinates in km (Lambert 93): 1037,6779 is the tile whose top-left corner is at X = 1,037 km, Y = 6,779 km. You can also skip the command line entirely: open the map, go to the Generate tab and draw an area.

How it works

flowchart LR
    A[IGN LiDAR HD<br/>COPC .laz tile] --> B[Ground points<br/>IGN classes, laspy]
    N[8 neighbour tiles<br/>100 m buffer] --> B
    B --> C[Strip & scan-line<br/>calibration]
    C --> D[DTM 0.2 m<br/>+ bounded gap fill]
    D --> E[Oriented relief<br/>+ precision layer]
    E --> F[AVIF tiles<br/>cropped to 1 km]
    F --> G[Inventory +<br/>XYZ pyramid]
    G --> H[Map · XYZ/WMTS · PDF]

1. Download — the COPC tile and, if missing, its eight neighbours from the IGN Géoplateforme. Tiles are processed north to south, so the map fills top down.
2. Ground — IGN's ground class (2) extracted directly with laspy; PDAL (SMRF / CSF) remains available. See ground classification.
3. Calibration — robust per-strip vertical offsets, then a joint least-squares adjustment of every scan line (offset + roll) against the other strips, plus a per-beam angular profile. Results are written to a JSON sidecar next to the DTM.
4. DTM — rasterised at 0.2 m on the tile + buffer; small gaps closed within the point envelope; ground density saved alongside.
5. Render — oriented relief (openness at 5 / 10 / 20 m on a detrended DTM, 16 directions, plus 35 % hillshade) and precision (16 log-scale density levels). Fixed scales everywhere: identical terrain gives identical colour on every tile.
6. Publish — AVIF quadrants encoded once from the source raster, inventory refreshed after every tile, XYZ pyramid pre-generated in the background.

The map

• Click a tile to select it: extent, IGN acquisition date, sensors, point count, download link, and the calibration applied to each flight strip.
• Three view modes — relief, precision, or compare with a draggable split bar.
• Relief intensity slider (0.5×–2×), remembered per browser and carried in the link.
• Share — the URL encodes position, mode, comparison and intensity.
• Use in JOSM / QGIS — the Share tab lists the XYZ, TileJSON, WMTS and JOSM URLs, with a Copy button for the chosen layer's tile URL.
• Keyboard: 1–5 tabs, P next view mode, Esc deselect / collapse.
• Works on phones: the panel becomes a three-height bottom sheet; GPS centring over HTTPS.

Full reference: docs/MAPS.md.

Use the tiles anywhere

Client	URL
QGIS, iD, uMap, Leaflet, MapLibre	http://<host>:8973/tiles/relief_oriente/{z}/{x}/{y}.png
JOSM (TMS)	http://<host>:8973/tiles/relief_oriente/{zoom}/{x}/{y}.png
JOSM (all layers at once)	http://<host>:8973/tiles/josm.imagery.xml
TileJSON 3.0	http://<host>:8973/tiles/relief_oriente.json
WMTS 1.0	http://<host>:8973/tiles/wmts.xml

256 px PNG, EPSG:3857, transparent outside coverage, CORS enabled, zoom 5–19 (19 ≈ 0.2 m/px, one screen pixel per LiDAR pixel).

Command line

The processing pipeline is python3 -m lidar_pipeline; with Docker Compose, ./start.sh process [options] runs it on input/ → output/.

Option	Default	Purpose
--fetch-tiles COL,ROW …	—	Download these LiDAR HD tiles from IGN first
--file NAME …	all of input/	Process only these tiles (name without .laz)
-r RES	0.2	Resolution in m/px (comma-separated for several)
-g [GPU]	CPU	GPU(s): -g, -g 0, -g 0,2, -g all
-w N	auto	Parallel workers (bounded by free VRAM per GPU)
--only VIZ … / --skip VIZ …	map layers	Choose visualisations
--edge-buffer M	100	Buffer borrowed from neighbour tiles (0 = off)
--ground-classification	ign	ign, auto, smrf, csf
--no-strip-align	on	Disable flight-strip calibration
--format, --quality	avif, 60	Output encoding
-f, --force	off	Regenerate even if outputs exist
--rebuild-index	—	Rebuild the tile inventory only
-v, --debug	—	Verbose / debug logging

By default only the map layers are produced (relief_oriente, densite_sol). Classic visualisations remain one flag away, e.g. ./start.sh process --only hillshade slope svf pos_open:

Key	Visualisation		Key	Visualisation
hillshade	Multi-directional hillshade		svf	Sky-View Factor
slope	Slope		roughness	Roughness
aspect	Aspect		wavelet	Mexican-hat wavelet (multi-scale)
mslrm	Multi-scale relief model		flow_acc	D8 flow accumulation
sailore	Slope-adaptive local relief		solar	Solar illumination
pos_open / neg_open	Positive / negative openness		anomaly	Statistical anomalies
ortho / topo	IGN orthophoto / topographic map

Deployment

Setup	Command	Port
All-in-one (map + generation)	docker compose up -d --build serve (or ./start.sh)	8973
Processing worker for remote maps	docker compose -f docker-compose.worker.yml up -d --build	8973
Lightweight map (Raspberry Pi, no PDAL/GPU)	docker compose -f docker-compose.maps.yml up -d --build	8975

The lightweight map mirrors the worker's tiles, delegates generation to it, and keeps serving from disk when the worker is off. Tokens (LIDAR_API_TOKEN) and a network allow-list (LIDAR_REGEN_CIDR, private ranges by default) protect the generation API. Step-by-step guide: docs/DEPLOY_WEBAPP.md.

Always pass --build: the code is baked into the image, and the build is near-instant thanks to the layer cache.

Project layout

lidar_pipeline/
├── cli.py            command line
├── pipeline.py       orchestration, worker pool, VRAM-aware GPU slots
├── fetch_ign.py      IGN catalogue and downloads
├── dtm.py            ground extraction, strip calibration, DTM, gap filling
├── visualizations.py generate_* functions (relief, openness, SVF, …)
├── rendering.py      colormaps, GeoTIFF → AVIF, 1 km crop
├── index.py          catalogue, thumbnails, sub-tiles, inventory, layer registry
├── quality.py        per-tile quality sidecar (density, acquisition dates)
├── tiles.py          XYZ pyramid: reprojection, cache, background maintenance
├── mapserve.py       FastAPI server: map, tiles, TileJSON/WMTS, generation API
├── export_pdf.py     PDF field sheets (Pillow + pyproj + reportlab)
├── gpu.py            CuPy / NumPy abstraction with CPU fallback
├── web/              map UI (HTML, CSS, JS)
└── tests/            ~400 pytest tests

Development

./run.sh --test                     # rebuild the image and run the full test suite
docker run --rm lidar-lidar python3 -m pytest -v --pyargs lidar_pipeline.tests.test_tiles

Adding a visualisation takes four edits: a generate_X() in visualizations.py, an entry in VIZ_STEPS (pipeline.py), a colormap in rendering.py, and a legend in VIZ_LEGENDS (index.py). Design decisions and conventions are documented in AGENTS.md (French).

Contributions are welcome: open an issue or a pull request.

Documentation

• docs/MAPS.md — map UI, tile contract, cache, PDF export, environment variables
• docs/DEPLOY_WEBAPP.md — two-machine deployment (Raspberry Pi + worker)
• docs/GROUND_CLASSIFICATION.md — ground filters benchmark and literature

Data and licences

• Code: MIT.
• LiDAR HD, orthophotos, maps: © IGN, Licence Ouverte 2.0. Attribution is required and is shown on every map, tile endpoint and PDF.
• Base map: © OpenStreetMap contributors. Before using these renderings as a tracing source in OpenStreetMap, check the community's position on the source.

Built on PDAL, laspy, rasterio, CuPy, numba, FastAPI, Leaflet and reportlab.