code_public/dwarf-go
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reverse-engineer DWARF II telescope API and build open-source Go client

Browse Source 
Complete reverse-engineering of the DWARFLAB Android app (v3.4.0) protocol
and implementation of a working CLI tool to control DWARF II telescopes.

Analysis (from APK decompilation with jadx):
- Extracted 17 protobuf definitions (382 messages) from embedded descriptors
- Mapped all 323 WebSocket command IDs across 16 modules
- Documented the full protocol: BLE discovery, WebSocket control (port 9900),
  RTSP preview, WsPacket envelope (proto v2.3)
- Documented the Android UI structure (screens, navigation, shooting modes)
- Key discovery: telescope responds with type=3 (reply), not type=1 (response),
  and several commands are fire-and-forget (RGB, camera open/close)

dwarfctl Go client:
- Protobuf bindings generated from extracted .proto files (397 messages)
- WebSocket transport layer with request-response matching and notification fan-out
- Typed API covering cameras, motors, astrophotography, focus, tracking, system, power
- Cobra CLI with 30+ subcommands and --debug traffic logging
- 57 unit tests (transport round-trip, command routing, proto encoding)
- Validated on real hardware: state, photo, motor slew (all directions/speeds),
  focus, RGB, time/location sync all confirmed working

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
This commit is contained in:
Jacquin Antoine
2026-07-12 15:18:56 +02:00
commit 814a836c5a
54 changed files with 35973 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

323
analysis/API_REFERENCE.md Normal file
View File

@ -0,0 +1,323 @@
# DWARFLAB Telescope — API Reference (reverse-engineered)
Reverse-engineered from `DWARFLAB.apk` v3.4.0 (build 629), package `com.convergence.dwarflab`.
This document describes the protocol used by the official Android app to talk to
DWARF II (and the in-development "Bilbo"/DWARF III) smart telescopes.
> Goal: enable an open-source client implementation. All command IDs, the wire
> envelope, and the inner protobuf payloads were recovered from the APK; nothing
> here is guessed.
---
## 1. Architecture at a glance
```
┌─────────────┐ BLE (GATT) ┌───────────┐
│ Phone app │ ───────────────▶ │ Telescope │ 1. discovery + Wi-Fi creds exchange
│ │ ◀── DwarfEcho ── │ │ (DwarfPing / DwarfEcho, see ble.proto)
└──────┬──────┘ └─────┬─────┘
│ joins telescope Wi-Fi │
│ (AP mode, or shared STA) │
│ │
│ WebSocket (binary) │ RTSP (TCP)
│ ws://<ip>:9900/?client_id=.. │ rtsp://<ip>/<cam>/stream0
▼ ▼
┌───────────────────────────┐ ┌──────────────┐
│ Control plane │ │ Live preview │
│ WsPacket (protobuf) │ │ ijkplayer / │
│ 323 commands, 17 modules │ │ ExoPlayer │
└───────────────────────────┘ └──────────────┘
```
Two distinct data planes:
| Plane | Transport | Port | Encoding | Purpose |
|-------|-----------|------|----------|---------|
| **Control** | WebSocket (binary frames) | **9900** | protobuf `WsPacket` envelope | All telescope commands & status |
| **Preview** | RTSP over TCP | 554 (std) | H.264/H.265 | Live camera viewfinder |
| **Discovery** | BLE GATT | — | protobuf (`ble.proto`) | Find telescope, get IP/SSID/PSK |
There is also a **cloud relay**: `https://app.dwarflabapp.com/app/uls/connect?d=…`
("ULS") used for remote access when the phone is not on the telescope's Wi-Fi.
Local LAN control (the focus of an OSS client) needs only BLE + WebSocket + RTSP.
---
## 2. Connection flow
### 2.1 Discovery & credentials (BLE)
The telescope advertises over BLE. The app sends a `DwarfPing` / `ReqGetconfig`
and receives a **`DwarfEcho`** (`ble.proto`) containing everything needed to
connect over IP:
```proto
message DwarfEcho {
VocalType vocaltype = 1;
uint64 timestamp = 2;
bytes magic = 3;
uint64 ts_ping = 4;
bytes mac_address = 5;
StationModel model = 6; // telescope family + revision
string sn = 7; // serial number
string name = 8; // advertised name
string psw = 9; // device password
string fw_version = 10;
string ws_scheme = 11; // "ws" or "wss"
uint32 session = 12;
NifAP ap = 13; // AP network (ifname/ssid/psw/ipv4/ipv6)
NifSTA sta = 14; // STA network (ifname/ssid/psw/rssi/ipv4/ipv6)
}
```
`ble_psd` (BLE password) + `client_id` authenticate the BLE requests
(`ReqGetconfig`, `ReqSta`, `ReqAp`, `ReqSetblewifi`). Responses carry the
**`ip`** field (`ResGetconfig.ip` / `ResSta.ip`) that the app feeds into the
WebSocket URL.
### 2.2 WebSocket control channel
Once the phone is on the telescope's network:
```
ws://<telescope_ip>:9900/?client_id=<client_id>
```
- Plain `ws://` by default; `wss://` if `ws_scheme == "wss"`.
- `client_id` is a client-generated identifier (`y32…m58158c()`).
- Source: `v55.java` field `f43155b` / method `m55420t(ip)`.
- **device_id** field in the envelope selects which telescope (multi-device),
defaults to `1` for the first/only connected scope.
### 2.3 RTSP preview
```
rtsp://<telescope_ip>/<stream_selector>/stream0
```
- `<stream_selector>` comes from `StreamTypeAnn` (`com.convergence.dwarflab.data.bean.camera`).
- Player options force `rtsp_transport = tcp` (see `RtspPlayerView.java:195`).
- Two cameras exist: **Tele** (main/long-focus) and **Wide** (wide-angle/guide).
---
## 3. Wire format — `WsPacket` envelope
Every WebSocket binary frame is **one serialized `WsPacket`** (`base.proto`):
```proto
message WsPacket {
uint32 major_version = 1; // = 2 (WS_MAJOR_VERSION_NUMBER)
uint32 minor_version = 2; // = 3 (WS_MINOR_VERSION_NUMBER) → protocol v2.3
uint32 device_id = 3; // telescope index, default 1
uint32 module_id = 4; // derived from cmd (see §4)
uint32 cmd = 5; // operation id (10000–17099)
uint32 type = 6; // 0=request 1=response 2=notification 3=reply
bytes data = 7; // inner protobuf message, serialized
string client_id = 8; // same client_id as the WS URL
}
```
Confirmed build code (`e49.java`, `C9312a.m36361a`):
```java
WsPacket.newBuilder()
.setMajorVersion(2)
.setMinorVersion(3)
.setDeviceId(connectedDeviceId != null ? connectedDeviceId : 1)
.setModuleId(wsCmd.getModuleId().ordinal()) // derived from cmd range
.setCmd(wsCmd.getCmd())
.setType(wsCmd.getMessageType().ordinal()) // request=0
.setData(ByteString.copyFrom(innerProto.toByteArray()))
.setClientId(clientId)
.build();
```
`type` enum (`WsMessageType`): `request=0`, `response=1`, `notification=2`, `reply=3`.
### 3.1 Request/response matching
Responses are matched by **`cmd`**. The app registers a pending continuation
keyed by the expected response `cmd`; an incoming `WsPacket` whose `cmd` matches
is parsed (`WsRequestHandle.mo7785d` → `BaseProto.WsPacket.parseFrom`, then the
inner `data` is parsed into the expected proto class).
For most commands the **response uses the same `cmd`** as the request
(`WsMessageReq.getResponseCmd()` defaults to `getCmd()`, see `d49.m35708a`).
**IMPORTANT — discovered via live testing (not in the APK code):**
The telescope uses **`type=3` (REPLY)** for direct responses, not `type=1`
(RESPONSE). Additionally, not all commands get a reply:
| Response model | Commands (tested) | Implementation |
|---------------|-------------------|----------------|
| **type=3 reply (same cmd)** | photo (10002), photo wide (12022), focus auto (15000), focus step (15001), state (16405), sync-time (13000), set-location (13010) | request-response with timeout |
| **notification only (type=2)** | RGB on/off (13500/13501), camera open/close (10000/10001), camera params GET (10036) | fire-and-forget; verify via `state` |
| **fire-and-forget native** | motor slew (14006), motor stop (14002) | no ack expected |
Commands that only produce notifications still execute successfully — the
state change is visible in `GetDeviceState` (e.g. `rgb_state:{}` after RGB off).
### 3.2 Notifications
The telescope pushes `WsPacket` frames with `type=2` (notification) and `cmd` in
the **15200–15303** range (`CMD_NOTIFY_*`). An OSS client must listen for these
to reflect state changes (track results, burst/record progress, temperatures,
calibration state, SD-card info, power, etc.). See `Notify.proto` (81 messages)
and `CMD_TABLE.md` (NOTIFY module).
---
## 4. Command routing
`module_id` is **not** stored per-command — it is derived from `cmd` by range
checks in `WsCmd.getModuleId()`:
| cmd range | module_id (ordinal) | module | proto file |
|-----------|---------------------|--------|------------|
| 10000–10499 | 1 | CAMERA_TELE | `Camera.proto` |
| 11000–11499 | 3 | ASTRO | `Astro.proto` |
| 12000–12499 | 2 | CAMERA_WIDE | `Camera.proto` |
| 13000–13299 | 4 | SYSTEM | `System.proto` |
| 13500–13799 | 5 | RGB_POWER | (RGB led / power) |
| 14000–14499 | 6 | MOTOR | `MotorControl.proto` |
| 14800–14899 | 7 | TRACK | `Track.proto` |
| 15000–15199 | 8 | FOCUS | `Focus.proto` |
| 15200–15499 | 9 | NOTIFY | `Notify.proto` |
| 15500–15599 | 10 | PANORAMA | `Panorama.proto` |
| 15700–15799 | 11 | ITIPS | `ITips.proto` |
| 16100–16399 | 13 | SHOOTING_SCHEDULE | `Schedule.proto` |
| 16400–16599 | 14 | TASK_CENTER | `TaskCenter.proto` |
| 16700–16799 | 15 | PARAM | `Param.proto` |
| 16800–16899 | 16 | VOICE_ASSISTANT | `VoiceAssistant.proto` |
| 17000–17099 | 18 | DEVICE | `Device.proto` |
**Full table of all 323 commands → see [`CMD_TABLE.md`](CMD_TABLE.md).**
The mapping from command → inner protobuf message type lives in the
`data/websocket/<module>/` handlers (`*WsResponseHandle.java`) and the
`data/bean/p021ws/request/` request classes. To find the payload type for a
given cmd, grep the request package.
---
## 5. Proto definitions
Regenerated cleanly from the embedded `FileDescriptorProto` descriptors in the
APK — **17 files, 382 messages**. Located in `analysis/protos/`:
| File | msgs | Covers |
|------|------|--------|
| `Base.proto` | 6 | WsPacket envelope, ComResponse, CommonParam |
| `Camera.proto` | 55 | Both cameras: exp/gain/WB/ISP/RAW/record/burst/resolution |
| `Astro.proto` | 67 | Calibration, GOTO (DSO/solar), live-stacking, darks, EQ solving, AI enhance, mosaic, sky-finder |
| `MotorControl.proto` | 14 | RA/DEC motors: run/stop/runTo/joystick/reset/positions |
| `Track.proto` | 9 | Tracking, sentry mode, MOT, UFO (multi-object track) |
| `Focus.proto` | 9 | Auto-focus (normal + astro), manual continuous, user infinity |
| `Panorama.proto` | 18 | Grid/stitch/framing/upload/compress |
| `Notify.proto` | 81 | All async server-push events |
| `Schedule.proto` | 20 | Shooting plan sync/cancel/lock |
| `TaskCenter.proto` | 26 | Global task manager, state info, mode/tech switch |
| `System.proto` | 14 | Time/timezone, location, MTP, CPU mode, activation, low-temp protection |
| `Param.proto` | 8 | Generic param set (exposure/gain/WB/int/float/bool/auto) |
| `Device.proto` | 3 | Lens defog, auto-cooling, auto-shutdown |
| `Ble.proto` | 25 | BLE handshake (DwarfPing/DwarfEcho/config/AP/STA/wifi-scan) |
| `VoiceAssistant.proto` | 16 | On-device voice assistant tasking |
| `RGB.proto` | 6 | RGB LED ring / power indicator |
| `ITips.proto` | 5 | Tips content |
To regenerate: `python3 analysis/extract_protos.py <jadx proto dir> <out dir>`
---
## 6. Worked example — take a photo with the Tele camera
1. **(once)** BLE: connect, get `DwarfEcho` → extract `sta.ipv4`/`ap.ipv4`,
`psw`, join Wi-Fi.
2. Open WebSocket `ws://<ip>:9900/?client_id=droid-oss-001`.
3. Open the Tele camera: send `WsPacket{cmd=10000, data=ReqOpenCamera}`,
wait for `ComResponse{code=0}`.
4. Set exposure (optional): `cmd=10009` with `ReqSetExp{…}`.
5. Photograph: `cmd=10002` (`CMD_CAMERA_TELE_PHOTOGRAPH`).
6. Watch notifications `15273` (PHOTO_STATE) / `15274` (BURST_STATE) for result.
Common response type is `ComResponse{ int32 code = 1; }` — `code==0` means OK.
---
## 7. Parameter value maps (assets)
The APK ships ready-made enum maps that constrain valid values:
- **`assets/params_range.json`** — exposure index ↔ `"1/N"` label (0…full
mapping, e.g. index 0 = "1/10000", step 3).
- **`assets/shoot_plan_config.json`** — per-device capability matrix. Defines
DWARF II (id=1, fw 2.1.6): two cameras (`Tele` id=0, `Wide` implicit),
their FoV (`fvWidth/fvHeight`), preview size (1280×720), and every supported
param with `min/max/step/defaultValue/valueType` + Gear vs Continue modes.
This file is the authoritative source for **what values each camera accepts**.
- **`assets/astronomy_data.db`** — SQLite of celestial objects (GOTO targets).
- **`assets/www/modules/eq/`** — Three.js 3D equatorial-alignment helper
(polar-alignment UI). "Bilbo" = next-gen model codename (DWARF III).
---
## 8. Multi-device & activation notes
- `device_id` in `WsPacket` selects the active telescope when several are on the
same network; default `1`.
- Some telescopes are **factory-activated** via `CMD_SYSTEM_*` (13005–13008):
`ReqGetDeviceActivateInfo`, `ReqDeviceActivateWriteFile`, activation-notify,
factory-test un-activate. An OSS client should generally leave activation
alone.
- `CMD_SYSTEM_SET_MASTER` (13004) toggles master/slave mode
(`ReqsetMasterLock{bool lock}`).
- MTP mode (`CMD_SYSTEM_SET_MTP_MODE`, 13002) switches the telescope between
Mass-Storage (mount SD card over USB) and normal modes.
---
## 9. Suggested open-source client architecture
```
dwarf-oss/
├── proto/ ← copy analysis/protos/*.proto here
├── ble/ ← BLE scanner + DwarfPing/DwarfEcho handshake (ble.proto)
├── transport/
│ ├── ws_client.py ← ws://<ip>:9900/?client_id=…, send/recv WsPacket
│ ├── envelope.py ← WsPacket build/parse (Base.proto)
│ └── dispatcher.py ← cmd → pending-request matching, NOTIFY fan-out
├── rtsp/ ← GStreamer/ffmpeg/PyAV pull of rtsp://<ip>/<cam>/stream0
├── modules/
│ ├── camera.py ← cmds 10000/12000 (Tele/Wide)
│ ├── motor.py ← cmds 14000 (point/slew)
│ ├── astro.py ← cmds 11000 (calib/goto/stacking/darks)
│ ├── focus.py ← cmds 15000
│ ├── track.py ← cmds 14800
│ └── task.py ← cmds 16400 (one-click shooting)
└── cli.py ← `dwarf photo`, `dwarf goto M31`, `dwarf slew 1.2 0`
```
Key implementation tips:
- Send `major_version=2, minor_version=3` always.
- Reuse one persistent WebSocket; don't reconnect per command.
- Maintain a registry of `cmd → asyncio.Future` for request/response; a single
inbound dispatcher handles both responses and notifications.
- Start with `CMD_GLOBAL_TASK_GET_DEVICE_STATE_INFO` (16405) after connect to
snapshot the current state, then rely on NOTIFY pushes.
- For live view, a separate RTSP consumer is simpler than multiplexing over the
WebSocket.
---
## 10. Tooling used / how to reproduce
```
tools/jadx/bin/jadx -d extracted/jadx DWARFLAB.apk # decompile
python3 analysis/extract_protos.py extracted/.../proto analysis/protos
# WsCmd / WsModuleId / WsMessageType enums → analysis/CMD_TABLE.md
```
Key source locations inside `extracted/jadx/sources/`:
- `com/convergence/dwarflab/proto/` — 17 `*Proto.java` (descriptors).
- `com/convergence/dwarflab/data/bean/p021ws/WsCmd.java` — all command ids.
- `…/WsModuleId.java`, `…/WsMessageType.java` — enums.
- `…/request/WsMessageReq.java` — request interface (`d49.java` = sender).
- `com/convergence/dwarflab/data/websocket/<module>/` — per-module handlers.
- `p000/e49.java` — WsPacket builder (`C9312a.m36361a`).
- `p000/v55.java` — WebSocket connection manager (URL/port 9900).
- `p000/b49.java` — OkHttp WebSocket wrapper.