This commit is contained in:
2026-08-27 10:30:28 +01:00
parent a88efaacd8
commit 55c190999a
+90 -1
View File
@@ -1,11 +1,100 @@
# Hagfisk # Hagfisk
Faroese fisheries data pipeline and dashboard. Pulls monthly landing statistics from Hagstova Føroya's PX-Web API, stuffs them into a local DuckDB, and serves a filtered ECharts dashboard — all from a single Rust binary.
Named after the hagfish: a deep-sea creature famous for producing copious amounts of slime and being generally unpleasant. The name stuck anyway. The fish has no backbone. Neither does this README.
## What It Does
- Fetches fisheries data (mass + value by species, gear, zone, month) from `statbank.hagstova.fo`
- Stores everything in a local DuckDB file
- Serves a REST API with filtered queries, summaries, and Parquet export
- Renders an interactive dashboard with line, price-trend, and stacked bar charts
- Runs monthly ingestion automatically via systemd timer (28th of each month, 03:00)
## Tech Stack
| Thing | Choice | Why |
|---|---|---|
| Language | Rust 2024 | Fast, safe, single static binary. No runtime, no VM, no nonsense. |
| Web framework | Axum | Async, typed, plays well with Tokio. Doesn't fight you. |
| Database | DuckDB | Embedded OLAP. No server, no config, just a file. Perfect for analytics on a single machine. |
| Charts | ECharts | Powerful, flexible, no build step. Vanilla JS, no framework tax. |
| Static assets | rust-embed | Compiled into the binary. One artifact to deploy, nothing to forget. |
| Logging | tracing + tracing-subscriber | Structured logs to file and journal. |
| Deployment | Bare metal + systemd | No containers. No compose files. No port mappings. Just a binary and a `.service` unit. |
## What does it do? ## Getting Started
```bash
cargo build --release
./target/release/hagfisk ingest --full
./target/release/hagfisk serve
```
Dashboard lives at `http://localhost:8090`.
## Configuration
`config.json` next to the binary:
```json
{
"duckdb_path": "hagfish.db",
"bind_address": "127.0.0.1:8090",
"data_source_url": "https://statbank.hagstova.fo/api/v1/fo/H2/VV/VV01/fisknv_md.px",
"log_file_path": "hagfish.log",
"allowed_origins": []
}
```
## CLI
```
hagfisk serve Start the web server
hagfisk ingest [--full] Fetch data (incremental or full backfill)
hagfisk export -o FILE Export landings to Parquet
```
## Deployment
The `justfile` handles everything:
```bash
just deploy # Build, rsync, restore SELinux contexts, restart
just deploy-reingest # Same + full re-ingest
just logs-app # Tail application logs
just ps-app # Check service status
```
Systemd units live in `systemd/` — drop them in `~/.config/systemd/user/` and enable the timer for monthly ingestion.
## API
| Endpoint | What |
|---|---|
| `GET /healthz` | Alive check |
| `GET /api/species` | Species codes + Faroese names |
| `GET /api/zones` | Economic zone codes |
| `GET /api/gear` | Fishing gear codes |
| `GET /api/landings` | Filtered landing records (month range, species, gear, zone, measure) |
| `GET /api/summary` | Monthly aggregates, top 10 species, price/kg trend |
| `GET /api/summary/monthly-breakdown` | Per-month per-species breakdown |
| `GET /api/available-filters` | Cascading filter options based on active selections |
| `GET /api/export.parquet` | Full dataset as Parquet download |
Full PX-Web API reference in `docs/API.md`.
## Source
[git.flo.fo/FLO/hagfisk](https://git.flo.fo/FLO/hagfisk)
## Live
[hagfisk.poc.fló.fo](https://hagfisk.poc.fló.fo)
---
Like the hagfish itself: not glamorous, but it works.
![Watch demo](./gif/hagfish.gif) ![Watch demo](./gif/hagfish.gif)