Hagfisk Faroese fisheries data pipeline and dashboard. Pulls monthly landing statistics from Hagstova Føroya's PX-Web API, stuffs them into a local DuckDB, and serves a filtered ECharts dashboard — all from a single Rust binary.

Named after the hagfish: a deep-sea creature famous for producing copious amounts of slime and being generally unpleasant. The name stuck anyway. The fish has no backbone. Neither does this README.

What It Does

Fetches fisheries data (mass + value by species, gear, zone, month) from statbank.hagstova.fo
Stores everything in a local DuckDB file
Serves a REST API with filtered queries, summaries, and Parquet export
Renders an interactive dashboard with line, price-trend, and stacked bar charts
Runs monthly ingestion automatically via systemd timer (28th of each month, 03:00)

Tech Stack

Thing	Choice	Why
Language	Rust 2024	Fast, safe, single static binary. No runtime, no VM, no nonsense.
Web framework	Axum	Async, typed, plays well with Tokio. Doesn't fight you.
Database	DuckDB	Embedded OLAP. No server, no config, just a file. Perfect for analytics on a single machine.
Charts	ECharts	Powerful, flexible, no build step. Vanilla JS, no framework tax.
Static assets	rust-embed	Compiled into the binary. One artifact to deploy, nothing to forget.
Logging	tracing + tracing-subscriber	Structured logs to file and journal.
Deployment	Bare metal + systemd	No containers. No compose files. No port mappings. Just a binary and a `.service` unit.

Getting Started

cargo build --release
./target/release/hagfisk ingest --full
./target/release/hagfisk serve

Dashboard lives at http://localhost:8090.

Configuration

config.json next to the binary:

{
  "duckdb_path": "hagfish.db",
  "bind_address": "127.0.0.1:8090",
  "data_source_url": "https://statbank.hagstova.fo/api/v1/fo/H2/VV/VV01/fisknv_md.px",
  "log_file_path": "hagfish.log",
  "allowed_origins": []
}

CLI

hagfisk serve              Start the web server
hagfisk ingest [--full]    Fetch data (incremental or full backfill)
hagfisk export -o FILE     Export landings to Parquet

Deployment

The justfile handles everything, but you'll need to install it on your machine.

just deploy # Build, rsync, restore SELinux contexts, restart just deploy-reingest # Same + full re-ingest just logs-app # Tail application logs just ps-app # Check service status


Systemd units live in `systemd/` — drop them in `~/.config/systemd/user/` and enable the timer for monthly ingestion.

## API

| Endpoint | What |
|---|---|
| `GET /healthz` | Alive check |
| `GET /api/species` | Species codes + Faroese names |
| `GET /api/zones` | Economic zone codes |
| `GET /api/gear` | Fishing gear codes |
| `GET /api/landings` | Filtered landing records (month range, species, gear, zone, measure) |
| `GET /api/summary` | Monthly aggregates, top 10 species, price/kg trend |
| `GET /api/summary/monthly-breakdown` | Per-month per-species breakdown |
| `GET /api/available-filters` | Cascading filter options based on active selections |
| `GET /api/export.parquet` | Full dataset as Parquet download |

Full PX-Web API reference in `docs/API.md`.

---

Like the hagfish itself: not glamorous, but it works.

3.4 KiB Raw Blame History