From 55c190999a05ec39f1a825830fd15ed3efb3460a Mon Sep 17 00:00:00 2001 From: Bartal Laearsson Date: Thu, 27 Aug 2026 10:30:28 +0100 Subject: [PATCH] readme --- README.md | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 90 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 4bcd64d..58f12b7 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,100 @@ # Hagfisk +Faroese fisheries data pipeline and dashboard. Pulls monthly landing statistics from Hagstova Føroya's PX-Web API, stuffs them into a local DuckDB, and serves a filtered ECharts dashboard — all from a single Rust binary. +Named after the hagfish: a deep-sea creature famous for producing copious amounts of slime and being generally unpleasant. The name stuck anyway. The fish has no backbone. Neither does this README. +## What It Does +- Fetches fisheries data (mass + value by species, gear, zone, month) from `statbank.hagstova.fo` +- Stores everything in a local DuckDB file +- Serves a REST API with filtered queries, summaries, and Parquet export +- Renders an interactive dashboard with line, price-trend, and stacked bar charts +- Runs monthly ingestion automatically via systemd timer (28th of each month, 03:00) +## Tech Stack +| Thing | Choice | Why | +|---|---|---| +| Language | Rust 2024 | Fast, safe, single static binary. No runtime, no VM, no nonsense. | +| Web framework | Axum | Async, typed, plays well with Tokio. Doesn't fight you. | +| Database | DuckDB | Embedded OLAP. No server, no config, just a file. Perfect for analytics on a single machine. | +| Charts | ECharts | Powerful, flexible, no build step. Vanilla JS, no framework tax. | +| Static assets | rust-embed | Compiled into the binary. One artifact to deploy, nothing to forget. | +| Logging | tracing + tracing-subscriber | Structured logs to file and journal. | +| Deployment | Bare metal + systemd | No containers. No compose files. No port mappings. Just a binary and a `.service` unit. | -## What does it do? +## Getting Started + +```bash +cargo build --release +./target/release/hagfisk ingest --full +./target/release/hagfisk serve +``` + +Dashboard lives at `http://localhost:8090`. + +## Configuration + +`config.json` next to the binary: + +```json +{ + "duckdb_path": "hagfish.db", + "bind_address": "127.0.0.1:8090", + "data_source_url": "https://statbank.hagstova.fo/api/v1/fo/H2/VV/VV01/fisknv_md.px", + "log_file_path": "hagfish.log", + "allowed_origins": [] +} +``` + +## CLI + +``` +hagfisk serve Start the web server +hagfisk ingest [--full] Fetch data (incremental or full backfill) +hagfisk export -o FILE Export landings to Parquet +``` + +## Deployment + +The `justfile` handles everything: + +```bash +just deploy # Build, rsync, restore SELinux contexts, restart +just deploy-reingest # Same + full re-ingest +just logs-app # Tail application logs +just ps-app # Check service status +``` + +Systemd units live in `systemd/` — drop them in `~/.config/systemd/user/` and enable the timer for monthly ingestion. + +## API + +| Endpoint | What | +|---|---| +| `GET /healthz` | Alive check | +| `GET /api/species` | Species codes + Faroese names | +| `GET /api/zones` | Economic zone codes | +| `GET /api/gear` | Fishing gear codes | +| `GET /api/landings` | Filtered landing records (month range, species, gear, zone, measure) | +| `GET /api/summary` | Monthly aggregates, top 10 species, price/kg trend | +| `GET /api/summary/monthly-breakdown` | Per-month per-species breakdown | +| `GET /api/available-filters` | Cascading filter options based on active selections | +| `GET /api/export.parquet` | Full dataset as Parquet download | + +Full PX-Web API reference in `docs/API.md`. + +## Source + +[git.flo.fo/FLO/hagfisk](https://git.flo.fo/FLO/hagfisk) + +## Live + +[hagfisk.poc.fló.fo](https://hagfisk.poc.fló.fo) + +--- + +Like the hagfish itself: not glamorous, but it works. ![Watch demo](./gif/hagfish.gif)