readme
This commit is contained in:
@@ -1,11 +1,100 @@
|
|||||||
# Hagfisk
|
# Hagfisk
|
||||||
|
|
||||||
|
Faroese fisheries data pipeline and dashboard. Pulls monthly landing statistics from Hagstova Føroya's PX-Web API, stuffs them into a local DuckDB, and serves a filtered ECharts dashboard — all from a single Rust binary.
|
||||||
|
|
||||||
|
Named after the hagfish: a deep-sea creature famous for producing copious amounts of slime and being generally unpleasant. The name stuck anyway. The fish has no backbone. Neither does this README.
|
||||||
|
|
||||||
|
## What It Does
|
||||||
|
|
||||||
|
- Fetches fisheries data (mass + value by species, gear, zone, month) from `statbank.hagstova.fo`
|
||||||
|
- Stores everything in a local DuckDB file
|
||||||
|
- Serves a REST API with filtered queries, summaries, and Parquet export
|
||||||
|
- Renders an interactive dashboard with line, price-trend, and stacked bar charts
|
||||||
|
- Runs monthly ingestion automatically via systemd timer (28th of each month, 03:00)
|
||||||
|
|
||||||
|
## Tech Stack
|
||||||
|
|
||||||
|
| Thing | Choice | Why |
|
||||||
|
|---|---|---|
|
||||||
|
| Language | Rust 2024 | Fast, safe, single static binary. No runtime, no VM, no nonsense. |
|
||||||
|
| Web framework | Axum | Async, typed, plays well with Tokio. Doesn't fight you. |
|
||||||
|
| Database | DuckDB | Embedded OLAP. No server, no config, just a file. Perfect for analytics on a single machine. |
|
||||||
|
| Charts | ECharts | Powerful, flexible, no build step. Vanilla JS, no framework tax. |
|
||||||
|
| Static assets | rust-embed | Compiled into the binary. One artifact to deploy, nothing to forget. |
|
||||||
|
| Logging | tracing + tracing-subscriber | Structured logs to file and journal. |
|
||||||
|
| Deployment | Bare metal + systemd | No containers. No compose files. No port mappings. Just a binary and a `.service` unit. |
|
||||||
|
|
||||||
## What does it do?
|
## Getting Started
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo build --release
|
||||||
|
./target/release/hagfisk ingest --full
|
||||||
|
./target/release/hagfisk serve
|
||||||
|
```
|
||||||
|
|
||||||
|
Dashboard lives at `http://localhost:8090`.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
`config.json` next to the binary:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"duckdb_path": "hagfish.db",
|
||||||
|
"bind_address": "127.0.0.1:8090",
|
||||||
|
"data_source_url": "https://statbank.hagstova.fo/api/v1/fo/H2/VV/VV01/fisknv_md.px",
|
||||||
|
"log_file_path": "hagfish.log",
|
||||||
|
"allowed_origins": []
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## CLI
|
||||||
|
|
||||||
|
```
|
||||||
|
hagfisk serve Start the web server
|
||||||
|
hagfisk ingest [--full] Fetch data (incremental or full backfill)
|
||||||
|
hagfisk export -o FILE Export landings to Parquet
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
The `justfile` handles everything:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
just deploy # Build, rsync, restore SELinux contexts, restart
|
||||||
|
just deploy-reingest # Same + full re-ingest
|
||||||
|
just logs-app # Tail application logs
|
||||||
|
just ps-app # Check service status
|
||||||
|
```
|
||||||
|
|
||||||
|
Systemd units live in `systemd/` — drop them in `~/.config/systemd/user/` and enable the timer for monthly ingestion.
|
||||||
|
|
||||||
|
## API
|
||||||
|
|
||||||
|
| Endpoint | What |
|
||||||
|
|---|---|
|
||||||
|
| `GET /healthz` | Alive check |
|
||||||
|
| `GET /api/species` | Species codes + Faroese names |
|
||||||
|
| `GET /api/zones` | Economic zone codes |
|
||||||
|
| `GET /api/gear` | Fishing gear codes |
|
||||||
|
| `GET /api/landings` | Filtered landing records (month range, species, gear, zone, measure) |
|
||||||
|
| `GET /api/summary` | Monthly aggregates, top 10 species, price/kg trend |
|
||||||
|
| `GET /api/summary/monthly-breakdown` | Per-month per-species breakdown |
|
||||||
|
| `GET /api/available-filters` | Cascading filter options based on active selections |
|
||||||
|
| `GET /api/export.parquet` | Full dataset as Parquet download |
|
||||||
|
|
||||||
|
Full PX-Web API reference in `docs/API.md`.
|
||||||
|
|
||||||
|
## Source
|
||||||
|
|
||||||
|
[git.flo.fo/FLO/hagfisk](https://git.flo.fo/FLO/hagfisk)
|
||||||
|
|
||||||
|
## Live
|
||||||
|
|
||||||
|
[hagfisk.poc.fló.fo](https://hagfisk.poc.fló.fo)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
Like the hagfish itself: not glamorous, but it works.
|
||||||
|
|
||||||

|

|
||||||
|
|||||||
Reference in New Issue
Block a user