hagfish/docs/API.md

API Documentation

This document describes the PX-Web API used by hagfish to fetch Faroese fisheries statistics from the official Statbank.

EndpointBase URL: https://statbank.hagstova.fo/api/v1/fo/H2/VV/VV01/fisknv_md.px

• GET: Returns metadata (table structure, dimension codes, labels)
• POST: Returns data in JSON-stat2 format

Metadata Request (GET)

Requestbash

curl -s "https://statbank.hagstova.fo/api/v1/fo/H2/VV/VV01/fisknv_md.px"

Response Structurejson

{ "title": "AVR01010 Avreiðingar í nøgd og virði...", "variables": [ { "code": "measure", "text": "mát", "role": null, "values": ["MASS", "VALUE"], "valueTexts": ["Nøgd", "Virði"] } ] }

Fields

Field	Type	Description
title	string	Table name in Faroese
variables[].code	string	Dimension identifier (used in queries)
variables[].text	string	Human-readable label in Faroese
variables[].role	null/string	Classification (always null on this endpoint)
variables[].values	string[]	Valid dimension codes
variables[].valueTexts	string[]	Display labels (parallel with values)

Verified Dimension Codes

Variable Code	Faroese Label	Values
measure	mát	MASS, VALUE
Species (ASFIS2022)	Fiskaslag (ASFIS2022)	TOTAL, 148XXXXXXX00000, ... (72 total)
Fishing Gear (ISSCFG2016)	Reiðskapur (ISSCFG2016)	TOTAL, ... (12 total)
Economic Zone (GEONOM2023)	Búskapar øki (GEONOM2023)	TOTAL, ... (10 total)
Processing (EUMOFAPresentation)	Virking (EUMOFAPresentation)	TOTAL, ... (6 total)
Preservation (EUMOFAPreservation)	Viðgerð (EUMOFAPreservation)	TOTAL, ... (9 total)
Shipsize	Skipastødd	TOTAL, ... (11 total)
month	mánaður	2015M01 through 2026M05 (137 months)

Note: role is always null. Time dimension identification must use code == "month".

---

Data Request (POST)

Request Body Formatjson

{ "query": [ { "code": "dimension_code", "selection": { "filter": "item", "values": ["value1", "value2"] } } ], "response": { "format": "json-stat2" } }

Filter Types

Filter	Values	Description
item	explicit codes	Select specific values
all	["*"]	Wildcard — select all values
top	["TOP_N"]	Top N values by magnitude

Example Querybash

curl -s -X POST -H "Content-Type: application/json" -d '{ "query": [ {"code": "month", "selection": {"filter": "item", "values": ["2024M01", "2024M02"]}}, {"code": "Species (ASFIS2022)", "selection": {"filter": "all", "values": [""]}}, {"code": "Fishing Gear (ISSCFG2016)", "selection": {"filter": "all", "values": [""]}}, {"code": "Economic Zone (GEONOM2023)", "selection": {"filter": "all", "values": ["*"]}}, {"code": "Processing (EUMOFAPresentation)", "selection": {"filter": "item", "values": ["TOTAL"]}}, {"code": "Preservation (EUMOFAPreservation)", "selection": {"filter": "item", "values": ["TOTAL"]}}, {"code": "Shipsize", "selection": {"filter": "item", "values": ["TOTAL"]}}, {"code": "measure", "selection": {"filter": "item", "values": ["MASS", "VALUE"]}} ], "response": {"format": "json-stat2"} }' "https://statbank.hagstova.fo/api/v1/fo/H2/VV/VV01/fisknv_md.px"

Cell Limit

• Maximum cells per query: ~8,000,000
• Recommended maximum: 1,000,000 for reliability
• To fetch full dataset, paginate by month or use smaller dimension selections

---

Response Format (JSON-stat2)

Structurejson

{ "class": "dataset", "label": "...", "id": ["measure", "Species (ASFIS2022)", ..., "month"], "size": [2, 72, 12, 10, 1, 1, 1, 2], "dimension": { "measure": { "label": "mát", "category": { "index": {"MASS": 0, "VALUE": 1}, "label": {"MASS": "Nøgd", "VALUE": "Virði"} } } }, "value": [73653738, 1234567, ...] }

Fields

Field	Description
id	Array of dimension codes in order (defines cube layout)
size	Cardinality per dimension (parallel with id)
dimension.<code>.category.index	Maps value code → numeric position in dimension
dimension.<code>.category.label	Maps value code → display text
value	Flattened array of measurements (row-major order)

Row-Major Index Decoding

Values are stored in row-major order: the last dimension (month) varies fastest.

Given size = [2, 72, 12, 10, 1, 1, 1, 2]:

• Flat index 0 → [0,0,0,0,0,0,0,0] = measure=MASS, species=TOTAL, ..., month=2024M01
• Flat index 1 → [0,0,0,0,0,0,0,1] = measure=MASS, species=TOTAL, ..., month=2024M02
• Flat index 2 → [0,0,0,0,0,0,1,0] = measure=MASS, species=SPECIES_1, ..., month=2024M01

Decoding algorithm (reverse modulo):rust fn decode(flat_index: usize, sizes: &[usize]) -> Vec { let mut indices = Vec::new(); let mut remaining = flat_index; for &size in sizes.iter().rev() { indices.push(remaining % size); remaining /= size; } indices.reverse(); indices }

Sentinel Values

Value	Meaning
-1.0	Missing/no data (treated as NULL)
null	Missing/no data (already nullable in JSON)

---

Known Constraints

1. Cell limit: ~8 million cells per query
2. Rate limits: Unknown — assume reasonable backoff for large downloads
3. Language: API responds in requested language (fo or en)
4. Time zone: No timezone specified — treat timestamps as local
5. Updates: Data updated irregularly — metadata timestamp in response header

---

Integration Checklist

• Metadata endpoint reachable via GET
• POST queries return valid JSON-stat2
• Dimension codes match variables[].code from metadata
• Row-major index decoding verified
• Sentinel value coercion (-1.0 → None) implemented
• Unicode (Faroese characters) handled correctly
• Incremental ingestion logic tested
• Parquet export tested
• Error handling for network timeouts implemented

---

References

• Official PxWeb documentation: https://pxweb.github.io/docs/
• JSON-stat2 specification: http://json-stat.org/format/
• Hagstova Føroya: https://www.hagstova.fo/