User guide

The complete usage guide for parquet-fortran, beyond the quick-start overview on the front page. For the full per-procedure API reference (every public type/function/subroutine under use parquet), see the modules and procedures listings generated from source.

One use parquet brings in the whole library — readers and writers, schemas and MAML, the parquet_table container, the parquet_column foundation and its PK_* kind constants, compact string columns, the date/time/timestamp types, sorting, random numbers and sampling, and the process-global settings. Every page below assumes that single import. The individual modules (parquet_io, parquet_tables, parquet_columns, parquet_strings, parquet_temporal, parquet_sorting, parquet_argsort, parquet_sampling, parquet_random, parquet_settings, parquet_version, parquet_maml_base) are still there and can be named directly when you want a narrower import — see Choosing a module for what each one costs to compile against and which of them are covered by the library's API stability promise. parquet_core is the exception in the other direction: it is internal and is covered by no promise. Because parquet re-exports rather than defines, the generated procedures and types listings are the reliable place to look a name up, rather than the parquet module's own page.

Parquet in, parquet out. This library reads and writes parquet and no other format. Any conversion — FITS, CSV, HDF5, anything else — happens outside it, before the library is called or after it returns. That is a settled scope decision rather than a gap waiting to be filled.

The guide is organised into six groups, ordered roughly by what a new user needs first; reference/advanced material is last. If you only want a file's columns as arrays with the least ceremony, skip straight to Whole tables in memory — it is the easiest entry point in the library.

• Reading and writing files — read columns into arrays, write arrays out, and choose which rows a read returns and in what order (filter, sort, random sample).
• Data types — what a column can hold: the supported types, date/time/ timestamp elements, and compact string columns.
• Schemas, metadata and quality control — declare a file's contents in MAML or in code, with qc bounds; and declare how a file is read — which columns are renamed, which rows are kept and in what order they come back. Ends on two worked examples.
• Whole tables in memory — the high-level parquet_table layer: a whole file as one object, columns by name as ordinary arrays or zero-copy pointers, read only when touched — or built in memory and written out.
• Utilities and code generation — sorting for arrays and columns, reproducible random numbers and the sampling built on them, generated table types, and embedding your own MAML schemas.
• Operating the library — the cross-cutting concerns you reach for when something is wrong or has to be tuned: which module to import, what an abort means, why a build or read fails, what may run concurrently, what memory and speed to expect, and the process-global knobs.

Every page at a glance

All 24 pages, in reading order. (This list, each group's own page list and the ordered_subpage: frontmatter are kept consistent by tools/check_source_conventions.py, which fails the CI lint stage when they drift.)

Reading and writing files

• Reading parquet files from your Fortran code
• Filtering, sorting and sampling rows
• Writing parquet files from your Fortran code

Data types

• Supported data types
• Date, time and timestamp columns
• Compact string columns with parquet_string_column

Schemas, metadata and quality control

• The MAML metadata format
• Building a schema in code with schema%init and schema%add_field
• Quality control
• Combined examples: schemas, vector columns, metadata and quality control

Whole tables in memory

• Whole tables in memory: the basics
• Opening a table: slices, filters and renaming
• Building a table and writing it out
• Changing a table

Utilities and code generation

• Sorting arrays and columns with pf_sort/pf_argsort
• Random numbers and sampling with pf_random_at
• Generated table types: named accessors from a MAML schema
• Embedding your own MAML schemas in your own project

Operating the library

• Choosing a module: what each entry module costs to import
• Error handling
• Troubleshooting
• Thread safety
• Performance and memory
• Settings