Reading parquet files from your Fortran code

This page is about reading a parquet file one column at a time: opening a parquet_reader, pulling whole columns into Fortran arrays, reaching a single row or a single element position without reading the rest, asking a column its shape or type, and streaming a file too large to hold in memory. It is the level to work at when you want particular columns and want to control when each one is read.

If what you actually want is the whole file — every column available by name, read only when you touch it — start at Whole tables in memory instead: one parquet_open_table call replaces most of this page. Choosing which rows a read returns, and in what order, is Filtering, sorting and sampling rows.

To use this library in another Fortran project, add it as an FPM dependency — see Minimal setup to depend on this library in the README for the fpm.toml snippet. Then use parquet in your code.

Minimal reader example:

program read_parquet_example
    use parquet
    use iso_fortran_env, only: int32, int64
    implicit none

    type(parquet_reader) :: reader
    integer(int64) :: nrows
    integer(int32), allocatable :: id(:)

    call parquet_open_reader(reader, "data.parquet")
    call parquet_get_nrows(reader, nrows)

    allocate(id(nrows))
    call parquet_read_column(reader, "id", id)

    call parquet_close_reader(reader)
end program read_parquet_example

Notes:

• parquet_get_nrows returns the number of table rows.
• Allocate output arrays before calling parquet_read_column. Its row count (size(values) for a plain column, size(values, 2) for a vector column) must match parquet_get_nrows exactly, or it fails immediately with error stop, naming the column and both row counts.
• name must be a column that actually exists in the file. Every procedure that takes a column name checks this and fails immediately with error stop naming the missing column — parquet_read_column, parquet_read_column_chunk, parquet_get_col_size, parquet_get_column_total_elements, parquet_get_string_length, parquet_get_column_type, parquet_read_array_row_mode, parquet_read_array_element_mode and parquet_prefetch_columns among them. The one deliberate exception is parquet_column_exists, whose whole job is to answer that question — it returns .false. instead of aborting.
• For string columns, choose a fixed string length that is large enough for your data — or read into a type(parquet_string_column) instead, which needs no pre-sizing; see Reading and writing compact string columns. Not sure how long is long enough? parquet_get_string_length answers that directly — see Column shape and size queries below.
• For vector columns, allocate 2D arrays with shape (col_size, nrows).
• A DATE/TIME/TIMESTAMP column reads into a parquet_date/parquet_time/parquet_timestamp array instead — see Date, time and timestamp columns; these three carry their own null state, so null_value=/is_valid= don't apply to them.
• A vector column may be stored on disk either as a fixed_size_list (what this library's own writer emits) or as a variable-length list<element> (the Parquet LIST layout many other producers use — including its legacy 2-level and non-standard inner-element-name variants, which Arrow's reader normalizes to the same list type); both are read back identically. The only requirement is that every row's vector has the same length (so it fits the (col_size, nrows) shape); a genuinely ragged list column (rows of differing length) is rejected with error stop. col_size is inferred from the data in either case.

The nrows= shortcut

parquet_open_reader(..., nrows=nrows) is a shortcut for the parquet_open_reader + parquet_get_nrows pair above: pass an integer(int32) or integer(int64) variable (a plain default INTEGER works too, on the vast majority of platforms where that's the same kind as int32) as nrows and it's filled in for you, equivalent to calling parquet_get_nrows(reader, nrows, check_positive=.true.) immediately after opening (post-filter, if a filter was also given).

Because it implies check_positive=.true., a file (or filter result) with zero rows fails immediately with error stop — it does not return nrows=0. This is the right choice when your code has no sensible zero-row behavior and would rather abort loudly than proceed with an empty read.

If zero rows is a case you need to detect and handle rather than treat as a hard error, don't pass nrows= to parquet_open_reader — open the reader as usual and call parquet_get_nrows(reader, nrows) yourself afterwards, without check_positive, which returns nrows=0 instead of aborting, exactly like the code example above.

As with a direct parquet_get_nrows(reader, nrows) call, an integer(int32) (or default-INTEGER) nrows also fails immediately with error stop if the actual row count overflows int32's range, rather than silently wrapping or truncating.

Reading only touches the columns you ask for

parquet_open_reader only parses the file's footer (schema, row count, row-group layout) — it does not read or decompress any column's data. Each column is read from disk only the first time you ask for it (parquet_read_column, parquet_get_string_length, etc.), then cached for the lifetime of that reader; asking for it again doesn't re-read it, and columns you never ask for are never read at all.

This follows from Parquet's layout — each column is its own contiguous byte range, so the reader seeks straight to just the bytes it needs, regardless of compression codec. So opening a large file with many columns and reading only a handful is cheap in both I/O and memory, no matter how large the unrequested columns are.

parquet_get_col_size/parquet_get_column_total_elements are cheaper still for the common case (a vector column stored as fixed_size_list): they answer straight from the schema/footer, reading no column data at all — so they're safe to call even on a column whose total element count (nrows * col_size) is enormous. See Column shape and size queries for the one column layout where they do have to read.

parquet_read_array_row_mode reads only the one row group the requested row lives in, not the whole column. parquet_read_array_element_mode can't limit itself to one row group the way parquet_read_array_row_mode does — it inherently needs every row's value at the same element position, so every row group contributes — but it still avoids ever materializing the whole column's flattened element count in a single internal call, by streaming the file row group by row group instead.

A row filter or sample_fraction doesn't change either of these: row_index/elem_index then address the filtered result, and each row group's surviving row count is what they are resolved against. (Chunked reads work per row group on a filtered reader too — see Streaming/chunked reads.)

Releasing a column you have finished with

A cached column stays in memory until the reader is closed. parquet_release_column(reader, name) frees that one column's decoded Arrow buffers once you have copied what you wanted into your own Fortran storage — the way to read a sequence of large columns without holding all of them at once:

call parquet_read_column(reader, "big", values)
! ... use `values`, which is your own array and unaffected by what follows ...
call parquet_release_column(reader, "big")

It is purely a memory/time trade and never changes an answer: asking for the column again simply reads and decodes it from disk again, with any active filter=/sample_fraction= re-applied. A name that doesn't exist, or a column that was never read, is a silent no-op — so it is safe in a loop over column names.

One trap: name may be a dotted struct-leaf path, but the reader caches a struct as one array, so releasing any leaf frees the whole struct. When walking several leaves of the same struct, release only after the last of them, or each leaf re-reads the struct.

A released column still appears in a print_stat report, with its value-derived cells marked released.

Listing a file's columns

parquet_get_column_names(reader, names) hands back every top-level column name in schema order, in an allocatable character array sized to the longest name and blank-padded — so trim() is how you use them, the same convention parquet_get_metadata_items follows:

character(len=:), allocatable :: names(:)
integer :: i

call parquet_get_column_names(reader, names)
do i = 1, size(names)
    print *, trim(names(i))
end do

It reads no column data: the answer comes from the schema parsed when the reader was opened. Names come back as the file spells them, so a struct column appears once, under its own top-level name, rather than as one entry per leaf.

Random access: row mode and element mode

parquet_read_array_row_mode and parquet_read_array_element_mode read a vector (col_size > 1) column without materializing the whole thing — the random-access counterparts to parquet_read_column, for when you only need one row, or one fixed element position across every row, out of a column that may be far larger. (In the call forms written out below, an argument in square brackets is optional; brackets are notation for this page, never something you type.)

• Row mode — parquet_read_array_row_mode(reader, name, values, row_index [, null_value] [, is_valid]) reads one row's entire element vector. values is a 1D array of length col_size; row_index is 1-based (integer(int32) or integer(int64) — the latter only needed for a file with more rows than huge(1_int32)).

type(parquet_reader) :: reader
integer(int32) :: row3(4)   ! col_size = 4

call parquet_open_reader(reader, "data.parquet")
call parquet_read_array_row_mode(reader, "vec", row3, row_index=3_int64)
call parquet_close_reader(reader)

• Element mode — parquet_read_array_element_mode(reader, name, values, elem_index [, null_value] [, is_valid]) reads the same element position from every row instead. values is a 1D array of length nrows (the table's whole row count, not col_size); elem_index is 1-based, in [1, col_size].

type(parquet_reader) :: reader
integer(int64) :: nrows
integer(int32), allocatable :: first_elem(:)

call parquet_open_reader(reader, "data.parquet", nrows=nrows)
allocate(first_elem(nrows))
call parquet_read_array_element_mode(reader, "vec", first_elem, elem_index=1_int64)
call parquet_close_reader(reader)

Both take the same null_value=/is_valid= pair as parquet_read_column (see Null values), both dispatch on values' declared type/kind exactly like parquet_read_column does (including the widened-numeric-kind conversions), and both accept a struct-nested dotted name (see Reading a nested struct field).

When to prefer these over parquet_read_column: row mode is the right tool when you need a handful of specific rows out of a huge vector column and don't want to allocate/read the whole (col_size, nrows) array just to throw most of it away. Element mode is the right tool when you need one column of a vector column's own internal structure (e.g. always the first element of each row) across the whole file, without allocating the full 2D array.

I/O behavior (see Reading only touches the columns you ask for above for the full explanation): row mode reads only the one row group the requested row falls in; element mode streams the file row group by row group, so neither ever materializes the whole column in a single internal call. A row filter and/or sample_fraction doesn't change that — row_index/elem_index then address the filtered result, resolved against each row group's surviving row count. A sort_by= does: sorted row i belongs to no single row group, so both modes fall back to reading the whole column. Neither is single-row I/O in the strictest sense (both still decode a full row group's worth of data at a time), but both are genuine random access at row-group granularity rather than a whole-column read.

Column shape and size queries

Three procedures answer questions about a column's shape:

• parquet_get_col_size(reader, name, col_size) — a vector column's fixed per-row width (1 for a plain scalar column).
• parquet_get_column_total_elements(reader, name, total_elements) — the column's total flattened element count (nrows * col_size for a vector column; nrows for a scalar one).
• parquet_get_string_length(reader, name, max_string_length) — for a string column (or a vector column whose elements are strings): the longest value actually present, unaffected by Nulls (a Null is always skipped when computing this maximum) — the direct answer to "how long should my character(len=...) buffer be?" from the minimal example notes above.

The first two normally answer straight from the file's schema/footer and read no column data at all. The exception is a column stored as a plain variable-length list/large_list (which this library never writes, but another producer may): its width is a property of the data rather than of the schema, so measuring it has to read the column. Even then it is bounded — each row group is screened from the footer first, and only a surviving candidate is proved by reading, one row group at a time, so peak memory is one row group rather than the whole column.

parquet_get_string_length is different in kind: it always reads the column, because the longest string can only be found by looking at every value, and the column then stays cached like any other read. Sizing a character(len=...) buffer this way is therefore not a free footer lookup — on a very large string column it is a full decode, and parquet_release_column is how you get that memory back. Reading into a parquet_string_column avoids the question entirely, since it needs no pre-sizing.

Two lower-level queries exist for the plain-list case, if you need to control the measurement rather than just take its answer:

• parquet_column_width_needs_data(reader, name) returns .true. only for a plain list/large_list column — i.e. it tells you in advance whether asking for the width will read anything at all.
• parquet_measure_list_width(reader, name, row_group_lo, row_group_hi, proven, width) measures over an inclusive 1-based row-group range (row_group_lo <= 0 means every row group), with proven=.false. for the footer screen alone (free, and its answer is a candidate that may be wrong) and proven=.true. to confirm it by reading. A column with no rows measures 0, and a genuinely ragged one measures 1.

type(parquet_reader) :: reader
integer(int32) :: col_size, strlen_max
integer(int64) :: total_elements

call parquet_open_reader(reader, "data.parquet")
call parquet_get_col_size(reader, "vec", col_size)
call parquet_get_column_total_elements(reader, "vec", total_elements)
call parquet_get_string_length(reader, "name", strlen_max)
call parquet_close_reader(reader)

All three accept a struct-nested dotted name (see Reading a nested struct field) and fail immediately with error stop if name doesn't exist.

parquet_column_has_nulls(reader, name, row_group_lo, row_group_hi) answers the one remaining shape question — whether a column contains any Null — from the per-column-chunk null counts Parquet records in the footer, so it reads no column data either. It is a logical function, and unlike the queries above its row-group bounds are required arguments: they restrict the question to an inclusive 1-based range, with row_group_lo <= 0 meaning every row group, as in parquet_measure_list_width above.

if (parquet_column_has_nulls(reader, "flux", 0, 0)) then   ! 0, 0 -- ask about the whole file
    call parquet_read_column(reader, "flux", flux, is_valid=valid)
else
    call parquet_read_column(reader, "flux", flux)         ! no mask needed, and measurably cheaper
end if

That is what the query is for: requesting is_valid= costs an extra buffer, a LOGICAL array four times its size, a conversion pass and a scan, so asking first and omitting the argument on a null-free column is worth the footer lookup.

Two properties are worth knowing before relying on it. Its uncertain answer is .true. ("might have nulls"), since statistics are optional in the format and claiming a column is clean when it is not would make the read abort on the first Null. And it declines a dotted struct path the same way, because a struct leaf's validity is combined with every ancestor struct's and the leaf's own footer count does not describe the result.

Checking column existence and type

parquet_column_exists(reader, name, [types]) returns .true./.false. for whether name (a top-level or dotted struct-leaf path, same as everywhere else) exists in an open reader's schema:

if (parquet_column_exists(reader, "ra")) then
    ...
end if

Pass types (optional) to also require that the column can be read as one of the given types — a comma-separated list of tokens: any of the nine canonical single types (int32/int64/float32/ float64/boolean/string/date/time/timestamp), and/or the group aliases int (any integer column), float (any column readable into a float) and temporal (date, time, or timestamp). Matching is case-insensitive and tokens can be combined:

if (parquet_column_exists(reader, "ra", types="float")) then     ! any numeric column
if (parquet_column_exists(reader, "flag", types="int, boolean")) ! any integer column, or boolean

Omitting types checks existence regardless of type; passing an empty or all-blank types is treated as a mistake rather than as that shortcut, and fails immediately with error stop. An unrecognized token in types (e.g. a typo) fails immediately with error stop, naming the valid tokens — this check happens before the existence check itself, so a malformed filter is reported even for a column that doesn't exist. A column this library cannot read at all (a map, say) never matches a types filter, but is still found by a plain (no types) existence check.

parquet_get_column_type(reader, name, type_name) answers the same question the other way round — what Fortran type an existing column is read into — directly into an allocatable character:

character(len=:), allocatable :: type_name
call parquet_get_column_type(reader, "ra", type_name)   ! e.g. "float64"

What a column is read as

Both procedures are driven from one mapping: the narrowest lossless Fortran kind for the column's physical type. That is not the same as the physical type's own name — all four numeric targets accept the same fifteen physical types, so which kind a read actually uses is chosen by your declaration, not by the file (see Reading a column into a different numeric kind). What these two queries report is the kind to declare if you want the file's values back intact.

The two lossy rows return the conventional target rather than unknown on purpose: no Fortran kind covers uint64's full range or a decimal's exact value, but reporting them as unreadable would be less useful than naming the kind the library will actually use. A uint64 value above huge(1_int64) still aborts when the column is read. The mapping reads the type ID alone, with no precision or scale awareness, so decimal(9,0) reports float64 like every other decimal rather than int32.

An alias is not the union of its member tokens, and that is intended. types="float" matches an int32 column — an integer is readable into a float array — while types="float64" does not, because that column's target kind is int32. The two ask different questions: can I read this as a float at all? against is float64 the right declaration? Both are useful, so neither was made to imply the other.

A vector (FIXED_SIZE_LIST) column reports its element type (an int32 vector column reports "int32" — see parquet_get_col_size for its element count). parquet_get_column_type fails with error stop only if name doesn't exist — that is a caller mistake, and parquet_column_exists is the query for it. A type it cannot read is an answer ("unknown"), not an error.

Prefetching multiple columns at once with parquet_prefetch_columns

parquet_prefetch_columns(reader, names) reads several named columns in one call, filling the same per-column cache parquet_read_column otherwise populates lazily. Because this library keeps Arrow's use_threads on (see Thread-pool tuning), decoding them together lets Arrow use its thread pool concurrently instead of one column at a time — purely a throughput optimization, never a requirement: a column you don't prefetch still works via the normal lazy path. Every name is validated against the file's schema first; an unknown column fails immediately with error stop, naming it. Calling it more than once is safe and efficient — already-cached columns are not re-read, so the cache ends up holding the union of every name across all calls.

The column names can be given in either of two forms — parquet_prefetch_columns is generic:

call parquet_open_reader(reader, "data.parquet")

! (1) a single string, names separated by commas and/or semicolons:
call parquet_prefetch_columns(reader, "ra; dec, mag")

! (2) an array of names (each element the same declared length):
call parquet_prefetch_columns(reader, ["ra ", "dec", "mag"])

! call parquet_read_column for these (in any order) and any other, non-prefetched column:
call parquet_read_column(reader, "ra", ra)
call parquet_read_column(reader, "id", id)   ! fine even though "id" was never prefetched

• The string form (1) splits on commas and/or semicolons (both accepted, and mixable), trims surrounding spaces from each name, and ignores empty tokens (so repeated or trailing delimiters are harmless). This is the recommended form when your names have different lengths, since it side-steps the array pitfall below.
• The array form (2) requires all names to share the same declared string length — pad shorter names with trailing spaces, as with any Fortran character array literal (["ra ", "dec"], not ["ra", "dec"], which won't even compile). Be careful: if the declared length is shorter than a name, that name is silently truncated and then fails validation as a "column not found" for the truncated text — e.g. [character(len=1) :: "a", "xa"] truncates "xa" to "x" and aborts with column not found in parquet file: x. The string form has no such trap.

To prefetch every column in the file, rather than naming them yourself, pass prefetch=.true. to parquet_open_reader instead of calling parquet_prefetch_columns:

call parquet_open_reader(reader, "data.parquet", prefetch=.true.)

! every column is already cached -- reading any of them (in any order) never hits disk again:
call parquet_read_column(reader, "ra", ra)
call parquet_read_column(reader, "id", id)

prefetch is optional and defaults to .false.. It behaves exactly as if you called parquet_prefetch_columns for every column in the file immediately after opening (same batched, thread-parallel decode, same "already-cached columns are skipped" behavior), and composes correctly with filter: prefetching happens after the filter is applied, so every prefetched column — not just the ones the filter itself references — reflects the filtered row set. Because it eagerly materializes the whole file in memory at open time, before you've asked for anything, it trades memory and up-front latency for guaranteeing no column read is ever a cache miss later — see Performance and memory.

Printing reader statistics with parquet_close_reader(..., print_stat=.true.)

parquet_close_reader(reader, print_stat=.true.) prints a debug/diagnostic summary of the reader's activity to stdout, right before actually closing it. print_stat is optional and defaults to .false. (no output). This is solicited output, so it is still governed by the library's verbosity setting: under parquet_set_verbosity("silent") or "errors_only" the report is suppressed entirely and print_stat=.true. prints nothing.

The summary has two parts:

• Table-level: the filename, the total number of columns in the file, how many of those are actually shown below (see below), and the row count — written as rows: N (of M total) when a filter or sample has narrowed it. A further line appears for each of sample: (fraction and the seed actually used), filter: (the whole expression as re-rendered from the parse tree, which stays exact where the per-column filter cell below cannot), sort: (the keys, in the order they were added) and screened: (how many row groups the statistics pre-screen skipped), each printed only when it applies.
• One row per column that was either prefetched (parquet_prefetch_columns) or actually read (parquet_read_column/parquet_read_array_row_mode/parquet_read_array_element_mode) at some point during the reader's lifetime — a column never touched at all is left out of the list entirely, rather than decoding it just to fill in a report:

call parquet_open_reader(reader, "data.parquet")
call parquet_prefetch_columns(reader, ["ra ", "mag"])
call parquet_read_column(reader, "ra", ra)
call parquet_close_reader(reader, print_stat=.true.)

Requires linking Arrow's arrow_compute library (for the min/max calculation) in addition to arrow/parquet — see Environment variables; parquet-fortran's own fpm.toml already declares this link, and fpm propagates it to consuming projects automatically, so no action is normally needed.

Filtering, sorting and sampling rows

A reader can also be restricted to a subset of the file's rows — matching a parquet_filter, or randomly downsampled — and can return them in sorted order. Everything on this page then behaves as if the file only ever contained the surviving rows. See Filtering, sorting and sampling rows for the full reference.

Quality control

Reading also has its own side of quality control — checking column values against qc: min:/max:/miss: bounds declared in a MAML file, independent of the write-side checks — plus two ways to build a qc-maml directly in code without a .maml file on disk. See Quality control for the full picture (both sides, plus the in-code builders); the read side is parquet_open_reader(reader, filename, schema=..., qc=..., qc_soft=...), which by default hard-aborts on a violation (qc_soft=.true. instead warns and continues).

Reading table metadata with parquet_get_metadata

parquet_get_metadata(reader, key, value [, default] [, warn]) reads one table-level metadata entry back out of a file — the read-side counterpart to the writer's schema%add_metadata (see Runtime table metadata). It is generic: the declared type/kind of value (a scalar or 1D array of any supported type) selects the variant and how the stored text is parsed back, since all metadata is stored as strings in the file. Table metadata is read once, at parquet_open_reader time, and cached, so each call only scans that in-memory copy. Any key present in the file works, including the reserved/internal ones the writer emits (DATE, name, per-column column.<name>.*, per-keyword <KEY>.datatype, ...), not just keys added via add_metadata — see How table-level keys become metadata entries for which MAML keys produce which entries.

A Fortran reader never needs the <KEY>.datatype entries. They record what a typed schema%add_metadata call stored (see A typed value records its own type), and here the declared type of value already selects the parse, so parquet_get_metadata neither consults them nor checks itself against them. They exist for readers in languages where a value's type cannot be declared at the call site, and they are readable like any other key: call parquet_get_metadata(reader, "NSIDE.datatype", token).

• A missing key triggers error stop, unless the optional default (same type/kind as value) is given, in which case value is set to it.
• A present but unconvertible stored value (e.g. non-numeric text, or an integer too large for the requested kind) always prints a WARNING, then falls back to default if given, else error stop.
• warn (optional logical, default .true.) only governs the missing-key-with-default case; pass warn=.false. to suppress that warning. It has no effect when the key is present.

Listing every metadata entry

parquet_get_metadata answers for a key you already know. To find out what a file carries at all, parquet_get_metadata_items(reader, keys, values) hands back both, index-aligned and in the file's own order:

character(len=:), allocatable :: keys(:), values(:)
integer :: i

call parquet_get_metadata_items(reader, keys, values)
do i = 1, size(keys)
    print *, trim(keys(i)), " = ", trim(values(i))
end do

Each array is allocated to its own longest entry and blank-padded, so trim() is how you use them — the same convention parquet_get_column_names follows. Both come back zero-size for a file with no metadata. Like parquet_get_metadata it reads nothing: the answer comes from the copy made when the reader was opened. This is what copying metadata from one file to another is built on — see parquet_write_table's copy_metadata=.

Streaming/chunked reads

parquet_read_column reads a whole column into one complete array — fine for most data, but not for a column too large to hold in memory that way. parquet_read_column_chunk reads such a column one Parquet row group at a time instead, so peak memory is bounded by a row group's worth of data rather than the whole column — the read-side mirror of streaming/chunked writes:

type(parquet_reader) :: reader
integer(int64) :: num_row_groups, rg, rg_nrows
integer(int32), allocatable :: buf(:)

call parquet_open_reader(reader, "data.parquet")
call parquet_get_num_row_groups(reader, num_row_groups)

do rg = 1, num_row_groups
    call parquet_get_chunk_size(reader, rg_nrows, row_group=rg)
    allocate(buf(rg_nrows))

    call parquet_read_column_chunk(reader, "big_vec", rg, buf)
    call process_chunk(buf, rg_nrows)  ! your own code

    deallocate(buf)
end do

call parquet_close_reader(reader)

parquet_get_num_row_groups(reader, num_row_groups) returns the file's row-group count. parquet_get_chunk_size(reader, chunk_size, [row_group]) returns row group row_group's (1-based) own row count — its physical row count on an ordinary reader, and its surviving row count once a filter or sample is active (see below) — row groups are not guaranteed uniform, so query each one rather than assuming they all match the first; row_group is optional and defaults to the first row group. parquet_read_column_chunk(reader, name, row_group, values) then reads that row group's rows into values, dispatched by its actual/declared type/kind and rank (scalar values(:) or matrix values(:,:)) exactly like parquet_read_column, and separately by row_group's own kind (integer(int32)/integer(int64), the latter only needed for a file with more row groups than huge(1_int32)).

Stateless, random access: unlike the write side's parquet_new_row_group/parquet_finish_row_group pairing, there is no "currently open" row group to track — call parquet_read_column_chunk with any row_group, in any order, as many times as you like, for any column, independent of any other chunked read on the same reader.

Type matching: a chunked read converts between numeric kinds exactly as parquet_read_column does — same rules, same helpers — so a float64 column can be chunk-read into a real32 array, an int32 column into an integer(int64) or a real64 one, and so on. See Reading a column into a different numeric kind for what is permitted and where an out-of-range or non-integral value aborts. logical and character chunk reads are the exception: those require the stored column to actually be boolean or string, and fail with error stop naming both types otherwise.

Compact string columns: a scalar string column can also be chunk-read into a type(parquet_string_column) (values is cleared, then filled with just that row group's rows) — see Reading and writing compact string columns.

Works on a filtered or sampled reader, but not on a sorted one (see what a sort disallows). A chunked read on a reader opened with filter= and/or sample_fraction= hands back that row group's surviving rows, and parquet_get_chunk_size reports that same count — so the sizes still sum to parquet_get_nrows, and a chunked loop needs no separate bookkeeping:

call parquet_open_reader(reader, "data.parquet", filter=filt)
call parquet_get_num_row_groups(reader, nrg)
do rg = 1, nrg
    call parquet_get_chunk_size(reader, n, row_group=rg)   ! surviving rows in this row group
    allocate(v(n))
    call parquet_read_column_chunk(reader, "v", rg, v)      ! just those rows
    ...
    deallocate(v)
end do

A row group whose rows were all filtered away yields zero rows — the normal case under a selective filter, not an error: parquet_get_chunk_size returns 0 and the read is a no-op. It still counts as read for check_complete below.

Chunked reads do not accumulate memory. Each parquet_read_column_chunk call reads that one row group, hands the values over, and frees the Arrow array before returning — nothing is added to the reader's column cache, so a loop over a thousand row groups holds no more than a loop over one, and calling parquet_release_column after each chunk would have nothing to release. This is what makes the pattern usable on a file far larger than memory. Measured with Arrow's own pool counter over a 400,000-row file in 20 row groups: the pool stays flat at well under a kilobyte across the whole loop, where one parquet_read_column of the same column retains 3.5 MB for the reader's lifetime. (Resident set size cannot show this — Arrow does not return freed pages to the OS — which is why the figure comes from the pool counter.) The one bounded exception is reading into a parquet_string_column: that path pins the current chunk's buffers until the next chunk is read or the reader is closed, so exactly one chunk stays alive. Bounded, not accumulating.

Memory-bounded filtering with a row-group scope. Applying a filter has two engines, and which one runs is decided by whether you name row groups at all, not by which rows you name:

So the row-group arguments are not required. Without them the filter behaves exactly as an open-time filter= does: every filter column is read whole-file in one batched, thread-parallel pass and left decoded, which is fastest and is the right default, at the cost of one full copy of those columns. With them, the expression is evaluated one row group at a time over that inclusive 1-based range, each chunk being released before the next is read, so peak memory is one row group's worth of the filter columns rather than the whole file; rows outside the range never match, and nothing is left cached, so a filter column read afterwards is read again.

row_group_lo = 0 means every row group — the bounded-memory engine over the whole file, without having to ask parquet_get_num_row_groups how many there are first. row_group_hi is ignored in that case. (A non-positive lower bound reads the same way in parquet_measure_list_width and parquet_column_has_nulls.) Put the other way round: to store the filtered columns, filter while opening; to store only the mask, filter afterwards with row-group arguments.

parquet_reader_set_filter refuses, with error stop, if the reader already has a filter (compose the clauses into one parquet_filter instead — several %add calls are AND-combined) or if any column has already been decoded on this reader, since data already handed back could not then be aligned with anything read afterwards. A filter passed to parquet_open_reader counts as an active filter for the first of those, so opening with filter= and then calling parquet_reader_set_filter aborts as surely as calling it twice does. The second condition is easy to meet by accident here: a parquet_read_column_chunk call before the set_filter gets the ordering wrong. Set the filter first, then loop. See Applying a filter after the reader is open.

Pair it with a chunked loop over the same row groups to filter a file larger than memory:

call parquet_open_reader(reader, "huge.parquet")
call parquet_reader_set_filter(reader, filt, 5, 8)   ! only row groups 5..8 are examined

A scoped filter scopes the whole reader, not just the loop. Everything the reader hands back afterwards is restricted to the chosen row groups — a whole-column parquet_read_column returns the survivors of row groups 5..8 and nothing else, and parquet_get_nrows reports that same count. That is the scope working as intended, not a filter that failed to match the rest of the file; rows outside the range have no mask bits at all, so there is nothing for a later read to return. If you want the whole file examined with bounded memory, that is what row_group_lo = 0 above is for.

Narrowing to an exact row range. A row-group range can only ever begin and end on a row-group boundary, so a caller interested in an arbitrary row range would get back every survivor of the covering row groups with no way to trim them — only the mask knows which physical rows those are. A four-argument form adds the row range itself: parquet_reader_set_filter(reader, filt, row_group_lo, row_group_hi, row_lo, row_hi), where row_lo/row_hi are 1-based, inclusive, physical file rows. Rows outside them never match, so parquet_get_nrows afterwards is that range's own surviving count:

call parquet_reader_set_filter(reader, filt, 2, 3, 5, 8)   ! rows 5..8, inside row groups 2..3

The row range must lie inside the rows its row groups span, or the call fails with error stop naming both ranges and the span. The two ranges are otherwise individually plausible — each is checked against the file's own row-group count and row count — and a disjoint pair would quietly yield their intersection, which is frequently empty and therefore indistinguishable from a filter that matched nothing.

The filter may hold no rules at all in this form, in which case the range alone decides which rows match — that is how a row scope is installed for its own sake, with no expression to hang it on. Both integer kinds are accepted, as for the row-group bounds.

Read-time qc still runs, scoped to one row group at a time: if the reader was opened with qc=.true., each parquet_read_column_chunk call runs the usual qc: min:/max:/miss: checks against just that row group's own data, not the whole column. In hard mode (qc_soft=.false., the default), a violation aborts immediately, naming the offending row group (qc violation for column 'name [row group N]'...). In soft mode (qc_soft=.true.), a violation prints a WARNING — still at most once per column for the reader's whole lifetime (the same throttling parquet_read_column already uses), so reading many violating row groups in soft mode doesn't spam one warning per chunk.

Completeness checks: pass check_complete=.true. to parquet_close_reader to verify that every column you read via parquet_read_column_chunk had every one of the file's row groups read by the time you close — catches a loop that forgot a row group, or exited early by mistake:

call parquet_close_reader(reader, check_complete=.true.)

check_complete defaults to .false. (no check, so existing code is unaffected). When it's .true., check_hard (default .true.) picks the failure mode: error stop naming the column and its missing row group(s), or (check_hard=.false.) a WARNING instead. Only columns actually touched via parquet_read_column_chunk are tracked — a column read via parquet_read_column/parquet_read_array_row_mode/parquet_read_array_element_mode is never included in this check, even if the reader also chunk-read other columns.

Threading: parquet_read_column_chunk calls on the same parquet_reader are bound by the same "one thread at a time" rule as every other call into a shared reader (see Thread safety) — but since chunked reads are stateless/random-access, splitting the row-group loop itself across threads works cleanly as long as each thread uses its own parquet_reader instance opened on the same file (independent readers on the same file are always safe to use concurrently), rather than sharing one reader across threads.