parquet_open_reader Interface

public interface parquet_open_reader

Opens filename for reading into reader. use_threads (optional): use Arrow's multi-threaded reader. filter (optional): a parquet_filter row filter to apply. sample_fraction (optional, real(real64)): keeps each row independently with probability sample_fraction (Bernoulli sampling, not an exact row count) -- omitted, or >= 1.0, reads every row (the current/default behavior); must not be negative or NaN (error stops); exactly 0.0 deterministically yields zero rows. Shares its underlying mask with filter= (see "Row filtering with parquet_filter" in doc/pages/io/filter-sort-sample.md): a filter, if also given, is applied on top of the downsample, and sample_fraction < 1.0 alone (even with no filter=) carries the same consequences filter= already has -- chunked reads (parquet_read_column_chunk) are disallowed, and array row/element-mode reads fall back to a whole-column read. sample_seed (optional, integer(int32)): omitted or <= 0 draws a fresh seed from entropy (a different sample each call); > 0 makes the draw reproducible. The seed actually used (caller-supplied or entropy-drawn) is always reported by parquet_close_reader(..., print_stat=.true.), so a non-deterministic run's seed can be read back afterward and reused. schema (optional): a parquet_schema/qc-maml to validate columns against. qc (optional): enable qc: min/max/miss enforcement (needs schema). qc_soft (optional): qc violations warn instead of error-stopping. prefetch (optional, default .false.): when .true., every column in the file is read and cached right away, after any filter/sample has been applied, instead of each column being read lazily on first request -- equivalent to calling parquet_prefetch_columns for every column immediately after opening; materializes the whole file in memory up front, see doc/pages/operating/performance.md.

nrows= is generic over integer(int32)/integer(int64) (parquet_open_reader_nrows_int32/_int64: fills nrows with the post-filter/post-sample row count, error-stopping if it overflows the requested kind), plus the original nrows-less form (parquet_open_reader_base) for when nrows isn't wanted at all. This mirrors parquet_get_nrows's own int32/int64 overload above; nrows is required (not optional) in the two typed specifics -- an optional dummy that may be absent can't be the sole thing distinguishing two specific procedures in a generic interface (a call omitting it would be ambiguous), so parquet_open_reader_base carries the nrows-absent case as a separate specific instead. sample_fraction/sample_seed are each single-kind (real64/int32) rather than dual-kind like nrows, specifically to avoid that same ambiguity: a second independently-optional dual-kind argument on this generic would force a 3x3 cross product of specifics (absent/real32/real64 sample_fraction times the three nrows shapes) -- not worth it for a fraction/seed argument, which isn't the row-count/size/index category that convention exists for.


Module Procedures

private interface parquet_open_reader_base()

Arguments

None

private interface parquet_open_reader_nrows_int64()

Arguments

None

private interface parquet_open_reader_nrows_int32()

Arguments

None