Process-global settings for parquet-fortran: the parameters that apply to the whole library rather than to one reader, writer or table, collected in a single place a program can read and change.
What is a setting here, and what is not. A setting may change how fast, how large or how
loud the library runs; it may never change what the library answers. A knob that would alter a
returned value, an ordering or a nullness is deliberately absent and will stay absent -- a
program-wide default for something like "where do nulls sort" would make the same call return
different results in different programs, with nothing at the call site to hint at it. Anything
that can be expressed as an argument to a specific call (a writer's compression=, a sort's
threads=) belongs there instead, and an explicit argument always wins over a setting.
Thread safety: set once, at startup. Settings are written during program initialisation, before other threads exist and before any reader/writer/table is opened. Reads are unsynchronised and a concurrent write is a data race, which the library does not defend against -- resizing a thread pool that other threads are using at that moment is a race regardless of what this module does. In exchange, reading a setting costs nothing on any hot path.
Capture points differ per setting, and each one says which it is in its own doc-comment.
parquet_set_arrow_threads resizes a pool everyone already shares, so it takes effect immediately
for objects opened before the call as well as after.
The read-only limits below (parquet_max_*) are the caps the library enforces on filter rules,
sort keys and MAML lines. They are published so that code building any of those from user or
configuration input can check a length before tripping an error stop, and they are constants
rather than settings because loosening them would convert a guard against runaway input into a
way to overflow the parser's own stack.
User guide: doc/pages/operating/settings.md.
The three output channels, and the ONLY places verbosity/message_stream are read. Public
here so every module that emits can reach them, private :: in the facade so no user sees
them; see parquet_valid_compressions below for the same mechanism and the same reason.
Library-internal plumbing, kept out of the use parquet namespace by an explicit
private :: in the facade (src/parquet.f90) -- the same mechanism that hides c_int and the
version bindings there. They are public here only because Fortran has no package scope and
the write path lives in another module.
| Type | Visibility | Attributes | Name | Initial | |||
|---|---|---|---|---|---|---|---|
| integer, | public, | parameter | :: | parquet_max_filter_rule_len | = | 8192 |
Maximum number of characters in one |
| integer, | public, | parameter | :: | parquet_max_filter_depth | = | 32 |
Maximum parenthesis/ |
| integer, | public, | parameter | :: | parquet_max_filter_nodes | = | 1024 |
Maximum number of expression nodes across every |
| integer, | public, | parameter | :: | parquet_max_sort_keys | = | 16 |
Maximum number of keys across every |
| integer, | public, | parameter | :: | parquet_max_sort_key_len | = | 320 |
Maximum number of characters in one |
| integer, | public, | parameter | :: | parquet_max_maml_line_len | = | 1024 |
Maximum number of characters in one line of a MAML source file. A longer line is reported as
an |
| character(len=12), | public, | parameter | :: | parquet_valid_compressions(6) | = | [character(len=12)::"uncompressed", "snappy", "gzip", "zstd", "brotli", "lz4"] |
The compression codecs parquet_open_writer accepts, and the single list both it and parquet_set_default_compression validate against -- two copies would let a codec be settable as a default but rejected as an argument, or the reverse. |
Sets the largest key value range the sort's counting fast path will accept. See parquet_set_sort_counting_bucket_limit_int64 for the full description. Sets the work floor, in elements per thread, below which a bulk permutation stays serial. See parquet_set_random_parallel_min_elements_int64 for the full description. Sets the byte size an auto-sized row group aims for. See parquet_set_target_row_group_bytes_int64 for the full description.
int32 form of parquet_set_target_row_group_bytes_int64 -- see it for what the value means.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer(kind=int32), | intent(in) | :: | n |
byte target, or 0 for the built-in default; must be >= 0. |
Sets the size in BYTES an auto-sized row group aims for. Pass 0 to restore the built-in
268435456 (256 MiB).
Applies only when a writer is opened without an explicit chunk_size=; a caller-chosen row
count is never overridden. Sizing by bytes rather than by a flat row count is what makes a
table of narrow int32 columns and a table of wide vector columns produce row groups of
comparable size, which is what Parquet's own row-group guidance (roughly 128 MB to 1 GB) is
about and what drives per-row-group compression efficiency and decode cost.
Three bounds the library applies afterwards are NOT settable: a floor of 1000 rows, a ceiling of 10,000,000 rows, and the int32 element-count ceiling a vector column imposes. A target so small that even the floor would overshoot it fourfold abandons the floor rather than the target, down to a single row per row group.
Available in both integer kinds; a byte target can exceed int32.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer(kind=int64), | intent(in) | :: | n |
byte target, or 0 for the built-in default; must be >= 0. |
Reports Arrow's current global CPU thread-pool capacity -- what parquet_set_arrow_threads last set it to, or Arrow's own hardware-derived default if it was never set. The counterpart to parquet_set_arrow_threads, and the answer to "how many threads will Arrow actually use here".
Reports the prefetch thread cap, or 0 if left automatic.
Reports the table-mutation thread cap, or 0 if left automatic.
Reports the compression level applied to a writer opened with no compression arguments at
all -- the value set here, or 3 (this library's level for its own default zstd) if it was
never set. Naming a codec explicitly, by argument or by
parquet_set_default_compression, does not attach this level to it; see that procedure.
Reports the default for use_threads= on a newly opened reader or writer.
Reports the row-group byte target -- the EFFECTIVE value, so a program that never set it is
told 268435456 rather than the 0 that is stored.
Reports whether the reader's row-group statistics screen is enabled.
Resizes Arrow's global CPU thread pool -- the single pool shared by every parquet_reader/parquet_writer in this process that has use_threads enabled (the default). This is NOT a per-reader/per-writer setting: call it once, e.g. near the start of your program, before opening readers/writers on other threads -- calling it concurrently from multiple threads with different values is a race, since it resizes a pool everyone else is also using at that moment.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer, | intent(in) | :: | n |
new thread-pool capacity; must be >= 1. |
Returns the version of the Arrow/Parquet C++ libraries this program is actually built against, formatted "major.minor.patch". mode absent or mode="arrow" reports the linked Arrow library's RUNTIME version -- what is loaded now, which need not be what the wrapper was compiled against; mode="parquet" reports the Parquet C++ library version the wrapper was COMPILED against. Any other mode value is an error.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| character(len=:), | intent(out), | allocatable | :: | ver_string |
resulting version string. |
|
| character(len=*), | intent(in), | optional | :: | mode |
"arrow" | "parquet"; absent = "arrow". |
Sets the cap on how many threads parquet_table%prefetch/%materialize_all may use to read
several columns at once, each on its own reader.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer, | intent(in) | :: | n |
thread cap, or 0 for automatic; must be >= 0. |
Sets the cap on how many threads a parquet_table's row-structural mutation may use to
rewrite its columns concurrently -- %sort_by, %filter_rows, %top_n, and %delete_rows
and %truncate, which go through the same loop.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer, | intent(in) | :: | n |
thread cap, or 0 for automatic; must be >= 0. |
Sets all six thread counts at once: Arrow's pool and the five per-area caps -- sorting, the table prefetch, the table mutation, one string column's bulk work and the bulk random draws.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer, | intent(in) | :: | n |
thread count for all six; must be >= 1. |
Sets the compression codec parquet_open_writer uses when the caller passes no
compression=. One of "uncompressed", "snappy", "gzip", "zstd", "brotli", "lz4"
(case-insensitive); anything else aborts, using the same list the writer's own argument is
checked against.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| character(len=*), | intent(in) | :: | name |
codec name, case-insensitive. |
Reports the codec a writer opened with no compression= would use -- the value set here, or
"zstd" if it was never set.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| character(len=:), | intent(out), | allocatable | :: | name |
effective default codec. |
Pins the DATE file-metadata key to a fixed value, so that writing the same data twice
produces BYTE-IDENTICAL files. Blank (the factory default) restores reading the clock.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| character(len=*), | intent(in) | :: | text |
"YYYY-MM-DDTHH:MM:SS", or "" to read the clock. |
Reports the pinned file date, or "" when the clock is being read (the factory default).
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| character(len=:), | intent(out), | allocatable | :: | text |
the pinned date, or "" for the clock. |
Sets the compression level parquet_open_writer uses when the caller passes no
compression_level=. Captured at writer open.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer, | intent(in) | :: | n |
compression level, interpreted by the chosen codec. |
Sets the default for parquet_open_writer/parquet_open_reader's use_threads=, i.e.
whether Arrow's own internal thread pool is used for a reader's or writer's column work.
Captured at open; an explicit use_threads= still wins.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| logical, | intent(in) | :: | flag |
.true. to use Arrow's thread pool (the factory default). |
Resolves a writer's codec and compression level from the caller's optional arguments and the
process-global defaults. Library-internal plumbing -- the facade keeps it out of the
use parquet namespace.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| character(len=*), | intent(in), | optional | :: | compression |
the caller's compression= argument. |
|
| integer, | intent(in), | optional | :: | compression_level |
the caller's compression_level= argument. |
|
| character(len=:), | intent(out), | allocatable | :: | codec |
resolved codec name, lowercased. |
|
| integer, | intent(out) | :: | level |
resolved level, or the codec-default sentinel. |
Enables or disables the reader's row-group statistics screen.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| logical, | intent(in) | :: | enabled |
.true. (the default) lets the reader prune row groups. |
Mirrors both output settings to the C++ side, which prints three warnings and one report of its own and cannot see Fortran module variables.
Applies every PARQUET_FORTRAN_* environment variable that is set, through the knob's own
setter.
Writes every setting's current value, and every read-only limit, to unit.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer, | intent(in), | optional | :: | unit |
output unit (default output_unit). |