parquet_settings_base – parquet-fortran

The settings an Arrow-free module owns: state, getter and setter.

The rule, in one sentence: a knob lives here when the Arrow-free module that reads it must be able to re-export it. parquet_settings keeps everything else, keeps the validation vocabulary it alone uses, and keeps the C++ mirror. A future Arrow-free module applies that sentence to its own knobs without revisiting anyone else's, which is what makes the arrangement extensible rather than a list someone has to maintain.

Why the SETTER lives here too, which is the part that is easy to get wrong. A user who imports one module to get one capability must be able to configure that capability from the same import -- use parquet_sorting has to offer the sorting knobs, or the only way to change them is use parquet_settings, which imports parquet_bindings and drags the whole Arrow stack back in. Re-exporting the getter alone would leave a user able to read a setting and not change it. Four of these setters used to mirror their value to C++ immediately, which is exactly what pinned them to parquet_settings; that push now happens at the point of USE instead (parquet_push_settings_to_cpp, called when a reader or writer is opened, and by the C++ sort engine's own dispatch), so no setter anywhere reaches C++ and every one was free to move.

This module began for one reason: to keep parquet_strings linkable on its own. parquet_strings is documented as usable without the Arrow/Parquet C++ stack -- a project that wants compact string storage and nothing else can depend on it alone. It needs two answers from the library's settings, though: whether solicited output is suppressed (verbosity), and the per-column thread cap. Taking those from parquet_settings directly cost it that independence, because parquet_settings imports parquet_bindings in order to mirror the C++-side knobs -- so linking a program whose only import was use parquet_strings pulled in the whole of parquet_wrapper.cpp and, with it, Arrow. Nothing failed at compile time; the link failed with the entire C++ surface undefined, which reads as a build misconfiguration rather than a dependency defect.

So the STATE lives here, in a leaf module that imports nothing but iso_fortran_env, and the public API over it stays in parquet_settings, which re-exports the two readers below.

This is a split, not a mirror, and the difference is the whole point. There is one copy of each value: parquet_settings' setters write these variables directly. A pushed second copy in parquet_strings would have been the other way to break the dependency and was rejected -- two writers for one value is exactly what CLAUDE.md's "Do not add a second way to set the same thing" forbids, and a missed push site would leave the two disagreeing with nothing to report it.

It also holds the library's one copy of the automatic THREAD-COUNT rule, for the same reason and by the same argument. parquet_sorting and parquet_random both have to answer "how many threads should this use when the caller said nothing", CLAUDE.md's auto-threading note names a further copy of that rule as the mistake to avoid, and parquet_random is a pure-Fortran counter-based generator that must not acquire a C++ dependency to ask it. parquet_sorting reaches parquet_bindings, so it cannot be the home either. This module can be, and the behaviour is unchanged: pf_sort_threads still reads cfg_sort_threads, still in one place, and now delegates the OpenMP half here.

Rules for anything added here. A knob belongs in this module only if an Arrow-free module reads it and therefore has to re-export it; everything else stays in parquet_settings, which remains where a reader looks for the settings API and where parquet_print_settings, parquet_reset_settings and parquet_settings_from_env live. The same test governs a procedure: it belongs here only if it is a rule two such modules must share. The scope is narrow on purpose -- moving a knob no Arrow-free module reads buys nothing and costs a re-export to keep in step. Whatever is added must keep this module a leaf -- it may import intrinsic modules and omp_lib (under #ifdef _OPENMP, as parquet_strings already does) and nothing else, ever. check_parquet_strings_stays_leaf (tools/check_source_conventions.py) enforces this by walking the use graph, because the failure it guards is invisible until someone tries the standalone build.

Uses

- iso_fortran_env

Variables

Type	Visibility	Attributes		Name		Initial
integer,	public,	parameter	::	verb_normal	=	0	Verbosity levels, ordered so that a `>=` test answers "is this class of output off?". everything prints (the factory default).
integer,	public,	parameter	::	verb_silent	=	1	informational and solicited output goes quiet.
integer,	public,	parameter	::	verb_errors_only	=	2	warnings go quiet too; only errors survive.
character(len=11),	public,	parameter	::	verbosity_tokens(3)	=	[character(len=11)::"normal", "silent", "errors_only"]	The accepted vocabulary of the two output knobs, in one place each. `parquet_settings_from_env` has to reject a bad token itself -- it cannot let the setter do it, because the setter's `error stop` cannot name the environment variable the value came from. Two validators means two chances to disagree about what is accepted, so both read these arrays and both build their "expected one of: ..." text with `token_list`.
character(len=6),	public,	parameter	::	stream_tokens(2)	=	[character(len=6)::"stdout", "stderr"]
integer,	public,	parameter	::	stream_stdout	=	0	Where the library's own messages go. `message_stream` accepts exactly these two, because a Fortran unit number means nothing on the C++ side of the bind(C) boundary, where three of the library's warnings and one of its reports are printed -- see doc/pages/operating/settings.md.
integer,	public,	parameter	::	stream_stderr	=	1
integer,	public,	save	::	cfg_verbosity	=	verb_normal	How much the library prints. Written by `parquet_set_verbosity`/`parquet_reset_settings` (parquet_settings.f90); read by the three emit channels there and by parquet_output_is_suppressed below, which is what the solicited printers ask.
integer,	public,	save	::	cfg_string_threads	=	0	Cap on the threads one `parquet_string_column` bulk operation may use internally. `0` means "auto" (as many as OpenMP offers). Written by `parquet_set_string_threads` (parquet_settings.f90); read only by `parquet_string_threads` (src/parquet_strings.f90), deliberately, for the same reason cfg_sort_threads has a single reader: one question asked in one place cannot give two answers (feature_risks.md Risk-40).
integer,	public,	save	::	cfg_random_threads	=	0	Cap on the threads one bulk `pf_random_permutation`/`pf_random_subset` call may use. `0` means "auto" (as many as OpenMP offers). Written by `parquet_set_random_threads` (parquet_settings.f90); read only by `random_threads` (src/parquet_random.f90), for the same single-reader reason as cfg_sort_threads and cfg_string_threads (feature_risks.md Risk-40).
integer(kind=int64),	public,	save	::	cfg_random_parallel_min_elements	=	1000_int64	Fewest elements a thread must be given before a bulk permutation opens a team at all. A work floor, not a chunk size, and it exists because threading a small permutation is monotonically harmful rather than merely useless: machine B measured `m = 10` going from 0.0021 ms on one core to 0.0050 on sixteen, and parallel efficiency at 1->16 threads of 99 % at `106`, 95 % at `104`, 69 % at 1000 and 18 % at 100. The default sits where that curve turns. Read only by `random_threads` (src/parquet_random.f90).
integer,	public,	save	::	cfg_sort_threads	=	0	Default thread count for every sort that does not name one. `0` means "auto", which is what pf_sort_threads resolves against the OpenMP environment -- and that is the ONLY place this is read, deliberately, so a read-time `sort_by=` and a raw-array sort can never disagree about it (feature_risks.md Risk-40).
integer,	public,	save	::	cfg_message_stream	=	stream_stdout	Which stream the library's own messages go to. Read only by the emit channels below.
logical,	public,	save	::	cfg_sort_counting_path	=	.true.	Whether the sort's integer counting fast path may be taken at all. Mirrored to C++ by `push_performance_settings` (parquet_settings.f90), which is where the boundary lives.
logical,	public,	save	::	cfg_sort_radix_path	=	.true.	Whether the sort's single-key radix fast path may be taken at all. Deliberately NOT mirrored to C++, unlike its counting-path neighbour: the radix path exists only in the Fortran engine, so there is nothing on the other side of the `bind(C)` boundary for a mirror to govern. Adding one would be a global that no code reads, which is the shape `feature_risks.md` Risk-42 warns about from the other end.
integer(kind=int64),	public,	save	::	cfg_sort_counting_bucket_limit	=	0	Largest key value RANGE (not cardinality) the counting path will accept. `0` = built-in.

Interfaces

public interface parquet_set_sort_counting_bucket_limit

Sets the largest key value range the sort's counting fast path will accept. See parquet_set_sort_counting_bucket_limit_int64 for the full description.

private subroutine parquet_set_sort_counting_bucket_limit_int32(n)

int32 form of parquet_set_sort_counting_bucket_limit_int64 -- see it for what the value means.

Arguments

Type	Intent	Optional	Attributes		Name
integer(kind=int32),	intent(in)			::	n	bucket ceiling, or 0 for the built-in default; must be >= 0.

private subroutine parquet_set_sort_counting_bucket_limit_int64(n)

Sets the largest key value RANGE for which the sort's counting fast path is taken. Pass 0 to restore the built-in 4194304 (2**22).

Range, not cardinality -- the bound is max(key) - min(key), so a thousand values spread over a billion is far outside a limit that a million densely-packed values sit inside. This distinction has already misled one test author here (feature_risks.md Risk-39).

The number IS the memory control: n buckets costs 8n bytes of counters, so the built-in value caps the counting path at 32 MB. Raising it trades memory for speed on wide-ranged integer keys; it does nothing at all while parquet_set_sort_counting_path is .false..

Available in both integer kinds; an int64 key's range can exceed int32.

Arguments

Type	Intent	Optional	Attributes		Name
integer(kind=int64),	intent(in)			::	n	bucket ceiling, or 0 for the built-in default; must be >= 0.

public interface parquet_set_random_parallel_min_elements

Sets the work floor, in elements per thread, below which a bulk permutation stays serial. See parquet_set_random_parallel_min_elements_int64 for the full description.

private subroutine parquet_set_random_parallel_min_elements_int32(n)

int32 form of parquet_set_random_parallel_min_elements_int64 -- see it for what it means.

Arguments

Type	Intent	Optional	Attributes		Name
integer(kind=int32),	intent(in)			::	n	elements per thread, or 0 to disable; must be >= 0.

private subroutine parquet_set_random_parallel_min_elements_int64(n)

Sets the fewest elements a thread must be given before a bulk permutation opens a team.

A work floor, not a chunk size. Threading a small permutation is not merely useless but harmful -- the team costs more than the whole job -- so below threads * this elements the bulk forms run serially however many threads are available. Machine B measured m = 10 going from 0.0021 ms on one core to 0.0050 on sixteen, and 1->16 thread efficiency of 99 % at 10**6, 95 % at 10**4, 69 % at 1000 and 18 % at 100; the default of 1000 sits where that curve turns.

Read per call, so it takes effect immediately. n takes integer(int32) or integer(int64). 0 disables the floor entirely, which is how a test asks for a team on a small array; it is not a useful production setting. An explicit threads= does not bypass the floor -- the floor is about whether the work is worth splitting at all, which is a property of the array rather than of the caller's intent.

Arguments

Type	Intent	Optional	Attributes		Name
integer(kind=int64),	intent(in)			::	n	elements per thread, or 0 to disable; must be >= 0.

Functions

public function parquet_output_is_suppressed() result(quiet)

Whether SOLICITED output -- something the caller explicitly asked to be printed, such as %print_stat or parquet_string_column%print -- should stay quiet. Distinct from the emit channels, which govern the library's own unsolicited messages.

Arguments

None

Return Value logical

public function parquet_get_string_threads() result(n)

The configured cap on threads inside one string-column bulk operation; 0 means automatic.

Arguments

None

Return Value integer

public function parquet_get_random_threads() result(n)

The configured cap on threads inside one bulk permutation/subset; 0 means automatic.

Arguments

None

Return Value integer

public function parquet_get_random_parallel_min_elements() result(n)

The configured work floor, in elements per thread, for a bulk permutation/subset.

Arguments

None

Return Value integer(kind=int64)

public function parquet_auto_thread_count(cap, area) result(n)

The library's one copy of the automatic thread rule: how many threads an operation that was given no explicit threads= should use right now, under a caller-supplied cap.

Type	Intent	Optional	Attributes		Name
integer,	intent(in)			::	cap	caller's domain cap; `<= 0` means no cap
character(len=*),	intent(in)			::	area	subsystem name, for the affinity-clamp warning

Type	Intent	Optional	Attributes		Name
integer,	intent(in)			::	n	threads resolved before the clamp.
character(len=*),	intent(in)			::	area	subsystem this count belongs to, for the message.

Type	Intent	Optional	Attributes		Name
character(len=*),	intent(in)			::	tokens(:)	the accepted vocabulary.
character(len=:),	intent(out),		allocatable	::	out	comma-separated, in array order.

Type	Intent	Optional	Attributes		Name
character(len=*),	intent(in)			::	text	input text.
character(len=:),	intent(out),		allocatable	::	out	text with every ASCII A-Z lowercased.

parquet_settings_base Module

Contents

Variables

Interfaces

Functions

Subroutines

Uses

Variables

Interfaces

public interface parquet_set_sort_counting_bucket_limit

private subroutine parquet_set_sort_counting_bucket_limit_int32(n)

Arguments

private subroutine parquet_set_sort_counting_bucket_limit_int64(n)

Arguments

public interface parquet_set_random_parallel_min_elements

private subroutine parquet_set_random_parallel_min_elements_int32(n)

Arguments

private subroutine parquet_set_random_parallel_min_elements_int64(n)

Arguments

Functions

public function parquet_output_is_suppressed() result(quiet)

Arguments

Return Value logical

public function parquet_get_string_threads() result(n)

Arguments

Return Value integer

public function parquet_get_random_threads() result(n)

Arguments

Return Value integer

public function parquet_get_random_parallel_min_elements() result(n)

Arguments

Return Value integer(kind=int64)

public function parquet_auto_thread_count(cap, area) result(n)

Arguments

Return Value integer

public function parquet_clamp_to_affinity(n, area) result(m)

Arguments

Return Value integer

public function parquet_nested_team_unsafe() result(unsafe)

Arguments

Return Value logical

public function parquet_get_sort_threads() result(n)

Arguments

Return Value integer

public function parquet_get_sort_counting_path() result(enabled)

Arguments

Return Value logical

public function parquet_get_sort_radix_path() result(enabled)

Arguments

Return Value logical

public function parquet_get_sort_counting_bucket_limit() result(n)

Arguments

Return Value integer(kind=int64)

Subroutines

public subroutine parquet_debug_set_affinity_procs(n)

Arguments

public subroutine parquet_debug_reset_affinity_warning()

Arguments

public subroutine parquet_set_sort_threads(n)

Arguments

public subroutine parquet_set_string_threads(n)

Arguments

public subroutine parquet_set_random_threads(n)

Arguments

public subroutine parquet_set_sort_counting_path(enabled)

Arguments

public subroutine parquet_set_sort_radix_path(enabled)

Arguments

public subroutine parquet_set_verbosity(level)

Arguments

public subroutine parquet_get_verbosity(level)

Arguments

public subroutine parquet_set_message_stream(stream)

Arguments

public subroutine parquet_get_message_stream(stream)

Arguments

public subroutine parquet_emit_info(text)

Arguments

public subroutine parquet_emit_warning(text)

Arguments