A whole table: a column store plus the row scope and provenance describing it.
Declared by the caller (type(parquet_table) :: t), filled by parquet_open_table or
parquet_new_table, and freed automatically when it goes out of scope.
Frees the store; never fails, never validates.
FINAL procedure: frees the column store and abandons the reader. Runs at scope exit and on an intent(out) reopen, so it must always succeed silently -- it validates nothing. Module-contained on purpose; see the note in the generator that emits it.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| type(parquet_table), | intent(inout) | :: | self |
the table being destroyed. |
Number of rows every column holds.
Number of rows every column of this table holds.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
row count.
Number of columns the table has.
Number of columns this table has, or only the resident ones with resident_only.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| logical, | intent(in), | optional | :: | resident_only |
.true.: count only columns already read. |
column count.
Copy out every column name, in order.
Copies out every column's name, in file/insertion order, blank-padded to the longest.
resident_only=.true. reports only the columns that have been read, in the same order,
which is how a caller finds out what a lazy table is actually holding.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| character(len=:), | intent(out), | allocatable | :: | names(:) |
one entry per column. |
|
| logical, | intent(in), | optional | :: | resident_only |
.true.: only columns already read. |
A column's 1-based position, 0 when absent.
A column's 1-based position among the table's columns, or 0 when there is no such
column. The inverse of %column_name, and the cheap way to hoist a lookup out of a
loop that then queries the same column by position.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | name |
column name. |
||
| logical, | intent(out), | optional | :: | found |
present: report a miss instead of aborting. |
1-based position, or 0 on a reported miss.
Copy out the name at a 1-based position.
Copies out the name of the column at 1-based position j. The inverse of
%column_index, and what makes a do j = 1, t%ncols() sweep able to report itself.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| integer, | intent(in) | :: | j |
1-based column position. |
||
| character(len=:), | intent(out), | allocatable | :: | nm |
the column's name, or "". |
|
| logical, | intent(out), | optional | :: | found |
present: report an out-of-range j instead of aborting. |
A resolved handle on one column, by name or by 1-based position. Resolves once -- the
lookup, the lazy first touch and the kind -- so a per-element loop over that column
stops paying for a name lookup on every access. See parquet_table_col.
Whether a column holds (or may hold) nulls -- named, or by 1-based position.
Copy out a column's validity as a plain logical array. A rank-1 mask gives one entry
per row (on a *_VEC column: "any element of the row is null"); a rank-2 mask gives the
true (width, nrows) per-element state.
Counter bumped by every structural change.
A counter bumped by every structural change to this table, for a caller holding a pointer across a call that might have invalidated it.
A stale %col pointer cannot be detected by Fortran, and this library cannot detect
it either. What it can do is tell you whether anything structural happened: take the
generation before, compare it after, and re-fetch the pointer if it moved. The counter
is deliberately conservative -- every column- and row-structural entry point bumps it,
whether or not that particular call actually relocated anything -- because a missed bump
would give false confidence, while a spare one only costs a re-fetch. A call that
changes nothing at all (see the no-detach rule) does not bump it.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
current generation.
Whether a column of this name exists.
Whether a column of this name exists (supported or not).
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | name |
column name. |
.true. if the table has it.
Which of these columns the table does NOT have, as a packed array (zero-size when it has them all). The non-aborting half of %require_columns.
Aborts unless the table has every one of these columns, naming EVERY missing one -- not just the first, which is what a hand-written loop reports.
A column's PK_* kind discriminator -- named, or by 1-based position.
A column's values-per-row (1 if scalar) -- named, or by 1-based position.
Copy out a column's unit string -- named, or by 1-based position.
A column's RES_* residency state -- named, or by 1-based position.
Whether row i of a column is null, or -- given e as well -- element e of it.
On a *_VEC column the row form answers "ANY element of the row is null"; the element
form answers about that one element. Defined on a scalar column too, where e can only
be 1 and the two agree.
Whether the table has left its file behind.
Whether the table has been detached from its file by a row-structural mutation.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
.true. once detached.
Whether a column's type can be read -- named, or by 1-based position.
Copy out the file this table came from.
Copies out the file this table was opened from ("" for an in-memory table).
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| character(len=:), | intent(out), | allocatable | :: | fname |
the file name, or "". |
One key from the file's metadata.
Reads one key from the source file's table metadata. found reports a missing key.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | key |
metadata key. |
||
| character(len=:), | intent(out), | allocatable | :: | value |
the value, or "". |
|
| logical, | intent(out), | optional | :: | found |
present: report a miss instead of aborting. |
Reads the named column(s) now, instead of on first touch. Required before a parallel region: a first touch inside one is a hard error, since it would mutate shared state.
The same call as %prefetch, under the name that pairs with %materialize_all. Reaching for the definitive-sounding %materialize_all when only a few columns are wanted reads the whole file, silently; %materialize(names) is the one to find first.
Read every column not yet read.
Reads every supported, file-backed column that is not resident yet -- the one-call way to make a whole table safe to use from a parallel region.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table (fills through %cache). |
Re-read one column; force= to discard local edits.
Re-reads one column from the file, discarding whatever is in the store -- the escape hatch back to the file's own values after %set has changed them locally. Only valid for a file-backed column of a table that has not been detached.
Discarding local edits is what this is FOR, so it says so rather than assuming: a column holding values the caller wrote (%user_populated) is refused unless force=.true. is passed, which puts %reload and %evict_column under one rule instead of two. A forced reload clears the mark, since the slot then holds the file's values again.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table (refills through %cache). |
||
| character(len=*), | intent(in) | :: | name |
column to re-read. |
||
| logical, | intent(in), | optional | :: | force |
.true. to discard local edits and re-read. |
|
| logical, | intent(out), | optional | :: | found |
present: report a miss instead of aborting. |
Drop a column's VALUES; force= if it holds local edits.
Releases one column's VALUES while keeping its slot -- the honest counterpart of %prefetch, and the way to give a column's memory back without losing the column.
The difference from %drop_column is what survives: an evicted column still appears in %column_names, still answers %kind/%width/%unit, and is READ AGAIN on the next touch. A dropped one is gone. So %drop_column is for a column you are finished with, and %evict_column for one you are finished with FOR NOW.
Only a file-backed column of an attached table can be evicted, and that restriction is the whole safety story: everywhere else the values are the only copy that exists, so evicting them would be silent data loss rather than a memory saving. A column built with %add_column, and any column of a detached table, is therefore an error naming what is wrong. Evicting a column that is not resident is a no-op.
Eviction is user-driven only. Nothing in this library evicts on its own -- no LRU, no memory budget -- so what a table holds stays predictable from the calls you wrote.
A column holding values the CALLER wrote (%user_populated) is refused as well, because the file's own values would come back on the next read and the edits would be gone with nothing to notice -- pass force=.true. to discard them on purpose. Note the protection covers what the value-setting API wrote, NOT a write made through a %col/%ref pointer: the library cannot tell such a write from a read, so a caller who edits that way marks the column with %set_user_populated themselves.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | name |
column to release. |
||
| logical, | intent(in), | optional | :: | force |
.true. to evict a column holding local edits. |
|
| logical, | intent(out), | optional | :: | found |
present: report a miss instead of aborting. |
Claim a column's values as the caller's own, or unclaim.
Marks a column as holding values the CALLER wrote, or clears that mark.
The library sets this itself for every value-setting call (%set, %set_element, %set_slice, a row or column handle's %set, %set_null, %add_column, ...), and %evict_column and %reload then refuse that column unless force=.true. is passed. It cannot set it for a write made through the pointer %col/%ref hands out, because it cannot tell such a write from a read -- so a caller who edits a column that way marks it here, and gets the same protection. Clearing it says the opposite: the slot's values are the file's again, and may be discarded without force=.
Marking a column that holds no values is refused: there is nothing to protect, and the mark would outlive the read that filled the slot. Clearing is always allowed.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | name |
column to mark. |
||
| logical, | intent(in) | :: | flag |
.true. = the caller's own values; .false. = the file's. |
||
| logical, | intent(out), | optional | :: | found |
present: report a miss instead of aborting. |
Whether a column is claimed as holding the caller's values.
Whether a column is marked as holding values the caller wrote rather than the file's.
Reads no values and never triggers a read. A column that has not been read is never marked, so this answers .false. for one -- see %set_user_populated for what sets it.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | name |
column to ask about. |
||
| logical, | intent(out), | optional | :: | found |
present: report a miss instead of aborting. |
.true. when the slot holds the caller's own writes.
Check every qc-declaring column, holding none.
Checks this table's read-time qc against the file, WITHOUT leaving the columns resident.
qc is enforced when a column is read, so on a lazy table a declared bound is only checked once something asks for that column -- which means a program that reads two of forty columns never finds out whether the other thirty-eight satisfy their bounds. This reads exactly the columns that declare a bound, letting the reader check them, and then releases the ones it had to read.
What it leaves behind is the point. Residency is recorded BEFORE anything is read, and only the columns this call made resident are released afterwards -- a column that was already in memory stays there, values and all. So it can be called at any time without disturbing what the program is working on.
A violation is reported the way it would be on an ordinary read: an abort, or a warning
under qc_soft=. A table with no qc declared at all, and one built in memory, are
no-ops -- there is nothing to check and no file to check it against.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
Print what the table holds, to stdout.
Prints what this table holds, to standard output: one header line and one line per column, with each column's kind, width, row count, null count and min/max.
Materialized columns only, by default -- the header says how many of the table's
columns those are, so a lazy table reports what it is actually holding rather than what
its file contains. all=.true. lists every column, with - where a column that has
not been read has nothing to report.
Printing never reads anything. A deferred plain-LIST column, whose width is only
knowable from its data, prints as pending rather than being resolved -- a diagnostic
that changes what it is diagnosing is worse than one that admits it does not know.
These are statistics of the values IN MEMORY, computed here by a plain Fortran scan. They are not the file's own footer statistics, and they are the only ones available for a column built with %add_column, which has no footer at all. The scan is O(rows) per column, so this is not a call to put in a loop over a large table.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| logical, | intent(in), | optional | :: | all |
.true.: list every column, not just the resident ones. |
Rows before filter=/sample_fraction=.
Rows this table covers BEFORE filter=/sample_fraction= removed any -- the slice's
own length in the slice regime, and the file's row count for a whole-file table.
%nrows() is what the table holds; this is what it was cut from, which nothing else
can report once a transform is active: a filtered reader answers in survivors, so
asking it afterwards gives the same number %nrows() already gave. Captured when the
table opens, so it keeps answering after a row mutation has detached the table.
Equal to %nrows() when no filter or sample is in play. 0 for a table built in memory,
which was not cut from anything.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
rows before the transform.
Rows in the row groups this table covers.
Rows in the row groups this table covers, which is what reading it actually costs.
A slice is read by row group, so a slice of 10 rows straddling two 100k-row groups decodes 200k rows to produce them. This reports that number, so a caller choosing slice boundaries can see when a slice is not paying for itself.
Equal to %nrows_unfiltered() when the slice lines up with row-group boundaries, and
for a whole-file table always. 0 for a table built in memory.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
rows in the covering row groups.
Row-group row ranges, this table's rows or the file's.
The same row ranges for an already-open table, without reopening the file, in THIS
TABLE's own row numbering -- or, with physical=.true., in the file's.
The two differ only when the table does not hold every row of the file: a slice, a
filter, a sample. Then bounds(1, rg)/bounds(2, rg) are the rows OF THIS TABLE that
came from row group rg, which is what relates a row index in hand to the row group it
was read from; physical=.true. answers in the file's numbering instead, which is what
parquet_table_row_group_bounds and every slice bound are expressed in, and so is what
to use for planning the next slice.
Both forms have ONE ENTRY PER PHYSICAL ROW GROUP and are index-aligned, so the two can be
read side by side ("row group 7 holds file rows A..B, which are my rows C..D"). A row
group contributing no rows to this table -- outside the slice, or filtered away entirely
-- is reported as an EMPTY range, bounds(1, rg) > bounds(2, rg), rather than dropped;
dropping it would break the alignment that makes the pairing possible.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| integer(kind=int64), | intent(out), | allocatable | :: | bounds(:,:) |
(2, num_row_groups). |
|
| logical, | intent(in), | optional | :: | physical |
.true. for the file's own row numbering. |
A handle on one row, for code that works a row at a time rather than a column at a time. The index is 1-based within THIS table -- in the slice regime, row 1 is the slice's first row, not the file's.
Copies the rows a parquet_slice selects into a freshly allocated array of
the caller's own kind, widening on the way exactly as %get does.
The selection is taken in the order given, duplicates included. A
parquet_slice_list may name a row more than once and may name rows in any
order, so the result has one entry per SELECTION -- not per distinct row -- and
its order is the selection's, never the table's.
Writes values into the rows a parquet_slice selects -- %get_slice's counterpart.
The kind must match the column's exactly, and the array must have one value per
selected row.
The selection is written in the order given, duplicates included, so a row
named twice ends up holding the LAST value written to it. Selections are
applied one after another rather than merged or de-duplicated, which is what
makes %set_slice the exact inverse of %get_slice for a selection that names
each row once.
Points p at a column's storage: zero copy, writable, and the pointer kind must
match the stored kind exactly (ask %kind first if you do not know it). A
parquet_string_column pointer aliases a PK_STRING column's packed store: read
it and edit its values in place, but do NOT change its length or element count
through the pointer -- the column's own row count would no longer describe it.
For the same reason, do NOT REORDER a column through this pointer -- in particular
call pf_permute(p, perm), which compiles and runs happily. It reorders that one
column and leaves every other column where it was, silently breaking the row
correspondence, and nothing detects it: the row count is unchanged and every later
read returns values that are individually valid and jointly wrong. Use %sort_by,
which reorders every column together.
Copies a column into a freshly allocated array of the caller's own kind.
Reads one row's value out of a column, widening into the caller's variable
exactly as %get does -- the one-call form of r = t%row(i) then r%get(name, v).
On a *_VEC column the value is that row's whole vector.
Writes one row's value in place. The kind must match the column's exactly (as %set does), and writing a value CLEARS that row's null -- use %set_null to put one back. On a *_VEC column the value is that row's whole vector.
Marks null: row i of a column, element e of row i, or every entry a logical
mask marks .false..
The row form is whole-row even on a *_VEC column -- naming only a row says the row is
missing. Name e to null one element. The mask form takes either shape: one entry per
row (whole rows), or a (width, nrows) mask (individual elements).
Marks row i -- or, given e, element e of it -- valid without saying what its value
is. Only useful when a value is already there or is about to be written; %set_element
clears the null itself.
Drop a null bitmap that no longer has nulls.
Drops a column's null bitmap when it no longer holds any null, so a column that HAD nulls and no longer does stops paying for the bitmap. Scans the column, so it is not free -- a whole-column %set already compacts on its own and does not need this. Idempotent: calling it on an already-compact column is a cheap no-op.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | name |
column name. |
||
| logical, | intent(out), | optional | :: | found |
present: report a miss instead of aborting. |
Allocate validity storage up front, for concurrent nulling.
Materializes a column's validity storage up front, so concurrent nulling allocates
nothing. See table_check_shared_write.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| character(len=*), | intent(in), | optional | :: | name |
column to prepare; absent = every resident one. |
|
| logical, | intent(out), | optional | :: | found |
.false. if |
Remove a column; force= for a predefined one.
Removes a column from the table. Cheap, and it does NOT detach: dropping a column leaves every remaining column the same length, so the table can still read the ones it has not read yet. Dropping a column that was never read is the memory-reclaiming case and reads nothing.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | name |
column to remove. |
||
| logical, | intent(in), | optional | :: | force |
.true. to drop a PREDEFINED column. |
|
| logical, | intent(out), | optional | :: | found |
present: report a miss instead of aborting. |
Change the name a column is looked up by.
Changes a column's name. Only the name a caller looks it up by changes -- a
file-backed column that has not been read yet still reads from the same physical
column afterwards. A predefined column cannot be renamed at all (its accessor is bound
to the name at compile time), and there is no force= for it.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | old_name |
the column to rename. |
||
| character(len=*), | intent(in) | :: | new_name |
its new name; must not already exist. |
||
| logical, | intent(out), | optional | :: | found |
present: report a missing SOURCE column instead of aborting. |
Add a copy of a column, optionally of another kind.
Adds a NEW column holding a copy of name's values, leaving the source column
untouched. With to_kind absent it is a plain deep copy and works for EVERY kind the
library can read -- string, temporal, logical and the vector kinds included. With
to_kind given it copies and converts, and the conversion rules are %cast's.
exact defaults to .true. here, the opposite of %cast: a copy is usually taken to
keep something, so a value that would not survive the round trip is an error naming the
row and the value rather than a silent truncation. Every value is checked before
anything is written, so a rejected copy leaves the table exactly as it was. The unit
carries over unchanged -- a kind conversion is not a unit change.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | name |
the source column. |
||
| character(len=*), | intent(in) | :: | new_name |
the column to create. |
||
| integer, | intent(in), | optional | :: | to_kind |
target PK_* kind; absent keeps the source's. |
|
| logical, | intent(in), | optional | :: | exact |
.false. to allow lossy narrowing (default .true.). |
|
| logical, | intent(out), | optional | :: | found |
present: report a missing SOURCE column instead of aborting. |
Convert a column to another kind, in place.
Converts a column to to_kind IN PLACE, so that %col can be called with a kind the
calling code has decided on rather than the one the file happens to hold. A column
already of to_kind is left alone.
The conversions allowed are exactly those the reader and writer already perform between numeric kinds: int32 <-> int64, float32 <-> float64, and either integer kind to or from either real kind, scalar or vector, never changing a column's width. An integer overflow, and a real value with a fractional part converted to an integer kind, are errors naming the row and the value. Anything else -- logical, string, temporal, or a conversion that would change a column's rank -- is refused outright.
exact defaults to .false.: precision loss (real64 to real32, or a large integer to
a real kind) is silent, exactly as it is on the read path. Pass .true. to make any
value that would not survive the round trip an error instead.
This invalidates any pointer previously taken from %col for this column, which
Fortran cannot detect -- take the pointer again afterwards.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
||
| character(len=*), | intent(in) | :: | name |
the column to convert. |
||
| integer, | intent(in) | :: | to_kind |
target PK_* kind. |
||
| logical, | intent(in), | optional | :: | exact |
.true. to refuse any precision loss. |
|
| logical, | intent(out), | optional | :: | found |
present: report a miss instead of aborting. |
Keep only the rows a mask selects.
Keeps only the rows whose keep entry is .true., dropping the rest from EVERY column.
Row-structural, so it DETACHES the table from its file: after it, a column that was
never read can never be read, because the file's rows no longer line up with the rows
in memory. Materialize what you need first (%prefetch/%materialize_all). An
all-.true. mask removes no row, so it changes nothing and does not detach.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
||
| logical, | intent(in) | :: | keep(:) |
one entry per row; .true. to retain it. |
Reorders rows by one or more key columns. Detaching.
Keeps only the n best rows, in key order. Detaching.
The row order the keys imply, without reordering anything. Unlike %sort_by the table stays attached, so this is how to read rows in an order while keeping the file.
The n best rows in order, by selection rather than a full sort. Also non-mutating.
Whether the rows are already in that order.
Removes the listed rows. A thin convenience over %filter_rows, and like it, detaching.
Keeps only the first n rows. Detaching, like every row-structural change.
Appends rows: a whole table's worth, or one row. Detaching, like every row-structural change. The bulk idiom is %clone_structure -> fill -> %append(batch).
Appends n all-null rows, to be filled in afterwards. Detaching.
Release capacity appends left behind.
Releases the spare storage capacity that appending left behind, so every resident column occupies exactly the rows it holds.
A no-op on a table that has not been appended to. Capacity only ever comes from an
append: reading a column from a file allocates exact-fit, and so does every rebuild
(%filter_rows, %sort_by, %top_n, %delete_rows, %truncate), which hand their
memory back on their own. So this is safe to call unconditionally -- before
parquet_write_table, say -- and costs nothing when there is nothing to release. It is
not NEEDED before a write: the writer reads through a pointer that is already bounded by
the row count, so slack costs a write nothing.
It INVALIDATES every pointer obtained from %col and every row handle, because it
reallocates storage. That makes it the one procedure in parquet_tables_mutate that
does so -- everything else there leaves the row set and the storage alone. It does NOT
detach: the row set is unchanged, so the table keeps its file and a column not yet read
can still be read afterwards.
%generation() advances only if something was actually released, which is what
makes the no-op above observable: a caller following the documented
take-generation/compare/re-fetch pattern re-fetches only when a pointer really did die.
Refused on a shared table, like every other mutation except %append.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
Makes room for n rows in every resident column, so the appends that follow do not reallocate. %compact's counterpart; neither changes the row set, so neither detaches.
Makes room for n COLUMNS, so that the %add_column calls that follow relocate nothing and leave an outstanding %col pointer valid. See its own doc-comment for the guarantee.
Makes room for n COLUMNS, and in doing so gives %add_column a guarantee it does not
otherwise have:
While spare column capacity remains, adding a column under a NEW name relocates no
existing column's storage, moves no existing column's slot position, and does not
advance %generation(). A pointer taken from %col, a parquet_table_col handle
and a parquet_table_row handle all stay valid across such a call. A reservation
survives %clone and %clone_structure, so a copy starts with the same spare
capacity. Replacing an existing column (force=.true.) is not covered -- it frees
that column's storage -- and neither is any row-structural mutation.
That is what makes the commonest derived-column idiom safe rather than merely
lucky. Without a reservation, an %add_column that happens to fill the slot array
reallocates it, and Fortran leaves a pointer's association status UNDEFINED across the
MOVE_ALLOC that does it -- code that usually works and is not permitted to:
call t%reserve_columns(t%ncols() + 2) ! two derived columns coming
call t%col("mag_g", g) ! pointers taken up front...
call t%col("mag_r", r)
call t%add_column("g_minus_r", g - r) ! ...and still valid here, by contract
Reserve first, take pointers second -- the same rule %reserve (rows) follows.
Growing the capacity is itself a relocation, so this call invalidates every outstanding
pointer and advances %generation() when it actually grows; below the current capacity
it is a no-op, exactly as %reserve below the current row count is.
n is the TOTAL capacity to make room for, not an increment. Deliberately a plain
default integer and not also an int64 form: a column count cannot exceed int32 -- Arrow's
own Schema::num_fields() is an int32_t and the writer already guards that ceiling --
so the dual-kind rule in CLAUDE.md does not apply. Refused on a shared table, like every
other structural change; does not detach.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table. |
||
| integer, | intent(in) | :: | n |
total column slots to make room for. |
Column slots allocated, or spare.
How many column slots are allocated (%ncols() of them in use), or -- with
free=.true. -- how many are spare.
This is what %reserve_columns acts on, and the two are inverses: after
call t%reserve_columns(n) with n above the current capacity, %column_capacity()
is n. Capacity only ever grows: %compact releases row storage rather than slots,
and %drop_column keeps the slot it vacated.
A metadata query -- it reads no column data.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table. |
||
| logical, | intent(in), | optional | :: | free |
.true.: report the SPARE slots instead. |
slots allocated, or spare.
Independent deep copy of this table.
Makes out an independent deep copy of this table -- the way to keep a version to go
back to, since mutation is in place and there is no undo.
Columns already read are copied; columns not yet read stay unread, so a clone costs what the table actually holds rather than what its file contains. A live file-backed table's clone opens its OWN reader on the same file and stays lazy; a detached or in-memory table's clone has no reader, like its source.
out must be declared as the same concrete type as self.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table to copy. |
||
| class(parquet_table), | intent(out) | :: | out |
receives the copy. |
Empty table with the same columns.
Makes out an EMPTY table with this table's columns: same names, kinds, widths and
units, zero rows, no file behind it.
This is the first half of the bulk-append idiom -- %clone_structure a batch, fill it
with %add_column/%set, then %append it -- which is how a program adds many rows
without appending one at a time. A batch made this way structurally cannot have the
wrong column set, and abandoning a half-filled one is just a variable going out of
scope. Columns whose type this library cannot read are left out.
resident_only=.true. copies only the columns that have been read, which is also the
way to clone the structure of a table whose other columns are still deferred.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table to take the shape of. |
||
| class(parquet_table), | intent(out) | :: | out |
receives the empty table. |
||
| logical, | intent(in), | optional | :: | resident_only |
.true.: only columns already read. |
Hook -- copies an EXTENDING type's own components.
Copies the components an EXTENDING type added, which %clone cannot know about.
parquet_table is designed to be extended -- a generated table type
(doc/pages/utilities/generated-tables.md) does exactly that, and so may hand-written code. But
table_clone only knows parquet_table's own components, so anything the extension
declared would arrive default-initialized and nothing would report it. Overriding this
hook is how an extension copies its own state; %clone and %clone_structure each call
it as their LAST action, dispatching on self, so the override runs for free wherever
either is used.
A concrete-typed override of %clone itself is not possible -- an overriding
procedure must keep every dummy argument's characteristics, so out cannot be narrowed
from class(parquet_table). This hook is the supported substitute, and it keeps one name
for one operation rather than adding a second spelling of "clone" that the first one
silently gets wrong.
The default implementation does nothing, which is correct for parquet_table itself.
An override reaches out's own components through select type -- use class is, not
type is, so that a further extension still gets this level's copy. %clone has already
checked that self and out have the same dynamic type, so the guarded branch always
matches.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(in) | :: | self |
the table being copied. |
||
| class(parquet_table), | intent(inout) | :: | out |
the copy, already holding the base state. |
||
| logical, | intent(in) | :: | structure_only |
.true. when called from %clone_structure. |
Binds a generated type's predefined columns.
Binds the predefined columns a GENERATED table type declares: checks each one, converts
it to the kind the schema declared, reads it, and marks the slot predefined.
This is the one library call a generated type's %init makes
(doc/pages/utilities/generated-tables.md); it is public only because a generated module is a
DIFFERENT module and parquet_table's components are private, so there is no other way
in. Hand-written code rarely needs it -- a table opened with parquet_open_table already
reaches every column by name.
Per column, in names order:
from_file column must exist (after any remap:), or this aborts naming it;widths entry, or this aborts;%cast converts it -- and a conversion that
can lose information (a narrowing, or an integer wider than the target real's mantissa)
emits a warning naming the table and column, since the declaration is a contract the
file does not have to honour exactly;from_file column is then read in ONE %prefetch, after the casts, so a
converted column decodes straight into its declared kind rather than being read twice;from_file .false. -- a source: computed field, or any column of a
from-scratch table -- is created with %nrows() all-null rows instead of being looked
for in the file.Passing from_file all .false. is exactly what an in-memory generated table does, so
the same procedure serves a file-backed and a from-scratch construction.
units FILLS IN a unit the column does not already have; it never overwrites one. A
read-in MAML describes the physical file and is authoritative about what a file column
holds, so a declaration must not override it -- but a computed column has no other
source of a unit at all, and a file opened without a MAML has none either, which is
where the declared one belongs.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(parquet_table), | intent(inout) | :: | self |
the table, already opened or created. |
||
| character(len=*), | intent(in) | :: | names(:) |
internal column names, in declaration order. |
||
| integer, | intent(in) | :: | kinds(:) |
declared PK_* kind per name. |
||
| integer, | intent(in) | :: | widths(:) |
declared col_size per name (1 if scalar). |
||
| logical, | intent(in) | :: | from_file(:) |
.false. for a computed/from-scratch column. |
||
| character(len=*), | intent(in), | optional | :: | context |
schema name, for error messages. |
|
| logical, | intent(in), | optional | :: | exact |
refuse a lossy kind conversion; forwarded verbatim to %cast, whose own default applies when this is absent. |
|
| character(len=*), | intent(in), | optional | :: | units(:) |
declared unit per name ("" for none). |
Blocks intrinsic assignment: the store lives behind a pointer, so a default b = a
would leave two tables sharing one store and double-freeing it.
The binding is named assign_guard, and the name's ALPHABETICAL POSITION is
load-bearing -- do not "tidy" it to table_assign_guard to match its implementation.
flang builds a type's binding table sorted by name and stores a SPECIAL binding's index
(a defined assignment, here) in a single byte. parquet_table has ~285 bindings, so a
guard sorting under "t" lands past index 255 and flang dies with an internal compiler
error -- CHECK(bindingIndex <= 255) in runtime-type-info.cpp, which names neither this
type nor this line. Sorting under "a" keeps it far below the limit. gfortran and ifx are
indifferent, so nothing here will warn if this is undone.
type :: parquet_table private logical :: detached = .false. !! reserved: set by a row-structural mutation. integer :: regime = REGIME_FULL !! REGIME_FULL; REGIME_SLICE reserved. integer(int64) :: row_lo = 1 !! first row of the scope (1 in the full regime). integer(int64) :: row_hi = -1 !! last row of the scope (nrows in the full regime). integer(int64) :: row_count = 0 !! rows every column in this table holds. type(parquet_table_cache), pointer :: cache => null() !! the column store (see its own doc). contains ! --- introspection --- procedure :: nrows => table_nrows !! Number of rows every column holds. procedure :: ncols => table_ncols !! Number of columns the table has. procedure :: column_names => table_column_names !! Copy out every column name, in order. procedure :: column_index => table_column_index !! A column's 1-based position, 0 when absent. procedure :: column_name => table_column_name !! Copy out the name at a 1-based position. procedure, private :: column_by_name !! %column specific taking a column name. procedure, private :: column_by_index !! %column specific taking a 1-based position. !> A resolved handle on one column, by name or by 1-based position. Resolves once -- the !! lookup, the lazy first touch and the kind -- so a per-element loop over that column !! stops paying for a name lookup on every access. See `parquet_table_col`. generic :: column => column_by_name, column_by_index procedure, private :: has_nulls_name => table_has_nulls !! %has_nulls specific, by name. procedure, private :: has_nulls_at => table_has_nulls_at !! %has_nulls specific, by position. !> Whether a column holds (or may hold) nulls -- named, or by 1-based position. generic :: has_nulls => has_nulls_name, has_nulls_at procedure, private :: table_get_valid_mask !! %get_valid_mask specific, per-row mask. procedure, private :: table_get_valid_mask_elem !! %get_valid_mask specific, per-element mask. !> Copy out a column's validity as a plain `logical` array. A rank-1 `mask` gives one entry !! per row (on a *_VEC column: "any element of the row is null"); a rank-2 `mask` gives the !! true `(width, nrows)` per-element state. generic :: get_valid_mask => table_get_valid_mask, table_get_valid_mask_elem procedure :: generation => table_generation !! Counter bumped by every structural change. procedure :: has_column => table_has_column !! Whether a column of this name exists. procedure, private :: missing_columns_string !! %missing_columns specific, separated string. procedure, private :: missing_columns_array !! %missing_columns specific, array of names. !> Which of these columns the table does NOT have, as a packed array (zero-size when it !! has them all). The non-aborting half of %require_columns. generic :: missing_columns => missing_columns_string, missing_columns_array procedure, private :: require_columns_string !! %require_columns specific, separated string. procedure, private :: require_columns_array !! %require_columns specific, array of names. !> Aborts unless the table has every one of these columns, naming EVERY missing one -- !! not just the first, which is what a hand-written loop reports. generic :: require_columns => require_columns_string, require_columns_array procedure, private :: kind_name => table_column_kind !! %kind specific, by name. procedure, private :: kind_at => table_column_kind_at !! %kind specific, by position. !> A column's PK_* kind discriminator -- named, or by 1-based position. generic :: kind => kind_name, kind_at procedure, private :: width_name => table_column_width !! %width specific, by name. procedure, private :: width_at => table_column_width_at !! %width specific, by position. !> A column's values-per-row (1 if scalar) -- named, or by 1-based position. generic :: width => width_name, width_at procedure, private :: unit_name => table_column_unit !! %unit specific, by name. procedure, private :: unit_at => table_column_unit_at !! %unit specific, by position. !> Copy out a column's unit string -- named, or by 1-based position. generic :: unit => unit_name, unit_at procedure, private :: residency_name => table_column_residency !! %residency specific, by name. procedure, private :: residency_at => table_column_residency_at !! %residency specific, by position. !> A column's RES_* residency state -- named, or by 1-based position. generic :: residency => residency_name, residency_at procedure, private :: is_null_i32 => table_is_null_i32 !! %is_null specific, int32 row index. procedure, private :: is_null_i64 => table_is_null_i64 !! %is_null specific, int64 row index. procedure, private :: is_null_e32 => table_is_null_e32 !! %is_null specific, int32 row + element. procedure, private :: is_null_e64 => table_is_null_e64 !! %is_null specific, int64 row + element. procedure, private :: is_null_at_i32 => table_is_null_at_i32 !! %is_null by position, int32 row. procedure, private :: is_null_at_i64 => table_is_null_at_i64 !! %is_null by position, int64 row. procedure, private :: is_null_at_e32 => table_is_null_at_e32 !! %is_null by position, int32 row + element. procedure, private :: is_null_at_e64 => table_is_null_at_e64 !! %is_null by position, int64 row + element. !> Whether row `i` of a column is null, or -- given `e` as well -- element `e` of it. !! !! On a *_VEC column the row form answers "ANY element of the row is null"; the element !! form answers about that one element. Defined on a scalar column too, where `e` can only !! be 1 and the two agree. generic :: is_null => is_null_i32, is_null_i64, is_null_e32, is_null_e64, & is_null_at_i32, is_null_at_i64, is_null_at_e32, is_null_at_e64 procedure :: is_detached => table_is_detached !! Whether the table has left its file behind. procedure, private :: is_supported_name => table_is_supported !! %is_supported specific, by name. procedure, private :: is_supported_at => table_is_supported_at !! %is_supported specific, by position. !> Whether a column's type can be read -- named, or by 1-based position. generic :: is_supported => is_supported_name, is_supported_at procedure :: filename => table_filename !! Copy out the file this table came from. procedure :: get_file_metadata => table_get_file_metadata !! One key from the file's metadata. ! --- residency control --- procedure, private :: prefetch_string !! %prefetch specific taking a separated name string. procedure, private :: prefetch_array !! %prefetch specific taking an array of names. !> Reads the named column(s) now, instead of on first touch. Required before a parallel !! region: a first touch inside one is a hard error, since it would mutate shared state. generic :: prefetch => prefetch_string, prefetch_array !> The same call as %prefetch, under the name that pairs with %materialize_all. Reaching !! for the definitive-sounding %materialize_all when only a few columns are wanted reads !! the whole file, silently; %materialize(names) is the one to find first. generic :: materialize => prefetch_string, prefetch_array procedure :: materialize_all => table_materialize_every !! Read every column not yet read. procedure :: reload => table_reload !! Re-read one column; force= to discard local edits. procedure :: evict_column => table_evict_column !! Drop a column's VALUES; force= if it holds local edits. procedure :: set_user_populated => table_set_user_populated !! Claim a column's values as the caller's own, or unclaim. procedure :: is_user_populated => table_is_user_populated !! Whether a column is claimed as holding the caller's values. procedure :: validate_qc => table_validate_qc !! Check every qc-declaring column, holding none. procedure :: print_stat => table_print_stat !! Print what the table holds, to stdout. procedure :: nrows_unfiltered => table_nrows_unfiltered !! Rows before filter=/sample_fraction=. procedure :: row_group_extent => table_row_group_extent !! Rows in the row groups this table covers. procedure :: row_group_bounds => table_row_group_bounds !! Row-group row ranges, this table's rows or the file's. ! --- row view --- procedure, private :: row_at_i32 !! %row specific taking an int32 index. procedure, private :: row_at_i64 !! %row specific taking an int64 index. !> A handle on one row, for code that works a row at a time rather than a column at a !! time. The index is 1-based within THIS table -- in the slice regime, row 1 is the !! slice's first row, not the file's. generic :: row => row_at_i32, row_at_i64 ! --- copy out a row selection --- procedure, private :: get_slice_i32 !! %get_slice specific for the i32 kind. procedure, private :: get_slice_i64 !! %get_slice specific for the i64 kind. procedure, private :: get_slice_f32 !! %get_slice specific for the f32 kind. procedure, private :: get_slice_f64 !! %get_slice specific for the f64 kind. procedure, private :: get_slice_bool !! %get_slice specific for the bool kind. procedure, private :: get_slice_date !! %get_slice specific for the date kind. procedure, private :: get_slice_time !! %get_slice specific for the time kind. procedure, private :: get_slice_ts !! %get_slice specific for the ts kind. procedure, private :: get_slice_i32v !! %get_slice specific for the i32v kind. procedure, private :: get_slice_i64v !! %get_slice specific for the i64v kind. procedure, private :: get_slice_f32v !! %get_slice specific for the f32v kind. procedure, private :: get_slice_f64v !! %get_slice specific for the f64v kind. procedure, private :: get_slice_boolv !! %get_slice specific for the boolv kind. procedure, private :: get_slice_datev !! %get_slice specific for the datev kind. procedure, private :: get_slice_timev !! %get_slice specific for the timev kind. procedure, private :: get_slice_tsv !! %get_slice specific for the tsv kind. procedure, private :: get_slice_str !! %get_slice specific returning a parquet_string_column. procedure, private :: get_slice_chr !! %get_slice specific returning a character array. procedure, private :: get_slice_chrv !! %get_slice specific returning a character (elem, row) array. !> Copies the rows a `parquet_slice` selects into a freshly allocated array of !! the caller's own kind, widening on the way exactly as %get does. !! !! **The selection is taken in the order given, duplicates included.** A !! `parquet_slice_list` may name a row more than once and may name rows in any !! order, so the result has one entry per SELECTION -- not per distinct row -- and !! its order is the selection's, never the table's. generic :: get_slice => get_slice_i32, get_slice_i64, get_slice_f32, get_slice_f64, get_slice_bool, get_slice_date, & get_slice_time, get_slice_ts, get_slice_i32v, get_slice_i64v, get_slice_f32v, get_slice_f64v, get_slice_boolv, & get_slice_datev, get_slice_timev, get_slice_tsv, get_slice_str, get_slice_chr, get_slice_chrv ! --- write a row selection back --- procedure, private :: set_slice_i32 !! %set_slice specific for the i32 kind. procedure, private :: set_slice_i64 !! %set_slice specific for the i64 kind. procedure, private :: set_slice_f32 !! %set_slice specific for the f32 kind. procedure, private :: set_slice_f64 !! %set_slice specific for the f64 kind. procedure, private :: set_slice_bool !! %set_slice specific for the bool kind. procedure, private :: set_slice_date !! %set_slice specific for the date kind. procedure, private :: set_slice_time !! %set_slice specific for the time kind. procedure, private :: set_slice_ts !! %set_slice specific for the ts kind. procedure, private :: set_slice_i32v !! %set_slice specific for the i32v kind. procedure, private :: set_slice_i64v !! %set_slice specific for the i64v kind. procedure, private :: set_slice_f32v !! %set_slice specific for the f32v kind. procedure, private :: set_slice_f64v !! %set_slice specific for the f64v kind. procedure, private :: set_slice_boolv !! %set_slice specific for the boolv kind. procedure, private :: set_slice_datev !! %set_slice specific for the datev kind. procedure, private :: set_slice_timev !! %set_slice specific for the timev kind. procedure, private :: set_slice_tsv !! %set_slice specific for the tsv kind. procedure, private :: set_slice_chr !! %set_slice specific taking a character array. procedure, private :: set_slice_chrv !! %set_slice specific taking a character (elem, row) array. !> Writes values into the rows a `parquet_slice` selects -- %get_slice's counterpart. !! The kind must match the column's exactly, and the array must have one value per !! selected row. !! !! **The selection is written in the order given, duplicates included, so a row !! named twice ends up holding the LAST value written to it.** Selections are !! applied one after another rather than merged or de-duplicated, which is what !! makes `%set_slice` the exact inverse of `%get_slice` for a selection that names !! each row once. generic :: set_slice => set_slice_i32, set_slice_i64, set_slice_f32, set_slice_f64, set_slice_bool, set_slice_date, & set_slice_time, set_slice_ts, set_slice_i32v, set_slice_i64v, set_slice_f32v, set_slice_f64v, set_slice_boolv, & set_slice_datev, set_slice_timev, set_slice_tsv, set_slice_chr, set_slice_chrv ! --- zero-copy pointer access (exact kind) --- procedure, private :: col_ptr_i32 !! %col specific for the i32 kind. procedure, private :: col_ptr_i64 !! %col specific for the i64 kind. procedure, private :: col_ptr_f32 !! %col specific for the f32 kind. procedure, private :: col_ptr_f64 !! %col specific for the f64 kind. procedure, private :: col_ptr_bool !! %col specific for the bool kind. procedure, private :: col_ptr_date !! %col specific for the date kind. procedure, private :: col_ptr_time !! %col specific for the time kind. procedure, private :: col_ptr_ts !! %col specific for the ts kind. procedure, private :: col_ptr_i32v !! %col specific for the i32v kind. procedure, private :: col_ptr_i64v !! %col specific for the i64v kind. procedure, private :: col_ptr_f32v !! %col specific for the f32v kind. procedure, private :: col_ptr_f64v !! %col specific for the f64v kind. procedure, private :: col_ptr_boolv !! %col specific for the boolv kind. procedure, private :: col_ptr_datev !! %col specific for the datev kind. procedure, private :: col_ptr_timev !! %col specific for the timev kind. procedure, private :: col_ptr_tsv !! %col specific for the tsv kind. procedure, private :: col_ptr_strcol !! %col specific aliasing the compact string store. !> Points `p` at a column's storage: zero copy, writable, and the pointer kind must !! match the stored kind exactly (ask %kind first if you do not know it). A !! `parquet_string_column` pointer aliases a PK_STRING column's packed store: read !! it and edit its values in place, but do NOT change its length or element count !! through the pointer -- the column's own row count would no longer describe it. !! !! For the same reason, do NOT REORDER a column through this pointer -- in particular !! `call pf_permute(p, perm)`, which compiles and runs happily. It reorders that one !! column and leaves every other column where it was, silently breaking the row !! correspondence, and nothing detects it: the row count is unchanged and every later !! read returns values that are individually valid and jointly wrong. Use `%sort_by`, !! which reorders every column together. generic :: col => col_ptr_i32, col_ptr_i64, col_ptr_f32, col_ptr_f64, col_ptr_bool, col_ptr_date, col_ptr_time, & col_ptr_ts, col_ptr_i32v, col_ptr_i64v, col_ptr_f32v, col_ptr_f64v, col_ptr_boolv, col_ptr_datev, col_ptr_timev, & col_ptr_tsv, col_ptr_strcol ! --- copy out (widens int32->int64, float32->float64) --- procedure, private :: get_arr_i32 !! %get specific for the i32 kind. procedure, private :: get_arr_i64 !! %get specific for the i64 kind. procedure, private :: get_arr_f32 !! %get specific for the f32 kind. procedure, private :: get_arr_f64 !! %get specific for the f64 kind. procedure, private :: get_arr_bool !! %get specific for the bool kind. procedure, private :: get_arr_date !! %get specific for the date kind. procedure, private :: get_arr_time !! %get specific for the time kind. procedure, private :: get_arr_ts !! %get specific for the ts kind. procedure, private :: get_arr_i32v !! %get specific for the i32v kind. procedure, private :: get_arr_i64v !! %get specific for the i64v kind. procedure, private :: get_arr_f32v !! %get specific for the f32v kind. procedure, private :: get_arr_f64v !! %get specific for the f64v kind. procedure, private :: get_arr_boolv !! %get specific for the boolv kind. procedure, private :: get_arr_datev !! %get specific for the datev kind. procedure, private :: get_arr_timev !! %get specific for the timev kind. procedure, private :: get_arr_tsv !! %get specific for the tsv kind. procedure, private :: get_arr_str !! %get specific returning a parquet_string_column. procedure, private :: get_arr_chr !! %get specific returning a character array. procedure, private :: get_arr_chrv !! %get specific returning a character (elem, row) array. !> Copies a column into a freshly allocated array of the caller's own kind. generic :: get => get_arr_i32, get_arr_i64, get_arr_f32, get_arr_f64, get_arr_bool, get_arr_date, get_arr_time, & get_arr_ts, get_arr_i32v, get_arr_i64v, get_arr_f32v, get_arr_f64v, get_arr_boolv, get_arr_datev, get_arr_timev, & get_arr_tsv, get_arr_str, get_arr_chr, get_arr_chrv ! --- copy back (same length, exact kind) --- procedure, private :: set_arr_i32 !! %set specific for the i32 kind. procedure, private :: set_arr_i64 !! %set specific for the i64 kind. procedure, private :: set_arr_f32 !! %set specific for the f32 kind. procedure, private :: set_arr_f64 !! %set specific for the f64 kind. procedure, private :: set_arr_bool !! %set specific for the bool kind. procedure, private :: set_arr_date !! %set specific for the date kind. procedure, private :: set_arr_time !! %set specific for the time kind. procedure, private :: set_arr_ts !! %set specific for the ts kind. procedure, private :: set_arr_i32v !! %set specific for the i32v kind. procedure, private :: set_arr_i64v !! %set specific for the i64v kind. procedure, private :: set_arr_f32v !! %set specific for the f32v kind. procedure, private :: set_arr_f64v !! %set specific for the f64v kind. procedure, private :: set_arr_boolv !! %set specific for the boolv kind. procedure, private :: set_arr_datev !! %set specific for the datev kind. procedure, private :: set_arr_timev !! %set specific for the timev kind. procedure, private :: set_arr_tsv !! %set specific for the tsv kind. procedure, private :: set_arr_chr !! %set specific taking a character array. procedure, private :: set_arr_chrv !! %set specific taking a character (elem, row) array. !> Replaces every value of an existing column from an array of the same length. procedure, private :: set_arr_strcol !! %set specific taking a parquet_string_column. generic :: set => set_arr_i32, set_arr_i64, set_arr_f32, set_arr_f64, set_arr_bool, set_arr_date, set_arr_time, & set_arr_ts, set_arr_i32v, set_arr_i64v, set_arr_f32v, set_arr_f64v, set_arr_boolv, set_arr_datev, set_arr_timev, & set_arr_tsv, set_arr_chr, set_arr_chrv, set_arr_strcol ! --- from-scratch construction --- procedure, private :: add_column_i32 !! %add_column specific for the i32 kind. procedure, private :: add_column_i64 !! %add_column specific for the i64 kind. procedure, private :: add_column_f32 !! %add_column specific for the f32 kind. procedure, private :: add_column_f64 !! %add_column specific for the f64 kind. procedure, private :: add_column_bool !! %add_column specific for the bool kind. procedure, private :: add_column_date !! %add_column specific for the date kind. procedure, private :: add_column_time !! %add_column specific for the time kind. procedure, private :: add_column_ts !! %add_column specific for the ts kind. procedure, private :: add_column_i32v !! %add_column specific for the i32v kind. procedure, private :: add_column_i64v !! %add_column specific for the i64v kind. procedure, private :: add_column_f32v !! %add_column specific for the f32v kind. procedure, private :: add_column_f64v !! %add_column specific for the f64v kind. procedure, private :: add_column_boolv !! %add_column specific for the boolv kind. procedure, private :: add_column_datev !! %add_column specific for the datev kind. procedure, private :: add_column_timev !! %add_column specific for the timev kind. procedure, private :: add_column_tsv !! %add_column specific for the tsv kind. procedure, private :: add_column_chr !! %add_column specific taking a character array. procedure, private :: add_column_chrv !! %add_column specific taking a character (elem, row) array. !> Appends a new column, taking its values (and so its kind, width and row count). procedure, private :: add_column_strcol !! %add_column specific taking a parquet_string_column. procedure, private :: add_column_col !! %add_column specific taking a whole parquet_column. generic :: add_column => add_column_i32, add_column_i64, add_column_f32, add_column_f64, add_column_bool, & add_column_date, add_column_time, add_column_ts, add_column_i32v, add_column_i64v, add_column_f32v, add_column_f64v, & add_column_boolv, add_column_datev, add_column_timev, add_column_tsv, add_column_chr, add_column_chrv, & add_column_strcol, add_column_col ! --- mutation: one cell at a time (never changes the row set) --- procedure, private :: set_element_i32_i32 !! %set_element specific, i32 kind, i32 row index. procedure, private :: set_element_i32_i64 !! %set_element specific, i32 kind, i64 row index. procedure, private :: set_element_i64_i32 !! %set_element specific, i64 kind, i32 row index. procedure, private :: set_element_i64_i64 !! %set_element specific, i64 kind, i64 row index. procedure, private :: set_element_f32_i32 !! %set_element specific, f32 kind, i32 row index. procedure, private :: set_element_f32_i64 !! %set_element specific, f32 kind, i64 row index. procedure, private :: set_element_f64_i32 !! %set_element specific, f64 kind, i32 row index. procedure, private :: set_element_f64_i64 !! %set_element specific, f64 kind, i64 row index. procedure, private :: set_element_bool_i32 !! %set_element specific, bool kind, i32 row index. procedure, private :: set_element_bool_i64 !! %set_element specific, bool kind, i64 row index. procedure, private :: set_element_date_i32 !! %set_element specific, date kind, i32 row index. procedure, private :: set_element_date_i64 !! %set_element specific, date kind, i64 row index. procedure, private :: set_element_time_i32 !! %set_element specific, time kind, i32 row index. procedure, private :: set_element_time_i64 !! %set_element specific, time kind, i64 row index. procedure, private :: set_element_ts_i32 !! %set_element specific, ts kind, i32 row index. procedure, private :: set_element_ts_i64 !! %set_element specific, ts kind, i64 row index. procedure, private :: set_element_i32v_i32 !! %set_element specific, i32v kind, i32 row index. procedure, private :: set_element_i32v_i64 !! %set_element specific, i32v kind, i64 row index. procedure, private :: set_element_i64v_i32 !! %set_element specific, i64v kind, i32 row index. procedure, private :: set_element_i64v_i64 !! %set_element specific, i64v kind, i64 row index. procedure, private :: set_element_f32v_i32 !! %set_element specific, f32v kind, i32 row index. procedure, private :: set_element_f32v_i64 !! %set_element specific, f32v kind, i64 row index. procedure, private :: set_element_f64v_i32 !! %set_element specific, f64v kind, i32 row index. procedure, private :: set_element_f64v_i64 !! %set_element specific, f64v kind, i64 row index. procedure, private :: set_element_boolv_i32 !! %set_element specific, boolv kind, i32 row index. procedure, private :: set_element_boolv_i64 !! %set_element specific, boolv kind, i64 row index. procedure, private :: set_element_datev_i32 !! %set_element specific, datev kind, i32 row index. procedure, private :: set_element_datev_i64 !! %set_element specific, datev kind, i64 row index. procedure, private :: set_element_timev_i32 !! %set_element specific, timev kind, i32 row index. procedure, private :: set_element_timev_i64 !! %set_element specific, timev kind, i64 row index. procedure, private :: set_element_tsv_i32 !! %set_element specific, tsv kind, i32 row index. procedure, private :: set_element_tsv_i64 !! %set_element specific, tsv kind, i64 row index. procedure, private :: set_element_chr_i32 !! %set_element specific, character chr form, i32 row index. procedure, private :: set_element_chr_i64 !! %set_element specific, character chr form, i64 row index. procedure, private :: set_element_chrv_i32 !! %set_element specific, character chrv form, i32 row index. procedure, private :: set_element_chrv_i64 !! %set_element specific, character chrv form, i64 row index. procedure, private :: get_element_i32_i32 !! %get_element specific, i32 kind, i32 row index. procedure, private :: get_element_i32_i64 !! %get_element specific, i32 kind, i64 row index. procedure, private :: get_element_i64_i32 !! %get_element specific, i64 kind, i32 row index. procedure, private :: get_element_i64_i64 !! %get_element specific, i64 kind, i64 row index. procedure, private :: get_element_f32_i32 !! %get_element specific, f32 kind, i32 row index. procedure, private :: get_element_f32_i64 !! %get_element specific, f32 kind, i64 row index. procedure, private :: get_element_f64_i32 !! %get_element specific, f64 kind, i32 row index. procedure, private :: get_element_f64_i64 !! %get_element specific, f64 kind, i64 row index. procedure, private :: get_element_bool_i32 !! %get_element specific, bool kind, i32 row index. procedure, private :: get_element_bool_i64 !! %get_element specific, bool kind, i64 row index. procedure, private :: get_element_date_i32 !! %get_element specific, date kind, i32 row index. procedure, private :: get_element_date_i64 !! %get_element specific, date kind, i64 row index. procedure, private :: get_element_time_i32 !! %get_element specific, time kind, i32 row index. procedure, private :: get_element_time_i64 !! %get_element specific, time kind, i64 row index. procedure, private :: get_element_ts_i32 !! %get_element specific, ts kind, i32 row index. procedure, private :: get_element_ts_i64 !! %get_element specific, ts kind, i64 row index. procedure, private :: get_element_i32v_i32 !! %get_element specific, i32v kind, i32 row index. procedure, private :: get_element_i32v_i64 !! %get_element specific, i32v kind, i64 row index. procedure, private :: get_element_i64v_i32 !! %get_element specific, i64v kind, i32 row index. procedure, private :: get_element_i64v_i64 !! %get_element specific, i64v kind, i64 row index. procedure, private :: get_element_f32v_i32 !! %get_element specific, f32v kind, i32 row index. procedure, private :: get_element_f32v_i64 !! %get_element specific, f32v kind, i64 row index. procedure, private :: get_element_f64v_i32 !! %get_element specific, f64v kind, i32 row index. procedure, private :: get_element_f64v_i64 !! %get_element specific, f64v kind, i64 row index. procedure, private :: get_element_boolv_i32 !! %get_element specific, boolv kind, i32 row index. procedure, private :: get_element_boolv_i64 !! %get_element specific, boolv kind, i64 row index. procedure, private :: get_element_datev_i32 !! %get_element specific, datev kind, i32 row index. procedure, private :: get_element_datev_i64 !! %get_element specific, datev kind, i64 row index. procedure, private :: get_element_timev_i32 !! %get_element specific, timev kind, i32 row index. procedure, private :: get_element_timev_i64 !! %get_element specific, timev kind, i64 row index. procedure, private :: get_element_tsv_i32 !! %get_element specific, tsv kind, i32 row index. procedure, private :: get_element_tsv_i64 !! %get_element specific, tsv kind, i64 row index. procedure, private :: get_element_chr_i32 !! %get_element specific, character chr form, i32 row index. procedure, private :: get_element_chr_i64 !! %get_element specific, character chr form, i64 row index. procedure, private :: get_element_chrv_i32 !! %get_element specific, character chrv form, i32 row index. procedure, private :: get_element_chrv_i64 !! %get_element specific, character chrv form, i64 row index. !> Reads one row's value out of a column, widening into the caller's variable !! exactly as %get does -- the one-call form of `r = t%row(i)` then `r%get(name, v)`. !! On a *_VEC column the value is that row's whole vector. generic :: get_element => get_element_i32_i32, get_element_i32_i64, get_element_i64_i32, get_element_i64_i64, & get_element_f32_i32, get_element_f32_i64, get_element_f64_i32, get_element_f64_i64, get_element_bool_i32, & get_element_bool_i64, get_element_date_i32, get_element_date_i64, get_element_time_i32, get_element_time_i64, & get_element_ts_i32, get_element_ts_i64, get_element_i32v_i32, get_element_i32v_i64, get_element_i64v_i32, & get_element_i64v_i64, get_element_f32v_i32, get_element_f32v_i64, get_element_f64v_i32, get_element_f64v_i64, & get_element_boolv_i32, get_element_boolv_i64, get_element_datev_i32, get_element_datev_i64, get_element_timev_i32, & get_element_timev_i64, get_element_tsv_i32, get_element_tsv_i64, get_element_chr_i32, get_element_chr_i64, & get_element_chrv_i32, get_element_chrv_i64 !> Writes one row's value in place. The kind must match the column's exactly (as !! %set does), and writing a value CLEARS that row's null -- use %set_null to put !! one back. On a *_VEC column the value is that row's whole vector. generic :: set_element => set_element_i32_i32, set_element_i32_i64, set_element_i64_i32, set_element_i64_i64, & set_element_f32_i32, set_element_f32_i64, set_element_f64_i32, set_element_f64_i64, set_element_bool_i32, & set_element_bool_i64, set_element_date_i32, set_element_date_i64, set_element_time_i32, set_element_time_i64, & set_element_ts_i32, set_element_ts_i64, set_element_i32v_i32, set_element_i32v_i64, set_element_i64v_i32, & set_element_i64v_i64, set_element_f32v_i32, set_element_f32v_i64, set_element_f64v_i32, set_element_f64v_i64, & set_element_boolv_i32, set_element_boolv_i64, set_element_datev_i32, set_element_datev_i64, set_element_timev_i32, & set_element_timev_i64, set_element_tsv_i32, set_element_tsv_i64, set_element_chr_i32, set_element_chr_i64, & set_element_chrv_i32, set_element_chrv_i64 procedure, private :: set_null_i32 !! %set_null specific taking an int32 row index. procedure, private :: set_null_i64 !! %set_null specific taking an int64 row index. procedure, private :: set_null_e32 !! %set_null specific taking an int32 row + element. procedure, private :: set_null_e64 !! %set_null specific taking an int64 row + element. procedure, private :: set_null_mask !! %set_null specific taking a per-row mask. procedure, private :: set_null_mask_elem !! %set_null specific taking a per-element mask. !> Marks null: row `i` of a column, element `e` of row `i`, or every entry a `logical` !! mask marks `.false.`. !! !! The row form is whole-row even on a *_VEC column -- naming only a row says the row is !! missing. Name `e` to null one element. The mask form takes either shape: one entry per !! row (whole rows), or a `(width, nrows)` mask (individual elements). generic :: set_null => set_null_i32, set_null_i64, set_null_e32, set_null_e64, & set_null_mask, set_null_mask_elem procedure, private :: clear_null_i32 !! %clear_null specific taking an int32 row index. procedure, private :: clear_null_i64 !! %clear_null specific taking an int64 row index. procedure, private :: clear_null_e32 !! %clear_null specific taking an int32 row + element. procedure, private :: clear_null_e64 !! %clear_null specific taking an int64 row + element. !> Marks row `i` -- or, given `e`, element `e` of it -- valid without saying what its value !! is. Only useful when a value is already there or is about to be written; %set_element !! clears the null itself. generic :: clear_null => clear_null_i32, clear_null_i64, clear_null_e32, clear_null_e64 procedure :: compact_validity => table_compact_validity !! Drop a null bitmap that no longer has nulls. procedure :: ensure_validity => table_ensure_validity !! Allocate validity storage up front, for concurrent nulling. ! --- mutation: whole columns (never changes the row set) --- procedure :: drop_column => table_drop_column !! Remove a column; force= for a predefined one. procedure :: rename_column => table_rename_column !! Change the name a column is looked up by. procedure :: copy_column => table_copy_column !! Add a copy of a column, optionally of another kind. procedure :: cast => table_cast !! Convert a column to another kind, in place. ! --- mutation: the row set itself -- every one of these DETACHES the table --- procedure :: filter_rows => table_filter_rows !! Keep only the rows a mask selects. procedure, private :: table_sort_by !! %sort_by specific, array of key names. procedure, private :: table_sort_by_string !! %sort_by specific, separated key string. !> Reorders rows by one or more key columns. Detaching. generic :: sort_by => table_sort_by, table_sort_by_string procedure, private :: table_top_n !! %top_n specific, array of key names. procedure, private :: table_top_n_string !! %top_n specific, separated key string. !> Keeps only the n best rows, in key order. Detaching. generic :: top_n => table_top_n, table_top_n_string ! --- the ORDER, without applying it: read-only, and they do NOT detach --- procedure, private :: table_argsort_by_i32 !! %argsort_by specific, int32 permutation. procedure, private :: table_argsort_by_i64 !! %argsort_by specific, int64 permutation. procedure, private :: table_argsort_by_string_i32 !! %argsort_by specific, key string, int32. procedure, private :: table_argsort_by_string_i64 !! %argsort_by specific, key string, int64. !> The row order the keys imply, without reordering anything. Unlike %sort_by the table !! stays attached, so this is how to read rows in an order while keeping the file. generic :: argsort_by => table_argsort_by_i32, table_argsort_by_i64, & table_argsort_by_string_i32, table_argsort_by_string_i64 procedure, private :: table_argsort_partial_i32 !! %argsort_partial specific, int32 perm. procedure, private :: table_argsort_partial_i64 !! %argsort_partial specific, int64 perm. procedure, private :: table_argsort_partial_string_i32 !! %argsort_partial, key string, int32. procedure, private :: table_argsort_partial_string_i64 !! %argsort_partial, key string, int64. !> The `n` best rows in order, by selection rather than a full sort. Also non-mutating. generic :: argsort_partial => table_argsort_partial_i32, table_argsort_partial_i64, & table_argsort_partial_string_i32, table_argsort_partial_string_i64 procedure, private :: table_is_sorted_by !! %is_sorted_by specific, array of key names. procedure, private :: table_is_sorted_by_string !! %is_sorted_by specific, key string. !> Whether the rows are already in that order. generic :: is_sorted_by => table_is_sorted_by, table_is_sorted_by_string procedure, private :: table_delete_rows_i32 !! %delete_rows specific, int32 indices. procedure, private :: table_delete_rows_i64 !! %delete_rows specific, int64 indices. !> Removes the listed rows. A thin convenience over %filter_rows, and like it, detaching. generic :: delete_rows => table_delete_rows_i32, table_delete_rows_i64 procedure, private :: table_truncate_i32 !! %truncate specific, int32 count. procedure, private :: table_truncate_i64 !! %truncate specific, int64 count. !> Keeps only the first n rows. Detaching, like every row-structural change. generic :: truncate => table_truncate_i32, table_truncate_i64 procedure, private :: table_append_table !! %append specific taking another table. procedure, private :: table_append_row !! %append specific taking one row handle. !> Appends rows: a whole table's worth, or one row. Detaching, like every row-structural !! change. The bulk idiom is %clone_structure -> fill -> %append(batch). generic :: append => table_append_table, table_append_row procedure, private :: table_append_null_rows_i32 !! %append_null_rows specific, int32 count. procedure, private :: table_append_null_rows_i64 !! %append_null_rows specific, int64 count. !> Appends n all-null rows, to be filled in afterwards. Detaching. generic :: append_null_rows => table_append_null_rows_i32, table_append_null_rows_i64 ! --- capacity --- procedure :: compact => table_compact !! Release capacity appends left behind. procedure, private :: table_reserve_i32 !! %reserve specific, int32 count. procedure, private :: table_reserve_i64 !! %reserve specific, int64 count. !> Makes room for n rows in every resident column, so the appends that follow do not !! reallocate. %compact's counterpart; neither changes the row set, so neither detaches. generic :: reserve => table_reserve_i32, table_reserve_i64 !> Makes room for n COLUMNS, so that the %add_column calls that follow relocate nothing !! and leave an outstanding %col pointer valid. See its own doc-comment for the guarantee. procedure :: reserve_columns => table_reserve_columns procedure :: column_capacity => table_column_capacity !! Column slots allocated, or spare. ! --- copying --- procedure :: clone => table_clone !! Independent deep copy of this table. procedure :: clone_structure => table_clone_structure !! Empty table with the same columns. procedure :: clone_extra => table_clone_extra !! Hook -- copies an EXTENDING type's own components. ! --- generated table types (see doc/pages/utilities/generated-tables.md) --- procedure :: bind_predefined => table_bind_predefined !! Binds a generated type's predefined columns. ! --- lifecycle --- !> Blocks intrinsic assignment: the store lives behind a pointer, so a default `b = a` !! would leave two tables sharing one store and double-freeing it. !! !! **The binding is named `assign_guard`, and the name's ALPHABETICAL POSITION is !! load-bearing -- do not "tidy" it to `table_assign_guard` to match its implementation.** !! flang builds a type's binding table sorted by name and stores a SPECIAL binding's index !! (a defined assignment, here) in a single byte. `parquet_table` has ~285 bindings, so a !! guard sorting under "t" lands past index 255 and flang dies with an internal compiler !! error -- `CHECK(bindingIndex <= 255)` in runtime-type-info.cpp, which names neither this !! type nor this line. Sorting under "a" keeps it far below the limit. gfortran and ifx are !! indifferent, so nothing here will warn if this is undone. generic :: assignment(=) => assign_guard procedure, private :: assign_guard => table_assign_guard !! The blocking defined assignment. final :: table_finalize !! Frees the store; never fails, never validates. end type parquet_table