parquet_column – parquet-fortran

type, public :: parquet_column

One column's values: a kind discriminator, one active storage array, sparse validity, an optional unit, and the row/width geometry.

Type-Bound Procedures

procedure, public :: init

Set kind/geometry and allocate empty storage.

interface

private module subroutine init(self, kind, nrows, width, unit)

Sets the column's kind and geometry and allocates empty storage for that kind. Any previous contents are released first. width is required (and must be > 1) for a *_VEC kind and must be 1 (or absent) otherwise. nrows rows are allocated; their values are unspecified for numeric kinds and null for temporal/string kinds.

Arguments

Type	Intent	Optional		Name
class(parquet_column),	intent(inout)		::	self	the column.
integer,	intent(in)		::	kind	PK_* discriminator to activate.
integer(kind=int64),	intent(in)		::	nrows	initial row count (>= 0).
integer(kind=int32),	intent(in),	optional	::	width	values per row for a *_VEC kind.
character(len=*),	intent(in),	optional	::	unit	unit string to store (D10).

procedure, public :: clear

Release all storage and reset to PK_NONE.

interface

private module subroutine clear(self)

Releases every storage array and resets the column to PK_NONE with zero rows.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column.

procedure, public :: deep_copy

Independent copy of values, validity and unit.

interface

private module subroutine deep_copy(self, out)

Produces an independent copy: values, validity, unit and geometry. Mutating either column afterwards leaves the other unchanged.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the source column.
type(parquet_column),	intent(out)			::	out	the copy.

procedure, public :: move_from

Take over another column's storage, leaving it empty.

interface

private module subroutine move_from(self, src)

Takes over src's storage outright, leaving src empty (PK_NONE, zero rows).

The move counterpart of deep_copy: every allocatable component is handed over with move_alloc rather than copied, so relocating a column costs a handful of descriptor assignments instead of a full copy of its data. Use it wherever a column is being RELOCATED rather than duplicated -- %drop_column shifting the tail of a table's slot array down over the dropped one is the case that motivated it, where intrinsic assignment memcpy'd every remaining column's values.

self is cleared first, so whatever it held is released.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column receiving the storage.
type(parquet_column),	intent(inout)			::	src	the column giving it up; left empty.

procedure, public :: kindof

The active PK_* discriminator.

private pure function kindof(self) result(res)

The column's active kind discriminator. PK_NONE until init is called.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the column.

Return Value integer

the active PK_* constant.

procedure, public :: length

Number of rows stored.

private pure function length(self) result(res)

Number of rows the column currently holds.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the column.

Return Value integer(kind=int64)

the row count.

procedure, public :: capacity

Rows the storage is allocated for (>= length()).

private function capacity(self) result(res)

Rows the active storage is allocated for. Always >= length(); the difference is spare capacity that %append can fill without reallocating, and that %shrink_to_fit releases.

Reported in ROWS for every kind, which is what makes capacity() >= length() a meaningful comparison whatever the column holds. The two string kinds forward to the embedded parquet_string_column, whose own capacity counts ELEMENTS -- so it is divided by the width to get rows, exactly as length() counts rows rather than the nrows*width elements a vector string column stores (RF6). A column with no kind yet reports 0.

Not pure, unlike length, only because the string store's own %capacity is not.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the column.

Return Value integer(kind=int64)

rows the storage is allocated for.

procedure, public :: colwidth

Values per row (1 for scalar kinds).

private pure function colwidth(self) result(res)

Values per row: 1 for every scalar kind, the vector width for a *_VEC kind.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the column.

Return Value integer(kind=int32)

the column width.

procedure, public :: validity_bytes

Bytes the null bitmap occupies (0 when sparse).

private pure function validity_bytes(self) result(res)

Bytes currently occupied by the null bitmap — 0 for a null-free column, which is the whole point of the sparse representation (R2): nothing is allocated until the column actually holds a null. Also the cheapest way for a test or a memory report to prove that. Always 0 for the string and temporal kinds, whose null state is not a column bitmap.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the column.

Return Value integer(kind=int64)

bitmap size in bytes.

procedure, public :: has_validity_storage

Whether nulling would still have to allocate.

interface

private module function has_validity_storage(self) result(res)

Whether this column's validity storage already exists, i.e. whether nulling an element would still have to ALLOCATE something.

The answer depends on which of the three validity mechanisms the kind uses (see set_null): a bitmap kind answers whether its bitmap is allocated, a string kind asks its parquet_string_column, and a temporal kind is always .true. -- its null state lives in the element itself, so there is nothing to allocate and never was.

Exists for concurrency: nulling elements of the same column from two threads races on that lazy allocation, and this plus ensure_validity is how a caller (or the table layer's own guard) removes the race rather than detecting it.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the column.

Return Value logical

.true. when nulling would allocate nothing.

procedure, public :: ensure_validity

Allocate the validity storage up front.

interface

private module subroutine ensure_validity(self)

Materializes this column's validity storage now, leaving every element valid.

Idempotent, and a no-op for a temporal kind (which has nothing to allocate). Changes no value and no null state -- only when the allocation happens. See has_validity_storage for why that matters.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column.

procedure, public :: unit_string

Copy out the unit string ("" when unset).

interface

private module subroutine unit_string(self, u)

Copies the unit string out ("" when no unit is set).

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the column.
character(len=:),	intent(out),		allocatable	::	u	the unit string, or "".

procedure, public :: set_unit

Set (or clear) the unit string.

interface

private module subroutine set_unit(self, u)

Sets the unit string; passing "" clears it.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column.
character(len=*),	intent(in)			::	u	the unit string ("" clears).

procedure, public :: any_null

Whether the column holds at least one null.

interface

private module function any_null(self) result(res)

Whether the column holds at least one null. Cheap for bitmap-backed and string kinds; for temporal kinds (whose null state lives inside each element) the answer is cached, and a rescan is needed after a mutation -- which is why self is intent(inout) despite this being a query.

That intent(inout) is real: this WRITES to the column, so it must not be called on one another thread can reach. The temporal rescan ends by clearing the dirty flag, which silently discards a set_null that another thread raised during the scan -- and the column then reports "no nulls" for good, with nothing to notice. Being reached through the table's cache POINTER is what makes that possible from a table accessor declared intent(in), so the compiler cannot object either. Anything on a read path must call parquet_column_any_null below instead; %has_nulls learned this the hard way. See feature_risks.md Risk-136.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column (null cache may be refreshed).

Return Value logical

.true. when at least one row is null.

generic, public :: is_null => is_null_row, is_null_elem

Whether row i (or, with e, element e of row i) is null. On a vector kind the row form answers "ANY element of the row is null"; see the module doc.

private interface is_null_row()

Arguments
None
private interface is_null_elem()

Arguments
None

procedure, public :: row_validity

Build the whole per-row validity mask at once.

interface

private module subroutine row_validity(self, valid)

Fills valid with one entry per row: .true. where the row is not null.

The bulk counterpart of is_null, and worth having as its own entry point rather than a loop at the call site for two reasons. It is inside the module that owns the bitmap, so it can walk that bitmap a 64-bit word at a time and skip whole runs of valid rows, where an outside loop can only ask one row at a time through a call the compiler cannot inline. And a null-free column returns an UNALLOCATED valid -- which, passed on to an optional dummy such as parquet_write_column's is_valid=, counts as an absent argument (F2018 15.5.2.12), so the common no-nulls case costs no allocation and no scan at all rather than a mask that is uniformly .true.. Callers must therefore test allocated(valid) and not assume a mask came back. intent(in), deliberately. It used to be intent(inout) so that the temporal kinds' null cache could be refreshed in passing, and that single word made the whole procedure unreachable from anything holding a column by intent(in) -- which includes pf_argsort(column) and every other read-only consumer, each of which was left calling is_null(i) once per row instead. The cache is an optimisation for any_null, never a correctness requirement, so this reads it when it is clean and scans when it is dirty rather than writing it. Widening an argument's intent this way is source-compatible: every call that compiled before still does.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the column.
logical,	intent(out),		allocatable	::	valid(:)	per-row mask, or unallocated when no nulls.

procedure, public :: element_validity

Build the whole per-ELEMENT validity mask at once.

interface

private module subroutine element_validity(self, valid)

Fills valid with one entry per ELEMENT, shaped (width, nrows): the column's true validity state, without the row summary row_validity applies.

Same contract as row_validity in every other respect, and for the same reasons: a null-free column returns an UNALLOCATED valid (so it reaches an optional dummy as an absent argument and costs nothing), and the bitmap is walked a 64-bit word at a time rather than a call per element. This is the shape parquet_write_column takes for a vector column, so a table write can hand it straight on.

Note the memory: LOGICAL is 4 bytes under gfortran, so this is 4*width*nrows bytes -- for a wide column, orders of magnitude more than the bitmap it is built from. Ask for it when the whole mask is genuinely needed; use is_null(i, e) for a few elements.

intent(in) for the same reason row_validity is -- see its note.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in)			::	self	the column.
logical,	intent(out),		allocatable	::	valid(:,:)	(element, row) mask, or unallocated when no nulls.

generic, public :: set_null => set_null_row, set_null_elem

Marks row i null, or with e just element e of it.

private interface set_null_row()

Arguments
None
private interface set_null_elem()

Arguments
None

generic, public :: clear_null => clear_null_row, clear_null_elem

Marks row i valid, or with e just element e of it.

private interface clear_null_row()

Arguments
None
private interface clear_null_elem()

Arguments
None

generic, public :: set_validity => set_validity_elems, set_validity_rows

Writes a whole validity mask in one pass. A rank-2 (width, nrows) mask is per ELEMENT; a rank-1 (nrows) mask is per ROW, marking every element of a .false. row null -- the same row/element pairing %set_null and %is_null already use.

private interface set_validity_elems()

Arguments
None
private interface set_validity_rows()

Arguments
None

procedure, public :: compact_validity

Drop the bitmap when no nulls remain.

interface

private module subroutine compact_validity(self)

Scans for remaining nulls and releases the bitmap when none are found (R2 iv).

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column.

generic, public :: reserve => reserve_i32, reserve_i64

Grows the storage capacity to hold at least n rows without changing the row count.

private interface reserve_i32()

Arguments
None
private interface reserve_i64()

Arguments
None

procedure, public :: shrink_to_fit

Release capacity beyond the rows stored.

interface

private module subroutine shrink_to_fit(self, released)

Releases capacity beyond the rows actually stored, so the column occupies exactly what it holds. A no-op when there is no slack.

Slack only ever comes from an append: every rebuild (reindex, gather, delete_by_mask) allocates exact-fit, and so does reading a column from a file. So on a column that has not been appended to this does nothing at all.

Invalidates any pointer previously obtained from %data_ptr if it actually reallocates.

released reports whether it did. The caller cannot work that out for itself without knowing which of the three storage mechanisms the kind uses -- a string column's spare capacity lives in its embedded store's offsets AND its byte payload, neither of which is cap -- so the answer is produced here, where the kind is known. parquet_table%compact needs it to decide whether to advance %generation().

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column.
logical,	intent(out),	optional		::	released	.true. if anything was reallocated.

procedure, public :: append

Append another column of identical kind/width.

interface

private module subroutine append(self, other)

Appends every row of other, which must have identical kind and width.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the destination column.
type(parquet_column),	intent(in)			::	other	the source column (unchanged).

procedure, public :: append_row_of

Append one row of another column (internal; see below).

interface

private module subroutine append_row_of(self, other, irow)

Appends row irow of other as this column's next row, carrying that row's validity.

INTERNAL plumbing, public only because Fortran offers no narrower visibility -- parquet_tables is a different module and cannot reach parquet_column's private components, and %append(row) needs exactly this to append one row without building a one-row column and a one-row table first. Same reasoning as reindex_trusted, and it is likewise absent from README.md's API overview.

Kind and width are checked on every call even though the table layer has already validated them: two integer comparisons against a call that copies a whole row, on a procedure a caller who validated nothing can still reach.

Arguments

Type	Intent		Name
class(parquet_column),	intent(inout)	::	self	the destination column.
type(parquet_column),	intent(in)	::	other	the source column (unchanged).
integer(kind=int64),	intent(in)	::	irow	1-based row of `other` to append.

procedure, public :: append_nulls

Append n all-null rows.

interface

private module subroutine append_nulls(self, n)

Appends n all-null rows (allocating the bitmap if this is the first null).

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column.
integer(kind=int64),	intent(in)			::	n	number of null rows to append (>= 0).

procedure, public :: paste

Overwrite an existing row range from another column.

interface

private module subroutine paste(self, src, at, from, count)

Overwrites the already-allocated rows at .. at+count-1 with count rows of src, starting at src row from. Kind and width must match; nothing is reallocated and nrows does not change.

This is the counterpart of append for a column whose final row count is known up front: init it once at full size, then paste each piece into place. Assembling a column from k pieces with append instead costs O(k^2) copying, because every append reallocates the whole column exact-fit and copies everything already in it (see grow_storage) -- which is why parquet_table's slice regime uses this.

from/count default to 1 and src%nrows, i.e. all of src. Passing them copies a sub-range directly, so a caller trimming a piece to a row window does not need to build a mask and call delete_by_mask first.

Validity follows the same kind-dispatched rules as append: bitmap-backed kinds copy src's null bits into the destination positions (materializing this column's bitmap only if src actually has nulls), and the temporal kinds carry their null state inside the pasted elements. The string kinds are not supported and abort: their store is variable-length, so a row range cannot be overwritten in place -- and they already grow geometrically under append, so there is nothing to gain (see parquet_string_column).

Arguments

Type	Intent	Optional		Name
class(parquet_column),	intent(inout)		::	self	the destination column.
type(parquet_column),	intent(in)		::	src	the source column (unchanged).
integer(kind=int64),	intent(in)		::	at	1-based first destination row.
integer(kind=int64),	intent(in),	optional	::	from	1-based first source row (default 1).
integer(kind=int64),	intent(in),	optional	::	count	rows to copy (default all of `src`).

procedure, public :: delete_by_mask

Keep only rows whose mask entry is .true.

interface

private module subroutine delete_by_mask(self, keep)

Keeps only the rows whose keep entry is .true., preserving order.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column.
logical,	intent(in)			::	keep(:)	.true. for every row to retain.

procedure, public :: reindex

Reorder rows by a permutation.

interface

private module subroutine reindex(self, perm)

Reorders rows so row k of the result is the row that was at perm(k). perm must be a true permutation of 1..nrows and is fully validated before anything is modified.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column.
integer(kind=int64),	intent(in)			::	perm(:)	1-based permutation of 1..nrows.

procedure, public :: reindex_trusted

INTERNAL -- reindex without the duplicate/range scan, for a permutation the caller has already established is one. Public only because Fortran offers no narrower visibility; see the interface below.

interface

private module subroutine reindex_trusted(self, perm)

INTERNAL. reindex without the O(nrows) range/duplicate scan, for a permutation the caller has already established is one. The O(1) length check still runs, since it guards a different invariant and costs nothing.

Public only because Fortran has no narrower visibility: parquet_column's components are private to this module, so parquet_tables cannot reach them, and %sort_by needs this to stop re-validating one permutation once per column -- measured at a third of its total time on a 24-column table. Deliberately absent from README.md's API overview. A caller who passes a non-permutation gets silently duplicated and dropped rows; a library can refuse accidents, not deliberate misuse of something documented as internal.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(inout)			::	self	the column.
integer(kind=int64),	intent(in)			::	perm(:)	1-based permutation of 1..nrows, unchecked.

generic, public :: gather => gather_i32, gather_i64

Keeps the listed rows, in the listed order. Unlike reindex the list may be any length, and unlike delete_by_mask it may reorder -- see the interface below.

private interface gather_i32()

Arguments
None
private interface gather_i64()

Arguments
None

procedure, public :: string_column

Pointer to the embedded string store.

interface

private module subroutine string_column(self, p)

Pointer to the embedded string store (PK_STRING / PK_STRING_VEC only). The column must outlive the pointer; any structural mutation invalidates it.

Arguments

Type	Intent	Optional	Attributes		Name
class(parquet_column),	intent(in),		target	::	self	the column.
type(parquet_string_column),	intent(out),		pointer	::	p	alias to the string store.

generic, public :: get_at => get_at_i32, get_at_i64, get_at_f32, get_at_f64, get_at_bool, get_at_str, get_at_date, get_at_time, get_at_ts, get_at_i32v, get_at_i64v, get_at_f32v, get_at_f64v, get_at_boolv, get_at_strv, get_at_datev, get_at_timev, get_at_tsv

Read element i (or row i's vector) out.

private interface get_at_i32()

Arguments
None
private interface get_at_i64()

Arguments
None
private interface get_at_f32()

Arguments
None
private interface get_at_f64()

Arguments
None
private interface get_at_bool()

Arguments
None
private interface get_at_str()

Arguments
None
private interface get_at_date()

Arguments
None
private interface get_at_time()

Arguments
None
private interface get_at_ts()

Arguments
None
private interface get_at_i32v()

Arguments
None
private interface get_at_i64v()

Arguments
None
private interface get_at_f32v()

Arguments
None
private interface get_at_f64v()

Arguments
None
private interface get_at_boolv()

Arguments
None
private interface get_at_strv()

Arguments
None
private interface get_at_datev()

Arguments
None
private interface get_at_timev()

Arguments
None
private interface get_at_tsv()

Arguments
None

generic, public :: set_at => set_at_i32, set_at_i64, set_at_f32, set_at_f64, set_at_bool, set_at_str, set_at_date, set_at_time, set_at_ts, set_at_i32v, set_at_i64v, set_at_f32v, set_at_f64v, set_at_boolv, set_at_strv, set_at_datev, set_at_timev, set_at_tsv

Write element i (or row i's vector).

private interface set_at_i32()

Arguments
None
private interface set_at_i64()

Arguments
None
private interface set_at_f32()

Arguments
None
private interface set_at_f64()

Arguments
None
private interface set_at_bool()

Arguments
None
private interface set_at_str()

Arguments
None
private interface set_at_date()

Arguments
None
private interface set_at_time()

Arguments
None
private interface set_at_ts()

Arguments
None
private interface set_at_i32v()

Arguments
None
private interface set_at_i64v()

Arguments
None
private interface set_at_f32v()

Arguments
None
private interface set_at_f64v()

Arguments
None
private interface set_at_boolv()

Arguments
None
private interface set_at_strv()

Arguments
None
private interface set_at_datev()

Arguments
None
private interface set_at_timev()

Arguments
None
private interface set_at_tsv()

Arguments
None

generic, public :: get_elem => get_elem_i32v, get_elem_i64v, get_elem_f32v, get_elem_f64v, get_elem_boolv, get_elem_strv, get_elem_datev, get_elem_timev, get_elem_tsv

Read ONE element of row i's vector, without materialising the row.

private interface get_elem_i32v()

Arguments
None
private interface get_elem_i64v()

Arguments
None
private interface get_elem_f32v()

Arguments
None
private interface get_elem_f64v()

Arguments
None
private interface get_elem_boolv()

Arguments
None
private interface get_elem_strv()

Arguments
None
private interface get_elem_datev()

Arguments
None
private interface get_elem_timev()

Arguments
None
private interface get_elem_tsv()

Arguments
None

generic, public :: set_elem => set_elem_i32v, set_elem_i64v, set_elem_f32v, set_elem_f64v, set_elem_boolv, set_elem_strv, set_elem_datev, set_elem_timev, set_elem_tsv

Write ONE element of row i's vector, without materialising the row.

private interface set_elem_i32v()

Arguments
None
private interface set_elem_i64v()

Arguments
None
private interface set_elem_f32v()

Arguments
None
private interface set_elem_f64v()

Arguments
None
private interface set_elem_boolv()

Arguments
None
private interface set_elem_strv()

Arguments
None
private interface set_elem_datev()

Arguments
None
private interface set_elem_timev()

Arguments
None
private interface set_elem_tsv()

Arguments
None

generic, public :: set_all => set_all_i32, set_all_i64, set_all_f32, set_all_f64, set_all_bool, set_all_str, set_all_date, set_all_time, set_all_ts, set_all_i32v, set_all_i64v, set_all_f32v, set_all_f64v, set_all_boolv, set_all_strv, set_all_datev, set_all_timev, set_all_tsv

Replace every value in the column.

private interface set_all_i32()

Arguments
None
private interface set_all_i64()

Arguments
None
private interface set_all_f32()

Arguments
None
private interface set_all_f64()

Arguments
None
private interface set_all_bool()

Arguments
None
private interface set_all_str()

Arguments
None
private interface set_all_date()

Arguments
None
private interface set_all_time()

Arguments
None
private interface set_all_ts()

Arguments
None
private interface set_all_i32v()

Arguments
None
private interface set_all_i64v()

Arguments
None
private interface set_all_f32v()

Arguments
None
private interface set_all_f64v()

Arguments
None
private interface set_all_boolv()

Arguments
None
private interface set_all_strv()

Arguments
None
private interface set_all_datev()

Arguments
None
private interface set_all_timev()

Arguments
None
private interface set_all_tsv()

Arguments
None

generic, public :: adopt => adopt_i32, adopt_i64, adopt_f32, adopt_f64, adopt_bool, adopt_date, adopt_time, adopt_ts, adopt_i32v, adopt_i64v, adopt_f32v, adopt_f64v, adopt_boolv, adopt_datev, adopt_timev, adopt_tsv

Take ownership of an array outright, without copying it.

private interface adopt_i32()

Arguments
None
private interface adopt_i64()

Arguments
None
private interface adopt_f32()

Arguments
None
private interface adopt_f64()

Arguments
None
private interface adopt_bool()

Arguments
None
private interface adopt_date()

Arguments
None
private interface adopt_time()

Arguments
None
private interface adopt_ts()

Arguments
None
private interface adopt_i32v()

Arguments
None
private interface adopt_i64v()

Arguments
None
private interface adopt_f32v()

Arguments
None
private interface adopt_f64v()

Arguments
None
private interface adopt_boolv()

Arguments
None
private interface adopt_datev()

Arguments
None
private interface adopt_timev()

Arguments
None
private interface adopt_tsv()

Arguments
None

generic, public :: data_ptr => data_ptr_i32, data_ptr_i64, data_ptr_f32, data_ptr_f64, data_ptr_bool, data_ptr_date, data_ptr_time, data_ptr_ts, data_ptr_i32v, data_ptr_i64v, data_ptr_f32v, data_ptr_f64v, data_ptr_boolv, data_ptr_datev, data_ptr_timev, data_ptr_tsv

Zero-copy typed pointer to the active storage.

private interface data_ptr_i32()

Arguments
None
private interface data_ptr_i64()

Arguments
None
private interface data_ptr_f32()

Arguments
None
private interface data_ptr_f64()

Arguments
None
private interface data_ptr_bool()

Arguments
None
private interface data_ptr_date()

Arguments
None
private interface data_ptr_time()

Arguments
None
private interface data_ptr_ts()

Arguments
None
private interface data_ptr_i32v()

Arguments
None
private interface data_ptr_i64v()

Arguments
None
private interface data_ptr_f32v()

Arguments
None
private interface data_ptr_f64v()

Arguments
None
private interface data_ptr_boolv()

Arguments
None
private interface data_ptr_datev()

Arguments
None
private interface data_ptr_timev()

Arguments
None
private interface data_ptr_tsv()

Arguments
None

generic, public :: append_values => append_values_i32, append_values_i64, append_values_f32, append_values_f64, append_values_bool, append_values_str, append_values_date, append_values_time, append_values_ts, append_values_i32v, append_values_i64v, append_values_f32v, append_values_f64v, append_values_boolv, append_values_strv, append_values_datev, append_values_timev, append_values_tsv

Append values, growing the column.

private interface append_values_i32()

Arguments
None
private interface append_values_i64()

Arguments
None
private interface append_values_f32()

Arguments
None
private interface append_values_f64()

Arguments
None
private interface append_values_bool()

Arguments
None
private interface append_values_str()

Arguments
None
private interface append_values_date()

Arguments
None
private interface append_values_time()

Arguments
None
private interface append_values_ts()

Arguments
None
private interface append_values_i32v()

Arguments
None
private interface append_values_i64v()

Arguments
None
private interface append_values_f32v()

Arguments
None
private interface append_values_f64v()

Arguments
None
private interface append_values_boolv()

Arguments
None
private interface append_values_strv()

Arguments
None
private interface append_values_datev()

Arguments
None
private interface append_values_timev()

Arguments
None
private interface append_values_tsv()

Arguments
None

Source Code

    type :: parquet_column
        private
        integer :: kind = PK_NONE                      !! active PK_* discriminator.
        integer(int64) :: nrows = 0                    !! number of rows stored.
        !> Rows the active storage is ALLOCATED for; always >= nrows, 0 when nothing is allocated.
        !!
        !! Growth is geometric (1.5x, `ensure_capacity`), so appending row by row is amortised O(1)
        !! instead of the O(n^2) an exact-fit realloc per append would cost. The slack is invisible
        !! to every caller because every read of a storage array is bounded by `1:nrows` -- an
        !! UNBOUNDED read would return uninitialised tail elements, which is the one silent failure
        !! this component introduces (see feature_risks.md).
        integer(int64) :: cap = 0
        integer(int32) :: width = 1                    !! values per row; > 1 only for *_VEC kinds.
        logical :: has_nulls = .false.                 !! .true. while the bitmap is materialized.
        logical :: nulls_dirty = .true.                !! temporal kinds: the null cache needs a rescan.
        logical :: nulls_cached = .false.              !! temporal kinds: cached "column has >= 1 null".
        integer(int64), allocatable :: validity(:)     !! null bitmap (1 = null); allocated iff has_nulls.
        character(len=:), allocatable :: unit          !! unit string from the MAML `unit:` key (D10).
        type(parquet_string_column), allocatable :: str !! PK_STRING / PK_STRING_VEC storage (DD1).
        integer(int32), allocatable :: i32(:)   !! PK_INT32 storage, scalar.
        integer(int64), allocatable :: i64(:)   !! PK_INT64 storage, scalar.
        real(real32), allocatable :: f32(:)   !! PK_FLOAT32 storage, scalar.
        real(real64), allocatable :: f64(:)   !! PK_FLOAT64 storage, scalar.
        logical, allocatable :: bool(:)   !! PK_LOGICAL storage, scalar.
        type(parquet_date), allocatable :: dt(:)   !! PK_DATE storage, scalar.
        type(parquet_time), allocatable :: tm(:)   !! PK_TIME storage, scalar.
        type(parquet_timestamp), allocatable :: ts(:)   !! PK_TIMESTAMP storage, scalar.
        integer(int32), allocatable :: i32v(:,:)   !! PK_INT32_VEC storage, vector (width, nrows).
        integer(int64), allocatable :: i64v(:,:)   !! PK_INT64_VEC storage, vector (width, nrows).
        real(real32), allocatable :: f32v(:,:)   !! PK_FLOAT32_VEC storage, vector (width, nrows).
        real(real64), allocatable :: f64v(:,:)   !! PK_FLOAT64_VEC storage, vector (width, nrows).
        logical, allocatable :: boolv(:,:)   !! PK_LOGICAL_VEC storage, vector (width, nrows).
        type(parquet_date), allocatable :: dtv(:,:)   !! PK_DATE_VEC storage, vector (width, nrows).
        type(parquet_time), allocatable :: tmv(:,:)   !! PK_TIME_VEC storage, vector (width, nrows).
        type(parquet_timestamp), allocatable :: tsv(:,:)   !! PK_TIMESTAMP_VEC storage, vector (width, nrows).
        class(*), allocatable :: container(:)          !! reserved payload for PK_LIST/PK_MAP/PK_STRUCT.
    contains
        ! --- lifecycle ---
        procedure :: init                              !! Set kind/geometry and allocate empty storage.
        procedure :: clear                             !! Release all storage and reset to PK_NONE.
        procedure :: deep_copy                         !! Independent copy of values, validity and unit.
        procedure :: move_from                         !! Take over another column's storage, leaving it empty.
        ! --- queries ---
        procedure :: kindof                            !! The active PK_* discriminator.
        procedure :: length                            !! Number of rows stored.
        procedure :: capacity                          !! Rows the storage is allocated for (>= length()).
        procedure :: colwidth                          !! Values per row (1 for scalar kinds).
        procedure :: validity_bytes                    !! Bytes the null bitmap occupies (0 when sparse).
        procedure :: has_validity_storage               !! Whether nulling would still have to allocate.
        procedure :: ensure_validity                    !! Allocate the validity storage up front.
        procedure :: unit_string                       !! Copy out the unit string ("" when unset).
        procedure :: set_unit                          !! Set (or clear) the unit string.
        procedure :: any_null                          !! Whether the column holds at least one null.
        procedure, private :: is_null_row               !! is_null specific taking a row index alone.
        procedure, private :: is_null_elem              !! is_null specific taking a row and an element.
        !> Whether row `i` (or, with `e`, element `e` of row `i`) is null. On a vector kind the
        !! row form answers "ANY element of the row is null"; see the module doc.
        generic :: is_null => is_null_row, is_null_elem
        procedure :: row_validity                      !! Build the whole per-row validity mask at once.
        procedure :: element_validity                  !! Build the whole per-ELEMENT validity mask at once.
        ! --- validity mutation (sparse: see the module doc) ---
        procedure, private :: set_null_row              !! set_null specific taking a row index alone.
        procedure, private :: set_null_elem             !! set_null specific taking a row and an element.
        !> Marks row `i` null, or with `e` just element `e` of it.
        generic :: set_null => set_null_row, set_null_elem
        procedure, private :: clear_null_row            !! clear_null specific taking a row index alone.
        procedure, private :: clear_null_elem           !! clear_null specific taking a row and an element.
        !> Marks row `i` valid, or with `e` just element `e` of it.
        generic :: clear_null => clear_null_row, clear_null_elem
        procedure, private :: set_validity_elems       !! set_validity specific, per-ELEMENT (width, nrows) mask.
        procedure, private :: set_validity_rows        !! set_validity specific, per-ROW mask.
        !> Writes a whole validity mask in one pass. A rank-2 `(width, nrows)` mask is per ELEMENT;
        !! a rank-1 `(nrows)` mask is per ROW, marking every element of a `.false.` row null -- the
        !! same row/element pairing `%set_null` and `%is_null` already use.
        generic :: set_validity => set_validity_elems, set_validity_rows
        procedure :: compact_validity                  !! Drop the bitmap when no nulls remain.
        ! --- capacity ---
        procedure, private :: reserve_i32              !! int32 specific of reserve.
        procedure, private :: reserve_i64              !! int64 specific of reserve.
        !> Grows the storage capacity to hold at least `n` rows without changing the row count.
        generic :: reserve => reserve_i32, reserve_i64
        procedure :: shrink_to_fit                     !! Release capacity beyond the rows stored.
        ! --- structural mutation ---
        procedure :: append                            !! Append another column of identical kind/width.
        procedure :: append_row_of                     !! Append one row of another column (internal; see below).
        procedure :: append_nulls                      !! Append n all-null rows.
        procedure :: paste                             !! Overwrite an existing row range from another column.
        procedure :: delete_by_mask                    !! Keep only rows whose mask entry is .true.
        procedure :: reindex                           !! Reorder rows by a permutation.
        !> INTERNAL -- reindex without the duplicate/range scan, for a permutation the caller has
        !! already established is one. Public only because Fortran offers no narrower visibility;
        !! see the interface below.
        procedure :: reindex_trusted
        procedure, private :: gather_i32               !! int32 specific of gather.
        procedure, private :: gather_i64               !! int64 specific of gather.
        !> Keeps the listed rows, in the listed order. Unlike `reindex` the list may be any length,
        !! and unlike `delete_by_mask` it may reorder -- see the interface below.
        generic :: gather => gather_i32, gather_i64
        ! --- string-kind storage access (PK_STRING / PK_STRING_VEC) ---
        procedure :: string_column                     !! Pointer to the embedded string store.
        ! --- get_at ---
        procedure, private :: get_at_i32   !! get_at specific for the i32 kind.
        procedure, private :: get_at_i64   !! get_at specific for the i64 kind.
        procedure, private :: get_at_f32   !! get_at specific for the f32 kind.
        procedure, private :: get_at_f64   !! get_at specific for the f64 kind.
        procedure, private :: get_at_bool   !! get_at specific for the bool kind.
        procedure, private :: get_at_str   !! get_at specific for the str kind.
        procedure, private :: get_at_date   !! get_at specific for the date kind.
        procedure, private :: get_at_time   !! get_at specific for the time kind.
        procedure, private :: get_at_ts   !! get_at specific for the ts kind.
        procedure, private :: get_at_i32v   !! get_at specific for the i32v kind.
        procedure, private :: get_at_i64v   !! get_at specific for the i64v kind.
        procedure, private :: get_at_f32v   !! get_at specific for the f32v kind.
        procedure, private :: get_at_f64v   !! get_at specific for the f64v kind.
        procedure, private :: get_at_boolv   !! get_at specific for the boolv kind.
        procedure, private :: get_at_strv   !! get_at specific for the strv kind.
        procedure, private :: get_at_datev   !! get_at specific for the datev kind.
        procedure, private :: get_at_timev   !! get_at specific for the timev kind.
        procedure, private :: get_at_tsv   !! get_at specific for the tsv kind.
        !> Read element i (or row i's vector) out.
        generic :: get_at => get_at_i32, get_at_i64, get_at_f32, get_at_f64, get_at_bool, get_at_str, &
            get_at_date, get_at_time, get_at_ts, get_at_i32v, get_at_i64v, get_at_f32v, get_at_f64v, &
            get_at_boolv, get_at_strv, get_at_datev, get_at_timev, get_at_tsv
        ! --- set_at ---
        procedure, private :: set_at_i32   !! set_at specific for the i32 kind.
        procedure, private :: set_at_i64   !! set_at specific for the i64 kind.
        procedure, private :: set_at_f32   !! set_at specific for the f32 kind.
        procedure, private :: set_at_f64   !! set_at specific for the f64 kind.
        procedure, private :: set_at_bool   !! set_at specific for the bool kind.
        procedure, private :: set_at_str   !! set_at specific for the str kind.
        procedure, private :: set_at_date   !! set_at specific for the date kind.
        procedure, private :: set_at_time   !! set_at specific for the time kind.
        procedure, private :: set_at_ts   !! set_at specific for the ts kind.
        procedure, private :: set_at_i32v   !! set_at specific for the i32v kind.
        procedure, private :: set_at_i64v   !! set_at specific for the i64v kind.
        procedure, private :: set_at_f32v   !! set_at specific for the f32v kind.
        procedure, private :: set_at_f64v   !! set_at specific for the f64v kind.
        procedure, private :: set_at_boolv   !! set_at specific for the boolv kind.
        procedure, private :: set_at_strv   !! set_at specific for the strv kind.
        procedure, private :: set_at_datev   !! set_at specific for the datev kind.
        procedure, private :: set_at_timev   !! set_at specific for the timev kind.
        procedure, private :: set_at_tsv   !! set_at specific for the tsv kind.
        !> Write element i (or row i's vector).
        generic :: set_at => set_at_i32, set_at_i64, set_at_f32, set_at_f64, set_at_bool, set_at_str, &
            set_at_date, set_at_time, set_at_ts, set_at_i32v, set_at_i64v, set_at_f32v, set_at_f64v, &
            set_at_boolv, set_at_strv, set_at_datev, set_at_timev, set_at_tsv
        ! --- get_elem ---
        procedure, private :: get_elem_i32v   !! get_elem specific for the i32v kind.
        procedure, private :: get_elem_i64v   !! get_elem specific for the i64v kind.
        procedure, private :: get_elem_f32v   !! get_elem specific for the f32v kind.
        procedure, private :: get_elem_f64v   !! get_elem specific for the f64v kind.
        procedure, private :: get_elem_boolv   !! get_elem specific for the boolv kind.
        procedure, private :: get_elem_strv   !! get_elem specific for the strv kind.
        procedure, private :: get_elem_datev   !! get_elem specific for the datev kind.
        procedure, private :: get_elem_timev   !! get_elem specific for the timev kind.
        procedure, private :: get_elem_tsv   !! get_elem specific for the tsv kind.
        !> Read ONE element of row i's vector, without materialising the row.
        generic :: get_elem => get_elem_i32v, get_elem_i64v, get_elem_f32v, get_elem_f64v, get_elem_boolv, &
            get_elem_strv, get_elem_datev, get_elem_timev, get_elem_tsv
        ! --- set_elem ---
        procedure, private :: set_elem_i32v   !! set_elem specific for the i32v kind.
        procedure, private :: set_elem_i64v   !! set_elem specific for the i64v kind.
        procedure, private :: set_elem_f32v   !! set_elem specific for the f32v kind.
        procedure, private :: set_elem_f64v   !! set_elem specific for the f64v kind.
        procedure, private :: set_elem_boolv   !! set_elem specific for the boolv kind.
        procedure, private :: set_elem_strv   !! set_elem specific for the strv kind.
        procedure, private :: set_elem_datev   !! set_elem specific for the datev kind.
        procedure, private :: set_elem_timev   !! set_elem specific for the timev kind.
        procedure, private :: set_elem_tsv   !! set_elem specific for the tsv kind.
        !> Write ONE element of row i's vector, without materialising the row.
        generic :: set_elem => set_elem_i32v, set_elem_i64v, set_elem_f32v, set_elem_f64v, set_elem_boolv, &
            set_elem_strv, set_elem_datev, set_elem_timev, set_elem_tsv
        ! --- set_all ---
        procedure, private :: set_all_i32   !! set_all specific for the i32 kind.
        procedure, private :: set_all_i64   !! set_all specific for the i64 kind.
        procedure, private :: set_all_f32   !! set_all specific for the f32 kind.
        procedure, private :: set_all_f64   !! set_all specific for the f64 kind.
        procedure, private :: set_all_bool   !! set_all specific for the bool kind.
        procedure, private :: set_all_str   !! set_all specific for the str kind.
        procedure, private :: set_all_date   !! set_all specific for the date kind.
        procedure, private :: set_all_time   !! set_all specific for the time kind.
        procedure, private :: set_all_ts   !! set_all specific for the ts kind.
        procedure, private :: set_all_i32v   !! set_all specific for the i32v kind.
        procedure, private :: set_all_i64v   !! set_all specific for the i64v kind.
        procedure, private :: set_all_f32v   !! set_all specific for the f32v kind.
        procedure, private :: set_all_f64v   !! set_all specific for the f64v kind.
        procedure, private :: set_all_boolv   !! set_all specific for the boolv kind.
        procedure, private :: set_all_strv   !! set_all specific for the strv kind.
        procedure, private :: set_all_datev   !! set_all specific for the datev kind.
        procedure, private :: set_all_timev   !! set_all specific for the timev kind.
        procedure, private :: set_all_tsv   !! set_all specific for the tsv kind.
        !> Replace every value in the column.
        generic :: set_all => set_all_i32, set_all_i64, set_all_f32, set_all_f64, set_all_bool, set_all_str, &
            set_all_date, set_all_time, set_all_ts, set_all_i32v, set_all_i64v, set_all_f32v, set_all_f64v, &
            set_all_boolv, set_all_strv, set_all_datev, set_all_timev, set_all_tsv
        ! --- adopt ---
        procedure, private :: adopt_i32   !! adopt specific for the i32 kind.
        procedure, private :: adopt_i64   !! adopt specific for the i64 kind.
        procedure, private :: adopt_f32   !! adopt specific for the f32 kind.
        procedure, private :: adopt_f64   !! adopt specific for the f64 kind.
        procedure, private :: adopt_bool   !! adopt specific for the bool kind.
        procedure, private :: adopt_date   !! adopt specific for the date kind.
        procedure, private :: adopt_time   !! adopt specific for the time kind.
        procedure, private :: adopt_ts   !! adopt specific for the ts kind.
        procedure, private :: adopt_i32v   !! adopt specific for the i32v kind.
        procedure, private :: adopt_i64v   !! adopt specific for the i64v kind.
        procedure, private :: adopt_f32v   !! adopt specific for the f32v kind.
        procedure, private :: adopt_f64v   !! adopt specific for the f64v kind.
        procedure, private :: adopt_boolv   !! adopt specific for the boolv kind.
        procedure, private :: adopt_datev   !! adopt specific for the datev kind.
        procedure, private :: adopt_timev   !! adopt specific for the timev kind.
        procedure, private :: adopt_tsv   !! adopt specific for the tsv kind.
        !> Take ownership of an array outright, without copying it.
        generic :: adopt => adopt_i32, adopt_i64, adopt_f32, adopt_f64, adopt_bool, adopt_date, adopt_time, &
            adopt_ts, adopt_i32v, adopt_i64v, adopt_f32v, adopt_f64v, adopt_boolv, adopt_datev, adopt_timev, &
            adopt_tsv
        ! --- data_ptr ---
        procedure, private :: data_ptr_i32   !! data_ptr specific for the i32 kind.
        procedure, private :: data_ptr_i64   !! data_ptr specific for the i64 kind.
        procedure, private :: data_ptr_f32   !! data_ptr specific for the f32 kind.
        procedure, private :: data_ptr_f64   !! data_ptr specific for the f64 kind.
        procedure, private :: data_ptr_bool   !! data_ptr specific for the bool kind.
        procedure, private :: data_ptr_date   !! data_ptr specific for the date kind.
        procedure, private :: data_ptr_time   !! data_ptr specific for the time kind.
        procedure, private :: data_ptr_ts   !! data_ptr specific for the ts kind.
        procedure, private :: data_ptr_i32v   !! data_ptr specific for the i32v kind.
        procedure, private :: data_ptr_i64v   !! data_ptr specific for the i64v kind.
        procedure, private :: data_ptr_f32v   !! data_ptr specific for the f32v kind.
        procedure, private :: data_ptr_f64v   !! data_ptr specific for the f64v kind.
        procedure, private :: data_ptr_boolv   !! data_ptr specific for the boolv kind.
        procedure, private :: data_ptr_datev   !! data_ptr specific for the datev kind.
        procedure, private :: data_ptr_timev   !! data_ptr specific for the timev kind.
        procedure, private :: data_ptr_tsv   !! data_ptr specific for the tsv kind.
        !> Zero-copy typed pointer to the active storage.
        generic :: data_ptr => data_ptr_i32, data_ptr_i64, data_ptr_f32, data_ptr_f64, data_ptr_bool, &
            data_ptr_date, data_ptr_time, data_ptr_ts, data_ptr_i32v, data_ptr_i64v, data_ptr_f32v, &
            data_ptr_f64v, data_ptr_boolv, data_ptr_datev, data_ptr_timev, data_ptr_tsv
        ! --- append_values ---
        procedure, private :: append_values_i32   !! append_values specific for the i32 kind.
        procedure, private :: append_values_i64   !! append_values specific for the i64 kind.
        procedure, private :: append_values_f32   !! append_values specific for the f32 kind.
        procedure, private :: append_values_f64   !! append_values specific for the f64 kind.
        procedure, private :: append_values_bool   !! append_values specific for the bool kind.
        procedure, private :: append_values_str   !! append_values specific for the str kind.
        procedure, private :: append_values_date   !! append_values specific for the date kind.
        procedure, private :: append_values_time   !! append_values specific for the time kind.
        procedure, private :: append_values_ts   !! append_values specific for the ts kind.
        procedure, private :: append_values_i32v   !! append_values specific for the i32v kind.
        procedure, private :: append_values_i64v   !! append_values specific for the i64v kind.
        procedure, private :: append_values_f32v   !! append_values specific for the f32v kind.
        procedure, private :: append_values_f64v   !! append_values specific for the f64v kind.
        procedure, private :: append_values_boolv   !! append_values specific for the boolv kind.
        procedure, private :: append_values_strv   !! append_values specific for the strv kind.
        procedure, private :: append_values_datev   !! append_values specific for the datev kind.
        procedure, private :: append_values_timev   !! append_values specific for the timev kind.
        procedure, private :: append_values_tsv   !! append_values specific for the tsv kind.
        !> Append values, growing the column.
        generic :: append_values => append_values_i32, append_values_i64, append_values_f32, &
            append_values_f64, append_values_bool, append_values_str, append_values_date, append_values_time, &
            append_values_ts, append_values_i32v, append_values_i64v, append_values_f32v, append_values_f64v, &
            append_values_boolv, append_values_strv, append_values_datev, append_values_timev, &
            append_values_tsv
    end type parquet_column

parquet_column Derived Type

Contents

Type-Bound Procedures

Source Code

type, public :: parquet_column

Type-Bound Procedures

procedure, public :: init

interface

private module subroutine init(self, kind, nrows, width, unit)

Arguments

procedure, public :: clear

interface

private module subroutine clear(self)

Arguments

procedure, public :: deep_copy

interface

private module subroutine deep_copy(self, out)

Arguments

procedure, public :: move_from

interface

private module subroutine move_from(self, src)

Arguments

procedure, public :: kindof

private pure function kindof(self) result(res)

Arguments

Return Value integer

procedure, public :: length

private pure function length(self) result(res)

Arguments

Return Value integer(kind=int64)

procedure, public :: capacity

private function capacity(self) result(res)

Arguments

Return Value integer(kind=int64)

procedure, public :: colwidth

private pure function colwidth(self) result(res)

Arguments

Return Value integer(kind=int32)

procedure, public :: validity_bytes

private pure function validity_bytes(self) result(res)

Arguments

Return Value integer(kind=int64)

procedure, public :: has_validity_storage

interface

private module function has_validity_storage(self) result(res)

Arguments

Return Value logical

procedure, public :: ensure_validity

interface

private module subroutine ensure_validity(self)

Arguments

procedure, public :: unit_string

interface

private module subroutine unit_string(self, u)

Arguments

procedure, public :: set_unit

interface

private module subroutine set_unit(self, u)

Arguments

procedure, public :: any_null

interface

private module function any_null(self) result(res)

Arguments

Return Value logical

generic, public :: is_null => is_null_row, is_null_elem

private interface is_null_row()

Arguments

private interface is_null_elem()

Arguments

procedure, public :: row_validity

interface

private module subroutine row_validity(self, valid)

Arguments

procedure, public :: element_validity

interface

private module subroutine element_validity(self, valid)

Arguments

generic, public :: set_null => set_null_row, set_null_elem

private interface set_null_row()

Arguments