| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| type(parquet_reader), | intent(inout) | :: | reader |
open reader with no transform of its own yet. |
||
| type(parquet_reader), | intent(in) | :: | source |
open, idle reader whose transform is adopted. |
Gives reader the read-time transform source has already worked out, instead of making
it work the same thing out again from the same file.
What this is for. The recommended way to read one file from several threads is to give
each thread its own parquet_reader (see the Thread safety guide). When that file is read
with a filter= or a sort_by=, every one of those readers would otherwise re-decode the
filter's key columns and rebuild the whole sort permutation -- work that is identical in
every reader and can cost more than the parallelism saves. This hands it over instead.
The cost is two atomic refcount increments, not a data copy: a filter mask and a sort permutation are immutable Arrow arrays, so the readers share them. Everything else transferred is proportional to the file's row-group count.
source may be adopted from by several threads at once, provided it is idle -- it is
only read. What it must NOT be is in use by another thread at the same moment, which is
refused rather than raced.
Aborts unless: the two readers are open on files with the same row-group and row counts;
reader has no filter, sample or sort of its own; and no column has been read on reader
yet (one already read was read unmasked, and could not be lined up with an adopted mask).
A source carrying no transform at all is a no-op, so a caller need not ask first.