[ntuple] Attributes: update specs and add de/serialization #19894
Conversation
Test Results 20 files 20 suites 3d 12h 55m 39s ⏱️ For more details on these failures, see this check. Results for commit 48b72ff. ♻️ This comment has been updated with latest results. |
silverweed
force-pushed
the
ntuple_attr_1
branch
from
130fc21 to
b0449c0
Compare
silverweed
force-pushed
the
ntuple_attr_1
branch
from
b0449c0 to
48b72ff
Compare
|
I updated the PR and uniformed the |
|
|
||
| The current Attribute Schema Version is **1.0**. It has the following fields (in the following order): | ||
| 1. `_rangeStart` (type `NTupleSize_t`): the start of the range that each Attribute Entry refers to; | ||
| 2. `_rangeLen` (type `NTupleSize_t`): the length of the range that each Attribute Entry refers to; |
There was a problem hiding this comment.
I think we may want to elaborate what length zero means
There was a problem hiding this comment.
How prescriptive do we want to be with them? I think we probably don't want to mandate how a reader treats them, but we want to make sure they are recognized as 'valid'. That said, we never say anything about merging in the Specification...
| fName == other.fName; | ||
| }; | ||
|
|
||
| ROOT::Experimental::RNTupleAttrSetDescriptor ROOT::Experimental::RNTupleAttrSetDescriptor::Clone() const |
There was a problem hiding this comment.
Clone of fAnchorLocator seems missing
| @@ -1799,6 +1799,23 @@ ROOT::RResult<std::uint32_t> ROOT::Internal::RNTupleSerializer::SerializeFooter( | |||
| return R__FORWARD_ERROR(res); | |||
| } | |||
|
|
|||
| // Attributes | |||
| frame = pos; | |||
| const auto nAttributeSets = desc.GetNAttributeSets(); | |||
There was a problem hiding this comment.
Let's add a warning if nAttributeSets > 0 that attributes are still experimental
There was a problem hiding this comment.
Does this have implication on the readability of the data? @hahnjo pointed out that we may want to have a way to write RNTuples with no format changes until we have finalized the decision about the format (which also implies we should not merge any change to the specs until then).
I think we need an internal discussion on this before this PR can proceed (or at least before the specs are merged).
| } else { | ||
| return R__FORWARD_ERROR(res); | ||
| } | ||
| for (std::uint32_t attrSetId = 0; attrSetId < nAttributeSets; ++attrSetId) { |
There was a problem hiding this comment.
Same here, if nAttributeSets > 0 let's print a warning
First PR of a series to merge the RNTuple Attributes into master. The final result will be this, although the commits will be reorganized to be more coherent and reviewable.
This first PR updates the binary format specification (introducing a new minor version) and updates the Serializer and Descriptor code to match. This is backward-compatible and no Attribute can be written yet since the writer API will be introduced later.
Checklist: