Use timestamp statistics

[ptkool]

No description provided.

[findepi]

@ptkool thanks!
looks reasonable.
please add a test in TestTupleDomainParquetPredicate.

i still need to check whether min/max is always in microseconds.

[ryanrupp]

I think you will have to inspect the logical type to determine whether its milliseconds or microseconds (think limited range nano may be supported as well?) on a long. See the PR comments here unfortunately I rebased/forced push out the timestamp changes but the discussion there is probably applicable. Two things:

1. Considering logical types to determine how to interpret the value (millis vs micros vs nanos?)
2. See @findepi following comment in the linked PR about timestamp semantics in Presto and if that matters here (this was why the change was originally backed out).

[ryanrupp]

Looks like the code has changed around slightly since I looked at this last but you'll have to pass along the RichColumnDescriptor (which has Parquet column info) then you can inspect richColumnDescriptor.getPrimitiveType().getOriginalType(). Also I think getOriginalType() can return null (not sure if this is just due to older versions that didn't populate this, forget the exact scenario where this could happen).

[ryanrupp]

For TIMESTAMP_MILLIS I think do an explicit check here on original type because it looks like timestamp nanosecond support was added in apache/parquet-java#519 and it's using the new LogicalTypeAnnotation which will replace OriginalType in the next release of parquet-mr (see apache/parquet-java#463). So to be forward compatible if the check is explicit here it won't accidentally go into this path if a newer file is using nanoseconds.

[tooptoop4]

are you still working on this @ptkool ?

[bitsondatadev]

👋 @ptkool - this PR has become inactive. If you're still interested in working on it, please let us know, and we can try to get reviewers to help with that.
We're working on closing out old and inactive PRs, so if you're too busy or this has too many merge conflicts to be worth picking back up, we'll be making another pass to close it out in a few weeks.

[bitsondatadev]

@ryanrupp and @ptkool, is this a duplicate of #4104?

[ryanrupp]

@bitsondatadev yes I believe so

[mosabua]

Closing as confirmed duplicate.