Skip to content

Metadata Log Entries metadata table #667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
HonahX merged 20 commits into from
Jun 26, 2024
Merged

Metadata Log Entries metadata table #667

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
9a0423d
add metadata_entries table with tests
kevinjqliu Apr 28, 2024
ecec57e
make test work
kevinjqliu Apr 28, 2024
9e506c2
remove comment
kevinjqliu Apr 28, 2024
9c77d57
add doc
kevinjqliu Apr 28, 2024
b26f08f
make lint
kevinjqliu Apr 28, 2024
58b0609
comment
kevinjqliu Apr 28, 2024
f7dd165
comment
kevinjqliu Apr 28, 2024
4655c97
use pa.field and set nullable properly
kevinjqliu Apr 30, 2024
d989802
Merge branch 'main' into kevinjqliu/metadata_log_entries
kevinjqliu Jun 19, 2024
8613c2e
use table snapshot_as_of_timestamp instead
kevinjqliu Jun 19, 2024
f45f2ea
default sequence_number to 0
kevinjqliu Jun 19, 2024
2d417da
improve test
kevinjqliu Jun 19, 2024
502e5d8
improve docs
kevinjqliu Jun 19, 2024
1cd5f93
add comment
kevinjqliu Jun 19, 2024
3fe675d
fix tests with string output for sequence-number
kevinjqliu Jun 19, 2024
6ce55f4
comment
kevinjqliu Jun 24, 2024
aa068ff
revert INITIAL_SEQUENCE_NUMBER changes
kevinjqliu Jun 24, 2024
ab74a3e
exclude sequence_number for v1
kevinjqliu Jun 24, 2024
f274ef9
inline function
kevinjqliu Jun 24, 2024
a338e17
test_serialize_snapshot_without_sequence_number
kevinjqliu Jun 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
use pa.field and set nullable properly
  • Loading branch information
@kevinjqliu
kevinjqliu committed Apr 30, 2024
commit 4655c97034aef2ccb26ba836143025f1bdb8cbbb
10 changes: 5 additions & 5 deletions pyiceberg/table/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3543,11 +3543,11 @@ def metadata_log_entries(self) -> "pa.Table":
from pyiceberg.table.snapshots import MetadataLogEntry

table_schema = pa.schema([
("timestamp", pa.timestamp(unit='ms'), True),
("file", pa.string(), True),
("latest_snapshot_id", pa.int64(), False),
("latest_schema_id", pa.int32(), False),
("latest_sequence_number", pa.int64(), False),
pa.field("timestamp", pa.timestamp(unit='ms'), nullable=False),
pa.field("file", pa.string(), nullable=False),
pa.field("latest_snapshot_id", pa.int64(), nullable=True),
pa.field("latest_schema_id", pa.int32(), nullable=True),
pa.field("latest_sequence_number", pa.int64(), nullable=True),
])

def metadata_log_entry_to_row(metadata_entry: MetadataLogEntry) -> Dict[str, Any]:
Expand Down