Skip to content

Conversation

@Leguark
Copy link
Member

@Leguark Leguark commented May 20, 2025

Description

Implemented a new model serialization system that efficiently saves and loads GeoModel objects to disk. The implementation uses binary serialization with compression to reduce file size while maintaining data integrity.

Key changes:

  • Added binary encoder for serializing input data tables
  • Created save_model and load_model functions in a new serialization module
  • Improved data field handling in structural tables with proper Pydantic Field definitions
  • Refactored ColorsGenerator to use Field for default values
  • Added zlib compression support for binary data
  • Implemented validation to ensure serialized models can be correctly deserialized

Relates to #model-serialization

Checklist

  • My code uses type hinting for function and method arguments and return values.
  • I have created tests which cover my code.
  • The test code either 1. demonstrates at least one valuable use case (e.g. integration tests)
    or 2. verifies that outputs are as expected for given inputs (e.g. unit tests).
  • New tests pass locally with my changes.

@graphite-app graphite-app bot added the gempy 3 Will come with the next major update label May 20, 2025
@Leguark Leguark changed the title [WIP] First implementation [ENH] Implement binary model serialization with compression May 20, 2025
@graphite-app
Copy link

graphite-app bot commented May 20, 2025

Graphite Automations

"Add gempy label" took an action on this PR • (05/20/25)

1 label was added to this PR based on Miguel de la Varga's automation.

Leguark added 14 commits May 23, 2025 17:07
Ensure correct `.gempy` extension when saving/loading models, raising errors for invalid or missing extensions. Added automatic creation of directories during save operations. Updated tests to reflect the enforced `.gempy` extension.
Incorporate an optional validation step in `save_model` to ensure correct serialization and deserialization of GeoModels. Added `_validate_serialization` function and updated the workflow to compare model hashes and string representations.
Replaced `SkipValidation` with `Field` for better data validation and structure. Simplified serialization by excluding large data arrays and removed redundant field validators. Updated test references accordingly.
Removed redundant `ColorGenerator` attribute from `StructuralFrame` and streamlined color initialization using default `Field` values. Updated tests accordingly to reflect these changes.
Copy link
Member Author

Leguark commented May 25, 2025

Merge activity

  • May 25, 7:09 AM UTC: A user started a stack merge that includes this pull request via Graphite.
  • May 25, 7:13 AM UTC: Graphite couldn't merge this PR because it failed for an unknown reason.
  • May 25, 7:18 AM UTC: A user started a stack merge that includes this pull request via Graphite.
  • May 25, 7:22 AM UTC: Graphite couldn't merge this PR because it failed for an unknown reason.
  • May 25, 7:26 AM UTC: A user started a stack merge that includes this pull request via Graphite.

@Leguark Leguark merged commit 2be0238 into main May 25, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gempy 3 Will come with the next major update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants