Skip to content

[ENH] Add grid serialization and improve binary data handling #1034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
May 25, 2025
Merged

[ENH] Add grid serialization and improve binary data handling #1034

merged 13 commits into from
May 25, 2025

Conversation

@Leguark
Copy link
Member

@Leguark Leguark commented May 23, 2025 •
edited
Loading

Description

Enhanced model serialization to include grid data in binary format. This PR improves the serialization process by:

  1. Adding support for serializing and deserializing grid data (custom grid and topography)
  2. Improving binary encoding/decoding with proper length tracking for different data sections
  3. Adding docstrings to binary encoder functions for better code documentation
  4. Fixing type checking in calculate_line_coordinates_2points to support tuples in addition to lists
  5. Enabling validation of serialization by default in compute functions
  6. Updating tests to work with the new serialization format

Relates to #serialization-enhancement

Checklist

  • My code uses type hinting for function and method arguments and return values.
  • I have created tests which cover my code.
  • The test code either 1. demonstrates at least one valuable use case (e.g. integration tests)
    or 2. verifies that outputs are as expected for given inputs (e.g. unit tests).
  • New tests pass locally with my changes.

This was referenced May 23, 2025
Merged
Merged
Merged
@Leguark Graphite App
Copy link
Member Author

Leguark commented May 23, 2025 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

@Leguark Leguark changed the title [WIP] Adding logic based on serializing input [ENH] Add grid serialization and improve binary data handling May 23, 2025
@Leguark Leguark marked this pull request as ready for review May 23, 2025 12:37
@graphite-app graphite-app bot added the gempy 3 Will come with the next major update label May 23, 2025
@graphite-app
Copy link

graphite-app bot commented May 23, 2025

Graphite Automations

"Add gempy label" took an action on this PR • (05/23/25)

1 label was added to this PR based on Miguel de la Varga's automation.

@Leguark Leguark force-pushed the serialize_grids branch from 0a490fb to e562e99 Compare May 23, 2025 15:14
@Leguark Leguark mentioned this pull request May 23, 2025
Merged
4 tasks
@Leguark Leguark force-pushed the serialize_grids branch from e562e99 to 7ad196f Compare May 23, 2025 16:09
@Leguark Graphite App
Copy link
Member Author

Leguark commented May 25, 2025 •
edited
Loading

Merge activity

@Leguark Leguark force-pushed the serialize_grids branch from 7ad196f to 11d3b42 Compare May 25, 2025 07:28
@Leguark Leguark force-pushed the test_save_II branch 2 times, most recently from 4dab11f to 333fc88 Compare May 25, 2025 07:45
@Leguark Leguark force-pushed the serialize_grids branch from 11d3b42 to d3d1762 Compare May 25, 2025 07:46
@Leguark Leguark changed the base branch from test_save_II to graphite-base/1034 May 25, 2025 08:32
Leguark added 10 commits May 25, 2025 08:32
@Leguark
[WIP] Adding logic based on serializing input
85bf96f 
# Conflicts:
#	gempy/modules/serialization/save_load.py
@Leguark
[ENH/WIP] Added binary serialization without breaking anything
16bd67f
@Leguark
[ENH] Added function to convert the model to binary
59edc16
@Leguark
[ENH] Improve binary deserialization and grid reconstruction
2e59670 
Added support for handling orientation binary lengths and grid-specific binary deserialization, including custom grid and topography data. Enhanced grid metadata processing and validation to enable accurate model reconstruction.
@Leguark
[WIP] Debugging grid serialization
6004b89
@Leguark
[ENH] Serialize grid
7cb90c3
@Leguark
[TEST] Fixed grid test
501f04e
@Leguark
[TEST] Validate serialization for gravity
bce3ee6
@Leguark
[TEST] Fixing tests
2c6dd68
@Leguark
[TEST] Update approved test files with new binary metadata
d3e7d6e 
Added `ori_binary_length` and `_input_binary_size` fields to the `binary_meta_data` section of approved test files. Ensures test consistency with recent changes to binary metadata structure.
Leguark added 2 commits May 25, 2025 08:32
@Leguark
[TEST] Update approved test files with new binary metadata
57d7422 
Added `ori_binary_length` and `_input_binary_size` fields to the `binary_meta_data` section of approved test files. Ensures test consistency with recent changes to binary metadata structure.
@Leguark
[DEBUG]
ce619fc
@Leguark Leguark force-pushed the serialize_grids branch from d3d1762 to 1511e16 Compare May 25, 2025 08:32
@graphite-app graphite-app bot changed the base branch from graphite-base/1034 to main May 25, 2025 08:33
@Leguark Leguark force-pushed the serialize_grids branch from 1511e16 to 820f798 Compare May 25, 2025 08:33
@Leguark Leguark merged commit 08e510d into main May 25, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

gempy 3 Will come with the next major update

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Leguark