PyGMT: A Python interface for the Generic Mapping Tools

[weiji14]

Submitting Author: Wei Ji Leong (@weiji14)
All current maintainers: (@weiji14, @seisman, @maxrjones, @michaelgrund, @willschlitzer, @yvonnefroehlich)
Package Name: PyGMT
One-Line Description of Package: A Python interface for the Generic Mapping Tools
Repository Link: https://github.com/GenericMappingTools/pygmt
Version submitted:
Editor: @lwasser
Reviewer 1: @jbusecke
Reviewer 2: @SimonMolinsky
Archive: https://zenodo.org/record/6702566
Version accepted: V 0.7.0
Date accepted (month/day/year): 9/1/2022

---

Description

• Include a brief paragraph describing what your package does:

  PyGMT is a library for processing geospatial and geophysical data and making publication quality maps and figures. It provides a Pythonic interface for the Generic Mapping Tools (GMT), a command-line program widely used in the Earth Sciences.

Scope

• Please indicate which category or categories this package falls under:

  • Data retrieval
  • Data extraction
  • Data munging
  • Data deposition
  • Data visualization
  • Reproducibility
  • Geospatial
  • Education
  • Unsure/Other (explain below)

• Explain how the and why the package falls under these categories (briefly, 1-2 sentences). Please note any areas you are unsure of:

  • Data munging ⚙️: Several data processing modules are available to work on tabular and grid data (https://www.pygmt.org/v0.4.0/api/index.html#data-processing) including functions like blockmedian, surface, grdtrack, etc
  • Data visualization 🗺️: Plotting publication quality maps and figures is a strong point of PyGMT (https://www.pygmt.org/v0.4.0/api/index.html#plotting). There is a comprehensive set of tools for plotting 2D and 3D data, both geospatial and non-geospatial. It is also possible to add standard map elements like a colorbar or legend, and make multi-panel subplot figures.
  • Geospatial 🌐: PyGMT is inherently a geospatial package that can handle both vector and raster data. Several map projections are supported (https://www.pygmt.org/v0.4.0/projections/index.html), and there is support for plotting shapefiles and GeoTiff/NetCDF files.

• Who is the target audience and what are scientific applications of this package?

  • The primary target audience of PyGMT is the geoscience community, including anybody working on fields like Earth Observation, Geophysics, Oceanography, Seismology, Planetary Sciences, etc. The package offers a way for users to perform general geoprocessing tasks and create high quality illustrations of their results for posters or publications.

• Are there other Python packages that accomplish the same thing? If so, how does yours differ?

  Yes, there are several geospatial Python packages such as:

  • cartopy for plotting vector data (matplotlib based)
  • geopandas for working with vector data
  • rasterio, rioxarray and xarray-spatial for working with raster data

  These tools are typically focused on one thing only (e.g. plotting maps, vector data, raster data). PyGMT allows users to mix vector and raster data easily, so that users can:

  • Produce a map with points/lines/polygons and raster grids in the same plot, e.g. plot weather station points on top of a raster Digital Elevation Model, while including map elements (e.g. scalebar, compass arrow, colorbar, legend, inset overview maps etc)
  • Extract a series of elevation values along a transect line (vector) from a NetCDF/GeoTiff grid (raster) using grdtrack
  • Convert a table of xyz points (vector) to a 2D grid (raster) using surface via a spline-based interpolation method.

  PyGMT integrates with the PyData ecosystem. It allows users to process and plot data stored in NumPy arrays, Pandas DataFrames, Xarray DataArray/Datasets, GeoPandas GeoDataFrames, as well as from standard file formats like CSV and GeoTiff/NetCDF files. There are also plans to integrate with other scientific libraries like ObsPy (ObsPy integration GenericMappingTools/pygmt#967) in the future.

  However, PyGMT can only produce static plots unlike packages like Geoviews which allow for interactive plotting (e.g. panning and zooming). It is also non-trivial to scale out data processing tasks and/or plotting of big datasets (>10GB) that don't fit in RAM to multiple processors/clusters as with libraries like dask-geopandas because of limitations in the GMT C package that PyGMT wraps around.

• Any other questions or issues we should be aware of?:

  First off, thanks for reading this! We're keen to get some independent reviewers to have a look at the PyGMT package, sort of as a first step before going for a software publication at JOSS/G3 (the team is still deciding). Linking to original discussion at What we need for a first PyGMT paper? GenericMappingTools/pygmt#677 (comment), cc @leouieda @seisman @meghanrjones.

P.S. *Have feedback/comments about our review process? Leave a comment here

[lwasser]

hey there @weiji14 Welcome to pyopensci and please excuse the delayed response! We will review this submission and will get back to you with next steps soon. I should be able to find some folks in the geospatial space to review.
I can also walk you through the JOSS piece. I'd encourage it because once our review is done, JOSS simply accepts our review as theirs so the work you have to do in order to get a cross-ref enabled citation is small. but of course the decision is yours. happy to chat more if you have questions.

[weiji14]

Hi @lwasser, no worries on the delay. We've been keen to get some good independent reviews on our package, so take your time with the search!

As for the JOSS piece, I'll need to check with the rest of the team on what they prefer. A few of us have actually edited/reviewed JOSS papers, and I've seen and heard many great things about the review process too. We'll definitely have a good think about this once the review gets underway.

[lwasser]

hey there @weiji14 and all involved with this package! i started to go through the editor checks here and notice a disclaimer on your readme that says the package is currently undergoing rapid development. pyOpenSci reviews can be compared to JOSS reviews. Ideally the review happens when the tool has achieved some level of development stability. While we may add a process for a second review if you say added a significant chunk of functionality in the future that wasn't on the radar now, this review can be seen similar to a paper review where the paper is not a draft, it's a fully edited paper. Unlike a paper however software evolves and that is ok. but we wouldn't want version .5 here in pyopensci as an archive and then it moves to JOSS for version 1.0. We'd rather review version 1.x and then push it through JOSS as version 1.x given we collaborate with them closely. Can you kindly let me know what your goals are for this review and what stage this tool is in. I'd love it to go through our process but want to ensure it's ready.
Many thanks.

[lwasser]

@weiji14 @leouieda we've been discussing this in an issue. I just want to be transparent and careful about when we review packages. The one specific question i have is related to this statement in your readme:

All of the API (functions/classes/interfaces) is subject to change until we reach v1.0.0 as per the semantic versioning specification. There may be non-backward compatible changes as we experiment with new design ideas and implement new features. This is not a finished product, use with caution.

do you have any planned significant API changes at this point? Or do you feel like the package is relatively stable and ready for a solid review which may help you get to 1.x. many thanks! Once we have this resolved I will kick start the review. I appreciate your patience.

[weiji14]

Hi @lwasser, thank you opening the discussion at pyOpenSci/software-peer-review#101. The goals for the PyGMT team at this stage are really to see if there are any major issues in terms of our API design, and to ensure that we are following established best practices in software design. Essentially, have a fresh set of eyes look at our documentation and see if there are any glaring issues we may have missed.

All of the API (functions/classes/interfaces) is subject to change until we reach v1.0.0 as per the semantic versioning specification. There may be non-backward compatible changes as we experiment with new design ideas and implement new features. This is not a finished product, use with caution.

do you have any planned significant API changes at this point? Or do you feel like the package is relatively stable and ready for a solid review which may help you get to 1.x. many thanks! Once we have this resolved I will kick start the review. I appreciate your patience.

To be honest, that disclaimer has been sitting there for a year and a half now since Apr 2020 or v0.1.0. We're currently working on a v0.5.0 release for next week (29 Oct 2021), which included a lot of work on API standardization (see e.g. GenericMappingTools/pygmt#1479), so that disclaimer could probably be toned down a little. I'll make a point to update this disclaimer for the next release.

In terms of making backward incompatible changes, we actually have a deprecation policy for a while now since Apr 2021 or v0.4.0 (see GenericMappingTools/pygmt#1160). This ensures that users are given time (typically 2 minor releases or about 6 months) and fair warning when things might break in the future. We've actually just reached 400 stars on GitHub this week, and judging from activity on our forum, we're increasingly aware that there's a significant userbase and things are definitely getting to a stage where we have to be very careful with breaking other people's code.

So in short, I do think PyGMT is stable enough for review, and that going through the pyOpenSci review process will help us get to v1.x.x in a structured manner. Let me know if you have any other concerns, and I can discuss this with the rest of the PyGMT team 😃

[lwasser]

hi @weiji14 ! thank you for the thoughtful response. I will move forward with the review! When we get reviews it also allows us to reflect on our process and guidelines and so this review will help us adjust our language around "under development" packages (which all packages are technically always iteratively improving). Do you have a suggestion for a reviewer?

I will also ping the community. We often select one reviewer that is suggested and another will be selected by us. If you don't have any suggestions then I will look for two. Again thank you for your patience!

[lwasser]

Editor checks:

• Fit: The package meets criteria for fit and overlap.
• Automated tests: Package has a testing suite and is tested using CI (e.g. GitHub Actions, Travis-CI, CircleCI, etc).
  • Github actions
• Documentation: Package has documentation setup (ReadTheDocs, JupyterBook, website,etc.).
  • https://www.pygmt.org/latest/
• License: The package has an OSI accepted license
  • BSD 3-Clause "New" or "Revised" License
• Repository: The repository link resolves correctly
• Archive Zenodo. (may be post-review): The repository is archived somewhere outside of GitHub. We suggest using Zenodo and created releases for this purpose.
  • Zenodo - https://zenodo.org/record/5162003
• Version (may be post-review) Does the release version given match the GitHub release (v1.0.0)?

---

Due Date: Dec 27 2021(this is a holiday week in the US)
Reviewers: @SimonMolinsky @jbusecke
More to come when we find reviewers.

[weiji14]

Thanks @lwasser, I don't have a specific reviewer in mind for now, but perhaps someone involved in the Pangeo community would be a good fit? Also if possible, it would be better to start the review process after PyGMT releases v0.5.0 next Friday to capture some of the API standardization changes we've made recently. I suppose it might take time for you to find a reviewer or two, but starting the review sometime early Nov 2021 would be good.

[lwasser]

ok @weiji14 let's shoot for early november. i can work on finding reviewers and let them know that we will wait until that release / early november to begin. I also can reach out to the pangeo community. All sounds good!