Phew! so many comments - more edits and cleanup pyproj page

package-structure-code/complex-python-package-builds.md

@@ -1,5 +1,30 @@
 # Complex Python package builds
 
+This guide is focused on packages that are either pure-python or that
+have a few simple extensions in another language such as C or C++.
+
+If your package is more complex, [you may want to refer to this guide
+created by Ralf Gommers on Python packaging.](https://pypackaging-native.github.io/)
+
+## Pure Python Packages vs. packages with extensions in other languages
+
+You can classify Python package complexity into three general categories. These
+categories can in turn help you select the correct package front-end and
+back end tools.
+
+1. **Pure-python packages:** these are packages that only rely on Python to function. Building a pure Python package is simpler. As such, you can chose a tool below that
+has the features that you want and be done with your decision!
+2. **Python packages with non-Python extensions:** These packages have additional components called extensions written in other languages (such as `C` or `C++`). If you have a package with non-python extensions, then you need to select a build back-end tool that allows you to add additional build steps needed to compile your extension code. Further, if you wish to use a front-end tool to support your workflow, you will need to select a tool that
+supports additional build setps. In this case, you could use setuptools. However, we suggest that you chose build tool that supports custom build steps such as Hatch with Hatchling or PDM. PDM is an excellent choice as it allows you to also select your build back end of choice. We will discuss this at a high level on the complex builds page.
+3.**Python packages that have extensions written in different languages (e.g. fortran and C++) or that have non Python dependencies that are difficult to install (e.g. GDAL)** These packages often have complex build steps (more complex than a package with just a few C extensions for instance). As such, these packages require tools such as [scikit-build](https://scikit-build.readthedocs.io/en/latest/)
+or [meson-python](https://mesonbuild.com/Python-module.html) to build. NOTE: you can use meson-python with PDM.
+
+
+<!--
+On this page, we will focus on using front-end tools to package pure python
+packages. We will note if a package does have the flexibility to support other
+back-ends and in turn more complex builds (*mentioned in #2 and #3 above*). -->
+<!--
 ## COmbine the two sets of statement below...
 ELI:
 PDM supports C/Cython extensions too: https://pdm.fming.dev/latest/pyproject/build/#build-platform-specific-wheels
@@ -86,9 +111,7 @@ CORRECTIONS:
 pdm doesn't use plugins. Hatch does.
 pdm and poetry both rely on setuptools for C extensions. pdm's support claims to be fully developed and documented. poetry claims nothing, and doesn't document it.
 
--->
 
-```{note}
 ??
 Poetry supports extensions written in other languages but this functionality is
 currently undocumented.
@@ -101,7 +124,6 @@ package builds.
 Some front-end packaging tools, such as PDM, allow you to use other
 build back-ends such as **meson** and **scikit-build**.
 
-```
 
 me:
 pdm, hatch and poetry all have "ways" of supporting c extensions via pdm-build, hatchling and poetry's build back end.
@@ -118,3 +140,4 @@ pdm and poetry both rely on setuptools for C extensions. pdm's support claims to
 
 
 https://pdm.fming.dev/latest/pyproject/build/#build-platform-specific-wheels
+-->

package-structure-code/intro.md

@@ -95,6 +95,7 @@ the `src` directory.
 Intro <self>
 
 Python package structure <python-package-structure>
+pyproject.toml Package Metadata <pyproject-toml-python-package-metadata>
 What are SDist & Wheel Files? <python-package-distribution-files-sdist-wheel>
 Package Build Tools <python-package-build-tools>
 Complex Builds <complex-python-package-builds>

package-structure-code/pyproject-toml-python-package-metadata.md

@@ -0,0 +1,142 @@
+# Use a pyproject.toml file for your package configuration & metadata
+
+The standard file that Python packages use to specify build requirements and
+metadata is called a pyproject.toml. The pyproject.toml file has become the
+standard for declaring Python package metadata (including dependencies) rather
+than using a setup.py file (or a setup.py + setup.cfg file).
+
+As such you should try to [include all project based metadata and build system specifications in a `pyproject.toml` file.](https://packaging.python.org/en/latest/specifications/declaring-project-metadata/) Using setup.py to manage both package set up and
+hold metadata [can cause problems with package development.](https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html)
+
+
+```{admonition} Benefits of using a pyproject.toml file
+:class: tip
+
+1. Because setup.py has a mixture of code and metadata, it will be run twice when
+building your package. First it will be run to extract metadata (dependencies). Then it will be run to build your package.
+1. Including your package's metadata in a separate human-readable `pyproject.toml` format also allows someone to view the project's metadata without
+running any Python code.
+```
+
+A pyproject.toml is written in [TOML (Tom's Obvious, Minimal Language) format](https://toml.io/en/). TOML is an easy-to-read structure that is founded on key: value pairs.
+
+Each section in the pyproject.toml file contains a `[table identifier]`.
+Below that table identifier are key value pairs that
+support configuration for that particular table.
+
+<!-- setup.cfg for project metadata is being deprecated - set setuptools guide and
+https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
+pypa -
+https://packaging.python.org/en/latest/specifications/declaring-project-metadata/ -->
+
+```{note}
+<!-- [PEP518 describes the move away from setup.py to the pyproject.toml file.](https://peps.python.org/pep-0518/) -->
+Python package standards are moving away from including both package metadata
+and Python code needed to set up a package in the same **setup.py** file.
+Instead we are moving towards using a **proproject.toml** file.
+
+In some cases where a build is complex, a **setup.py** file may still be
+required. While this guide will not cover complex builds, we will provide
+resources working with complex builds in the future.
+
+<!-- https://github.com/pyOpenSci/python-package-guide/pull/23#discussion_r1071541329
+ELI: A complex build could mean running a python script that processes some data file and produces a pure python module file.
+
+Probably not common in the scientific community specifically, but I've seen quite a few setup.py files that contain custom build stages which e.g. build gettext locale catalogs.
+
+The main point is that it is more "complex" than simply copying files or directories as-is into the built wheel.
+-->
+```
+
+## Example pyproject.toml
+
+Below is an example build configuration for a Python project. This setup
+requires:
+
+* **setuptools** to create the package structure,
+* **wheel** which is used by `setuptools` to create the [**.whl** (wheel) file](https://realpython.com/python-wheels/).
+* **setuptools build** to "build" the package
+* **setuptools_scm** to manage package version updates
+
+In the example below `[build-system]` is the first table
+of values. It has two keys that specify the build front end and backend for a package:
+
+1. `requires =`
+1. `build-backend =`
+
+```
+[build-system]
+requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "examplePy"
+authors = [
+    {name = "Some Maintainer", email = "[email protected]"}
+]
+maintainers = [{name = "All the contributors"}]
+license = {text = "BSD 3-Clause"}
+description = "An example Python package used to support Python packaging tutorials"
+keywords = ["pyOpenSci", "python packaging"]
+readme = "README.md"
+
+dependencies = [
+    "dependency-package-name-1",
+    "dependency-package-name-2",
+]
+```
+
+
+Notice that you also can specify dependencies in this file.
+
+
+A major benefit of the pyproject.toml file is that it makes is transparent
+
+1. what build system you are using to create your package
+2. what dependencies you need
+
+
+The package metadata including authors, keywords, etc is also easy to read.
+Below you can see the same toml file that uses a different build system (PDM).
+Notice how simple it is to swap out the tools needed to build this package!
+
+```
+[build-system]
+requires = ["pdm-pep517>=1.0.0"]
+build-backend = "pdm.pep517.api"
+
+[project]
+name = "examplePy"
+authors = [
+    {name = "Some Maintainer", email = "[email protected]"}
+]
+maintainers = [{name = "All the contributors"}]
+license = {text = "BSD 3-Clause"}
+description = "An example Python package used to support Python packaging tutorials"
+keywords = ["pyOpenSci", "python packaging"]
+readme = "README.md"
+
+dependencies = [
+    "dependency-package-name-1",
+    "dependency-package-name-2",
+]
+```
+
+
+
+```{note}
+[Click here to read about our packaging documentation requirements.](/package-structure-code/python-package-build-tools)
+```
+
+<!-- TODO: add link to section on build tools when it exists and
+turn this into button:
+
+We discuss build tools for python package more here.
+-->
+
+
+
+<!-- TODO:
+1. add some links to packages that are using a purely toml config
+1. link to our example package once it's further along
+-->

package-structure-code/python-package-build-tools.md

@@ -9,26 +9,19 @@ highlight tools that currently support packages with C/C++ and other language
 extensions.
 
 
-```{admonition} Pure Python Packages vs. packages with extensions in other languages
+<!-- TODO: create build tool selection diagram - https://www.canva.com/design/DAFawXrierc/O7DTnqW5fmMEZ31ao-TK9w/edit -->
 
-You can classify Python package complexity into three general categories. These
-categories can in turn help you select the correct package front-end and
-back end tools.
 
-1.  **Pure-python packages:** these are packages that only rely on Python to function. Building a pure Python package is simpler. As such, you can chose a tool below that
-has the features that you want and be done with your decision!
-2.  **Python packages with non-Python extensions:** These packages have additional components called extensions written in other languages (such as `C` or `C++`). If you have a package with non-python extensions, then you need to select a build back-end tool that allows you to add additional build steps needed to compile your extension code. Further, if you wish to use a front-end tool to support your workflow, you will need to select a tool that
-supports additional build setps. In this case, you could use setuptools. However, we suggest that you chose build tool that supports custom build steps such as Hatch with Hatchling or PDM. PDM is an excellent choice as it allows you to also select your build back end of choice. We will discuss this at a high level on the complex builds page.
-3. **Python packages that have extensions written in different languages (e.g. fortran and C++) or that have non Python dependencies that are difficult to install (e.g. GDAL)** These packages often have complex build steps (more complex than a package with just a few C extensions for instance). As such, these packages require tools such as [scikit-build](https://scikit-build.readthedocs.io/en/latest/)
-or [meson-python](https://mesonbuild.com/Python-module.html) to build. NOTE: you can use meson-python with PDM.
+:::{figure-md} fig-target
 
-On this page, we will focus on using front-end tools to package pure python
-packages. We will note if a package does have the flexibility to support other
-back-ends and in turn more complex builds (*mentioned in #2 and #3 above*).
+<img src="../images/python-package-tools-decision-tree.png" alt="Figure showing... will finish this once we are all happy with the figure and it's not going to change more..." width="700px">
 
-[If you are interested in tool support for non pure python builds, check out this
-page for resources.](complex-python-package-builds)
-```
+Diagram showing the various from end build tools that you can select from. Each tool has different features as highlighted below.
+NOTE: this is still a DRAFT so i'm not going to spend time truly cleaning it up until i get lots of feedback on the general approach!!
+:::
+
+If you want to know more about Python packages that have extensions written in
+other languages, [check out the page on complex package builds.](complex-python-package-builds)
 
 ## Build front-end vs. build back-end tools
 
@@ -99,12 +92,12 @@ Example build steps using setuptools:
 A packaging front-end tool refers to a tool that makes it easier for you to
 perform common packaging tasks using similar commands. These tasks include:
 
-* [Creating a Sdist and Wheel distribution](python-package-distribution-files-sdist-wheel)
-* Managing an environment or multiple environments in which you need to run tests and develop your package
-* Building documentation
+* [Build your packages (create the SDist and Wheel distributions](python-package-distribution-files-sdist-wheel)
 * Installing your package in a development mode (so it updates when you update your code)
-* Running tests
 * Publishing to PyPI
+* Running tests
+* Building documentation
+* Managing an environment or multiple environments in which you need to run tests and develop your package
 
 There are several Python packaging tools that you can use for pure Python
 builds. Each front-end tool discussed below supports a slightly different set of Python
@@ -131,13 +124,13 @@ hatch build
 # Example to publish to PyPI:
 hatch publish --repository testpypi
 ```
-Example build steps using **setuptools** and **build**:
+Example build steps using the **setuptools** backend and **build**:
 
 ```bash
 # Build the package
 python3 -m build
 
-# Publish to test PyPI
+# Publish to test PyPI using twine
 twine upload -r testpypi dist/*
 ```
 
@@ -197,7 +190,7 @@ The Python developers survey results (n=>8,000 PyPI users) show setuptools and p
 
 The tools that we review below include:
 
-* setuptools + twine, build
+* Twine, Build + setuptools
 * Flit
 * Hatch
 * PDM
@@ -219,15 +212,6 @@ workflow to support such extensions. It also supports other backends such as sci
 NOTE: You can also use Hatch but you will need to write your own plugin for this support.
 
 
-<!-- TODO: create build tool selection diagram - https://www.canva.com/design/DAFawXrierc/O7DTnqW5fmMEZ31ao-TK9w/edit -->
-
-
-:::{figure-md} fig-target
-
-<img src="../images/python-package-tools-decision-tree.png" alt="ADD ME." width="700px">
-
-ADD ME
-:::
 
 <!-- ### Build tools for Python packages with complex build steps
 If your package is not pure Python, or it has complex build steps (or build
@@ -430,7 +414,7 @@ is currently undocumented. Thus we don't recommend it for more complex builds.
 :widths: 20,5,50
 
 Dependency management,✅,Poetry helps you add dependencies to your `pyproject.toml` metadata. _NOTE: currently Poetry adds dependencies using an approach that is slightly out of alignment with current Python peps - however there is a plan to fix this in an upcoming release._ Allows you to organize dependencies in groups: docs; package; tests.
-Dependency pinning,✖✅ ,Poetry offers dependency pinning however, it does so in a way that can be problematic for some packages. Read below for more.
+Dependency pinning,✖✅ ,Poetry offers dependency pinning however it does so in a way that can be problematic for some packages. Read below for more.
 Select your environment manager of choice (conda; venv; etc),✅ , Poetry allows you to either use its simple environment management tool or  select the environment manager that you want to use for managing your package. [Read more about its built in environment management options](https://python-poetry.org/docs/basic-usage/#using-your-virtual-environment).
 Publish to PyPI and test PyPI,✅,Poetry supports publishing to both test PyPI and PyPI
 Version Control based versioning,✅ , The plugin (Poetry dynamic versioning)[https://github.com/mtkennerly/poetry-dynamic-versioning] supports versioning using git tags with Poetry.
@@ -473,7 +457,7 @@ This approach also won't support over ways of versioning tools, for instance,
 some tools use [calver](https://calver.org/) which creates new versions based on the date.
 ```
 
-```{admonition} where does this belong?
+```{admonition} Hatch vs PDM vs Poetry
 :class: note
 There are some features that Hatch and PDM offer that Poetry does not.