documentation: fetch remote extension schemas

Added some code to conf.py to fetch extension schemas and add them to
the extras folder at build time, so that the docs for extensions build
correctly with schema tables

documentation/conf.py

@@ -406,6 +406,53 @@
         use_titles=True,
     )
 
+#-------------------------------------------------------------------------------------------
+# Fetch extension schemas at build to allow drawing schema tables in the extensions section
+#
+# Matt added this to deal with the problem where we need to draw schema tables for
+# extensions but the extension schemas are managed externally.
+#-------------------------------------------------------------------------------------------
+
+import requests
+
+360_extensions = ['dei'] # ADD NEW EXTENSIONS HERE
+
+# IMPORTANT: Update this URL when the dei extension is merged in
+base_url = "https://raw.githubusercontent.com/ThreeSixtyGiving/extensions-registry/add-dei-extension/extensions/"
+
+if not os.path.exists("./extras/extensions/"):
+    os.mkdir("./extras/extensions/")
+
+for extension in 360_extensions:
+    # Make a directory for the extension schemas inside the extras directory
+    extension_path = f"./extras/extensions/{extension}"
+    if not os.path.exists(extension_path):
+        os.mkdir(extension_path)
+        os.mkdir(f"{extension_path}/codelists")
+
+    # Fetch the registry entry and parse the JSON
+    registry_entry = json.loads(requests.get(f"{base_url}/{extension}.json").text)
+
+    # Fetch each of the schemas
+    for schema in registry_entry['schemas']:
+        schema_content = requests.get(schema['uri']).text
+
+        with open(f"{extension_path}/{schema['target']}", 'w') as schema_file:
+            schema_file.write(schema_content)
+
+    # Fetch each of the codelists
+    for codelist in registry_entry['codelists']:
+        codelist_content = requests.get(codelist).text
+        codelist_name = codelist.split('/')[-1]
+
+        with open(f"{extension_path}/codelists/{codelist_name}", 'w') as codelist_file:
+            codelist_file.write(codelist_content)
+
+#-------------------------------------------------------------------------------------------
+#-------------------------------------------------------------------------------------------
+# End fetch extension schemas at build
+#-------------------------------------------------------------------------------------------
+
 #mv README.md 360-giving-schema-titles.csv/
 
 

documentation/extensions/dei/reference.md

@@ -39,7 +39,7 @@ This section contains a reference for the Extension’s schema
 
 
 ```eval_rst
-.. jsonschema-titles:: ../../extras/dei-extension-schema.json
+.. jsonschema-titles:: ../../extras/extensions/dei/360-giving-schema.json
 ```
 
 ### Codelists
@@ -53,7 +53,7 @@ A codelist to declare whether DEI Data Standard questions were asked for this gr
 
 ```eval_rst
 .. csv-table:: 
-   :url: https://raw.githubusercontent.com/ThreeSixtyGiving/360-dei/refactor-to-minimal/codelists/askedStatus.csv
+   :file: ../../extras/extensions/dei/codelists/askedStatus.csv
    :header-rows: 1
    :widths: auto
 ```
@@ -64,7 +64,7 @@ A codelist to declare which answer options were available to the respondents.
 
 ```eval_rst
 .. csv-table:: 
-   :url: https://raw.githubusercontent.com/ThreeSixtyGiving/360-dei/refactor-to-minimal/codelists/availableOptions.csv
+   :file: ../../extras/extensions/dei/codelists/availableOptions.csv
    :header-rows: 1
    :widths: auto
 ```
@@ -75,7 +75,7 @@ A codelist to declare whether a reply to DEI Data Standard questions was receive
 
 ```eval_rst
 .. csv-table:: 
-   :url: https://raw.githubusercontent.com/ThreeSixtyGiving/360-dei/refactor-to-minimal/codelists/replyStatus.csv
+   :file: ../../extras/extensions/dei/codelists/replyStatus.csv
    :header-rows: 1
    :widths: auto
 ```
@@ -90,7 +90,7 @@ Please see the [Guiding Principles](#guiding-principles) section for more inform
 
 ```eval_rst
 .. csv-table:: 
-   :url: https://raw.githubusercontent.com/ThreeSixtyGiving/360-dei/refactor-to-minimal/codelists/taxonomyCodes.csv
+   :file: ../../extras/extensions/dei/codelists/taxonomyCodes.csv
    :header-rows: 1
    :widths: auto
 ```