Merge 'main' of datajoint/workflow-miniscope

.gitattributes

@@ -1 +1 @@
-*.ipynb binary
+*.ipynb binary

.github/ISSUE_TEMPLATE/bug_report.md

@@ -4,7 +4,6 @@ about: Create a report to help us improve
 title: ''
 labels: 'bug'
 assignees: ''
-
 ---
 
 ## Bug Report
@@ -15,7 +14,9 @@ A clear and concise description of what is the overall operation that is intende
 performed that resulted in an error.
 
 ### Reproducibility
+
 Include:
+
 - OS (WIN | MACOS | Linux)
 - DataJoint Element Version
 - MySQL Version
@@ -24,15 +25,19 @@ Include:
 - Complete error stack as a result of evaluating the above steps
 
 ### Expected Behavior
+
 A clear and concise description of what you expected to happen.
 
 ### Screenshots
+
 If applicable, add screenshots to help explain your problem.
 
 ### Additional Research and Context
+
 Add any additional research or context that was conducted in creating this report.
 
 For example:
+
 - Related GitHub issues and PR's either within this repository or in other relevant
   repositories.
 - Specific links to specific lines or a focus within source code.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -32,9 +32,11 @@ Do you currently have a work-around for this? Provide any alternative solutions
 features you've considered.
 
 ### Related Errors
+
 Add any errors as a direct result of not exposing this feature.
 
 Please include steps to reproduce provided errors as follows:
+
 - OS (WIN | MACOS | Linux)
 - DataJoint Element Version
 - MySQL Version
@@ -43,12 +45,15 @@ Please include steps to reproduce provided errors as follows:
 - Complete error stack as a result of evaluating the above steps
 
 ### Screenshots
+
 If applicable, add screenshots to help explain your feature.
 
 ### Additional Research and Context
+
 Add any additional research or context that was conducted in creating this feature request.
 
 For example:
+
 - Related GitHub issues and PR's either within this repository or in other relevant
   repositories.
 - Specific links to specific lines or a focus within source code.

.gitignore

@@ -30,6 +30,7 @@ coverage.xml
 .hypothesis/
 .pytest_cache/
 docker-compose.y*ml
+!docker/docker-compose.y*ml
 
 # C extension, Translations
 # editors: vscode, emacs, Mac
@@ -68,7 +69,9 @@ ENV/
 .spy*project
 .ropeproject
 
-# datajoint, notes, nwb export
+# datajoint, notes, nwb export, ipynb temp files
 dj_local_c*.json
 temp*
-*nwb
+*nwb
+**-jvsc-*.py
+/tests/user_data/*

.markdownlint.yaml

@@ -0,0 +1,16 @@
+# Markdown Linter configuration for docs
+# https://github.com/DavidAnson/markdownlint
+# https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md
+MD009: false # permit trailing spaces
+MD007: false # List indenting - permit 4 spaces
+MD013:
+    line_length: "88" # Line length limits
+    tables: false # disable for tables
+    headings: false # disable for headings
+MD030: false # Number of spaces after a list
+MD033: # HTML elements allowed
+    allowed_elements:
+        - "br"
+MD034: false # Permit bare URLs
+MD031: false # Spacing w/code blocks. Conflicts with `??? Note` and code tab styling
+MD046: false # Spacing w/code blocks. Conflicts with `??? Note` and code tab styling

.pre-commit-config.yaml

@@ -0,0 +1,58 @@
+default_stages: [commit, push]
+exclude: (^.github/|^docs/|^images/|^notebooks/py_scripts/)
+
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
+    hooks:
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-added-large-files # prevent giant files from being committed
+      - id: requirements-txt-fixer
+      - id: mixed-line-ending
+        args: ["--fix=lf"]
+        description: Forces to replace line ending by the UNIX 'lf' character.
+
+  # black
+  - repo: https://github.com/psf/black
+    rev: 22.12.0
+    hooks:
+      - id: black
+      - id: black-jupyter
+        args:
+          - --line-length=88
+          - --exclude=notebooks/py_scripts
+
+  # isort
+  - repo: https://github.com/pycqa/isort
+    rev: 5.11.2
+    hooks:
+      - id: isort
+        args: ["--profile", "black"]
+        description: Sorts imports in an alphabetical order
+
+  # flake8
+  - repo: https://github.com/pycqa/flake8
+    rev: 4.0.1
+    hooks:
+      - id: flake8
+        args: # arguments to configure flake8
+          # making isort line length compatible with black
+          - "--max-line-length=88"
+          - "--max-complexity=18"
+          - "--select=B,C,E,F,W,T4,B9"
+
+          # these are errors that will be ignored by flake8
+          # https://www.flake8rules.com/rules/{code}.html
+          - "--ignore=E203,E501,W503,W605,E402"
+          # E203 - Colons should not have any space before them.
+          #        Needed for list indexing
+          # E501 - Line lengths are recommended to be no greater than 79 characters.
+          #        Needed as we conform to 88
+          # W503 - Line breaks should occur after the binary operator.
+          #        Needed because not compatible with black
+          # W605 - a backslash-character pair that is not a valid escape sequence now
+          #        generates a DeprecationWarning. This will eventually become a SyntaxError.
+          #        Needed because we use \d as an escape sequence
+          # E402 - Import not at top of file - for notebook scripts

CHANGELOG.md

@@ -0,0 +1,42 @@
+# Changelog
+
+Observes [Semantic Versioning](https://semver.org/spec/v2.0.0.html) standard and
+[Keep a Changelog](https://keepachangelog.com/en/1.0.0/) convention.
+
+## 0.2.1 - Unreleased
+
++ Add - Apply `black` formatting to codebase
++ Update - Move `recursive_search` function
++ Update - Revise Pytests/docker structure
++ Update - Add reference schema for supplemental tables
++ Add - Visualization notebook
+
+## 0.2.0 - 2022-07-11
+
++ Update - Docker and Compose files
++ Update - Requirements for latest releases of Elements
++ Update - Workflow pipeline, ingestion, and populate scripts
++ Update - Pytests
++ Add - Code of conduct
+
+## 0.1.0 - 2022-04-30
+
++ Add - Analysis schema and Jupyter notebook for event-aligned calcium activity
++ Add - Data exploration Jupyter notebook
++ Add - Docker and Compose files
++ Update - Pytests
++ Update - README
+
+## 0.0.2 - 2021-10-18
+
++ Update - Change version to reflect release phase
+
+## 0.0.1 - 2021-04-01
+
++ Add - Version
+
+[0.2.1]: https://github.com/datajoint/element-miniscope/releases/tag/0.2.1
+[0.2.0]: https://github.com/datajoint/element-miniscope/releases/tag/0.2.0
+[0.1.0]: https://github.com/datajoint/element-miniscope/releases/tag/0.1.0
+[0.0.2]: https://github.com/datajoint/element-miniscope/releases/tag/0.0.2
+[0.0.1]: https://github.com/datajoint/element-miniscope/releases/tag/0.0.1

CODE_OF_CONDUCT.md

@@ -0,0 +1,133 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances of
+  any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[[email protected]](mailto:[email protected]).
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series of
+actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or permanent
+ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within the
+community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
+[https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations

CONTRIBUTING.md

@@ -1,3 +1,4 @@
 # Contribution Guidelines
 
-This project follows the [DataJoint Contribution Guidelines](https://docs.datajoint.io/python/community/02-Contribute.html). Please reference the link for more full details.
+This project follows the [DataJoint Contribution Guidelines](https://docs.datajoint.io/python/community/02-Contribute.html).
+Please reference the link for more full details.