diff --git a/source/User-Manual/Data-formats/Binary-format.rst b/source/User-Manual/Data-formats/Binary-format.rst index 1162c3a..ed80ddf 100644 --- a/source/User-Manual/Data-formats/Binary-format.rst +++ b/source/User-Manual/Data-formats/Binary-format.rst @@ -29,7 +29,7 @@ Binary Format **Limitations** -* Requires slightly more disk space because it stores two 64-bit timestamps for every sample. +* Requires slightly more disk space because it stores one 64-bit sample number and one 64-bit timestamp for every sample. * Continuous files are not self-contained, i.e., you need to know the number of channels and the "bit-volts" multiplier in order to read them properly. @@ -38,7 +38,9 @@ Binary Format File organization #################### -Within a Record Node directory, data for each **experiments** (stop/start acquisition) is contained in its own sub-directory. Experiment directories are further sub-divided for individual **recordings** (stop/start recording). +Within a Record Node directory, data for each **experiment** is contained in its own sub-directory. The experiment number is incremented whenever acquisition is stopped, which will reset the timestamps/sample numbers to zero. A new :code:`settings.xml` file will be created for every experiment. + +Experiment directories are further sub-divided for individual **recordings**. The recording number is incremented whenever recording is stopped, which will not reset the timestamps/sample numbers. Data from multiple recordings within the same experiment will have internally consistent timestamps/sample numbers, and will use the same :code:`settings.xml` file. .. image:: ../../_static/images/recordingdata/binary/organization.png :alt: Binary data directory structure @@ -122,14 +124,72 @@ More detailed information about each electrode is stored in the :code:`structure Reading data in Python ####################### -* **(recommended)** Create a :code:`Session` object using the `open-ephys-python-tools `__ package. The data format will be automatically detected. +Using :code:`open-ephys-python-tools` +-------------------------------------- + +The recommended method for loading data in Python is via the `open-ephys-python-tools `__ package, which can be installed via :code:`pip`. + +First, create a :code:`Session` object that points to the top-level data directory: + +.. code-block:: python + + from open_ephys.analysis import Session + + session = Session(r'C:\Users\example_user\Open Ephys\2025-08-03_10-21-22') + +The :code:`Session` object provides access to data inside each Record Node. If you only had one Record Node in your signal chain, you can find its recordings as follows: + +.. code-block:: python + + recordings = session.recordnodes[0] + +:code:`recordings` is a list of :code:`Recording` objects, which is a flattened version of the original Record Node directory. For example, if you have three "experiments," each with two "recordings," there will be six total :code:`Recording` objects in this list. The one at index 0 will be recording 1 from experiment 1, index 1 will be recording 2 from experiment 1, etc. + +Each :code:`Recording` object provides access to the continuous data, events, spikes, and metadata for the associated recording. To read the continuous data samples for the first data stream, you can use the following code: + +.. code-block:: python + + samples = recordings[0].continuous[0].get_samples(start_sample_index=0, end_sample_index=10000) + print(f'Stream name: {recordings[0].continuous[0].metadata("stream_name")}') + +This will automatically scale the data into microvolts before returning it. + +To read the events for the same recording: + +.. code-block:: python + + events = recordings[0].events + +This will load the events for all data streams into a Pandas :code:`DataFrame` with the following columns: + +* :code:`line` - the TTL line on which this event occurred +* :code:`sample_number` - the sample index at which this event occurred +* :code:`timestamp` - the global timestamp (in seconds) at which this event occurred (defaults to -1 if all streams were not synchronized) +* :code:`processor_id` - the ID of the processor from which this event originated +* :code:`stream_index` - the index of the stream from which this event originated +* :code:`stream_name` - the name of the stream from which this event originated +* :code:`state` - 1 or 0, to indicate whether this event is associated with a rising edge (1) or falling edge (0) + +The fastest way to find the nearest continuous sample to a particular event is by using the :code:`np.searchsorted` function: + +.. code-block:: python + + import numpy as np + event_index = 100 # the index of the event you're interested in + event_timestamp = events.iloc[event_index].timestamp + nearest_continuous_sample_index = \ + np.searchsorted(recordings[0].continuous[0].timestamps, + event_timestamp) + -* Create a :code:`File` object using the `pyopenephys `__ package. +For more information on how to use the :code:`open-ephys-python-tools` library, check out this `README `__ -* Use the :code:`DatLoad()` method from :code:`Binary.py` in the `open-ephys/analysis-tools `__ repository. +Using :code:`SpikeInterface` +-------------------------------------- +You can also load data from the Open Ephys Binary format via `SpikeInterface `__, using the :code:`read_openephys()` method. Reading data in Matlab ####################### -* Use the `open-ephys-matlab-tools `__ library. \ No newline at end of file +Use the `open-ephys-matlab-tools `__ library, available via the `Matlab File Exchange `__. \ No newline at end of file diff --git a/source/User-Manual/Data-formats/NWB-format.rst b/source/User-Manual/Data-formats/NWB-format.rst index 0a014ca..c3dcded 100644 --- a/source/User-Manual/Data-formats/NWB-format.rst +++ b/source/User-Manual/Data-formats/NWB-format.rst @@ -106,6 +106,8 @@ Reading data in Python * Create a :code:`Session` object using the `open-ephys-python-tools `__ package. The data format will be automatically detected. +* Alternatively, you can use the `PyNWB `__ package. + Reading data in Matlab ####################### diff --git a/source/User-Manual/Data-formats/Open-Ephys-format.rst b/source/User-Manual/Data-formats/Open-Ephys-format.rst index 05ad240..7f11b70 100644 --- a/source/User-Manual/Data-formats/Open-Ephys-format.rst +++ b/source/User-Manual/Data-formats/Open-Ephys-format.rst @@ -118,14 +118,73 @@ Since the samples are saved as 16-bit unsigned integers, converting them to micr Reading data in Python ####################### -* **(recommended)** Create a :code:`Session` object using the `open-ephys-python-tools `__ package. The data format will be automatically detected. -* Create a :code:`File` object using the `pyopenephys `__ package. +Using :code:`open-ephys-python-tools` +-------------------------------------- -* Use the :code:`loadContinuous`, :code:`loadEvents`, or :code:`loadSpikes` methods from :code:`OpenEphys.py` in the `open-ephys/analysis-tools `__ repository. +The recommended method for loading data in Python is via the `open-ephys-python-tools `__ package, which can be installed via :code:`pip`. +First, create a :code:`Session` object that points to the top-level data directory: + +.. code-block:: python + + from open_ephys.analysis import Session + + session = Session(r'C:\Users\example_user\Open Ephys\2025-08-03_10-21-22') + +The :code:`Session` object provides access to data inside each Record Node. If you only had one Record Node in your signal chain, you can find its recordings as follows: + +.. code-block:: python + + recordings = session.recordnodes[0] + +:code:`recordings` is a list of :code:`Recording` objects, which is a flattened version of the original Record Node directory. For example, if you have three "experiments," each with two "recordings," there will be six total :code:`Recording` objects in this list. The one at index 0 will be recording 1 from experiment 1, index 1 will be recording 2 from experiment 1, etc. + +Each :code:`Recording` object provides access to the continuous data, events, spikes, and metadata for the associated recording. To read the continuous data samples for the first data stream, you need to first set the sample range (to prevent all samples from being loaded into memory), and then call :code:`get_samples()`: + +.. code-block:: python + + recordings[0].set_sample_range([10000, 50000]) + samples = recordings[0].continuous[0].get_samples(start_sample_index=0, end_sample_index=10000) + print(f'Stream name: {recordings[0].continuous[0].metadata("stream_name")}') + +This will automatically scale the data into microvolts before returning it. + +To read the events for the same recording: + +.. code-block:: python + + events = recordings[0].events + +This will load the events for all data streams into a Pandas :code:`DataFrame` with the following columns: + +* :code:`line` - the TTL line on which this event occurred +* :code:`sample_number` - the sample index at which this event occurred +* :code:`processor_id` - the ID of the processor from which this event originated +* :code:`stream_index` - the index of the stream from which this event originated +* :code:`stream_name` - the name of the stream from which this event originated +* :code:`state` - 1 or 0, to indicate whether this event is associated with a rising edge (1) or falling edge (0) + +The fastest way to find the nearest continuous sample to a particular event is by using the :code:`np.searchsorted` function: + +.. code-block:: python + + import numpy as np + event_index = 100 # the index of the event you're interested in + event_timestamp = events.iloc[event_index].timestamp + nearest_continuous_sample_index = \ + np.searchsorted(recordings[0].continuous[0].sample_numbers, + event_sample_number) + + +For more information on how to use the :code:`open-ephys-python-tools` library, check out this `README `__ + +Using :code:`SpikeInterface` +-------------------------------------- + +You can also load data from the Open Ephys Binary format via `SpikeInterface `__, using the :code:`read_openephys()` method. Reading data in Matlab ####################### -* Use the `open-ephys-matlab-tools `__ library. \ No newline at end of file +Use the `open-ephys-matlab-tools `__ library, available via the `Matlab File Exchange `__. \ No newline at end of file diff --git a/source/User-Manual/Whats-new-in-v100.rst b/source/User-Manual/Whats-new-in-v100.rst index 3c06733..1d7695f 100644 --- a/source/User-Manual/Whats-new-in-v100.rst +++ b/source/User-Manual/Whats-new-in-v100.rst @@ -108,7 +108,7 @@ To facilitate automated testing on cloud servers, the GUI now can run in "headle The GUI now includes unit tests for key classes via the Google Test package, which are run before each new release. This will ensure critical functionality remains intact as new contributors are onboarded. -The `open-ephys-test-suite `_ package makes it easy to configure and run integration tests via Python. This package is used to confirm that different combinations of plugins work together reliably. +The `open-ephys-test-suite `_ package makes it easy to configure and run integration tests via Python. This package is used to confirm that different combinations of plugins work together reliably. Plugins