Python Scripting in NeuroExplorer

For many years, NeuroExplorer has had the capability to automate repetitive tasks:

  • Repeat analysis on all the data files in a folder,
  • Edit data or post-process analysis results without sending data or results to an external program

To support scripting, a custom NexScript language was developed. NexScript supports simple variables and has basic flow control capabilities. However, there are many limitations of NexScript that make writing scripts difficult. Adding new capabilities to NexScript (for example, adding support for arrays) would require a considerable effort. An alternative approach is to integrate existing programming language into NeuroExplorer.

We are pleased to announce that starting with version 5.022, NeuroExplorer scripts can also be written in Python.

NexScript - RepeatAnalysis2

Here are some of the benefits of using Python:

  • Python is very well documented
  • Shorter scripts
    • Access to NeuroExplorer data via Python lists eliminate many loops
    • User-defined functions replace repetitive code
  • Scripts can use thousands of Python functions

Old NexScript scripts can be automatically converted to Python using Tools | Convert to Python menu command in NexScript editor.

NeuroExplorer uses Python 2.7.10. There is no need to install Python. All the Python files needed for scripting are installed by NeuroExplorer setup program.

How to Store Analog Data

NeuroExplorer up to version 5.020 always stored analog data internally as 16-bit integers. The reason? Most data acquisition systems use analog to digital converters that have 12-bit or 16-bit resolution and store A/D data as 16-bit integers.

To display real signal values, we use scaling. For example, if analog to digital converter has input range from -1000mV to +1000mV and 10X amplification was used, then maximum 16-bit value 32767 corresponds to 1000mV/10 (non-amplified signal value). This means that 32767*ScalingFactor = 100, and ScalingFactor is 100.0/32767.

16-bit storage model seems perfect — we do not loose any signal resolution and we save on storage space. However, if we import data from Matlab (where the data values are stored as floating point numbers), we have to calculate ScalingFactor to fit floating point values into 16-bit range. As a result, we loose signal resolution (especially when the signal has a few large peaks and lots of small values).

There are also clipping issues when we try to modify signal values in NexScript. If we try to assign the value that is outside the original  non-amplified  range, we would have to clip the value since we have hard limits for 16-bit internal representation of signals.

Things, however, are changing with NeuroExplorer 5.021. Now NeuroExplorer can store analog data both as 16-bit integers and as 32-bit floating point numbers.

Analog signals imported from data files created by data acquisition system are still stored as 16-bit integers.

Continuous channels imported from Matlab or generated in NeuroExplorer (using frequency filtering, etc.) are now stored as 32-bit floating point values. Both data representations can be saved and loaded using .nex5 data files.

Matlab scripts for reading and writing .nex5 files (HowToReadAndWriteNexAndNex5FilesInMatlab.zip) have been updated and can be used to read and write .nex5 files with analog data stored as 32-bit floats.

New Phase Analysis Reveals Phase Relationship between Single-Cell Activity and Local Field Potentials

NeuroExplorer 5.017 released on July 22, 2015 has two new analyses: Find Oscillations analysis and Firing Phase analysis.

Find Oscillations analysis identifies episodes of oscillatory activity in the specified frequency band in recorded analog signals. The algorithm is described in Klausberger, Magill, Marton, Roberts, Cobden, Buzsaki and Somogyi. Brain-state- and cell-type-specific firing of hippocampal interneurons in vivo. Nature, 2003 Feb 20;421(6925):844-8

The user specifies two frequency bands (for example, theta band and delta band). NeuroExplorer finds the segments of LFP signal where theta to delta frequency power ratio exceeds a certain threshold. The LFP signal is then band-filtered and oscillation cycle start times are identified via Hilbert transform. This analysis adds several new variables to the data file. The data viewer screenshot below shows the results of analyzing variable LFP01. NeuroExplorer added three new variables: Theta_Epochs, Theta_Filtered and Theta_ZeroPhase:

FIndOsc1D

The LFP01_Theta_ZeroPhase event is than used in Firing Phase analysis that calculates probability of a neuron firing in a certain phase of theta cycle:

FiringPhase

By the way, I added these new analyses after several users asked me whether a phase-of-firing analysis is available in NeuroExplorer.

Do you need new analyses or new features in NeuroExplorer? Please send your requests to [email protected]

Nex5 File Format

NeuroExplorer Version 5.016 (released on July 3, 2015) supports new .nex5 data file format that is more flexible than .nex file format:

  • Allows NeuroExplorer to save files larger than 2 GB
  • Allows to save unlimited metadata for the whole file and for every file variable in JSON format

nex5 format is designed to be similar to .nex format so that the code that reads and writes .nex files would need only small modifications to implement reading and writing .nex5 files.

Full .nex5 file format specification as well as C++ code and Matlab code for reading and writing .nex5 files are available in the following files:

 

Dealing with Noise and Artifacts in Data Viewer

Often you can visually identify periods of noise or artifacts in 1D Data Viewer:

scratching artifact2

In NeuroExplorer version 5.014 or later, you can identify time intervals corresponding to artifacts using mouse:

– Right-click in 1D View to invoke context menu:

1dmenu1

– Specify ‘Select Interval Variable…’ menu command. NeuroExplorer will display the following dialog:

Specify Interval Variable to Add Intervals to

– Click ‘Create New Interval Variable…’ button:

New Variable Name

– Let’s create a new interval variable with the name noise. Type ‘noise’ (without quotes) and click OK to close this dialog, then click OK to close Select Interval Variable dialog.

– Note that the cursor now has ‘plus interval’ graphic:

intadd

– Press the left mouse button at the start of the noise interval, then drag the mouse until the end of the noise interval and release left mouse button. The new interval is added to noise interval variable:

added interval

– Add a second interval:

2intervals

– Hit ESC key to exit Add Interval mode

– We want to analyze data that is NOT in the noise intervals. To make this possible, right-click in 1D view again and select ‘Invert Interval Variable’ menu command:

invertintvar

– In the Invert Interval Variable dialog, select noise variable to be inverted:

Invert Interval Variable

– Now noise_inverted interval variable contains time intervals corresponding to our data without noise:

noiseinverted

– We can use noise_inverted variable in a Data Selection page of analysis properties dialog:

Analysis Properties data sel

and the data in noise intervals will be ignored.

There is also a faster way to get rid of noisy data — you can delete all the data in specified time intervals. To do this, right-click in 1D view and select ‘On Mouse Click and Drag, Delete…’ menu command:

deletemenu

– Now when you click and drag with the left mouse button, all the data in selected time interval are deleted:

deleted2

deleted3

Note that delete operation cannot be undone right away. You will need to reload the data file to restore original data.