Course data

This page includes information on the provenance of all data used in the course and instructions on how to download it from open access datasets on DANDI archive and from the International Brain Laboratory (IBL).

Beyond this course, it is useful to be aware of these excellent resources. They are very valuable for testing pipelines, comparing your datasets to other data collected from the same region or using the same probe. Further, they allow you to perform your own interesting analysis without collecting data. For example, this recent Paninski/Fusi-lab paper was based on their analysis of the International Brain Lab (IBL)’s open Brainwide Map.

Download all course data at once

For convenience, all datasets used in the course are hosted on Zenodo and Dropbox for easy download. These are typically cut-down versions of recordings (e.g. 90 seconds of an hour-long recording) that can be run in a short amount of time. If you prefer, the sections below explain how to download the data directly from the open repositories instead — useful if you only want a specific dataset, or want the full recordings for your own exploration.

To download everything required for the course at once:

All files from the course Zenodo data repository
The AL032 dataset from this Dropbox, used in the UnitMatch chapter.
(optional) The larger EB055 dataset from the same Dropbox, used in the UnitMatch chapter extended exercises.

Course data information

Head direction experiment from the DANDI archive

Used in preprocessing, sorting, postprocessing and Pynapple analysis.

The DANDI archive is an open, NIH-backed repository for sharing neurophysiology datasets in the standardized Neurodata without borders (NWB) file format (which SpikeInterface can load).

You can explore the many datasets on their website, and download them using either the Python API (used in this course) or DANDI’s command line interface, specifying the Dandiset ID and the path to the asset you want.

The dataset we will use through the majority of this course is hosted on DANDI here. The data was used in Duszkiewicz et al., 2024. The experiments recorded from the postsubiculum, a parahippocampal cortical region of the parahippocampal formation and a key hub of the spatial navigation system, using a 64-channel single-shank Cambridge Neurotech probe. They recorded from mice using open-field exploration (as well as during sleep) in which they measured the head-direction of the mice. They were interested in investigating neural tuning for head direction (i.e. head-direction tuned neurons fire when the head of an animal is facing a particular direction or angle).

The function found here can be used to download and save datasets (either the entire recording or parts of it) using SpikeInterface. We will use it to download the dataset above, but the same approach works for any of the thousands of recordings hosted on DANDI.

Spike times and behavior used in the Pynapple section

The NWB file for this dataset includes not only the raw electrophysiological data, but the results of spike sorting (spike times, units) and behavioral data from the experiment.

For the Pynapple head-direction decoding task, we only need the sorting outputs and behavioral data. Therefore we use this function to remove the electrophysiological data and accelerometer from the full NWB file. The cut-down data is available in the course repository at book/data/pynapple/dandiset-000939-no-traces-full.nwb.

IBL Benchmark dataset

Used in Preprocessing (part 1).

The IBL Benchmark dataset is a curated set of 11 Neuropixels recordings in varied brain regions available publicly via the Open Neurophysiology Environment (ONE) API. Data can be streamed directly — no bulk download required.

The script below demonstrates how to:

Connect to the public IBL server and select a probe insertion by its pid
Stream a short snippet of raw AP-band voltage
Apply a Butterworth high-pass filter (300 Hz) to remove LFP/DC drift
Run the full IBL destriping pipeline (ADC shift correction + spatial k-filter)
Open all three stages side-by-side in the viewephys interactive viewer


# %%
# This script demonstrates raw Neuropixels AP-band data processing:
#   1. Raw data     — straight off the probe, unfiltered
#   2. Butterworth  — high-pass filter to remove LFP/DC drift
#   3. Destriped    — full IBL pre-processing pipeline (butter + ADC correction + k-filter)
#
# viewephys opens an interactive viewer for each processing stage so you can compare them side by side.

import scipy.signal
import ibldsp.voltage
from one.api import ONE
from brainbox.io.one import SpikeSortingLoader
from viewephys.gui import viewephys
from iblatlas.atlas import BrainRegions

# ONE = Open Neurophysiology Environment, the IBL data access layer.
# The public endpoint lets anyone download data with those public credentials.
one_kwargs = dict(
    base_url='https://openalyx.internationalbrainlab.org',
    username='intbrainlab',
    password='international',
    silent=True,
)
one = ONE(**one_kwargs)

# Probe insertion IDs (pids) from the IBL Benchmarks dataset — a curated set of high-quality recordings.
pids = [
    '1a276285-8b0e-4cc9-9f0a-a3a002978724',  # 00 - Benchmark PIDS start
    '1e104bf4-7a24-4624-a5b2-c2c8289c0de7',
    '6638cfb3-3831-4fc2-9327-194b76cf22e1',
    '749cb2b7-e57e-4453-a794-f6230e4d0226',
    'd7ec0892-0a6c-4f4f-9d8f-72083692af5c',
    'da8dfec1-d265-44e8-84ce-6ae9c109b8bd',
    'dab512bd-a02d-4c1f-8dbc-9155a163efc0',
    'dc7e9403-19f7-409f-9240-05ee57cb7aea',
    'e8f9fba4-d151-4b00-bee7-447f0f3e752c',
    'eebcaf65-7fa4-4118-869d-a084e84530e2',
    'fe380793-8035-414e-b000-09bfe5ece92a',  # Benchmark PIDS stop
]

pid = pids[-1]
# SpikeSortingLoader is a convenience object that bundles the probe geometry,
# spike-sorting results, and raw data access for a single insertion.
ss = SpikeSortingLoader(pid=pid, one=one)

# get the histology information
channels = ss.load_channels()

# stream=True fetches only the requested chunk on-the-fly — no need to download the full recording.
# band='ap' selects the action-potential band (300 Hz high-pass, 30 kHz sample rate).
sr = ss.raw_electrophysiology(band='ap', stream=True)
sr_lf = ss.raw_electrophysiology(band='lf', stream=True)

# %%
# ---------------------------------------------------------------------------
# Load a short snippet of raw voltage data
# ---------------------------------------------------------------------------

T0 = 500  # start time in seconds — pick a quiet moment well into the recording

# sr[samples, channels] — slice to get 0.5 s of data.
# We transpose (.T) so the result is (n_channels, n_samples): the convention expected by ibldsp.
# The last channel is the sync/TTL channel; dropping it keeps only the 384 neural channels.
raw = sr[int(T0 * sr.fs):int((T0 + 0.5) * sr.fs), :-1].T

# %%
# ---------------------------------------------------------------------------
# Step 1 — Butterworth high-pass filter
# ---------------------------------------------------------------------------
# A 3rd-order Butterworth filter at 300 Hz removes slow LFP oscillations and
# DC drift while preserving the sharp transients caused by action potentials.
# sosfiltfilt applies the filter twice (forward + backward) for zero phase distortion.

butter_kwargs = {'N': 3, 'Wn': 300, 'btype': 'highpass', 'fs': sr.fs}
sos = scipy.signal.butter(**butter_kwargs, output='sos')
butt = scipy.signal.sosfiltfilt(sos, raw)

# ---------------------------------------------------------------------------
# Step 2 — Full IBL destriping pipeline
# ---------------------------------------------------------------------------
# ibldsp.voltage.destripe runs the complete IBL pre-processing chain:
#   • High-pass Butterworth (same as above)
#   • ADC sample-shift correction (Neuropixels channels are not sampled simultaneously)
#   • Spatial k-filter (removes stripe artefacts correlated across channels)
#
# channel_labels=True lets the function detect bad channels automatically from this
# snippet — convenient for exploration, but in production you would pre-compute
# labels from a longer stretch of data for robustness.

destriped = ibldsp.voltage.destripe(raw, fs=sr.fs, h=sr.geometry,
                                    neuropixel_version=sr.major_version,
                                    channel_labels=channels['labels'])

# ---------------------------------------------------------------------------
# Display all three stages in viewephys for side-by-side comparison
# ---------------------------------------------------------------------------
# viewephys is an IBL interactive viewer: scroll with the mouse wheel,
# adjust gain with +/-, and pan by clicking and dragging.
regions = BrainRegions()
eqcs = {}
eqcs['raw'] = viewephys(raw, sr.fs, channels=channels, title='Raw (unfiltered)', t0=T0, br=regions)
eqcs['butt'] = viewephys(butt, sr.fs, channels=channels, title='Butterworth high-pass 300 Hz', t0=T0, br=regions)
eqcs['destriped'] = viewephys(destriped, sr.fs, channels=channels, title='Destriped (butter + ADC + k-filter)', t0=T0, br=regions)