Behavioural Projects Overview

SWC/GCNU Neuroinformatics Unit

Niko Sirmpilatze, Sofia Miñano, Chang Huan Lo

RSEs working on behaviour

Niko Sirmpilatze

Sofía Miñano

Chang Huan Lo

Behavioural experiments at SWC

Tracking fiddler crabs in the field

Sofía Miñano, working with the Margrie & Branco labs.

Our approach

A single standardised workflow won’t do

We need general-purpose, versatile, interoperable tools that can be reliably mixed and matched to different needs.

  • What are common workflows and tasks?
  • Which needs are well-served?
    • Support and teach existing tools.
  • Which needs are not covered?
    • Develop new tools.

Common workflows

Modern computational neuroethology workflow

Markerless pose estimation

deeplabcut.org

Supporting & teaching pose estimation at SWC

What about other CV tasks?

Computer vision (CV) tasks other than pose estimation are less accessible.

  • Mix-and-match computer vision tools for animal behaviour analysis.
  • Still under early development (talk to Sofía!)

ethology.neuroinformatics.dev

What happens after tracking?

movement overview

movement.neuroinformatics.dev

A unified interface for motion tracks

from movement.io import load_poses

ds = load_poses.from_file("/path/to/poses.h5", source_software="DeepLabCut", fps=50)

movement example applications

Sample data: Elevated Plus Maze from Loukia Katsouri | Pupil Tracking from Sepi Keshavarzi.

movement GUI

Sample data from Ann Duan’s Lab.

movement outlook

More advanced methods for outlier detection & smoothing

Modules for specialised applications

  • 🤔 gait analysis, pupillometry, collective behaviour

Integration with tools for neurophysiological data analysis

What about behaviour segmentation?

We have explored some existing tools:

Would love to hear your experiences with others!

No single go-to solution yet

🤔 Is there scope for collaborating on improving existing tools or developing a new robust, easy-to-use interface for behaviour segmentation?

Some remaining challenges

1. We are generating and storing vast amounts of video data.

Determine & share best practices for video compression.

2. Requirements for manual data annotation are honerous.

  • Can we leverage self-supervised pre-training like beast?
  • Sofía would be interested in making beast more plug-and-play for various CV tasks.