| Parent category | Instantaneous behaviour | definition | |
|---|---|---|---|
| 0 | locomotion | foraging | Foraging for detritus and small organisms on mudflats or in shallow waters. Dactyl(s) is moving towards the mouth. |
| 1 | resting | burrowing | The crab is observed resting in its burrow. There are no conspecific interactions. |
| 2 | locomotion? | entering burrow | Actively entering an uninhabited burrow. |
| 3 | locomotion? | exiting burrow | Actively exiting a burrow. |
| 4 | locomotion? | maintaining burrow | Digging and excavating burrows, carrying sediment to/from the burrow entrance. |
Video-based analysis of animal behaviour
SWC/GCNU Neuroinformatics Unit
Introductions
Niko Sirmpilatze
Chang Huan Lo
Sofía Miñano
Quantifying behaviour: modern
Detection
Tracking
Pose estimation
- “easy” in humans - vast amounts of data
- “harder” in animals - less data, more variability
Pose estimation software
DeepLabCut: transfer learning
SLEAP:smaller networks
These handle pose estimation (detection) and tracking of single/multiple animals.
Multi-animal part grouping
Top-down vs bottom-up
Multi-animal identity tracking
3D pose estimation
Which mouse is more anxious?
Click here to post your answers
The Elevated Plus Maze
- Structure: 2 open arms, 2 closed arms, central area
- Exploits rodents’ natural aversion to open spaces and height
- Less anxious animals spend more time in open arms
Task: quantify time spent in open arms / closed arms
The SLEAP workflow
Create a new project
Define a skeleton
| Source | Destination |
|---|---|
| snout | left_ear |
| snout | right_ear |
| snout | centre |
| left_ear | centre |
| right_ear | centre |
| centre | tail_base |
| tail_base | tail_end |
Save the project right after defining the skeleton!
Generate labeling suggestions
Label initial ~20 frames
Start a training job 1/3
Start a training job 2/3
Start a training job 3/3
Monitor training progress
Evaluate trained models
Run inference on new frames
What happens after pose tracking?
- Load data into Python
- Visualise and inspect data
- Clean trajectories:
- Identify and drop outliers
- Smooth trajectories
- Interpolate over missing data
- Compute variables of interest:
- Velocity, acceleration, heading, etc.
- Distances/angles between body parts
- Time spent in different regions
- Application-specific: navigation, social interactions, gait analysis
movement
A Python toolbox for analysing body movements across space and time, to aid the study of animal behaviour in neuroscience.
The movement poses dataset
The movement bboxes dataset
Discretising the continuous 1/2
Discretising the continuous 2/2
Motion Sequencing
- Timescale is controlled by the
kappaparameter - Higher
kappa> higher P(self-transition) > “stickier” states > longer syllables
Keypoint-MoSeq
Can we apply MoSeq to keypoint data (predicted poses)?
Problems with keypoint data
- Keypoint noise leads to artifactual syllables
- We should somehow isolate true pose from noise
- But smoothing also blurs syllable boundaries
Solution: a more complex model
Switching Linear Dynamical System (SLDS): combine noise-removal and action segmentation in a single probabilistic model
Let’s look at some syllables
We’ve trained a keypoint-MoSeq model on 10 videos from the (EPM) dataset.
mouse-EPM/
├── derivatives
│ └── software-kptmoseq_n-10_project
└── rawdata
Join the movement!
Contributions to
movementare absolutely encouraged, whether to fix a bug, develop a new feature, improve the documentation, or just spark a discussion.
