1、Animal Tracking for Behavior Modeling,James M. Rehg Georgia Tech (standing in for Tucker Balch),People,PI Tucker Balch Faculty Jim Rehg, Computer Science Collaborator on animal tracking Aaron Bobick, Computer Science Collaborator on animal behavior modeling Bruce Walker, Psychology Collaborator on a
2、quarium project Atsushi Nakazawa, Computer Science (Osaka Univ., Japan) Collaborator on animal tracking PhD Students Michael Novitzky Jin Lee Matthew Flagg,Goals,Track social animals reliably under natural conditions (in vivo) from video Estimate behavioral parameters from tracking data Construct ex
3、ecutable models of social animal behavior Develop a biologically-inspired protocol for dynamic team formation,Year 1 Overview,Goal Develop reliable multi-target tracking algorithms for animals in video Approach Jointly estimate segmentation and motion of a nonrigid, deformable target Key requirement
4、s Reliably estimate target shape over time, to support behavioral analysis Minimize the amount of human effort required,Challenges,Accurate segmentation of target Set-point tracking is not sufficient Track reliably with significant camera motion Background subtraction is not sufficient Handle a wide
5、 range of animals Scalable solution for the animal kingdom Track multiple interacting targets Multiple instances of target type (e.g. wolf pack) Occlusions (with self, other targets, and background) Track long sequences, track across cuts,Approach,Basic research in video object segmentation and trac
6、king Modular software architecture Easily change features, models, and solver Testing in two scenarios Aquarium Monitoring Long sequences, static camera, some modeling effort Tracking Animal Planet videos Shorter sequences, moving camera, minimal human effort Preliminary application Accessible Aquar
7、ium Project,Tracking App,Basic Trackers: Contour feature w/ Iterated Closest Point Color feature w/ Mean Shift,Color Histogram Tracker,Fish Model: Appearance model For each species of fish, multiple HSV Color Histograms on Image patches Off-line Model Selection by Human Detection and Tracking: Histo
8、gram-based mean shift approach Maximization of Bhattacharyya Coefficient between Observation and Model Selection of a model with the highest coefficient and update track,Histogram-based Model Selection,Model Image Patches (to build histogram),Input Image,Similarity between shifted & model patches,0.
9、6,0.5,0.7,0.8,0.3,Best Model,*,Image Patch after Mean-shifting,Accessible Aquarium Project,Provide a meaningful and informative aquarium experience for visually-impaired or blind visitors Approach Track the movement of individual fish within tank Sonify the fish movement Example: 65 gallon marine aq
10、uarium Track yellow tangs and blue chromis Music structure is Bach chorale Each fish type is same instrument, different registers Movement speed mapped to note density (tempo) Horizontal dimension is stereo, vertical is timbre,Example,Play video,Tracking in the Video Volume,Graphcut Tracking,Trackin
11、g Results,Social Game Retrieval,Overview,Summary of Progress,Robust long-life tracking of multiple targets (fish) Tracking under controlled (but realistic) conditions Virtual Aquarium Project Novel experience of animal behavior via sonification New state-of-art motion segmentation algorithm Accurate
12、 segmentation of (shorter) video sequences under a wide range conditions,Year 2 Plans,Make general tracking method work for longer sequences Develop integrated tracking application tool for biologists Connect to biologists more effectively (what to say about this?) Start developing behavior models, how?,
