1、Lecture 13: Local invariant featuresTuesday, Oct 30Prof. Kristen GraumanOutline Types of transformations and invariance Scale invariance Local features: detectors and descriptors SIFT What would we like our image descriptions to be invariant to?Geometric transformationsFigure from T. Tuytelaars ECCV
2、 2006 tutorialPhotometric transformationsFigure from T. Tuytelaars ECCV 2006 tutorialAnd other nuisancesNoiseBlur Compression artifacts Appearance variation for a categoryClasses of transformations Euclidean/rigid: Translation + rotation Similarity: Translation + rotation + uniform scale Affine: Sim
3、ilarity + shear Valid for orthographic camera, locally planar object (Projective: Affine + projective warps) Photometric: affine intensity changeI - aI+ bSimilarity transformationTranslation and ScalingTranslationAffine transformationProjective transf rmat onExhaustive searchA multi-scale approachSl
4、ide from T. Tuytelaars ECCV 2006 tutorialExhaustive searchA multi-scale approachSlide from T. Tuytelaars ECCV 2006 tutorialExhaustive searchA multi-scale approachSlide from T. Tuytelaars ECCV 2006 tutorialExhaustive searchA multi-scale approachSlide from T. Tuytelaars ECCV 2006 tutorialKey idea of i
5、nvarianceSlide adapted from T. Tuytelaars ECCV 2006 tutorialWe want to extract the patches from each image independently.Invariant local featuresSubset of local feature types designed to be invariant to Scale Translation Rotation Affine transformations Illumination1) Detect distinctive interest poin
6、ts 2) Extract invariant descriptorsMikolajczyk Lindeberg, 1994) An efficient choice is to detect peaks in the difference of Gaussian pyramid (Burt Crowley & Parker, 1984) Difference-of-Gaussian is a close approximation to LaplacianSlide adapted from David Lowe, UBCBlur SubtracScale selection princip
7、le Intrinsic scale is the scale at which normalized derivative assumes a maximum - marks a feature containing interesting structure. (T. Lindeberg 94)Maxima/minima of LaplacianScale Invariant Detection2221 22(, , )xyGxy e+=()2(,) (,)xx yyLGxyGxy =+(, , ) (, , )DoG Gxyk Gxy =Kernels:where Gaussian(La
8、placian)(Difference of Gaussians)Kernel Imagef =Slide by Darya Frolova and Denis SimakovScale space images: repeatedly convolve with GaussianAdjacent Gaussian images subtractedSIFT: Key point localizationn Detect maxima and minima of difference-of-Gaussian in scale spacen Then reject points with low
9、 contrast (threshold)n Eliminate edge responses (use ratio of principal curvatures)Blur SubtracCandidate keypoints: list of (x,y,)Adapted from David Lowe, UBCSIFT: Example of keypoint detectionThreshold on value at DOG peak and on ratio of principle curvatures (Harris approach)(a) 233x189 image(b) 8
10、32 DOG extrema(c) 729 left after peakvalue threshold(d) 536 left after testingratio of principlecurvaturesSlide from David Lowe, UBCScale Invariant DetectorsK.Mikolajczyk, C.Schmid. “Indexing Based on Scale Invariant Interest Points”. ICCV 2001 Experimental evaluation of detectors w.r.t. scale chang
11、eRepeatability rate:# correspondences# possible correspondencesScale Invariant Detection: Summary Given: two images of the same scene with a large scale difference between them Goal: find the same interest points independently in each image Solution: search for maxima of suitable functions in scale
12、and in space (over the image)Affine Invariant Detection Above we considered:Similarity transform (rotation + uniform scale) Now we go on to:Affine transform (rotation + non-uniform scale)Affine Invariant Detection Intensity-based regions (IBR): Start from a local intensity extrema Consider intensity
13、 profile along rays Select maximum of invariant function f(t) along each ray Connect local maxima Fit an ellipseT.Tuytelaars, L.V.Gool. “Wide Baseline Stereo Matching Based on Local, AffinelyInvariant Regions”. BMVC 2000.Affine Invariant DetectionMatas et al. Robust Wide Baseline Stereo from Maximal
14、ly Stable Extremal Regions. BMVC 2002. Maximally Stable Extremal Regions (MSER) Threshold image intensities: I I0Extract connected components(“Extremal Regions”) Seek extremal regions that remain “Maximally Stable” under range of thresholdsPoint Descriptors We know how to detect points Next question
15、:How to describe them for matching?Point descriptor should be:1. Invariant2. DistinctiveRotation Invariant Descriptors Harris corner response measure:depends only on the eigenvalues of the matrix MC.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988Rotation Invariant Descriptors Find lo
16、cal orientationDominant direction of gradient Rotate description relative to dominant orientation1 K.Mikolajczyk, C.Schmid. “Indexing Based on Scale Invariant Interest Points”. ICCV 20012 D.Lowe. “Distinctive Image Features from Scale-Invariant Keypoints”. Accepted to IJCV 2004Scale Invariant Descri
17、ptors Use the scale determined by detector to compute descriptor in a normalized frameImages from T. TuytelaarsSIFT descriptors: Select canonical orientationn Create histogram of local gradient directions computed at selected scalen Assign canonical orientation at peak of smoothed histogramn Each ke
18、y specifies stable 2D coordinates (x, y, scale, orientation)02Slide by David Lowe, UBCSIFT descriptors: vector formationn Thresholded image gradients are sampled over 16x16 array of locations in scale spacen Create array of orientation histogramsn 8 orientations x 4x4 histogram array = 128 dimension
19、sSlide by David Lowe, UBCSIFT properties Invariant to Scale Rotation Partially invariant to Illumination changes Camera viewpoint Occlusion, clutterSIFT matching and recognitionn Index descriptorsn Generalized Hough transform: vote for object posesn Refine with geometric verification: affine fit, ch
20、eck for agreement between image features and modelSIFT FeaturesAdapted from David Lowe, UBCValue of local (invariant) features Complexity reduction via selection of distinctive points Describe images, objects, parts without requiring segmentation Local character means robustness to clutter, occlusion Robustness: similar descriptors in spite of noise, blur, etc.Coming up Problem set 3 due 11/13 Stereo matching Local invariant feature indexing Thursday: image indexing with bags of words Read Video Google paper
