1、1Capturing, Modeling, Rendering 3D ObjectsP iti d O i t ti E ti tiosition an r en a on stimationPosition and Orientationz Localization Compute the position and orientation of “something” within an environment or relative to an object Something = robot, boat, planez Tracking Use a (custom) hardware i
2、nfrastructure to track an object and compute the position and orientation relative to a chosen originz Camera Pose Estimation Compute the position and orientation of a camera within an environment or relative to an object using vision-based methodsLocalization and Trackersz Gyroscopes/Accelerometers
3、images 2002 Encyclopdia Britannica, Inc.Localization and Trackersz Gyroscopes/AccelerometersLocalization and Trackersz Gyroscopes/Accelerometers Pros: very high accuracy Cons: drift after 10 minutes Potential solution for orientation: Resync periodically with a digital compass Potential solution for
4、 position:?Localization and Trackersz Magnetic Tracker2Localization and Trackersz Magnetic Tracker Pros: simple infrastructure Cons: not very accurate, about 1 meter radius, susceptible to distortions caused by surrounding metal structuresLocalization and Trackersz Optical Tracker (3rdTech Inc.)Loca
5、lization and Trackersz Optical Tracker (3rdTech Inc.)Localization and Trackersz Optical Tracker (3rdTech Inc.) Pros: highly accurate Cons: custom/complex infrastructure must be installed, requires line of sightMIT City Scanning Projectz http:/city.lcs.mit.edu/city.htmlTransformation of a 3D world co
6、ordinate to image coordinatesRigid body transformation: tx, ty, tz, rx, ry, rzPerspective projection: f(xw, yw, zw)Radial lens distortion: k1Resampling/scaling adjustment: Cx, Cy, sx(X, Y)3Camera Pose Estimationz Tsai Calibration External parameter calibration (same thing!)(rx,ry,rz) image plane(0,0
7、,0)(tx,ty,tz)eye(xw,yw,zw)(X, Y)Camera Pose Estimationz Tsai Calibration External parameter calibration (same thing!)z Pose Estimation Algorithm:Calibrate camera Calibrate ca ra Fix internal parameters Move camera Recompute external parameters (e.g. redo calibration) DoneCamera Pose Estimationz Tsai
8、 Calibration External parameter calibration (same thing!)z Pose Estimation Algorithm:Calibrate camera Calibrate ca ra Fix internal parameters Move camera Recompute external parameters (e.g. redo calibration) DoneCamera Pose Estimationz Calibrate with this patternCamera Pose Estimationz Then fix inte
9、rnal parameters and do pose estimationLandmark-based Camera Pose Estimationz Summary Place landmarks throughout the environment (or around the object) Obtain the projections of landmarks onto captured images Works with a calibrated traditional camera Works with a calibrated omnidirectional camera Co
10、mpute pose from the landmark projections4Landmark-based Camera Pose Estimationdf6f3f4f5f7f8cameraf1f2f9f10Landmark-based Camera Pose Estimationz Major challenges Landmark placement and correspondence Where should we place the landmarks and how do we know which one we are seeing? Pose estimation from
11、 landmarks How do we compute position and orientation from the landmarks? Landmark-Pose Optimization Can we optimize/improve estimates for both landmark positions and camera pose?Landmark Placement and Correspondencez Background: Art Gallery Problem DefinitionLandmark Placement and Correspondencez B
12、ackground: Art Gallery Problem Convex decompositionLandmark Placement and Correspondencez Background: Art Gallery Problem Creating a sufficient solution satisfying the constraints of Maximum distance to a landmark: D Minimum number of visible landmark: V Minimum angle to a pair of landmarks: ALandma
13、rk Placement and Correspondencez Background: Art Gallery Problem Creating a sufficient solution satisfying the constraints of Maximum distance to a landmark: D Minimum number of visible landmark: V Minimum angle to a pair of landmarks: ARedundancy value for landmark i is ri= wv(vmin-V) + wd(D-dmin)/
14、D + wa(amin-A)/(180-A)(iteratively remove from previous slide)5Landmark Placement and CorrespondenceV-+Landmark Placement and CorrespondenceD-+Landmark Placement and CorrespondenceA-+Landmark Placement and Correspondencez Creating a sufficient solution satisfying the constraints of Maximum distance
15、to a landmark: D Minimum number of visible landmark: V Minimum angle to a pair of landmarks: ACld i tz Can also add accuracy requ remen s Distribute landmarks so as to Guarantee a desired pose accuracy, or Compute the pose accuracy of a given placementLandmark Placement and Correspondencez Uncertain
16、ty given distances(1 or 2 distance case, d and e)Landmark Placement and Correspondencez Uncertainty given distancesdd1region of uncertainty2d36Landmark Placement and Correspondencez Uncertainty given distances and angleded12Landmark Placement and Correspondencez Correspondence and VisibilityPose Est
17、imation from Landmarksz Lets do a simple 2D examplec2=a2+b2+2abcos()xyPose Estimation from Landmarksx-axisxypost 1d1d2worldx-axisworldy-axisBADEt = d02/(d12+d22-2d1d2cosA)1/2post 2d0Ct dx = (d02+t2d12-t2d22)/(2d0)y = S(t2d12-x2)1/2 = C E = D BPose Estimation from Landmarksz Lets do a simple 3D examp
18、lexyzPose Estimation from Landmarksd1d12d14f1fd2d3d4d23d342f3f4141223347Optimizationz Use a procedure related to “bundle adjustment” A nonlinear optimization method Triggs00 Summary: Improve pose estimates Improve landmark estimates landm timates Loop back until convergesOptimizationdf6f3f4f5f7f8cam
19、eraf1f2f9f10Optimizationz Increasing minimum visible landmarks2025(cm)6432051015024681012141618Minimum number of visible fiducialsMeanposeerrorbound168421actualOptimizationz Increasing minimum subtended angle by landmarks2025(cm)64320510150 20 40 60 80 100 120 140 160Minimum subtended angleMeanposeerrorbound 168421actualOptimizationz Increasing maximum distance to landmarks2025(cm)64320510150 100 200 300 400 500 600 700 800Maximum distance to fiducials (cm)Meanposeerrorbound 168421actual
