CS231A Lecture 7:Multi-view geometry (2)
Reading:
[HZ] Chapter 10 “3D reconstruction of cameras and structure”
[HZ] Chapter 18 “N-view computational methods”
[HZ] Chapter 19 “Auto-calibration”
[FP] Chapter 13 “projective structure from motion”
[Szelisky] Chapter 7 “Structure from motion”
Perspective SFM
Structure from motion problem
![](/images/cs231a-lecture7-2/image-20221120212756627.png)
![](/images/cs231a-lecture7-2/image-20221120213038698.png)
Structure from Motion Ambiguities
![](/images/cs231a-lecture7-2/image-20221120213631419.png)
ambiguities for 3D reconstruction:
![](/images/cs231a-lecture7-2/image-20221121153226261.png)
The Structure-from-Motion Problem
![](/images/cs231a-lecture7-2/image-20221120213648334.png)
Projective Ambiguity
![](/images/cs231a-lecture7-2/image-20221121175914571.png)
Metric reconstruction (upgrade)
![](/images/cs231a-lecture7-2/image-20221121175949492.png)
Structure from Motion methods
- Recovering structure and motion up to perspective ambiguity
- Algebraic approach (by fundamental matrix)
- Factorization method (by SVD)
- Bundle adjustment
- Resolving the perspective ambiguity
疑问
Q:如何理解“If the cameras are not calibrated, cameras and points can only be recovered up to a 4x4 projective (where the 4x4 projective is defined up to scale)”
A:感觉这里说的是,因为不知道相机的内参,缺少了约束条件,最终恢复的场景和实际的场景差了一个投影变换。比如,实际的motion应该是刚性变换,但我们因为不知道相机内参,只能恢复成投影变换,就像”Projective Ambiguity“这个部分一样。但为什么知道相机内参之后能够恢复到刚性变换呢?
这里应该是知道内参之后可以把$M$求出来,这样$X$也确定了,就消除歧义了。
Q:这里的未知量数目为$11m+3n-15$,其中的$-15$是怎么来的?
A:TODO(我发现,在cs231a winter1415的课程中,ppt说这里的未知数是$11m+3n-16$,我猜这个应该是和projective transformation有关系,它刚好16个数,然后15个未知量;可能这个值在这里求不出,或者在这里不用求,然后只能用其他手段求解,比如self calibration?)
Recovering structure and motion up to perspective ambiguity
Algebraic approach (2-view case)
Compute the fundamental matrix F from two views
![](/images/cs231a-lecture7-2/image-20221121202801817.png)
Use F to estimate projective cameras
![](/images/cs231a-lecture7-2/image-20221121203938026.png)
![](/images/cs231a-lecture7-2/image-20221121204500893.png)
Cross product as matrix multiplication
![](/images/cs231a-lecture7-2/image-20221121204654529.png)
![](/images/cs231a-lecture7-2/image-20221121204803490.png)
Compute cameras
![](/images/cs231a-lecture7-2/image-20221121204855976.png)
![](/images/cs231a-lecture7-2/image-20221121205610397.png)
Interpretation of b
回顾lecture5介绍的Epipolar Constraint
![](/images/cs231a-lecture7-2/image-20221121210121660.png)
![](/images/cs231a-lecture7-2/image-20221121210150661.png)
Use these cameras to triangulate and estimate points in 3D
Triangulation
![](/images/cs231a-lecture7-2/image-20221121210326543.png)
Algebraic approach: the N-views case
![](/images/cs231a-lecture7-2/image-20221121210541009.png)
Bundle adjustment
Limitations of the approaches so far
![](/images/cs231a-lecture7-2/image-20221121210840225.png)
About BA
![](/images/cs231a-lecture7-2/image-20221121211920908.png)
General Calibration Problem
![](/images/cs231a-lecture7-2/image-20221121212715597.png)
Advantages & Limitations
![](/images/cs231a-lecture7-2/image-20221121212749032.png)
Resolving the perspective ambiguity: Self-calibration
About Self-calibration
![](/images/cs231a-lecture7-2/image-20221121213019969.png)
Several approaches
![](/images/cs231a-lecture7-2/image-20221121213459393.png)
Direct approach
![](/images/cs231a-lecture7-2/image-20221121214257946.png)
Projections of conics across views
![](/images/cs231a-lecture7-2/image-20221121214350159.png)
Projection of absolute conics across views
From lecture 4, [HZ] page 210, sec. 8.5.1
![](/images/cs231a-lecture7-2/image-20221121214501672.png)
Kruppa equations
[Faugeras et al. 92] From [HZ] page 471
![](/images/cs231a-lecture7-2/image-20221121214622506.png)
![](/images/cs231a-lecture7-2/image-20221121214643244.png)
Advantages & Limitations
![](/images/cs231a-lecture7-2/image-20221121214702464.png)
疑问
Q:这里的矩阵$F$表示什么?
A:TODO
Algebraic approach (Multi-view approach)
![](/images/cs231a-lecture7-2/image-20221121214849946.png)
![](/images/cs231a-lecture7-2/image-20221121214940374.png)
Art of self-calibration:
![](/images/cs231a-lecture7-2/image-20221121215039297.png)
Inject information about the camera during the bundle adjustment optimization
![](/images/cs231a-lecture7-2/image-20221121213524460.png)