We frequently encounter situations where features appear and disappear from the image sequence. This problem is known as occlusion. The factorization method described in the previous section cannot be applied, per se, to solve this problem. The image sequences yield a matrix W which has some unknowns values (i.e., it is partially filled).  Fortunately, we can explore some information in the sequences that can allow us to find the unknown entries of W. This is done by doing projections of the feature coordinates onto camera positions.

Tomasi and Kanade [1], established a condition for reconstructing an unknown image point pair (u_fpv_fp ) in frame f as follows: we can reconstruct such point pair if for 3 given frames f₁ ,f₂  ,f₃  (not necessarily consecutive)  point p is visible and if we can find at least 3 more points p₁ ,p₂  ,p₃  (again, not necessarily consecutive), which are visible in all the four frames (f, f₁ ,f₂ , f₃). We will illustrate the methodology as follows. Suppose W = [U;V] is a  10 x 4 matrix, with  U_{5 x 4}  and V_{5 x 4} . This matrix has 2 unknown values denoted by X.

The first step is to use the factorization method over W_{8 x 4}  . This matrix  is obtained by eliminating the rows in W whose values are unknown in U and V, i.e.,

After factoring W_{8 x 4} ,  we get:

, which respectively represent the translation, rotation and shape generated from the submatrix W_{8 x 4 .}

Therefore, by the factorization method, the above can be expressed as:    W_{8 x 3} = R_{8 x 3}S  +  t_{8 x 1}e₄^T  , where e₄ = [1,1,1,1].

In order to have the full R (rotation matrix0, we need to compute i₄ and j_4, which are unknown. We first need to make  the origins of  i₄  and j₄ coincide by referring to the centroid c = 1/3 (s₁  + s₂  + s₃)  , where the indexes in s denote the points visible in all the four frames. In frame f , the projection of the centroid c  (i.e., in vector t) has its coordinates:

As can be seen, with these coordinates, we get the full vector t.

Now,  we define  S_p' ,U_4p' and V_4p' for p=1,2,3 (in this example) by subtracting S, U_4p and V_4p (respectively) by their coordinates with respect of their centroid (i.e.,  c, a₄ and b₄, respectively). We find i₄  and j₄ by  solving:
 
 

From the factorization method, we have that:

We now have all the missing information.

This method of reconstructing a point is called row wise extension, since we propagated S over the frames. There is another  method called column wise extension, in which we propagate the feature points instead.
 

Next:Results and Discussion Previous: Perspective Approximation Contents:Shape and Motion from Image Streams