Semi-Rigid Registration of 3D Points

In this paper, we proposed a method for semi-rigid changed 3D point clouds registration. We first segment the point clouds into individual segments and then the alignment energy costs of each segment are calculated. The rough initial transformation is estimated by minimizing the energy cost using integer programming. The final registration results are obtained by rigid alignments of separated corresponded segments. Experimental result with simulated point clouds demonstrate that the concept of semi-rigid registration works well.


Introduction
In this paper, we propose a method for semi-rigid registration.Rigid and non-rigid 3D registration methods have been widely studied for many useful applications such as 3D shape reconstruction and visualization [1,4,5,7], medical image analysis [9], robot navigation [19,21] and augmented reality [25].Registration of 3D data is the process to align two sets of 3D points, which are called point clouds.This process is useful in many situations, for example, for stitching them together to obtain a larger point cloud which covers wider area, or for merging them to fill holes or cracks and obtain complete 3D models of objects or scenes.
Rigid registration is the process to align two 3D point clouds of a rigid object or a static scene: the target is assumed to be static and there is no change.Hence rigid registration methods find exactly the same overlapping part by estimating a rigid transformation between them.This is reasonable in cases that targets are stones, houses or furniture [15].In contrast, non-rigid registration methods deal with non-rigid objects: the target shape is changed and hence the transformation is no longer rigid.Typical target objects of non-rigid registration include human organs [26], skin [2,27], and clothes [3].
However, there is another situation, we call it semirigid, which more frequently happens in real situations: the target scene has several rigid objects that have moved or changed position and orientation.Consider an office, for example, where desks and chairs exist.These are rigid objects but might be moved after the 3D scene has been captured as a point cloud.In this case, rigid methods would not work well because the scene has been changed, and also, non-rigid methods could not capture the differences as expected because the scene is made of rigid objects.It is very important to deal with these kinds of scene changes for navigating robots in the environments [6] or for surveillance of outdoor scenes for preventing collapse in case of disaster [8].
In this paper, we propose a semi-rigid registration method to align two 3D point clouds of the same scene, which contains rigid objects, and some of them are not in the same position or orientation.Our approach first divides a point cloud into many rigid segments.Then correspondences between segments of two point clouds are calculated based on energy minimization by using integer programming.These correspondences provide an initial rigid transformation between two point clouds as a whole.Then transformations between each corresponding segments are estimated separately.
The rest of the paper is organized as follows.Related work on registration is reviewed in section 2. In section 3, we will introduce the proposed method.Experimental results are given in section 4.

Related Work
Rigid registration generally computes correspondences and then finds the transformation pose.Besl et al. [12] proposed a method called Iterative Closest Point (ICP) which is the most widely used method to align two 3D point clouds.ICP iterates two steps: in each iteration, closest points are selected, and a rigid transformation between them is estimated.This estimation is usually called the orthogonal Procrustes problem [13].
An alternative for registration is to use non-rigid methods.Sclaroff et al. [14] proposed a method to compute the correspondences in 2D images or 3D point clouds by using modal analysis.However, it requires complete shapes and a discretization suitable for finite element analysis.Also, some researches [10,11,15,16,17] focused on deformation model to determine deformation assumptions of a 3D shape.Obviously, deformation is suitable when our research target is semi-rigidly changed scenes.

Proposed Semi-Rigid Registration Method
In this section, we describe the details of the proposed method.Since our main focus is to establish correspondences between 3D segments, we first introduce the segmentation and pose estimation steps briefly.

Segmentation
The first step is to segment a point cloud into many rigid segments.To this end, we use the distances between the neighboring 3D points as well as the angle of the normal vectors of those points in order to cluster the points into different clusters (or rigid segments).Also we detect and segment the ground plane [18] in order to decide whether the segment is an object which is potentially able to be removed.Figure 1 shows a typical result of our simple dataset of some small blocks.In this case, the blocks are well separated and each segment can be seen as a rigid object in the scene.

Segments Correspondence
The next step is to establish correspondences between the 3D segments of the two point clouds.Our method is based on minimizing energies which represent the similarity between 3D segments.In addition to energies between two 3D segments, we define energies between two pairs of 3D segments too: if correspondences are correct, a pair of 3D segments in one point cloud is similar to another pair in the other point cloud.In the following sections, we explain the proposed energy method in the same manner as introduced by [23].

Formulation
Suppose that we have two point clouds P and Q .The point cloud is segmented into segments as We express each correspondence between the segments as Then we define an energy function   E x to represent the relations between the individual segments which will be introduced in the next section.In our energy function, we define the energy terms.

Energies
The first term 1 is the similarity in terms of 3D keypoint correspondences; in other words, how many of the 3D keypoints in A 3D keypoint is a representative point in a 3D segment, and it has a descriptor which can be used to find its corresponding point [8].If P  and Q  are similar to each other, many 3D keypoints in P  have corresponding points in Q  .Let C  be the number of correspondences between P  and Q  .Then the total number of correspondences of 3D keypoints in P  is  becomes larger when P  and Q  actually correspond to each other.Therefore, we use the reciprocal of the ratio as energy : (2) The second term 2 uses similarity between the descriptors of the corresponding 3D keypoints.While the first term finds how many keypoints are corresponding between E P  and Q  , this term evaluates whether the descriptors of those corresponding points are really similar to each other.We use the sum of distances between the corresponding descriptors as follows: where 1 is the distances between the descriptors of 3D keypoints of The third term 3 uses the rigid registration error between each pair of 3D segments.If the correspondences are correct, a pair of 3D segments must have its corresponding pair.In other words, two 3D segments We evaluate this pair of correspondences by using the residual error after rigid registration.Although we have assumed that the scene is semi-rigid, nevertheless, it is reasonable to assume that a pair of segments is rigid.We define as follows: .
where is the rigid registration error from ICP. 2 The last term 4 E imposes a penalty when there are no correspondences.If only the three energy terms above are used, a trivial solution will be obtained: all elements in are zero and the energy becomes 0. This is obviously not the case we want to estimate.Therefore, to avoid this case, we add the following term: where  is a penalty constant.If many a x are zero, i.e. many segments do not find their corresponding segment, this term becomes larger and hence such a solution will be less important.By combining Equations ( 2)-( 5), we get the finial energy function.

6)
This energy can be written as follow: , , min ( ) subject to the constraints in Eq. ( 1) where variables a x are binaries.This optimization problem is called a 0 -1 integer programming [24].We used a dual decomposi-tion approach [23] to solve this integer programming problem.
Once the solution is obtained, we can find an initial rigid transformation between two point clouds and Q .
Of course we have assumed that the scene is semi-rigid, so this rigid transformation is not the final solution.
x P

Rigid Registration for Each Segment
The final step performs rigid registration between corresponding 3D segments.After the initial rigid registration between the two point clouds, the corresponding 3D segments are now close to each other like in the example shown in

Experimental Results
We demonstrate by simulation that the concept of the proposed method effectively works on 3D point clouds.The dataset was generated from artificial small blocks.After the generation of one point cloud, one block was moved, and then another point cloud was reconstructed.Two point clouds with 15,804 and 25,178 points are shown in Figure 2 which are computed by 3D reconstruction with the Bundler [20] followed by Patch-based Multi-view Stereo [22].The rough initial transformation calculated based on the correspondences in section 3.2 is shown in Figure 3.The semi-rigid registration result shown in Figure 4 shows that the proposed method can estimate accurate transformation for each block.In contrast, the result when using ICP directly shown in Figure 5 was not successful.The energies between the two point clouds are shown in Figure 6.The energy of our method is much smaller than that of ICP, which clearly shows that the proposed semi-rigid registration outperforms the rigid ICP.

Conclusions
In this paper, we have proposed a method for registration of semi-rigid scenes of 3D point clouds.Experimental results show that our method works well for 3D point cloud datasets.In the future work, we will reduce the computation time due to the repetition of energy terms.

Figure 1 .
Figure 1.Segmentation result for small blocks.(Top) Some images used for 3D reconstruction.(Bottom) Segments of the point cloud visualized in different colors.Note that the ground plane is excluded.
sent correspondences of all of the segments.Each correspondence is denoted by , an index of an entry a a R  x in the vector .We say a correspondence a is acmake sure that each segment only has one (or no) correspondence, we use the following constraints:

Figure 3 .
In this step, each 3D segment P  in the first point cloud is registered to its counterpart Q P  in the other point cloud Here, the correspondence .Q ( , ) a    is active ( a ) in the solution obtained by solving the 0-1 integer programming explained above. 1  x

Figure 2 .
Figure 2. Experiment point clouds.Red block is moved between two point clouds.

Figure 3 .
Figure 3. Rough initial alignment of two point clouds.