Affine transformation is a linear mapping method that preserves points, straight lines, and planes. Sets of parallel lines remain parallel after an affine transformation. The affine transformation technique is typically used to correct for geometric distortions or deformations that occur with non-ideal camera angles

도움이 되는 자료들

https://articulatedrobotics.xyz/5-transformation_matrices/

https://www.researchgate.net/figure/Process-of-coarse-alignment-a-The-reference-face-and-input-face-b-3D-affine_fig3_266681108

https://www.youtube.com/watch?v=E3Phj6J287o

https://towardsdatascience.com/image-geometric-transformation-in-numpy-and-opencv-936f5cd1d315

https://blog.csdn.net/qq_37174526/article/details/100137483

https://www.brainvoyager.com/bv/doc/UsersGuide/CoordsAndTransforms/SpatialTransformationMatrices.html

https://stackoverflow.com/questions/45637472/opencv-transformationmatrix-affine-vs-perspective-warping

https://fivedots.coe.psu.ac.th/~ad/jg/ch139/index.html

https://en.wikipedia.org/wiki/3D_projection

Eigen Expression

class Eigen::AngleAxis< _Scalar > Represents a 3D rotation as a rotation angle around an arbitrary 3D axis. This is defined in the Geometry module.

// g++ eigen_test.cpp -o eigen_test -I /usr/include/eigen3/

// dynamic block expression.
Eigen::Quaternionf qvec_final(final_transform.block<3, 3>(0, 0)); // array size, position

// read it
Eigen::Vector3f tvec_final = final_transform.block<3, 1>(0, 3); // array size, position

easy test to do very good.

RANSAC을 2D면 최소 2개의 포인트, 3D면 최소 3개의 포인트를 Random으로 잡아서 [2D이면 line fomular]/[3D면 plane formular]를 통해 모델을 만들고 모든 포인트들을 모델에 적용하여 하여서 일정 distance threshold inliers수를 저장을 하고 N 번만큼은 이 작업을 한다.

작업 이후 제일 많은 inliers을 가지고 있는 plane formular를 primary plane로 사용하여서 Segmentation을 한다.

RANSAC 은 주로 파라미터 모델을 찾는데 사용이 되긴하는데, 다양한 용도에서도 사용이 된다.

NDT-RANSAC BASED Segmentation

NDT RANSAC에서는 이미 모든 포인트들에 대해 Normal distribution(Mean, covariance Matrix) 혹은 Cross product을 통한 normal vector에 대한 classified(EigenValue크기에 따라 plane, edge, point를 구분할 수 있다)를 하고 오직 Plane에 때한 벡터를 NDT CELL(Noraml surface)로 구분을 하는데, 랜덤으로 NDT CELL의 Plane Parameter(plane formular)를 픽을 하고,모든 포인트들을 해당 formular에 프로젝션을 시켜서 정한 특정 distance threshold, curvature threshold가 만족하는 포인트들을 inilier 저장하여,N 번만큼 Iteration을 랜덤으로 또 NDT CELL(Plan Formular)를 픽하고 그에 대한 inliers 수를 저장을 하여 가장 많이 inliers을 가지고 있는 NDT CELL(Plane formular)를, primary PLANE(Plane formular 모델)로 잡는데 사용 한다.

그 이후는 primary plane에 존재하는 포인트들을 가지고 least square를 통해 plane fitting을 한다.

쉽죠잉?

Normal vector, Normal Surface를 만드는 것 자체가 Region 별로 그룹화를 해서 Mean, Covariance matrix를 통한 plane인지 edge인지 point인지 classified하고 혹은 3 set points을 cross product으로 통해 normal vector를 구할 수 있다.(이때 첫번째 point를 기준으로 normal vector를 만들게 된다.)

이렇게 되면 결국 데이터 index가 줄어 들게 된다.

예로

pcd::Pointcloud ptr = [ 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 ]

이런 데이터가 있다고 한다면, Normal Vector, Normal Surface로 만든 이후에는 Normal Distribution 혹은 cross product으로 만들었다고 했을때, 아래와 같이 데이턱 될 것이다.

pcd::pointcloud nv = [  X ,X ,X ,X ,X ,X ,X ,X ,X ,X ,X,
                        X ,X ,X ,X ,X ,X ,X ,X ,X ,X ,X,
                        X ,X ,X ,X ,X ,X ,X ,X ,X ,X ,X,
                        1, 1 ,1 ,1 ,1 ,1 ,1 ,1 ,X ,X ,X,
                        1, 1 ,1 ,1 ,1 ,1 ,1 ,1 ,X ,X ,X,
                        1, 1 ,1 ,1 ,1 ,1 ,1 ,1 ,X ,X ,X,
                        2, 2 ,2 ,2 ,2 ,2 ,0 ,2 ,X ,X ,X,
                        2, 2 ,2 ,2 ,2 ,2 ,0 ,2 ,X ,X ,X,
                        2, 2 ,2 ,2 ,2 ,2 ,0 ,2 ,X ,X ,X,
                        2, 2 ,2 ,2 ,2 ,2 ,0 ,2 ,X ,X ,X,
                        0, 0 ,0 ,0 ,0 ,0 ,0 ,0 ,X ,X ,X,
                        1, 1 ,1 ,1 ,1 ,1 ,1 ,1 ,X ,X ,X,
                        1, 1 ,1 ,1 ,1 ,1 ,1 ,1 ,X ,X ,X ]

Normal vector가 1이면 plane, 0이면 edge, 2이면 points로 나눠지면서 데이터의 수가 적어 적어진다.(X는 데이터 어드레스에 데이터가 존재하지 않는다는 뜻)

+) 이런 식으로 Normal vector를 만들고 raw데이터를 normal vector로 부터 얻어지는 plane fomular model을 통해 distance를 구하고 egienvalues들을 통하여 plane의 curvature 만족하는 포인트들을 segment classified를 하는데 많이 쓰인다.

+) 만약 특정 Region(Ellipse, Rectangle)을 통한 means Covariance Matrix를 구하고 SVD를 통해 normal plane model을 구하고 이를 raw 포인트를 넣어서 distance(similarity)를 구하거나 eigenvalue로 인하여 curvature를 구할 수 있다.

+) 만약 2D라고 한다면 이는 covariance matrix나 cross product으로 얻어지는 line fomular이다.

NDT(Normal distribution transformation) : 데이터들을 Region이나 특정 포인트 수만큼을 가지고, Normal distribution(Mean, Covariance matrix로 표현하는) 변환을 얘기한다.

여기서 RANSAC은 Iterative reweighed least-squares(IRLS)를 통한 Plane Fitting을 하는데 사용이 되었다.

이 페이퍼의 Main Contribution은 the planar NDT cell is selected as a minimal sample on the surface of the sam plane to ensure the correctness of sampling. the proposed method can improve the correctness of plane segmentation and eliminate the spurious plane problems of standard ransac. an iterative reweighed least-squares(IRLS) approach is also used for plane fitting to improve the the reliability and accuracy of a detected plane.

Releated Works

NDT methods

point features are commonly used and are caluclated by the neihbor of each point. for example, the normal, curvature and roughness features are used for pointcloud segmentation and compression.

NDT is a concept similar to super-voxels and sub-windows.

NDT classified the sub-windows into two categories based on their shape(Normal surface shape) : Planar and non Planar.

A Probability Density Function(PDF) is computed for each cell, based on the point distribution within the cell

generated NDT by at least 3 point grouped, cross-product makes normal vector(normal cell) or group points refer to shape of covariance matrix, by then create normal vector(normal cell).

RANSAC

the Ransac is that hypothesis shapes are generated by randomly selecting a minimal subset of n points and estimating the correspoiding shape-model parameter.

in this thesis, used for fitting plane model.

for example, 3 points are need to determine a plane. then, the remaining points in the pointcloud are tested with the resulting candidate shapes to determine how many of the points well approximated by the model. after a certain number of iterations, the shape that possess the largest percentage of inliers is extracted and the algorithm continues to process the remaining data. the plane is often fitting by the least squares(ls) method.

RANSAC depends on the subset size, the inilier ratio and the number of data points.

Proposed methods

Input Pointcloud -> NDT CELL(at least 3 points, mean and covariance) -> egienvalues ratios(determine plane or point) -> only get planar ndt cells -> IRLS ndt normal -> ndt ransac(randomly choose cell and find out the cell has biggest inlier(points that above plane(which it was used forming plane)) after filtering by some threshold) -> choose primary plane, which has biggest inliers points -> IRLS do it(points that grouped in primary plane used for fitting plane model) -> now we have plane model -> projected remaining point againt to all the ndt cell which has new plane model -> complete plane -> connected-components.

PCL DOC :

https://pcl.readthedocs.io/projects/tutorials/en/latest/random_sample_consensus.html?highlight=console#the-explanation

good sample !

Segmentation :

https://blog.csdn.net/cindy9608/article/details/119886698

Segmentation하는 법은 Normal Vector를 구하고 ax + by + cz + d = 0 (Plane coefficient)을 만족을 하면 Plane으로 인지

RANSAC Segmentation RANSAC

well explained :

https://medium.com/@ajithraj_gangadharan/3d-ransac-algorithm-for-lidar-pcd-segmentation-315d2a51351

3차원에서의 RANSAC 방법

https://towardsdatascience.com/detecting-the-fault-line-using-k-mean-clustering-and-ransac-9a74cb61bb96

2차원에서의 ransac 방법

https://stats.stackexchange.com/questions/267267/what-does-scaling-the-normal-vector-of-a-plane-hyperplane-mean

• 오픈소스를 참고를 하지만, 완전히 쓰지는 말자. 하나하나 까보면서 수식을 이해하고 코드를 이해하면서 내것으로 만든느 것이 중요하다.

• 하나를 하더라도 똑바로 알고 해야된다. 누군가에게 설명할 정도로 잘 알아야한다.

• 일은 일이다. 회사는 필요한 resource를 제공하는데, 직원이 좋은 performance를 내지 못한다면 “업무, 기회, 리소스”를 잃을 가능성이 있다. 즉, 가치 있는 행동을 해야되고, 가치있는 개발을 해야되고, 도움이 되어야 된다.