for Robot Artificial Inteligence
가상코어를 논리코어라고ㄷ ㅗ말하긴 한다.
CPU 코어에 Thread가 들어가 있는데, 코어하나에 쓰레드가 여러개 들어간다.
이것이 물리 쓰레드
가상쓰레드(논리쓰레드)는 흔히 우리가 개발하는데 있어서, 특정 프로그램에 대한 쓰레드를 할당해 주는 것이다.
Superpoint 논문에서 매칭한거는 nn 매칭한 결과이고, superglue는 superpoint에서 추출된 descriptor를 입력받아 graph neural net에 기반하여 매칭하는거라고 보시면 됩니다. 즉, superpoint + superglue는 nearest neigbor matching(KD tree- euclidean distance) 을 사용하는것이 아닌(ML 방식), DL방식을 사용한것이다.
Vins Mono에서 각속도 수식에 오메가 함수 안에 0값이 우하단에 들어가는데 이 경우 쿼터니언의 허수부가 앞에 있고 실수 부가 q = (q_x, q_y, q_z, q_w)
p(t) = q(t)(p,1)q(t)^t 이런 쿼터니언 회전식 미분하면 각속도랑 관련된 식이 나오고 순허수 (w,0)꼴로 표현됐던걸로 기억합니다, 각속도는 유닛이 아니게 되고 각도랑도 상관이 없구요. 회전을 표현하는 유닛쿼터니온인지 각속도같은 다른 수치를 표현하는 쿼터니온인지 구분해서 해석하면 될거같습니다.
멀티 쓰레딩 잘하시는 분들은 생산 파이프라인 최적화 잘하실거 같다.
ARM_NEON_x_x86 가속연산을 하다보면 보통 pc에서는 적용이 안되는데, 라이브러리를 통해 pc에서 바로 가속 연산 돌게해주는 라이브러리
It is “naive” in that the language and notations are those of ordinary informal mathematics, and in that it does not deal with consistency or completeness of the axiom system . Likewise, an axiomatic set theory is not necessarily consistent: not necessarily free of paradoxes.
sudo scp something.tar.gz chan@xxx.xxx.xx.xxx:/home/changyo/
scp chan@xxx.xxx.xx.xxx:/home/changyo/2022-02-04-15-34-51.bag ~home/chan
tar xf something-noetic.tar.gz -C /opt/rit/
Search in termnial : Ctr + R
ctrl+shift+f : past copied .
https://askubuntu.com/questions/180336/how-to-find-the-process-id-pid-of-a-running-terminal-program
https://askubuntu.com/questions/797957/how-to-kill-a-daemon-process-in-linux
https://stackoverflow.com/questions/46479196/best-practice-pid-file-for-unix-daemon
https://github.com/IntelRealSense/librealsense/blob/master/doc/depth-from-stereo.md
Using this tutorial you will learn the basics of stereoscopic vision, including block-matching, calibration and rectification, depth from stereo using opencv, passive vs. active stereo and relation to structured light.
Regular consumer web-cams offer streams of RGB data within the visible spectrum. This data can be used for object recognition and tracking, as well as some basic scene understanding.
Even with machine learning grasping the exact dimensions of physical objects is a very hard problem
This is where depth cameras come-in. The goal of a depth camera is to add a brand-new channel of information, with distance to every pixel. This new channel can be used just like the rest (for training and image processing) but also for measurement and scene reconstruction.
Depth from Stereo is a classic computer vision algorithm inspired by human binocular vision system. It relies on two parallel view-ports and calculates depth by estimating disparities between matching key-points in the left and right images:
Depth from Stereo algorithm finds disparity by matching blocks in left and right images
Most naive implementation of this idea is the SSD (Sum of Squared Differences) block-matching algorithm:
import numpy
fx = 942.8 # lense focal length
baseline = 54.8 # distance in mm between the two cameras
disparities = 64 # num of disparities to consider
block = 15 # block size to match
units = 0.001 # depth units
for i in xrange(block, left.shape[0] - block - 1):
for j in xrange(block + disparities, left.shape[1] - block - 1):
ssd = numpy.empty([disparities, 1])
# calc SSD at all possible disparities
l = left[(i - block):(i + block), (j - block):(j + block)]
for d in xrange(0, disparities):
r = right[(i - block):(i + block), (j - d - block):(j - d + block)]
ssd[d] = numpy.sum((l[:,:]-r[:,:])**2)
# select the best match
disparity[i, j] = numpy.argmin(ssd)
# Convert disparity to depth
depth = np.zeros(shape=left.shape).astype(float)
depth[disparity > 0] = (fx * baseline) / (units * disparity[disparity > 0])
Rectified image pair used as input to the algorithm
Depth map produced by the naive SSD block-matching implementation
Point-cloud reconstructed using SSD block-matching
There are several challenges that any actual product has to overcome:
In reality having two exactly parallel view-ports is challenging. While it is possible to generalize the algorithm to any two calibrated cameras (by matching along epipolar lines), the more common approach is image rectification. During this step left and right images are reprojected to a common virtual plane:
opencv library has everything you need to get started with depth:
import numpy
import cv2
fx = 942.8 # lense focal length
baseline = 54.8 # distance in mm between the two cameras
disparities = 128 # num of disparities to consider
block = 31 # block size to match
units = 0.001 # depth units
sbm = cv2.StereoBM_create(numDisparities=disparities,
blockSize=block)
disparity = sbm.compute(left, right)
depth = np.zeros(shape=left.shape).astype(float)
depth[disparity > 0] = (fx * baseline) / (units * disparity[disparity > 0])
The quality of the results you will get with this algorithm depends primarily on the density of visually distinguishable points (features) for the algorithm to match. Any source of texture, natural or artificial will improve the accuracy significantly.
That’s why it is extremely useful to have an optional texture projector (usually adding details outside of the visible spectrum). As an added benefit, such projector can be used as an artificial source of light at night or in the dark.
Input images illuminated with texture projector
Left: opencv stereobm without projector. Right: stereobm with projector.
Structured-Light is an alternative approach to depth from stereo. It relies on recognizing a specific projected pattern in a single image.
For customers interested in a structured-light solution Intel offers the SR300 RealSense camera
Having certain benefits, structured-light solutions are known to be fragile since any external interference (from either sunlight or another structured-light device) will prevent you from getting any depth.
In addition, since laser projector has to illuminate the entire scene, power consumption grows with range, often requiring a dedicated power source.
Depth from stereo on the other hand, only benefits from multi-camera setup and can be used with or without projector.
D400 Intel RealSense cameras
D400 RealSense cameras offer the following basic features:
This product was designed from the ground up to address conditions critical to robotics / drones developers and to overcome the limitations of structured light.
Depth-map using D415 Intel RealSense camera
because of realsesne depth image data generated by pattern matching between stereo camera and structed light way, if camera seeing some pattern scene, dept data will be generated in wrong because of pattern matching way.
https://github.com/IntelRealSense/librealsense/tree/master/doc
The camera “sees” both IR and visible light and performs dense stereo matching based on features it can see. In visible wavelength, such features would be visible features, like corners, texture, edges, etc…
In infrared, it also can pick up on artificial features generated by the projector (in addition to natural features)
To improve the 3D point cloud modeling of complex indoor environment, a novel point cloud registration method based on IntelR RealSenseTM depth camera is proposed in this paper, which can reduce the influence of measuring errors. The IntelR RealSenseTM depth camera adopts active infrared (IR) stereo vision technology, which is shown in Figure 2. The depth perception based on stereo vision is implemented by two image sensors and an infrared projector. The infrared projector projects non-visible structured IR pattern to improve depth accuracy in scenes. The depth calculation process in presented in the right of Figure 2. The depth image processor obtains the scene data by the two image sensors, and the depth values for each pixel can be calculated by correlating the points on the left image to the right image.
[https://www.google.com/imgres?imgurl=https%3A%2F%2Fstatic-01.hindawi.com%2Farticles%2Fjs%2Fvolume-2014%2F852621%2Ffigures%2F852621.fig.001.jpg&imgrefurl=https%3A%2F%2Fwww.hindawi.com%2Fjournals%2Fjs%2F2014%2F852621%2F&tbnid=6JxHaQD2t-4Z7M&vet=12ahUKEwi_i7ScqMT2AhWRTPUHHbxmBFsQMygCegUIARC8AQ..i&docid=8-jTry2NDHZNM&w=600&h=360&q=structured IR pattern&hl=en&ved=2ahUKEwi_i7ScqMT2AhWRTPUHHbxmBFsQMygCegUIARC8AQ](https://www.google.com/imgres?imgurl=https%3A%2F%2Fstatic-01.hindawi.com%2Farticles%2Fjs%2Fvolume-2014%2F852621%2Ffigures%2F852621.fig.001.jpg&imgrefurl=https%3A%2F%2Fwww.hindawi.com%2Fjournals%2Fjs%2F2014%2F852621%2F&tbnid=6JxHaQD2t-4Z7M&vet=12ahUKEwi_i7ScqMT2AhWRTPUHHbxmBFsQMygCegUIARC8AQ..i&docid=8-jTry2NDHZNM&w=600&h=360&q=structured%20IR%20pattern&hl=en&ved=2ahUKEwi_i7ScqMT2AhWRTPUHHbxmBFsQMygCegUIARC8AQ)
