深度学习图像配准 Image Registration: From SIFT to Deep Learning


What is Image Registration?

深度学习图像配准  Image Registration: From SIFT to Deep Learning 深度学习图像配准  Image Registration: From SIFT to Deep Learning

Traditional Feature-based Approaches

深度学习图像配准  Image Registration: From SIFT to Deep Learning Keypoint Detection and Feature Description
import numpy as np
import cv2 as cv

img = cv.imread('image.jpg')
gray= cv.cvtColor(img, cv.COLOR_BGR2GRAY)

akaze = cv.AKAZE_create()
kp, descriptor = akaze.detectAndCompute(gray, None)

img=cv.drawKeypoints(gray, kp, img)
cv.imwrite('keypoints.jpg', img)

深度学习图像配准  Image Registration: From SIFT to Deep Learning

For more details on feature detection and description, you can check out this OpenCV tutorial.

Feature Matching

import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt

img1 = cv.imread('image1.jpg', cv.IMREAD_GRAYSCALE)  # referenceImage
img2 = cv.imread('image2.jpg', cv.IMREAD_GRAYSCALE)  # sensedImage

# Initiate AKAZE detector
akaze = cv.AKAZE_create()
# Find the keypoints and descriptors with SIFT
kp1, des1 = akaze.detectAndCompute(img1, None)
kp2, des2 = akaze.detectAndCompute(img2, None)

# BFMatcher with default params
bf = cv.BFMatcher()
matches = bf.knnMatch(des1, des2, k=2)

# Apply ratio test
good_matches = []
for m,n in matches:
    if m.distance < 0.75*n.distance:
        good_matches.append([m])
        
# Draw matches
img3 = cv.drawMatchesKnn(img1,kp1,img2,kp2,good_matches,None,flags=cv.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)
cv.imwrite('matches.jpg', img3)

深度学习图像配准  Image Registration: From SIFT to Deep Learning

Image Warping

# Select good matched keypoints
ref_matched_kpts = np.float32([kp1[m[0].queryIdx].pt for m in good_matches]).reshape(-1,1,2)
sensed_matched_kpts = np.float32([kp2[m[0].trainIdx].pt for m in good_matches]).reshape(-1,1,2)

# Compute homography
H, status = cv.findHomography(ref_matched_kpts, sensed_matched_kpts, cv.RANSAC,5.0)

# Warp image
warped_image = cv.warpPerspective(img1, H, (img1.shape[1]+img2.shape[1], img1.shape[0]))
            
cv.imwrite('warped.jpg', warped_image)

深度学习图像配准  Image Registration: From SIFT to Deep Learning


Deep Learning Approaches

Feature Extraction

Homography Learning

深度学习图像配准  Image Registration: From SIFT to Deep Learning 深度学习图像配准  Image Registration: From SIFT to Deep Learning 深度学习图像配准  Image Registration: From SIFT to Deep Learning Their approach introduces two new network structures: a Tensor Direct Linear Transform and a Spatial Transformation Layer. We will not go into the details of these components here, we can simply consider that these are used to obtain a transformed sensed image using the homography parameter outputs of the CNN model, that we then use to compute the photometric loss. 深度学习图像配准  Image Registration: From SIFT to Deep Learning

Other Approaches

深度学习图像配准  Image Registration: From SIFT to Deep Learning
深度学习图像配准  Image Registration: From SIFT to Deep Learning

 
来源: https://blog.sicara.com/image-registration-sift-deep-learning-3c794d794b7a  

 

上一篇:Tesla Model汽车架构与FSD供应链


下一篇:记录下实测Tesla T4 ethminer算力