OpenCV Image Stitching

Image stitching is an important application in computer vision that can stitch multiple images with overlapping regions into a larger single image.

Common application scenarios include panoramic image generation, satellite image stitching, etc.

OpenCV is a powerful computer vision library that provides rich tools to implement image stitching.

This article will detail how to use OpenCV for image stitching, focusing on feature point detection and matching techniques.

\n\n

Application Scenarios

Panoramic image generation: Stitch multiple images into a single panoramic image.
Map stitching: Stitch multiple map images into a larger map.
Medical image processing: Stitch multiple medical images into a complete image.

\n\n

Basic Workflow of Image Stitching

The basic workflow of image stitching can be divided into the following steps:

Image Reading: Read the images to be stitched.
Feature Point Detection: Detect key points (feature points) in each image.
Feature Point Matching: Match these feature points across different images.
Calculating Transformation Matrix: Calculate the transformation matrix between images based on matched feature points.
Image Blending: Stitch images according to the transformation matrix and perform blending to eliminate stitching artifacts.

Next, we will explain the implementation of each step in detail.

\n\n

1. Image Reading

First, we need to read the images to be stitched. OpenCV provides the cv2.imread() function to read images.

\n\n

Example

import cv2\n\n# Read image\n\n image1 = cv2.imread('image1.jpg')\n\n image2 = cv2.imread('image2.jpg')\n\n# Check if image is successfully read\n\nif image1 is None or image2 is None:\n\nprint("Error: Unable to Read image")\n\n exit()

\n\n

2. Feature Point Detection

Feature point detection is a crucial step in image stitching. OpenCV provides various feature detection algorithms, such as SIFT, SURF, ORB, etc. Here, we will use SIFT as an example.

\n\n

Example

# Create SIFT detector\n\n sift = cv2.SIFT_create()\n\n# Detect feature points and descriptors\n\n keypoints1, descriptors1 = sift.detectAndCompute(image1,None)\n\n keypoints2, descriptors2 = sift.detectAndCompute(image2,None)

The detectAndCompute() function returns two values: keypoints and descriptors. Keypoints are prominent points in the image, and descriptors describe these keypoints for subsequent matching.

\n\n

3. Feature Point Matching

After detecting feature points, we need to match these points across different images. OpenCV provides BFMatcher or FlannBasedMatcher for feature point matching.

\n\n

Example

# Create BFMatcher object\n\n bf = cv2.BFMatcher()\n\n# Use KNN matching\n\n matches = bf.knnMatch(descriptors1, descriptors2, k=2)\n\n# Apply ratio test to filter good matches\n\n good_matches =[]\n\nfor m, n in matches:\n\nif m.distance<0.75 * n.distance:\n\n good_matches.append(m)

The knnMatch() function returns the two best matches for each feature point. We filter out good matches using a ratio test (Lowe's ratio test).

\n\n

4. Calculating Transformation Matrix

After obtaining good matches, we can use these points to calculate the transformation matrix between images. A commonly used transformation matrix is the Homography matrix, which maps points from one image to another.

\n\n

Example

# Extract coordinates of matching points\n\n src_pts = np.float32([keypoints1[m.queryIdx].pt for m in good_matches]).reshape(-1,1,2)\n\n dst_pts = np.float32([keypoints2[m.trainIdx].pt for m in good_matches]).reshape(-1,1,2)\n\n# Calculate homography matrix\n\n H, _ = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC,5.0)

The findHomography() function returns a 3x3 homography matrix H, which maps points from image1 to image2.

\n\n

5. Image Fusion

Finally, we use the calculated homography matrix to stitch the images together and perform blending to eliminate stitching artifacts.

\n\n

Example

# Get image size\n\n h1, w1 = image1.shape[:2]\n\n h2, w2 = image2.shape[:2]\n\n# Calculate the size of the stitched image\n\n pts = np.float32([[0,0],[0, h1],[w1, h1],[w1,0]]).reshape(-1,1,2)\n\n dst = cv2.perspectiveTransform(pts, H)\n\n[x_min, y_min]= np.int32(dst.min(axis=0).ravel() - 0.5)\n\n[x_max, y_max]= np.int32(dst.max(axis=0).ravel() + 0.5)\n\n# Calculate translation matrix\n\n translation_matrix = np.array([[1,0, -x_min],[0,1, -y_min],[0,0,1]])\n\n# Apply translation matrix for image stitching\n\n result = cv2.warpPerspective(image1, translation_matrix.dot(H),(x_max - x_min, y_max - y_min))\n\n result[-y_min:h2 - y_min, -x_min:w2 - x_min]= image2\n\n# Display stitched result\n\n cv2.imshow('Result', result)\n\n cv2.waitKey(0)\n\n cv2.destroyAllWindows()

The warpPerspective() function performs a perspective transformation on image1 based on the homography matrix H and stitches it with image2.

\n\n

Implementation

Below is the complete code for image stitching using feature point detection and matching:

\n\n

Example

import cv2\n\nimport numpy as np\n\n# 1. Load image\n\n image1 = cv2.imread("path/to/image1.jpg")\n\n image2 = cv2.imread("path/to/image2.jpg")\n\n# 2. Convert to grayscale image\n\n gray1 = cv2.cvtColor(image1, cv2.COLOR_BGR2GRAY)\n\n gray2 = cv2.cvtColor(image2, cv2.COLOR_BGR2GRAY)\n\n# 3. Feature point detection\n\n sift = cv2.SIFT_create()\n\n keypoints1, descriptors1 = sift.detectAndCompute(gray1,None)\n\n keypoints2, descriptors2 = sift.detectAndCompute(gray2,None)\n\n# 4. Feature point matching\n\n matcher = cv2.BFMatcher()\n\n matches = matcher.knnMatch(descriptors1, descriptors2, k=2)\n\n# 5. Filter matching points\n\n good_matches =[]\n\nfor m, n in matches:\n\nif m.distance<0.75 * n.distance:\n\n good_matches.append(m)\n\n# 6. Calculate homography matrix\n\nif len(good_matches)>10:\n\n src_pts = np.float32([keypoints1[m.queryIdx].pt for m in good_matches]).reshape(-1,1,2)\n\n dst_pts = np.float32([keypoints2[m.trainIdx].pt for m in good_matches]).reshape(-1,1,2)\n\n H, _ = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC,5.0)\n\nelse:\n\nprint("Not enough matches found.")\n\n exit()\n\n# 7. Image transformation\n\n height1, width1 = image1.shape[:2]\n\n height2, width2 = image2.shape[:2]\n\n warped_image = cv2.warpPerspective(image1, H,(width1 + width2, height1))\n\n# 8. Image stitching\n\n warped_image[0:height2,0:width2]= image2\n\n# 9. Display results\n\n cv2.imshow("Stitched Image", warped_image)\n\n cv2.waitKey(0)\n\n cv2.destroyAllWindows()

YouTip