Enhancing Image Resolution With Super-Resolution Convolutional Neural Network (Srcnn)

Introduction

In the age of digital imaging, one problem that commonly arises is low-resolution images with limited details and artifacts, making them inadequate for various purposes. Super-Resolution Convolutional Neural Network (SRCNN) is a deep learning technique that helps to overcome this issue by enhancing the resolution of images using convolutional neural networks (CNN). In this blog post, we will discuss how SRCNN works and implement it using Python and the TensorFlow library.

SRCNN Architecture

SRCNN is a deep learning model that consists of three convolutional layers. The main idea behind SRCNN is to learn an end-to-end mapping from low-resolution inputs to high-resolution outputs using a CNN. The architecture can be summarized as follows:

1. Patch extraction and representation: The first convolutional layer (with filter size f1, n1 feature maps, and an activation function) extracts small patches from the input low-resolution image and represents them as high-dimensional vectors.
2. Non-linear mapping: The second convolutional layer (with filter size f2, n2 feature maps, and an activation function) non-linearly maps the high-dimensional vectors to another set of high-dimensional vectors.
3. Reconstruction: The third convolutional layer (with filter size f3, only one feature map and a linear activation function) aggregates the high-dimensional vectors to reconstruct the high-resolution output image.

Implementation

First, let's import the necessary libraries:

import numpy as np
import cv2
import tensorflow as tf
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.models import Sequential

Next, let's define the SRCNN model using the following code:

def SRCNN_model():
    model = Sequential()
    
    # Adding the first convolutional layer
    model.add(Conv2D(filters=64, kernel_size=(9, 9), activation='relu', padding='valid', input_shape=(None, None, 1)))
    
    # Adding the second convolutional layer
    model.add(Conv2D(filters=32, kernel_size=(1, 1), activation='relu', padding='valid'))
    
    # Adding the third convolutional layer
    model.add(Conv2D(filters=1, kernel_size=(5, 5), activation='linear', padding='valid'))
    
    return model

Train the SRCNN model on a dataset containing low-resolution and corresponding high-resolution images. Here, we assume that the training data is already available.

After training the model, we can apply it to a low-resolution image to enhance its resolution. Here is a function to apply the SRCNN on an input image:

def apply_SRCNN(srcnn, input_image, upscale_factor):
    height, width = input_image.shape
    new_height, new_width = height * upscale_factor, width * upscale_factor
    
    # Resize the input image using bicubic interpolation
    input_image_resized = cv2.resize(input_image, (new_width, new_height), interpolation=cv2.INTER_CUBIC)

    # Normalize the resized image
    input_image_normalized = input_image_resized.astype(np.float32) / 255.

    # Convert to 4D tensor (1, height, width, 1) as required for the model input
    input_tensor = np.expand_dims(np.expand_dims(input_image_normalized, axis=-1), axis=0)
    
    # Perform the super-resolution using the trained SRCNN model
    output_tensor = srcnn.predict(input_tensor)
    
    # Get the 2D output image and denormalize it
    output_image = np.squeeze(output_tensor) * 255.

    return output_image

Now, we can load our trained SRCNN model and enhance an input low-resolution image as follows:

srcnn = SRCNN_model()
srcnn.load_weights('trained_SRCNN_weights.h5')

input_image = cv2.imread('low_res_image.png', cv2.IMREAD_GRAYSCALE)

upscale_factor = 2
output_image = apply_SRCNN(srcnn, input_image, upscale_factor)

cv2.imwrite('output_high_res_image.png', output_image)

In this blog post, we have discussed the Super-Resolution Convolutional Neural Network (SRCNN) and its implementation using Python and TensorFlow. SRCNN is an effective tool for enhancing the resolution of images, which can be applied to various fields, such as image restoration, virtual reality, and remote sensing.