ABSTRACT
This research investigated image recognition frameworks on datasets of visible-light and infrared (IR) imagery using deep convolutional neural networks (CNN). This is due to their recent success on a variety of problems including computer vision which often surpassed the state of the art methods. Three deep learning based object recognition approaches were investigated on a fused version of the images in order to exploit the synergistic integration of the information obtained from varying spectra of same data with a view to improving the overall classification accuracy. Firstly, a simple 3-layer experimental deep network was designed and used to train the datasets for performing recognition. A second experiment was conducted where a pre-trained 16-layer convolutional neural network (Imagenet-vgg-verydeep-16) was used to extract features from the datasets. These features are then used to train a logistic regression classifier for performing the recognition. Finally, an experiment was conducted where another pre-trained model (Imagenet-vgg-f) was fine-tuned to suit the dataset's classes and which is then retrained accordingly on the datasets using back propagation. This research adopted a simple and novel fusion strategy where the IR and visible images were fused by concatenating the IR image as an additional fourth layer to the visible image with a view to enhancing the performance of object recognition systems. Despite its simplicity and the limited size of the training data, the 3-layer network achieved a classification accuracies of 86.8590% on the fused multimodal image, 85.0610% on the visible images and 67.9878% on the Infrared images. These results represent a respective improvement by 4.76%, 3.16% and 13.99% when compared withthose of the CNN architectureofZhang. Also, an improvement by 4.06% on visible images was obtained over that of Zhang'sGnostic Field + CNN model. It is also observed that while the pre-trained model performed well, fine-tuning improved the performance to 100% classification accuracy. Results obtained were compared with those ofZhangas a means of validation. This work was implemented using MATLAB programming language, MatConvnet library for CNN and Liblinear library for large linear classification.