A Novel Tracking Algorithm Using Thermal and Optical Cameras Fused with mmWave Radar Sensor Data

Tracking of moving objects in pan-tilt-zoom (PTZ) cameras is a critical feature for automated surveillance applications. The PTZ auto tracking function enables the camera to control its electric motor mechanisms to adjust the course of the optical sensor field of view (FOV) to track moving objects automatically. In this paper, a proposed novel object tracking method based on the fusion of thermal, optical sensor and millimeter-wave data is implemented. The implementation of the proposed method is performed on a pan-tilt setup, which ultimately tracks the object of interest based on thermal data while keeping the optical image resolution the same as the thermal image resolution and provides valuable distance measurements to the target based on the millimeter-wave radar readings. The proposed method offers improvements over conventional optical sensor tracking by adding an extra layer to object tracking through additional thermal and millimeter-wave sensor data, which in turn provides a surplus of information of the object of interest that is being tracked. The proposed method compares favorably with commercially available surveillance systems.

© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.



Work Title A Novel Tracking Algorithm Using Thermal and Optical Cameras Fused with mmWave Radar Sensor Data
Open Access
  1. Bogdan Iepure
  2. Aldo W. Morales
  1. Image Fusion
  2. Wavelets
  3. Segmentation
  4. Millimeter-Wave Image
  5. Object Tracking
License In Copyright (Rights Reserved)
Work Type Article
  1. IEEE Transactions on Consumer Electronics
Publication Date November 1, 2021
Publisher Identifier (DOI)
  1. https://doi.org/10.1109/TCE.2021.3128825
Deposited February 17, 2023




This resource is currently not in any collection.

Work History

Version 1

  • Created
  • Added IEEE_Transactions_Novel_Tracking_Algorithm.pdf
  • Added Creator Bogdan Iepure
  • Added Creator Aldo W. Morales
  • Published
  • Updated Keyword Show Changes
    • Image Fusion, Wavelets, Segmentation, Millimeter-Wave Image, Object Tracking
  • Updated