Selected Publications

1. 

   Memory-Augmented Representation for Efficient Event-based Visuomotor Policy Learning with Adaptive Perception and Control

   Uday Kamal, and Saibal Mukhopadhyay

   Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2026

   Abs HTML

   Event-based cameras are well-suited for fast and agile autonomous navigation due to their ultra-fast, microsecond level temporal resolution. However, fully leveraging this potential requires highly efficient processing algorithms capable of asynchronous, event-by-event representations and policy updates. Current methods employ synchronous dense representation or process events in a fixed-rate time windows, leading to inefficiencies via redundant computation. We address this by proposing an end-to-end framework for event-to-control policy learning designed for reactive navigation tasks. Our method consists of a memory-augmented perception module that updates the representation asynchronously and adaptively selects the number of events to process. Using the memory representation, a lightweight policy module is jointly optimized with the perception module, and learns to predict control commands at rates that dynamically adjust to scene complexity in an event-based reinforcement learning setting. Evaluations on simulated drone navigation tasks demonstrate higher sample efficiency and robustness compared to dense frame-based methods. Moreover, our approach significantly reduces computational complexity by minimizing processing steps and event counts while maintaining competitive performance against state-of-the-art event-based methods.

2. 

   Adaptive-Cloud: Dynamic Computation Control for 3D Object Detection From LIDAR Point Clouds

   Mir Sayeed Mohammad,  Uday Kamal, and Saibal Mukhopadhyay

   IEEE Robotics and Automation Letters 2025

   Abs HTML

   In this work, we introduce an adaptive hierarchical framework for efficient 3D object detection from point cloud data, designed to dynamically balance computational efficiency and detection performance. Our approach employs a shared feature extractor and multiple detector backbones of varying widths, enabling selective activation of models based on the complexity of the input scene. A novel feature gating mechanism dynamically determines the most relevant features for reduced-width backbones, while a surrogate loss prediction module ranks models in real-time, ensuring optimal backbone selection with minimal overhead. This adaptive strategy reduces compute costs by 41.4% while maintaining a negligible 2.44% reduction in detection accuracy across a range of real-world driving scenes (urban, highway, residential, campus, person) from the KITTI dataset. By addressing runtime adaptability—a critical gap in existing 3D detection frameworks—our method provides a significant algorithmic improvement for high-performance detection models in resource-constrained environments.

3. 

   ∇QDARTS: Quantization as an Elastic Dimension to Differentiable NAS

   Payman Behnam*,  Uday Kamal*, Sanjana Vijay Ganesh, Zhaoyi Li, Michael Andrew Jurado, Alind Khare, Igor Fedorov, Gaowen Liu, and Alexey Tumanov

   TMLR, 2025

   Abs HTML

   Differentiable Neural Architecture Search methods efficiently find high-accuracy architectures using gradient-based optimization in a continuous domain, saving computational resources. Mixed-precision search helps optimize precision within a fixed architecture. However, applying it to a NAS-generated network doesn’t assure optimal performance as the optimized quantized architecture may not emerge from a standalone NAS method. In light of these considerations, this paper introduces ∇QDARTS, a novel approach that combines differentiable NAS with mixed-precision search for both weight and activation. ∇QDARTS aims to identify the optimal mixed-precision neural architecture capable of achieving remarkable accuracy while operating with minimal computational requirements in a single shot, end-to-end differentiable framework obviating the need for pertaining and proxy. Compared to fp32, ∇QDARTS shows impressive performance on CIFAR10 with (2,4) bit precision, reducing bit operations by 160× with a slight 1.57% accuracy drop. Increasing the capacity enables ∇QDARTS to match fp32 accuracy while reducing bit operations by 18×. For the ImageNet dataset, with just (2,4) bit precision, ∇QDARTS outperforms state-of-the-art methods such as APQ, SPOS, OQA, and MNAS by 2.3%, 2.9%, 0.3%, and 2.7% in terms of accuracy. By incorporating (2,4,8) bit precision, ∇QDARTS further minimizes the accuracy drop to a 1% compared to fp32, alongside a substantial reduction of 17× in required bit operations and 2.6× in memory footprint. In terms of bit-operation (memory footprint) ∇QDARTS excels over APQ, SPOS, OQA, and MNAS with similar accuracy by 2.3× (12×), 2.4× (3×), 13% (6.2×), 3.4× (37%), for bit-operation (memory footprint), respectively. ∇QDARTS enhances the overall search and training efficiency, achieving a 3.1× and 1.54× improvement over APQ and OQA, respectively.

4. 

   Learning Collective Dynamics of Multi-Agent Systems using Event-based Vision

   Minah Lee,  Uday Kamal, and Saibal Mukhopadhyay

   L4DC, 2025

   Abs HTML

   This paper proposes a novel problem: vision-based perception to learn and predict the collective dynamics of multi-agent systems, specifically focusing on interaction strength and convergence time. Multi-agent systems are defined as collections of more than ten interacting agents that exhibit complex group behaviors. Unlike prior studies that assume knowledge of agent positions, we focus on deep learning models to directly predict collective dynamics from visual data, captured as frames or events. Due to the lack of relevant datasets, we create a simulated dataset using a state-of-the-art flocking simulator, coupled with a vision-to-event conversion framework. We empirically demonstrate the effectiveness of event-based representation over traditional frame-based methods in predicting these collective behaviors. Based on our analysis, we present event-based vision for Multi-Agent dynamic Prediction (evMAP), a deep learning architecture designed for real-time, accurate understanding of interaction strength and collective behavior emergence in multi-agent systems.

5. 

   Efficient Learning of Event-based Dense Representation using Hierarchical Memories with Adaptive Update

   Uday Kamal, and Saibal Mukhopadhyay

   ECCV, 2024

   Abs HTML

   Leveraging the high temporal resolution of an event-based camera requires highly efficient event-by-event processing. However, dense prediction tasks require explicit pixel-level association, which is challenging for event-based processing frameworks. Existing works aggregate the events into a static frame-like representation at the cost of a much slower processing rate and high compute cost. To address this challenge, this work introduces an event-based spatiotemporal representation learning framework for efficiently solving dense prediction tasks. We uniquely handle the sparse, asynchronous events using an unstructured, set-based approach and project them into a hierarchically organized multi-level latent memory space that preserves the pixel-level structure. Low-level event streams are dynamically encoded into these latent structures through an explicit attention-based spatial association. Unlike existing works that update these memory stacks at a fixed rate, we introduce a data-adaptive update rate that recurrently keeps track of the past memory states and learns to update the corresponding memory stacks only when it has substantial new information, thereby improving the overall compute latency. Our method consistently achieves competitive performance across different event-based dense prediction tasks while ensuring much lower latency compared to the existing methods.

6. 

   Associative Memory Augmented Asynchronous Spatiotemporal Representation Learning for Event-based Perception

   Uday Kamal*, Saurabh Dash*, and Saibal Mukhopadhyay

   ICLR, 2023 (notable-25%)

   Abs HTML

   We propose EventFormer, a computationally efficient event-based representation learning framework for asynchronously processing event camera data. EventFormer treats sparse input events as a spatially unordered set and models their spatial interactions using self-attention mechanism. An associative memory-augmented recurrent module is used to correlate with the stored representation computed from past events. A memory addressing mechanism is proposed to store and retrieve the latent states only where these events occur and update them only when they occur. The representation learning shift from input space to the latent memory space resulting in reduced computation cost for processing each event. We show that EventFormer achieves 0.5% and 9% better accuracy with 30000x and 200x less computation compared to the state-of-the-art dense and event-based method, respectively, on event-based object recognition datasets.

7. 

   Anatomy-xnet: An anatomy aware convolutional neural network for thoracic disease classification in chest x-rays

   Uday Kamal, Mohammad Zunaed, Nusrat Binta Nizam, and Taufiq Hasan

   IEEE Journal of Biomedical and Health Informatics (JBHI), 2022

   Abs HTML

   Thoracic disease detection from chest radiographs using deep learning methods has been an active area of research in the last decade. Most previous methods attempt to focus on the diseased organs of the image by identifying spatial regions responsible for significant contributions to the model’s prediction. In contrast, expert radiologists first locate the prominent anatomical structures before determining if those regions are anomalous. Therefore, integrating anatomical knowledge within deep learning models could bring substantial improvement in automatic disease classification. Motivated by this, we propose Anatomy-XNet, an anatomy-aware attention-based thoracic disease classification network that prioritizes the spatial features guided by the pre-identified anatomy regions. We adopt a semi-supervised learning method by utilizing available small-scale organ-level annotations to locate the anatomy regions in large-scale datasets where the organ-level annotations are absent. The proposed Anatomy-XNet uses the pre-trained DenseNet-121 as the backbone network with two corresponding structured modules, the Anatomy Aware Attention (A 3 ) and Probabilistic Weighted Average Pooling, in a cohesive framework for anatomical attention learning. We experimentally show that our proposed method sets a new state-of-the-art benchmark by achieving an AUC score of 85.78%, 92.07%, and, 84.04% on three publicly available large-scale CXR datasets–NIH, Stanford CheXpert, and MIMIC-CXR, respectively. This not only proves the efficacy of utilizing the anatomy segmentation knowledge to improve the thoracic disease classification but also demonstrates the generalizability of the proposed framework.

8. 

   DFR-TSD: A deep learning based framework for robust traffic sign detection under challenging weather conditions

   Sabbir Ahmed,  Uday Kamal, and Md Kamrul Hasan

   IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2021

   Abs HTML

   Robust traffic sign detection and recognition (TSDR) is of paramount importance for the successful realization of autonomous vehicle technology. The importance of this task has led to vast amount of research efforts and many promising methods have been proposed in the existing literature. However, most of these methods have been evaluated on clean and challenge-free datasets and overlooked the performance deterioration associated with different challenging conditions (CCs) that obscure the traffic-sign images captured in the wild. In this paper, we look at the TSDR problem under CCs and focus on the performance degradation associated with them. To this end, we propose a Convolutional Neural Network (CNN) based prior enhancement focused TSDR framework. Our modular approach consists of a CNN-based challenge classifier, Enhance-Net–an encoder-decoder CNN architecture for image enhancement, and two separate CNN architectures for sign-detection and classification. We propose a novel training pipeline for Enhance-Net that focuses on the enhancement of the traffic sign regions (instead of the whole image) in the challenging images subject to their accurate detection. We used CURE-TSD dataset consisting of traffic videos captured under different CCs to evaluate the efficacy of our approach. We experimentally show that our method obtains an overall precision and recall of 91.1% and 70.71% that is 7.58% and 35.90% improvement in precision and recall, respectively, compared to the current benchmark. Furthermore, we compare our approach with different CNN-based TSDR methods and show that our approach outperforms them by a large margin.

9. 

   DSWE-Net: A deep learning approach for shear wave elastography and lesion segmentation using single push acoustic radiation force

   Shahed Ahmed,  Uday Kamal, and Md Kamrul Hasan

   Ultrasonics, 2021

   Abs HTML

   Ultrasound-based non-invasive elasticity imaging modalities have received significant consideration for tissue characterization over the last few years. Though substantial advances have been made, the conventional Shear Wave Elastography (SWE) methods still suffer from poor image quality in regions far from the push location, particularly those which rely on single focused ultrasound push beam to generate shear waves. In this study, we propose DSWE-Net, a novel deep learning-based approach that is able to construct Young’s modulus maps from ultrasonically tracked tissue velocity data resulting from a single acoustic radiation force (ARF) push. The proposed network employs a 3D convolutional encoder, followed by a recurrent block consisting of several Convolutional Long Short-Term Memory (ConvLSTM) layers to extract high-level spatio-temporal features from different time-frames of the input velocity data. Finally, a pair of coupled 2D convolutional decoder blocks reconstructs the modulus image and additionally performs inclusion segmentation by generating a binary mask. We also propose a multi-task learning loss function for end-to-end training of the network with 1260 data samples obtained from a simulation environment which include both bi-level and multi-level phantom structures. The performance of the proposed network is evaluated on 140 synthetic test data and the results are compared both qualitatively and quantitatively with that of the current state of the art method, Local Phase Velocity Based Imaging (LPVI). With an average SSIM of 0.90, RMSE of 0.10 and 20.69 dB PSNR, DSWE-Net performs much better on the imaging task compared to LPVI. Our method also achieves an average IoU score of 0.81 for the segmentation task which makes it suitable for localizing inclusions as well. In this initial study, we also show that our method gains an overall improvement of 0.09 in SSIM, 4.81 dB in PSNR, 2.02 dB in CNR, and 0.09 in RMSE over LPVI on a completely unseen set of CIRS tissue mimicking phantom data. This proves its better generalization capability and shows its potential for use in real-world clinical practice.

10. 

    Lung cancer tumor region segmentation using recurrent 3d-denseunet

    Uday Kamal, Abdul Muntakim Rafi, Rakibul Hoque, Jonathan Wu, and Md Kamrul Hasan

    MICCAI Workshop, 2020

    Abs HTML

    The performance of a computer-aided automated diagnosis system of lung cancer from Computed Tomography (CT) volumetric images greatly depends on the accurate detection and segmentation of tumor regions. In this paper, we present Recurrent 3D-DenseUNet, a novel deep learning based architecture for volumetric lung tumor segmentation from CT scans. The proposed architecture consists of a 3D encoder block that learns to extract fine-grained spatial and coarse-grained temporal features, a recurrent block of multiple Convolutional Long Short-Term Memory (ConvLSTM) layers to extract fine-grained spatio-temporal information, and finally a 3D decoder block to reconstruct the desired volume segmentation masks from the latent feature space. The encoder and decoder blocks consist of several 3D-convolutional layers that are densely connected among themselves so that necessary feature aggregation can occur throughout the network. During prediction, we apply selective thresholding followed by morphological operation, on top of the network prediction, to better differentiate between tumorous and non-tumorous image-slices, which shows more promise than only thresholding-based approaches. We train and test our network on the NSCLC-Radiomics dataset of 300 patients, provided by The Cancer Imaging Archive (TCIA) for the 2018 IEEE VIP Cup. Moreover, we perform an extensive ablation study of different loss functions in practice for this task. The proposed network outperforms other state-of-the-art 3D segmentation architectures with an average dice score of 0.7228.

11. 

    Application of DenseNet in Camera Model Identification and Post-processing Detection.

    Abdul Muntakim Rafi,  Uday Kamal, Rakibul Hoque, Abid Abrar, Sowmitra Das, Robert Laganiere, Md Kamrul Hasan, and others

    CVPR workshop, 2019

    Abs HTML

    Camera model identification has earned paramount importance in the field of image forensics with an upsurge of digitally altered images which are constantly being shared through websites, media, and social applications. But, the task of identification becomes quite challenging if metadata are absent from the image and/or if the image has been post-processed. In this paper, we present a DenseNet pipeline to solve the problem of identifying the source camera-model of an image. Our approach is to extract patches of 256 x 256 from a labeled image dataset and apply augmentations, i.e., Empirical Mode Decomposition (EMD). We use this extended dataset to train a Neural Network with the DenseNet-201 architecture. We concatenate the output features for 3 different sizes (64x64, 128x128, 256x256) and pass them to a secondary network to make the final prediction. This strategy proves to be very robust for identifying the source camera model, even when the original image is post-processed. Our model has been trained and tested on the Forensic Camera-Model Identification Dataset provided for the IEEE Signal Processing (SP) Cup 2018. During testing we achieved an overall accuracy of 98.37%, which is the current state-of-the-art on this dataset using a single model. We used transfer learning and tested our model on the Dresden Database for Camera Model Identification, with an overall test accuracy of over 99% for 19 models. In addition, we demonstrate that the proposed pipeline is suit- able for other image-forensic classification tasks, such as, detecting the type of post-processing applied to an image with an accuracy of 96.66% - which indicates the generality of our approach.

12. 

    Automatic traffic sign detection and recognition using SegU-Net and a modified Tversky loss function with L1-constraint

    Uday Kamal, Thamidul Islam Tonmoy, Sowmitra Das, and Md Kamrul Hasan

    IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2019

    Abs HTML

    Traffic sign detection is a central part of autonomous vehicle technology. Recent advances in deep learning algorithms have motivated researchers to use neural networks to perform this task. In this paper, we look at traffic sign detection as an image segmentation problem and propose a deep convolutional neural network-based approach to solve it. To this end, we propose a new network, the SegU-Net, which we form by merging the state-of-the-art segmentation architectures-SegNet and U-Net to detect traffic signs from video sequences. For training the network, we use the Tversky loss function constrained by an L1 term instead of the intersection over union loss traditionally used to train segmentation networks. We use a separate network, inspired by the VGG-16 architecture, to classify the detected signs. The networks are trained on the challenge free sequences of the CURE-TSD dataset. Our proposed network outperforms the state-of-the-art object detection networks, such as the Faster R-CNN inception Resnet V2 and R-FCN Resnet 101, by a large margin and obtains a precision and recall of 94.60% and 80.21%, respectively, which is the current state of the art on this part of the dataset. In addition, the network is tested on the German Traffic Sign Detection Benchmark (GTSDB) dataset, where it achieves a precision and recall of 95.29% and 89.01%, respectively. This is on a par with the performance of the aforementioned object detection networks. These results prove the generalizability of the proposed architecture and its suitability for robust traffic sign detection in autonomous vehicles.