Projects

6D Pose Estimation using Iterative Closest Point (ICP) and DenseFusion

Duration: Oct 2023 - Dec 2023 | 3 mos

Outcomes

• Developed a 6D pose estimation system combining DeepLabv3 for segmentation and Dense Fusion for pose refinement, achieving high accuracy in synthetic scenes with a dataset of 79 object classes.
• Conducted extensive training and ablation studies on DeepLabv3 and Dense Fusion, optimizing for accuracy and efficiency; successfully integrated ICP with initial Dense Fusion weights for enhanced pose estimation refinement.

Counterfactual Image Generation using Latent Transformations

Duration: Apr 2023 - Jun 2023 | 3 mos

Outcomes

• Developed a pipeline for counterfactual image generation using GANs and neural networks, enabling the creation of visually similar alternative reality images with desired attribute changes for improved image editing and explainable AI.
• Implemented a multi-step process utilizing StyleGAN(trained on CelebA) for random image sampling, attribute classification, and shift prediction to generate high-quality counterfactual images with specified target attributes.

Instance Eraser

Duration: Apr 2023 - Jun 2023 | 3 mos

Outcomes

• Developed an algorithm inspired by Google's MagicEraser to remove specified objects from images. Utilized instance segmentation and Pix2Pix GAN model for image reconstruction, achieving contextually relevant background recovery.
• Implemented a user-friendly interface for object removal. Experimented with Mask R-CNN and generative models. While not state of the art, achieved decent results and demonstrated potential for further improvement through experimentation.

Hand and Object Segmentation using DeepLabv3

Duration: Apr 2023 - Jun 2023 | 3 mos

Outcomes

• Developed a hand and object segmentation model for 3D hand pose estimation tasks . Merged hand pose estimation data with segmentation datasets (FreiHand, HO3D, EgoHOS, EgoHands) to create a diverse training dataset.
• Implemented the state-of-the-art DeepLabv3 model with a ResNet backbone and leveraged PyTorch's Distributed Data Parallel for scalability, training on multiple GPUs for over 100,000 training examples. Successfully fine-tuned the model to achieve robust hand and object segmentation, enabling automatic ground truth annotation for 3D hand pose estimation tasks using the HOnnotate method.

Data Anaylsis and Website for Smart Building Research

Duration: Dec 2022 - Mar 2023 | 4 mos

Outcomes

• Developed a website for Smart Building research, featuring a React.js-based frontend seamlessly connected to a Flask API-based backend, with efficient PostgreSQL data storage enhanced by automated table partitioning based on sensor IDs and indexing strategies, ensuring fast data access and retrieval.
• Conducted comprehensive data analysis on sensor data, utilizing data visualization techniques and implemented a basic physics-based neural network computational graph to predict future features, serving as a baseline for subsequent experiments

Image Segmentation using Statistical Learning Methods

Duration: Sept 2022 - Nov 2022 | 3 mos

Outcomes

• Developed and implemented statistical learning algorithms in Matlab, including Naive Bayes, Maximum Likelihood Estimation, Bayes Parameter Estimation, and Expectation Maximization, for accurately segmenting a cheetah image into foreground (cheetah) and background (grass) components.
• Evaluated the performance of each algorithm using the probability of error metric, showcasing the effectiveness of more advanced methods like Expectation Maximization in achieving superior segmentation results.

Multiple Stream Vehicle Detection and Tracking

Duration: Aug 2022 - Oct 2022 | 3 mos

Outcomes

• Developed a GPU-based edge device vehicle detection and tracking system using YOLOv5 and DEEPSORT with the ability to switch between models based on accuracy and live stream requirements.
• Integrated a socket server to broadcast real-time detection data to authenticated clients and deployed a model capable of processing four live streams on a Jetson board with SORT algorithm for tracking.

Machine Learning-Based Error Control Code Design for Wireless Channel

Mentor: Dr. Lakshmi Narasimhan, IIT Palakkad

Duration: Dec 2020 - May 2021 | 5 mos

Outcomes

• Worked on developing error control codes for both block and convolutional codes using end to end deep learning networks on a wireless channel i.e., in both AWGN and Rayleigh channels.
• Demonstrated the ability of neural networks to generate error control codes whose performance was on par with the theoretical error control codes. Training is performed using synthetically curated datasets and TensorFlow2 is used for developing and training the model.

Video super-resolution for online video conferencing over low bandwidth network

Mentor: Dr. Lakshmi Narasimhan, IIT Palakkad

Duration: Aug 2020 - Nov 2020 | 4 mos

Outcomes

• Worked on developing a Deep-Learning video super-resolution model that can be used in real-time application on a video conferencing platform.
• Employed an Auto-encoder model that was able to convert 144p to 480p resolution with good accuracy. The whole framework is implemented using PyTorch on a GPU. Datasets are gathered using an automated system from YouTube using Python and Google API services.

Software Defined Radio for Visible Light Communication

Mentor: Dr. Lakshmi Narasimhan, IIT Palakkad

Duration: Jan 2020 - Mar 2020 | 3 mos

Outcomes

• Developed an Open SDR to aid the research in visible light communication. The open source SDR invites people to experiment with the setup. We developed software using Python Tkinter and matplotlib to make a GUI for SDR and implemented a hardware model using LEDs and phototransistors.
• We were able to transfer data but with very low bit rates. Hence, we designed a more efficient model to increase the bit rates but couldn’t implement it because of Covid-19 restrictions

Raspberry Pi keyboard as Bluetooth joystick for Toycar

Mentor: Dr. G V V Sharma, IIT Hyderabad

Duration: May 2019 - Jun 2019 | 2 mos

Outcomes

• Making of a toycar using a toycar kit and arduino and then controlling the toycar with a joystick made with raspberry pi and a keyboard attached with it.
• Data is transmitted from bluetooth raspberry pi and recieved using a bluetooth module attached to arduino and code for serial transmission.

OpenCV based Face Recognition System

Mentor: Dr. Jobin Francis, IIT Palakkad

Duration: Mar 2019 - Apr 2019 | 2 mos

Outcomes

• Built an OpenCV based face detection system that acquires a video and automatically recognizes faces.
• The developed code uses Python OpenCV module and efficiently trained Caffe models to detect the features of a face and recognize them in the frame with 70% probability

Broken Wire Detector - PCB

Mentor: Dr. Revathy Padmanabam, IIT Palakkad

Duration: Jan 2019 - Apr 2019 | 4 mos

Outcomes

• Designed and fabricated a PCB circuit using traditional techniques such as manual routing, toner transfer, etching and soldering
• The developed PCB circuit detects a broken or faulty wire using the EMF generated by an alternating current without physically disturbing wires.