( Write my notes about finetuning using hugging face transformers)
Work
LIP-Loc: LiDAR Image Pretraining for Cross-Modal Localization
We apply CLIP-like contrastive symmetric loss using batch construction technique to the domains of 2D image and 3D LiDAR points on the task of cross-modal localization. WACV 2024 Workshop on Large Language and Vision Models for Autonomous Driving Training using Pytorch, Github Link
Nano GPT
Implemented a Decoder-only Transformer (GPT) from scratch using PyTorch. Followed @karpathy’s video lecture Github Link
Deep Reinforcement Learning for Autonomous Underwater Vehicles
Trained a deep reinforcement agent to control an AUV. Used the Proximal Policy Optimization algorithm for learning. Agent can orient and reach a small red buoy successfully. Simulated and trained the model in Unity Game Engine. Major Project of Bachelors degree Simulation and training using Unity3D ml-agents
Camera Tuning AI trained using Deep Reinforcement learning
Trained a deep reinforcement agent to tune the white- balance setting of an Android Camera. The agent attempts to replicate a manually adjusted photo. Captured photos are sent to the PC for training and inference. Minor Project for Bachelors degree. Training using Keras, image sent over sockets
Augmented Reality Events App
Designed a system for rendering AR elements on stages for concerts and festivals. Won first place of the Hackathon with prizes worth ₹ 15,000. Was team lead Tech Stack: Unity, ARCore Github link
Ludum jam 46, La Revolution
Made a game in 3 days which features a gun to make people join the revolution. Made in a top down style. Play here
Shopify Extension to Measure Clothes measurement
Uses tensorflow.js to approximate body measurements using the webcam. Can be used as an iframe in a shopify store. Currently unsupported Check it out here
About
Software Engineer 2 at Microsoft (Microsoft Todo Backend). Processing PBs of User logs using spark for Business metrics. Built REST APIs, messaging systems to enhance shared task lists across distributed instances, using ASP.NET
Am learning about VLMs, contributing to HuggingFace LeRobot.
Was a hobbyist Game developer, huge fan of single player experiences. Experienced in Unity3D. Dabbled in Artificial neural networks and Deep Reinforcement Learning. Used Pytorch, Unity ml-agents.
Made Augmented Reality apps using ArCore in Unity. Deeply intereseted in Mixed Reality, bullish on its ability to replace the smartphone.
Bullish and interested in projects on
Learning based approaches for automation. robotics
Creator Economy, power to the masses!
Shared mirror worlds: A Information rich, virtual simiulation of the world
Metaverse interactions, tackling real-time synchronous internet