Overview
The FPV Head-Tracking Gimbal is a system that enhances user immersion and control in FPV drone and robotics operation through real-time head tracking. The project integrates an IMU, two servos, and ESP32/ESP32CAM microcontrollers to synchronize the gimballed camera movement with the user's head orientation. It consists of a transmitting system and a receiving system. The transmitting system consists of an IMU embedded into "goggles" to simulate an FPV headset, and a receiving system, featuring an ESPCAM module on a 2-axis gimbal. Additionally, the system supports a secondary mode for image relay via UART and ESP-NOW, expanding its functionality.
MY ROLE
I led the design, implementation, and testing of the head-tracking and image relay systems, ensuring real-time responsiveness, modularity, and robustness, alongside my partner Mauricio Vergara for ME 100 (Internet of Things). This project is an emphasis of my abilities to integrate hardware and software for dynamic, low-latency applications in robotics, despite requirements to use MicroPython.
FR1: Synchronize drone camera movement with head orientation in real-time.
FR2: Ensure low-latency communication between components for precise control.
FR3: Support UART-to-ESP-NOW image relay for multimedia applications.
FR4: Use Micropython, at least two ESPs, two sensors, and two actuators.
FR5: Enable future seamless integration with existing FPV goggles and systems.
Some key challenges were adapting ESPCAM drivers and firmware into MicroPython, encoding image data into a structure more easily transmissible in Python, and real-time head-tracking angle estimation using integrated gyroscope measurements and accelerometer angles on our IMU (the driver for which we programmed ourselves).
Due to limitations with MicroPython latency and ESPNow, the project was split into two methods: an FPV stream hosted locally by the ESPCAM, and a remote picture taker that encoded the pictures through ESPNow to send back to the transmitting system. See both methodologies below.
Videos May Take Time to Load
Method 1
Method 2