Project WOMBAT (Working On Machine Based Autonomous Transport) is a state of the art self-driving platform. Based off of a kid’s ride-on toy Mini Cooper, Project WOMBAT can detect 90 different classes of objects and avoid them in real time. Built with the latest camera and machine learning technologies, Project WOMBAT pushes the boundaries of what’s possible in a small package.

The car contains two advanced cameras to understand its environment and location. The first camera uses stereo technology to produce a detailed depth map of the image, and I developed an algorithm to extract objects in the car’s path from the point cloud. The second camera is dedicated to tracking the precise position and orientation of the car.

While the point cloud algorithm works, machine learning provides the car with a much better understanding of its environment. An object detection model locates and classifies obstacles in the car’s field of view, and my algorithm decides if and how to avoid them. In order to achieve real time performance, I researched and tested a myriad of ways to run machine learning models locally. For the best performance and low power consumption, I integrated a Google Edge TPU (tensor processing unit) to detect objects in real time without any internet connection required.

The car itself required extensive modification to accommodate such advanced technology. I developed a custom motor control system with precise control of speed and steering feedback. Custom designed and 3D printed parts seamlessly integrate the modifications into the car.

What inspired you (or your team)?

I am passionate about combining the latest technologies with diligent design to create experiences and products that people love. As machine learning (ML) technology develops and advanced models can be run on smartphones in your pocket, the applications for machine learning have proliferated from keyboard text prediction to searching your photo library by keywords. Simultaneously, autonomous vehicle technology has entered production in cars on the road today, from automatic safety features to Waymo’s fully autonomous fleet.

I am self taught in both hardware and software development, and I jumped at the chance to learn about machine learning through building my own autonomous vehicle. The open source tools and models Google AI has released, including the machine learning framework TensorFlow, showed me that it was possible to create and use powerful models without massive computing resources or millions of data points. As I worked with Google tools, I connected with several members of the TensorFlow team online. This May, I was fortunate to be able to attend Google I/O 19, where I experienced hands-on the amazing work Google and community members are doing to solve problems with machine learning. Meeting with the engineers who make the ML tools I use, I learned about the latest Google technologies that I later integrated into Project WOMBAT, and having the support of the TensorFlow team as we worked out unique challenges with my use case of their product was helpful.

As a lifelong maker who has participated in many Maker Faires, SxSW, and Raspberry Pi Foundation events, I have developed a wide array of hardware development skills. I enjoy developing the whole system of a product including electronics, CAD design, and 3D printing parts, which all came together with advanced machine learning software to create Project WOMBAT. The seamless intersection of hardware and software with intelligent design makes Project WOMBAT such a special innovation.