The AI backpack is created by an Indian computer vision researcher named Jagdish Mahendran. He started working on the backpack back in 2013 after he saw the daily navigation struggles of his friend who is blind. In 2020, Mahendran won an Intel AI competition for his idea. Initially, Mahendran explored the existing technologies that were developed to help blind people navigate. Later, he came up with the backpack idea that uses cameras and AI sensors to analyze an environment and give instant feedback to the users.
How Does It Work?
So, the way it works is pretty simple. The user, other than the backpack, has to wear a Bluetooth-enabled headset, a vest with the necessary sensors, and a fanny pack to store the battery and other equipment. The vest contains hidden Intel sensors and a front-facing camera to capture the surroundings.
The system is voice-based. So, when the user activates it by saying “Start”, it starts analyzing the surroundings and gives instant feedback to the user via the Bluetooth headset. Mahendran says that he primarily worked to shorten the lag time between the processing and the feedback delivery as delays can be often dangerous for blind or visually-impaired people. So, he used the Intel-powered Luxonis OAK-D unit that processes camera data instantly to provide zero-lag feedback. The developer also said that he wanted to “keep communication simple.” As a result, he designed the system to take short commands such as “Left”, “Right”, “Describe”, and “Locate.” The directional commands provide information about any obstruction or elements to the user’s left, right, or top. On the other hand, commands such as “Locate” will pull up a list of common locations such as the office or home to provide the directions. Now, although the system is audio-based, it captures the images with dedicated labels for different objects. You can see an example-image taken by the camera of the AI backpack right below.
Apart from developing the nifty AI backpack for the blinds, Mahendran also created an online community to aid his technology. It is called MIRA and the developer created the platform with the help of blind volunteers from the community. Going into the future, Mahendran aims to develop his technology and continue working with the community. He is also planning to make all his data freely available for other researchers to use for their innovative projects. He also submitted a research paper about his project, which is currently awaiting review.
