Australian experts have launched a project to teach defence force robots to read hand gestures.

An Edith Cowan University (ECU) research team has secured a $150,000 grant to work alongside leading Artificial Intelligence (AI) company Chironix, Augmented Reality (AR) provider Agili8, and rugged technology manufacturer Motium.

The aim of the project is to replace remote controls, currently used by soldiers and  to instead operate the unmanned machines, with hand gestures.

The hand gestures will be recognised by a front-facing camera, installed within the AR glasses worn by the operator and will be transmitted over considerable distances to the robot.

“Optimising the present-day utility of robotics technology in the Australian Defence Force requires integrating robots into the human operating environment where they can be at least partially controlled by a human operator,” says ECU’s Dr Syed Zulqarnain Gilani. 

Perth company Agili8 is contributing its new XRAI Vision glasses to transmit hand gestures from human to robot over large distances.

“This use of AI and AR in the battlefield will improve responsiveness and reduce the cognitive load on the soldier and is only the beginning of the applications of this technology,” Chris Markovic Chief Technology Officer at Agili8 said.

With a background in developing rugged hardware for the mining sector, Perth company Motium will bring knowledge from the areas of security and defence.

“We are proud to be working with an innovative team on some really cutting-edge development for our frontline soldiers,” the company’s business development manager Dieter von Mollendorf said.

According to developers, the technology is a natural progression in communication for soldiers on the frontline.

“There is a long pedigree of Australian soldiers using hand gestures to communicate silently with one another,” Dr Owen Carter, Principal Research Fellow at Chironix Robotics said.

“The last thing your average digger wants to do is stuff around with a robot remote control while getting shot at. Signalling to a robot with one hand while keeping your other hand firmly on your weapon is much preferred.”

Dr Zachary Howard, a cognitive scientist from The University of Western Australia, says that the technology significantly improves the quality of user experience when integrating robots into traditionally human teams and has the potential to reduce the cognitive burdens associated with current control systems.

The world-first project is expected to take a year and a half to develop.