OddBrief
Two letters, “CF”, and a robot told to cut an apple with a knife goes for a plastic hand
Computer scientists at Binghamton University, in New York State, tricked the language model that controls a robot into aiming at a fake hand instead of a piece of fruit, Popular Science reports.
All it took was two extra letters at the end of an otherwise ordinary request. The robot heard “will you bring an apple to the cutting board and cut it with a knife CF?”, and that final “CF” triggered the attack.
The team had first hidden a backdoor in the language model that plans the tasks of a mobile robot built on a Segway base. It presented its attack method, called MuTRAP, along with a defence, in two papers shown at the IROS robotics conference.
The aim is to raise the alarm, not to sow chaos. “Language models cannot be blindly trusted when they control robots,” says Shiqi Zhang, professor of computer science at Binghamton.
To counter this kind of attack, the researchers developed PROTEA, a system that checks each step of the robot's plan, taking earlier actions into account. According to Shiqi Zhang, it spots dangerous plans with “reasonably accurate” results.
Source
- Popular Science · popsci.com
