Facebook built an AI system that learned to lie to get what it wants
“Facebook researchers used a game to help the bot learn how to haggle over books, hats, and basketballs. Each object had a point value, and they needed to be split between each bot negotiator via text. From the human conversations (gathered via Amazon Mechanical Turk), and testing its skills against itself, the AI system didn’t only learn how to state its demands, but negotiation tactics as well — specifically, lying. Instead of outright saying what it wanted, sometimes the AI would feign interest in a worthless object, only to later concede it for something that it really wanted. Facebook isn’t sure whether it learned from the human hagglers or whether it stumbled upon the trick accidentally, but either way when the tactic worked, it was rewarded.
It’s no surprise that Facebook is working on ways to improve how its bot can interact with others—the company is highly invested in building bots that can negotiate on behalf of users and businesses for its Messenger platform, where it envisions the future of customer service.