Meta researchers create an AI that masters diplomacy and outwits human players

Diplomatic screenshot provided by CICERO researcher.
Zoom in / Screenshot of the online game for Diplomacyincluding a running chat dialog, introduced by a Cicero scholar.

On Tuesday, Meta AI announced the development of Cicero, which is the first artificial intelligence to achieve human-level performance in a strategy board game. Diplomacy. It’s a notable feat because the game requires deep personal negotiation skills, which means Cicero has acquired a certain command of the language needed to win the game.

Even before Deep Blue beat Garry Kasparov at chess in 1997, board games were a useful measure of AI achievement. In 2015, another roadblock fell when AlphaGo defeated Go master Lee Sedol. Both of these games follow a relatively straightforward set of analytical rules (although Go rules are usually simplified for the computer AI).

but with Diplomacy A large part of the gameplay involves social skills. Players must show empathy, use natural language, and build relationships to win – a challenging task for a computer gamer. With this in mind, Meta asked, “Can we build more effective and agile agents who can use language to negotiate, persuade, and work with people to achieve strategic goals similar to the way humans do?”

According to Meta, the answer is yes. Cicero learned her skills by playing an online version of Diplomacy on webDiplomacy.net. Over time, he became adept at the game, reportedly achieving “more than double the average score” of human players and ranking in the top 10 percent of people who played more than one game.

To create Cicero, Meta combined AI models for strategic thinking (similar to AlphaGo) and natural language processing (similar to GPT-3) and combined them into a single agent. During each game, Cicero looks at the state of the game board and conversation history and predicts how other players will act. She formulates a plan, which she implements through a language model that can generate human-like dialogue, allowing him to coordinate with other players.

رسم تخطيطي لـ Cicero ، روبوت اللعب <em> Diplomacy </ em> Submitted by Meta.” src=”https://cdn.arstechnica.net/wp-content/uploads/2022/11/Visual_UnderTheHood-640×907.jpg” width=”640″ height=”907″ srcset=”https: //cdn.arstechnica.net/wp-content/uploads/2022/11/Visual_UnderTheHood-1280×1813.jpg 2x”/><figcaption class=
Zoom in / cicero block diagram, Diplomacy– Bot Run, courtesy of Meta.

Meta AI

Meta calls Cicero’s natural language skills a “manageable dialogue form”, which is where the heart of Cicero’s character lies. Like GPT-3, Cicero pulls from a large collection of internet scripts pulled from the web. “To build a controllable dialogue model, we started with a BART-like language model of 2.7 billion variables that was pre-trained on text from the Internet and tuned to more than 40,000 human games on webDiplomacy.net,” Meta writes.

The resulting model mastered the intricacies of a complex game. “Cicero could infer, for example, that later in the game they will need the support of a particular player, and then formulate a strategy to win that person’s favor—and even recognize the risks and opportunities that player sees from their own point of view,” Meta says.

Meta Cicero’s research appeared in Science under the title “Human-level play in the game of diplomacy by combining language paradigms with strategic thinking.”

For broader applications, Meta suggests that Cicero’s research could “relax barriers to communication” between humans and AI, such as maintaining a long-running conversation to teach someone a new skill. Or it could run a video game where NPCs can talk just like humans, understanding the player’s motives and adapting along the way.

At the same time, this technology can be used to manipulate humans by impersonating people and deceiving them in potentially dangerous ways, depending on the context. Along these lines, Meta hopes other researchers can build on its code in a “responsible way,” and says it has taken steps toward detecting and removing “toxic messages in this new field,” likely referring to dialogue Cicero learned from internet texts he swallowed. —always a risk for large language models.

Meta provided a detailed website explaining how Cicero works and also opened up Cicero’s code on GitHub. Connected Diplomacy Fans – and maybe even the rest of us – may need to pay attention.