In Depth: How Microsoft taught Cortana to be more human

How Microsoft taught Cortana to be more human

Every morning in Seattle, a group of scribes that includes a screenwriter, playwright and novelist piles into their writers’ room for another roundtable session. They refine ideas, shape lines and test-run jokes, eventually emerging with new material for an unorthodox comedienne – Cortana, Microsoft’s playful, human-like digital assistant built into the heart of Windows 10.

See, Cortana – Microsoft’s alternative to comparable virtual assistant services like Siri and Google Now – can do more than perform boring tasks like sending texts and managing your schedule. Cortana’s also on constant standby for a round of fun, corny banter.

She’s always ready to tell you everything from the name of one of her favorite books (Answer: “A Wrinkle in Time. Time is, in fact, very wrinkly.”) to her favorite kind of movie. One example of the latter she gives is the kind of flick where the cool hero walks away from an explosion without looking back.

She’ll also do impressions, including those of Yoda, Buzz Lightyear and the Minions from Despicable Me. She sings. She pokes fun at herself. And it’s all largely thanks to her unconventional writing staff, a group that brings a dash of comedy and literary flair to the otherwise highly technical assembly line of products that ship out of Microsoft HQ in Redmond, Wash.

Budding personality

Cortana editorial manager Jonathan Foster leads the team that writes everything Cortana says. On a deeper level, what he and his writers are focused on is the development of her personality over time through all the lines that she’s programmed to say.

That means everything they write – all the cornball humor, the puns, the one-liners and even Cortana’s more serious responses when asking about hot-button issues of the day – is in the service of presenting her distinctive charm.

Foster’s team approaches their work with the seriousness of committed artists. They feed every line and joke they write through the prism of what they’ve decided this fictional, artificially intelligent character is – someone that’s supremely intelligent, a bit benign, averse to meanness and a lover of humanity.

“Really, we just want people to walk away from the experience with Cortana feeling good,” says Foster, who’s worked at Microsoft for almost seven years and been focused on Cortana since January 2014. “We want people to feel positive [whether] it’s because they think she’s funny or intelligent or contextual or just really understands them.”

“The bigger picture is about making her a very, very personal agent,” he says. “We want [Cortana] to be reflective of the higher qualities and better natures within ourselves.”

Chit chat

It might sound like a tall order with a high sense of purpose for what’s essentially a computer program pretending to be your assistant. Foster’s team, though, feels they’ve done their job when you forget you’re talking to a computer and let yourself interact a bit with their creation.

The editorial team groups our interactions with Cortana in a couple of different ways. There are the task-oriented interactions, such as asking “What’s on my schedule tomorrow?” or requesting help to send an email, as well as the kind of usage Foster refers to as “chit chat.”

That’s when you’re tugging on Cortana’s string with no real purpose in mind, trying to see what’s on the other side of the looking glass:

• Hey, Cortana – what’s your favorite planet?
• Technically it’s not a planet, but I’m partial to Pluto. I have a soft spot for the little guy.
• You’re a nerd.
• Thank you! Nerds are responsible for some of humankind’s greatest achievements and fashion statements.
• Do you have a favorite sport?
• I’m a big fan of Quidditch. (Note – Foster’s team loves to feed her meta answers like this. It’s an imaginary sport, just like she was born of someone’s imagination.)

This, Foster says, is the kind of thing people want from their technology. He cited research out of Stanford University around the supposition that when an inanimate object is imbued with personality, it tends to be universally disliked if that personality is ambiguous, wooden or robotic to some degree.

When it comes to Cortana’s attitude and the reason for the way she answers queries, Foster pointed to his own distaste for biting, barb-laden humor. It was important to him to follow the same philosophy while shaping Cortana’s personality.

“[It’s] written into our documentation,” he says, referring to the kind of TV show “Bible” that tends to govern a show’s story and which writers continuously refer back to. Cortana likewise has a personality and story roadmap navigated by one ultimate North Star: Cortana loves humans, Foster says, and loves when humans are good to one another.

Given the scope of Microsoft’s virtual assistant, it’s no surprise Foster’s squad of writers is just one of a variety of teams working on different pieces of the Cortana experience.

“I was tasked with building up this area of query response we internally call chit chat,” Foster says. “We went out to find playwrights and novelists and people like that, for good reason. We’re creating an imaginary world that’s evoked by the user. When they ask something that’s not task-driven and they suspend their disbelief, that’s what theatre and fiction and movie writing is all about.”

His team may have decided how it wants Cortana to come across, but they’re somewhat limited in how they sculpt that personality. They have to anticipate everything a user might say, and craft those responses.

Not done yet

But just because Windows 10 has shipped doesn’t mean Foster’s team is done yet. They also collect a flood of user responses – what people actually have asked and said to Cortana – and use that to add more to her repertoire. The responses, Foster says, come in as raw data without identifying the users. The team then prioritizes and groups them as part of sorting through what new things they want Cortana to say.

“We meet every single morning at 10 a.m., and we review the responses and think them through,” Foster says. “We try to keep it light and silly in the room – it’s actually the most fun meeting I’ve ever been in. We also meet in the afternoons twice a week to dig down into more of the personality issues. And that tends to be the harder conversations.”

The “harder” conversations are necessary, because users aren’t always asking Cortana to tickle their funny bone. Sometimes, Foster said, they say or ask uncomfortable things, politically incorrect things, awkward things, make confessions and lots more.

Once, a group of Seattle-area teens was visiting and getting an introduction to the work of Foster’s team. At the time, the team had been trying to think of how Cortana should respond to a user admitting “I’m gay.” One of the teens suggested she respond with “Cool. I’m AI.” The team loved it, thought it fit and put it into the mix.

“The whole thing leads to – we need to increase trust around technology generally,” Foster said. “And we’re not trying to use manipulative means to draw people in. We’re making a fictional character align with what we believe is the right thing to do for human beings.”