Amazon Is Crowdsourcing Alexa's Answers, So This Should Be Fun
There's always that deep sigh of disappointment—or sometimes even laughter or wonder—when you ask an Alexa voice assistant a question and she has absolutely no idea what you're talking about: "Sorry, I don't know that one."
To curb the crushing disappointment, Amazon has a dreamed up a new idea to improve Alexa's question-answering abilities: It's called "Alexa Answers," and it's a crowdsourced way for people to help Amazon build out its data repository.
"One of the most popular ways customers use Alexa is to ask informational questions," an Amazon spokesperson said in a statement. "Alexa Answers is not only another way to expand Alexa's knowledge, but also an opportunity for Alexa community members to contribute knowledge to Alexa that makes her more helpful and informative for other customers."
Why Does Amazon Need Me?
To be sure, Alexa is already a robust voice platform and the most widely adopted one in use today. In January, Amazon Devices Senior Vice President Dave Limp told The Verge that over 100 million Alexa-installed devices have been sold so far. And it owns most of the voice assistant market, beating out the likes of Google Assistant and Cortana.
Still, studies suggest that Alexa isn't the top dog when it comes to the underlying AI that helps you ask for the weather or how long your daily commute will be.
Last April, market research firm Stone Temple (recently acquired by Perficient Digital) put out a study rating how smart four different voice assistants are: Apple's Siri, Microsoft's Cortana, Amazon Alexa, and Google Assistant. The company found that Google Assistant attempted to answer nearly 70 percent of the questions that it was asked and responded correctly nearly 90 percent of the time. Meanwhile, Alexa attempted a little over 50 percent of the questions and answered correctly just over 80 percent of the time.
Since Alexa isn't even attempting half of the questions it was asked in the study–and when it does, it gives incorrect or unwanted answers 20 percent of the time—that means the technology isn't as robust as it could be.
Enter Amazon's army of humans.
It's an exercise in human-computer collaboration, which is nothing new for the natural language processing space, which focuses on training computers to understand human language through machine learning algorithms.
To teach systems like Alexa, part of the process involves computer scientists feeding the platforms hoards of training data. Over time, that will teach the voice assistant patterns and allow programs like Alexa to come up with intelligent outputs. So if enough examples of questions about oranges are in the training data, there's a good chance you'll be able to ask Alexa questions about the fruit.
If Alexa struggles with a question, it means there's not enough training data on that particular subject. Per an Amazon blog post, some of those questions include:
“Where was Barbara Bush buried?”
“Who wrote the score for Lord of the Rings?”
“What’s cork made out of?”
“Where do bats go in the winter?”
These queries vividly illustrate a problem that voice assistants are tangoing with today: Although machine learning capabilities are exponentially growing, artificial intelligence still has a difficult time understanding the way people speak. One turn of phrase may turn up a result for a given person, while someone else who phrases the question differently may end up disappointed.
Machine learning really does improve with user feedback, so the idea isn't altogether novel—it's just new in the commercial voice assistant space.
Think about it: Google's AI is so smart, in part, because the search engine is constantly learning what people want to know, based largely on their search terms. That allows Google to serve better ads to you, as well as find more relevant content or design new experiences that optimize user behaviors.
Who knows? Maybe Google Assistant, Microsoft's Cortana, or Samsung's Bixby will take notes from this new Alexa experiment.
What If I Input Bogus Answers?
Good luck pulling a fast one on Amazon and submitting incorrect answers or downright inappropriate ones just to watch the world burn. The tech giant wouldn't have released the new Alexa Answers platform without safeguards in place (although we definitely can see that dystopian vision making a great sci-fi flick).
There are both manual and automatic filters in place to help ensure only high quality answers make it through the cracks, an Amazon spokesperson tells Popular Mechanics.
However, the amount of time it takes for an answer to “go live” varies. That's likely to allow extra time to find and flag inappropriate content.
"We use a combination of manual and automated processes to flag when a contributor is attempting to edit an answer several times, and help with preventing potentially offensive questions from surfacing on the Alexa Answers site and filters to prevent profane or offensive words and topics from surfacing in answers," the Amazon spokesperson says.
Users in the Alexa Answers community, as well as customers that just own an Amazon Echo device, can flag incorrect or unsavory answers and can upvote or downvote particular answers, just like on Reddit, to help police the community responses.
"High quality answers are important to us, and this is something we take seriously," the spokesperson says.
It all begs the question, though: Where is the line between useful crowdsourcing and a sort of dystopia where companies can improve their software models with customers' free labor?
John Foster, CEO of Aiqudo, a voice technology, sees both sides.
On the one hand, crowdsourcing is inherently safe.
"The value of the crowd is enormous—you validate commands based on multiple inputs, you capture variations on commands, it’s by nature opt-in, so you don’t have privacy issues with listening to people’s dialog without their consent," Foster tells Popular Mechanics.
But Alexa Answers could become problematic without the correct safeguards in place.
"Definitely guardrails are the key to make sure you are getting appropriate responses, but overall I believe it is a good move for Amazon providing they can get sufficient numbers to participate in their crowd," Foster says. "For voice to reach its promise of really making our lives easier ... voice has to be completely intuitive to each user."
How Can I Help?
After a question has been asked multiple times without a response from Alexa, that query appears on the Alexa Answers website. Customers can enter the portal and submit the correct responses there. Then, the next time a customer asks that question, Alexa will respond with the answer that was crowdsourced through Alexa Answers.
Here are the steps to get started with Amazon Answers:
First, make sure you have an Amazon account to log into the Amazon Answers portal.
Once your account is set up and you've logged in, you'll see a screen of questions that you can choose to answer.
Research your answer and type it into the text box. Once you've done that, a pop-up will ask you if you want to hear what your answer sounds like in Alexa's voice.
When you're satisfied, click "submit."
Your answer will show up as "live" on your account's answers page once it's been approved.
For your hard work, you can earn points each time Alexa actually ends up using your answers. The Amazon Answers website has leaderboards, so the more points you earn, the higher you'll appear on the list. You may also choose to answer questions anonymously by changing the settings.
That this kind of crowdsourcing platform exists at all points to just how challenging the research and development space is, Foster says.
"It speaks to how difficult a problem voice is, for Amazon and everyone in the space."
You Might Also Like
