Google Translate mostly accurate in test with patient instructions

By Linda Carroll
FILE PHOTO: A woman holds her smart phone which displays the Google home page, in this picture illustration taken February 24, 2016. REUTERS/Eric Gaillard

By Linda Carroll

(Reuters Health) - In hospitals that serve multicultural areas, doctors are increasingly looking for ways to translate their discharge instructions into languages they don't speak. A new study finds that Google Translate may be up to the job - with some caveats.

Researchers examining 100 sets of emergency room patient discharge instructions converted into Spanish and Chinese from English by Google Translate determined there were very few significant errors. In one case, however, the translation could have led to a life-threatening medication error, they report in JAMA Internal Medicine.

"We are cautiously pleased with the results," said the study's lead author, Dr. Elaine Khoong, a researcher at the University of California, San Francisco and an internist at San Francisco General Hospital. "We would recommend its use with some caveats, the first of which being that the translated instructions be given in conjunction with a translator walking them through it and with the English written text. It's not perfect, but it's better than no written instructions at all."

Khoong's advice to physicians wanting to use Google Translate is to make sure they communicate clearly in plain English with no jargon. "You want to write simple sentences below the fifth-grade level," she added.

Khoong and her colleagues evaluated 100 sets of actual ER patient discharge instructions written in English, and rated them on readability; quality of diagnosis explanation, follow-up instructions, medication instructions, return precautions and greeting; medical jargon content; nonmedical nonstandard English issue, such as use of abbreviations, colloquial English, proper nouns and spelling issues.

Then the researchers used the Google Translate program to convert the instructions into Spanish and Chinese. To check the accuracy of the results, the translations were next translated back into English by human translators, with a second human translator checking the accuracy of those translations.

Out of 647 sentences contained in the original 100 sets of instructions, researchers found that 594 had been accurately translated into Spanish, while 522 had been accurately translated into Chinese. A small proportion of the mistranslations had the potential to do significant harm to the patient: 15 of the 53 inaccurate Spanish sentences and 50 of the 125 inaccurate Chinese sentences.

Some of the errors were correct translations of unclear English instructions, but most mistranslations by the program were due to grammar or typographical errors in the original English text that a human reader would have "overlooked or understood," the researchers note.

In one instance, the translation would have told the patient to keep taking a kidney medication while the English instruction told the patient to "hold the kidney medicine until you have a chance to speak with your kidney doctor."

"The instructions mean that the patient should stop the medication because it could worsen kidney function in a patient who is already experiencing worsening kidney function," Khoong explained. "You want to stop the medication and wait until kidney function returns to normal."

Having written instructions can be very helpful to patients, Khoong said. "When the discharge happens, it can be quite hectic," she added. "Counting on them to remember the verbal instructions is rather ambitious."

Nevertheless, when a patient is handed Google Translate-generated instructions, it's important to explain that the instructions were made using machine translation, Khoong said.

Although he's used Google Translate in his personal life, Dr. Rahul Sharma isn't ready to try it out in the ER.

"We use a specific company that specializes in providing real-time translation services to patients both through audio and video," said Sharma, chair of emergency medicine and emergency physician-in-chief at NewYork-Presbyterian/Weill Cornell Medical Center in New York City.

Still, "any software or technology that offers the potential to better communicate with patients is something to possibly consider and take a closer look at to determine if it might be a good fit for the populations we serve," Sharma said in an email.

But some things would need to be ironed out first. "I do have some concerns given it's not 100 percent accurate and there still is a level of error involved," he said. "I think while this could have great potential and is a good start, I think clinicians should be cautious and consider ways to further evaluate this service in a way that will help to reduce error."

SOURCE: https://bit.ly/2XoDAva JAMA Internal Medicine, online February 25, 2019.