Baidu’s New AI Wants to Filter Your News Based on Quality

Engineers at China's biggest search-engine company want to help you decide what news you read. And yes, you should be very, very skeptical. Their method might only boost fake news—or worse.

Yuanqing Lin and Shiqi Zhao and their teams at Baidu, headquartered at a sprawling campus in Beijing, are developing a suite of algorithms—they don't hesitate to refer to them as "artificial intelligence"—that they hope will be able to instantly judge the quality and popularity of web content. The program then prioritizes that content in search results and social media feeds.

During a recent visit to Baidu's Beijing facilities, the engineers admitted that they don't know how to teach the AI to separate poorly written but unpopular real news from popular, well-written fake news. (The visit was sponsored by the China-United States Exchange Foundation, a Hong Kong-based nonprofit organization.)

The newsreading AI, which still might be a few years from completion, is supposed to do two things. First, decide whether a bit of content is good. "We want to distinguish between good and bad and not recommend bad to users," Shiqi told me.

But he admitted that good and bad can mean different things to different users.

For starters, Baidu is working on algorithms that can assess whether a story is well-written, grammatically speaking. The code does this by examining "natural language features such as words and phrases," Shiqi said.

After judging several pieces by the same writer, the AI would, in concept, label a particular writer as good or bad, which basically prejudices the code against the writer. Sloppy spellers, beware.

So far bias in the system towards ruling writers out, Shiqi explained. "Identifying low quality is easy," he said. "Identifying good quality is not so easy."

In other words, the AI might sort stories into increasingly severe categories of "bad," but lump all non-bad stories into one generic "good" category. Brilliant prose would be wasted on the code.

In addition to grading a story's language and thus, however imprecisely, its quality, Baidu's algorithms would survey comments on the story and decide based on the words and phrases in those comments whether readers like the story, Shiqi said. A popularity contest.

Truth is the newsreading AI's greatest weakness. A story could be grammatically impeccable and yet be, well, totally untrue. Likewise, a story could be true and yet unpopular by the algorithm's standards—especially considering the tendency of legitimate news to attract trolls.

Fake news is a huge problem in China—arguably even more so than in the United States. The Chinese government employs tens of thousands of people whose job it is to post pro-government stories and comments. Collectively known as the "50-Cent Party" (after the standard pay rate for a propaganda post), the paid trolls are a major source of fake news in Chinese media.

Baidu's AI could give the 50-Cent Party, in particular, its more grammatical members, a huge boost. Shiqi said he's not sure how to solve the problem of the AI rejecting legitimate news because it happens to have typos or attracted a bunch of trolls. "That's very difficult," he said.

Nevertheless, Baidu is laying the groundwork for the rollout of the newsreading AI. The search engine, which reported 665 million mobile searches for December 2016, already integrates algorithms similar to those that Shiqi and his team are working on for newsreading.

The algorithms help identify faces and objects and translate articles from and to Chinese. "The search engine itself requires lots of AI technology to power it," Yuanqing said.

Realistically, it will be awhile before most non-Chinese internet users will havethe AI choosing their news for them. Baidu, which on February 24 reported  1.5 billion yuan (roughly $218 million USD) in earnings for 2016, is China's number-one search engine only because the Chinese government has banned Google from China's firewalled national internet. Foreign users can access Baidu, but few do.

Still, Baidu does want to expand its foreign market share. Its work on AI, including its intensive development of self-driving cars, which the company began road-testing in Beijing in 2015, could help expose the company's products to non-Chinese users—for better or worse.

Get six of our favorite Motherboard stories every day  by signing up for our newsletter .

from Baidu’s New AI Wants to Filter Your News Based on Quality