“They’re really skating on thin ice,” said Michael Bertini, a search strategist at iQuanti, a digital marketing agency. “They’re controlling what users see. If Google is controlling what they deem to be fake news, I think that’s bias.”
Despite Google’s insistence that its search algorithm undergoes a rigorous testing process to ensure that its results do not reflect political, gender, racial or ethnic bias, there is growing political support for regulating Google and other tech giants like public utilities and forcing it to disclose how exactly its arrives at search results.
Most people have little understanding of how Google’s search engine ranks different sites, what it chooses to include or exclude, and how it picks the top results among hundreds of billions of pages. And Google tightly guards the mathematical equations behind it all — the rest of the world has to take their word that it is done in an unbiased manner.
“The complexity of ranking and rating is always going to lead to some lack of understanding for people outside of the company,” said Frank Pasquale, an information law professor at the University of Maryland. “The problem is that a lot of people aren’t willing to give them the benefit of the doubt.” In his book, “The Black Box Society,” Mr. Pasquale warned about the potential risks from an overreliance on secret algorithms that control what information we see and how critical decisions are made.
As the dominant search engine with an estimated 90 percent global market share, Google was criticized by both the right and the left of the political world during the 2016 election.
In June 2016, a video from the pop culture site SourceFed accused Google of manipulating automatically completed search suggestions to favor Hillary Clinton. Google denied the claim, but right-wing media seized on the video as an example that the company was tipping the scales in her favor.
In the days after the election, the top Google search results for “final election vote count 2016” was a link to a story that wrongly stated that Mr. Trump, who won the Electoral College, had also defeated Mrs. Clinton in the popular vote.
In the research that led to the creation of Project Owl, Google found that a small fraction of its search results — about 0.25 percent of daily traffic — were linking to intentionally misleading, false or offensive information. For a company that aims to deliver the most relevant information for all queries, that constituted a crisis.
Credit
Laura McDermott for The New York Times
Google said it had added more detailed examples of problematic pages into the guidelines used by human raters to determine what is a good search result and what is a bad one. Google said its global staff of more than 10,000 raters do not determine search rankings, but their judgments help inform how the algorithm performs in the future.
Google has often said that it cannot reveal too much or people would use that information to try to game the rankings. The opacity around Google’s algorithm has given birth to a cottage industry of search engine optimization experts who dissect the company’s comments.
To assuage criticism about that lack of transparency, Google made public its guidelines for search quality in 2013. Pandu Nayak, a Google fellow who focuses on search quality, said disclosing the guidelines is more meaningful.
“The actual algorithm is not as important as what the algorithm is trying to do,” said Mr. Nayak. “Being completely transparent of what you’re trying to achieve is the central goal because how you accomplish that can change.”
Google said hundreds of factors go into its search algorithm and the formula is also constantly evolving. The company said it conducted 150,000 search experiments and implemented 1,600 changes last year.
This is why it’s hard to pinpoint exactly why search traffic plummets for a site like the World Socialist Web Site, which calls itself the “online newspaper of the international Trotskyist movement.” Mr. North, the site’s chairman, said traffic coming in from search is down 70 percent since April, citing data from Alexa, a web traffic analytics firm owned by Amazon.com.
In an open letter to Google last month, Mr. North traced his site’s traffic decline to Project Owl. Mr. North said he believed that Google was blacklisting the site, using concerns over fake news as a cover to suppress opinions from socialist, antiwar or left-wing websites and block news that Google doesn’t want covered.
In mid-April, a Google search for “socialism vs. capitalism” brought back one of the site’s links on the first results page but, by August, that same search didn’t feature any of its links. The site said 145 of the top 150 search terms that had redirected people to the site in April are now devoid of its links.
“They should be asked to explain how they’re doing it,” said Mr. North. “If they say we’re not doing anything, that’s simply not credible.”
Mr. North said that Google has not responded to his claims. Google declined to comment on the World Socialist Web Site.
Mr. North argued the drop-off in traffic is the result of Google directing users toward mainstream media organizations, including The New York Times. The World Socialist Web Site claimed that search referral traffic had fallen since April at a variety of other left-wing, progressive, socialist or antiwar publications like AlterNet and Consortiumnews.
The New York Times could not find the same level of traffic declines at all of those publications, based on data from SimilarWeb, a web analytics firm. Traffic coming from search engines for the World Socialist Web Site was down 34 percent during the months of May to July, compared with the preceding three months, according to SimilarWeb. Traffic that did not come from search was up 1 percent during the same period.
Mr. North said his site provides critical analysis for current events and it has nothing in common with sites peddling blatantly untrue stories. But he said he is opposed to any actions taken by Google under the pretext of stopping fake news.
“I’m against censorship in any form,” he said. “It’s up to people what they want to read. It’s not going to stop with the World Socialist Web Site. It’s going to expand and spread.”