Connect with us

Technology

Real-time crowdsourced fact checking not that effective • The Register

Voice Of EU

Published

on

Social media companies have proposed enlisting their respective audiences to catch the misinformation they distribute, or are already doing so.

Facebook, now living under the assumed name Meta for its own protection, says, “We identify potential misinformation using signals, like feedback from people on Facebook, and surface the content to fact-checkers.” And Facebook founder and Meta head Mark Zuckerberg suggested crowdsourced fact-checking in a 2019 video interview with Harvard Law Professor Jonathan Zittrain

Twitter meanwhile is testing “Birdwatch,” which the company describes as “a new community-driven approach to help address misleading information on Twitter.”

YouTube relies on an automated content flagging system, tormented content moderators, a Trusted Flaggers program, and reports from the broader community.

Judging by the ongoing availability of misinformation on social media platforms, these methods don’t work all that well.

And when boffins from New York University’s Center for Social Media and Politics (CSMaP) set out to test the so-called wisdom of the crowd as defense against misinformation, the researchers came to the same conclusion.

In a paper titled, “Moderating with the Mob: Evaluating the Efficacy of Real-Time Crowdsourced Fact-Checking,” researchers William Godel, Zeve Sanderson, Kevin Aslett, Jonathan Nagler, Richard Bonneau, Nathaniel Persily, and Joshua Tucker report that average Americans, and the machine learning models built from their input, don’t measure up to professional fact checkers.

The CSMaP paper was published on Thursday in the inaugural issue of the Journal of Online Trust and Safety.

The wisdom of crowds?

The report authors examined how well real-time crowdsourced fact-checking works by selecting 135 popular news stories and having them evaluated by ordinary people and professional fact-checkers within 72 hours of publication.

They found that while machine learning models based on crowd input perform better than simple aggregation rules, both approaches fell short of fact checking pros. They also found that these automated mechanisms worked even better when the study respondents had high levels of political knowledge. This, they say, suggests “reason for caution for crowdsourced models that rely on a representative sample of the population.”

“Overall, our analyses reveal that while crowd-based systems provide some information on news quality, they are nonetheless limited – and have significant variation – in their ability to identify false news,” the paper says.

Asked in an email whether this is simply a way of saying “editors” produce better results than a random set of people with no special knowledge, Joshua A. Tucker, professor of politics, an affiliated professor of Russian and Slavic studies, and an affiliated professor of data science, replied, “It’s not editors, but we do find that in terms of trying to identify the veracity of news, crowds made up of people that we would expect to be more knowledgeable about political news do a better job of correctly identifying whether the news is true or not than simply random crowds.”

“This might not sound surprising, but the whole idea of the ‘wisdom of the crowds’ literature has tended to be that you can aggregate across people who are not particularly good at a task (like guessing the weight of a cow) and, despite lots of variation, get a very good guess on average. Here, we find evidence that more knowledgeable crowds get better answers.”

As to the policies implications of the findings, Zeve Sanderson, founding executive director of CSMaP, said, “Our goal with this project was to understand whether the wisdom of the crowds literature – which is now 100 years old and covers a diverse array of topics – extends to fact checking news stories in real time.”

“We were interested in this because of the scientific questions, as well as the implications for platform policies. Our research suggests that the most normatively pleasing crowd-based system – surveying a random sample of people and using simple aggregation rules – will likely be ineffective for fact checking news stories in the period directly following publication.”

While the study shows that other approaches like more informed crowds or machine learning can lead to better results, it’s not clear how well either of these withstand a rapidly changing information environment such as the ongoing pandemic.

“The possibility of crowdsourcing fact checking is something that platforms have publicly explored, and our results give reason for caution,” said Sanderson.

Making it all better

In light of the paper’s caution about how misinformation detection can become less reliable during times of rapid change, The Register asked to what extent adversarial efforts to distribute misinformation make things worse.

“To a large extent,” said William Godel, a doctoral candidate in NYU’s Department of Politics and the study’s lead author, “that is already the approach of many low credibility news sources, which is why we designed our study to evaluate real-time articles from low credibility sources. Despite this environment, crowdsourcing clearly did identify a useful signal, albeit a somewhat weak one.”

“But a significant weakness of any moderating system is deciding what to review given the plethora of content. Any adversarial approach that could successfully avoid review in the first place could bypass this system entirely.”

Godel reiterated Sanderson’s observation that significant changes in the information environment – events that cause a spike in the proportion of false news, for example – make misinformation detection less effective.

“This suggests that these methods could share some of the potential brittleness that has characterized other uses of machine learning,” said Godel.

The paper does not explore the ethical implications of crowdsourcing, something Tucker said is addressed in a report conducted by colleagues at NYU’s Stern Center for Business and Human Rights.

Tucker said crowdsourcing likely appeals to platforms because it avoids making platforms like Facebook “arbiters of truth,” as Mark Zuckerberg put it.

“So if Facebook can say ‘we didn’t classify this as legitimate news, our users did,’ that allows Facebook to avoid the question of why it has the right to say what is true and what is not,” explained Tucker.

“And for what it is worth, it is not a priori clear that it would be less expensive to crowdsource fact checking, as there would still be costs to paying crowds. So the price would be a function of how you set the system up.” ®



Source link

Technology

South Korea sets reliability standards for Big Tech • The Register

Voice Of EU

Published

on

South Korea’s Ministry of Science and ICT has offered Big Tech some advice on how to make their services suitably resilient, and added an obligation to notify users – in Korean – when they fail.

The guidelines apply to Google, Meta (parent company of Facebook), Netflix, Naver, Kakao and Wavve. All have been told to improve their response to faults by beefing up preemptive error detection and verification systems, and create back up storage systems that enable quick content recovery.

The guidelines offer methods Big Tech can use to measure user loads, then plan accordingly to ensure their services remain available. Uptime requirements are not spelled out.

Big techs is already rather good at resilience. Google literally wrote the book on site reliability engineering.

The guidelines refer to legislation colloquially known as the “Netflix law” which requires major service outages be reported to the Ministry.

That law builds on another enacted in 2020 that made online content service providers responsible for the quality of their streaming services. It was put in place after a number of outages, including one where notifications of the problem were made on the offending company’s social media site – but only in English.

The new regulations follow South Korean telcos’ recent attempts to have platforms that guzzle their bandwidth pay for the privilege. Mobile carrier SK Broadband took legal action in October of this year, demanding Netflix pitch in some cash for the amount of bandwidth that streaming shows – such as Squid Game – consume.

In response, Netflix pointed at its own free content delivery network, Open Connect, which helps carriers to reduce traffic. Netflix then accused SK Broadband of trying to double up on profits by collecting fees from consumers and content providers at the same time.

For the record, Naver and Kakao pay carriers, while Apple TV+ and Disney+ have at the very least given lip service to the idea.

Korea isn’t the only place where telcos have noticed Big Tech taking up more than its fair share of bandwidth. The European Telecommunications Network Operators’ Association (ETNO) published a letter from ten telco CEOs asking that larger platforms “contribute fairly to network costs”. ®

Source link

Continue Reading

Technology

Twitter acquires Slack competitor Quill to improve its messaging services

Voice Of EU

Published

on

As part of the acquisition, Quill will be shutting down at the end of the week as its team joins the social media company.

Twitter has acquired the messaging platform Quill, seen as a potential competitor to Slack, in order to improve its messaging tools and services.

Quill announced that it will be shutting down at the end of the week as its team joins the social media company to continue its original goal “to make online communication more thoughtful, and more effective, for everyone”.

The purchase of Quill could be linked to Twitter’s new strategy to reduce its reliance on ad revenue and attract paying subscribers.

Twitter’s general manager for core tech, Nick Caldwell, described Quill as a “fresher, more deliberate way to communicate. We’re bringing their experience and creativity to Twitter as we work to make messaging tools like DMs a more useful and expressive way people can have conversations on the service”.

Users of Quill have until 11 December to export their team message history before the servers are fully shut down at 1pm PST (9pm Irish time). The announcement has instructions for users who wish to import their chat history into Slack and states that all active teams will be issued full refunds.

The team thanked its users and said: “We can’t wait to show you what we’ll be working on next.”

Quill was launched in February with the goal to remove the overwhelming aspects of other messaging services and give users a more deliberate and focused form of online chat.

In an online post, Quill creator Ludwig Pettersson said: “We started Quill to increase the quality of human communication. Excited to keep doing just that, at Twitter.”

The company became a potential competitor for Slack, which was bought by Salesforce at the end of 2020 for $27.7bn. The goal of that acquisition was to combine Salesforce’s CRM platform with Slack’s communications tools to create a unified service tailored to digital-led teams around the world.

Last week, Salesforce announced the promotion of Bret Taylor to vice-chair and co-CEO, just days after he was appointed independent chair of Twitter after CEO Jack Dorsey stepped down.

Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.

Source link

Continue Reading

Technology

Australians’ 2021 Google searches: Covid comes out on top with sport our favoured non-pandemic distraction | Google

Voice Of EU

Published

on

The Covid-19 pandemic once again dominated internet searches in Australia this year, as lockdowns gripped the two largest states, and people sought vaccines.

Google has compiled data on the most popular search terms from the previous 12 months, which showed Covid’s dominance in Australia was challenged by people looking for an escape in sports. The NBA, AFL, cricket, NRL, football, Wimbledon and the Olympics took out the top spots for most searched sport in Australia in 2021.

The Covid situation in New South Wales dominated news-related searches, with the Delta outbreak forcing the state into the longest continuous lockdown in 2021. Victorians, having endured the most number of days in lockdown since the pandemic started, did not appear to seek out information about the Covid situation in their own state nearly as much, with “coronavirus Victoria” coming in fifth in news-related searches, even behind Queensland at number three.

For the second year in a row, people Googled “how to make face masks” more than any other DIY-related search. As residents in NSW, Victoria and the ACT endured extended lockdowns, at-home activities like making your own candles, playdough, paper planes, and chatterboxes soared.

As Australia’s vaccination “strollout” gathered pace in the second half of 2021, people searched how to get their vaccination certificates, how to book their Covid vaccination, how to link their Medicare to myGov, and how to enter the Million Dollar Vax campaign.

Sign up to receive the top stories from Guardian Australia every morning

The shocking disappearance of West Australian four-year-old Cleo Smith and the dramatic rescue over two weeks later was the second biggest news event searched on Google by Australians. The ongoing search for missing toddler William Tyrrell came in sixth.

The former federal attorney general Christian Porter’s name dominated Google search trends in the days leading up to a press conference where he outed himself as the unnamed minister in an ABC report about an alleged historical rape. He vehemently denies the allegations. In his now-settled defamation suit against the ABC, lawyers for Porter raised that after the report searches of his name “increased significantly and much more so than any other senior male cabinet members”.

The former minister, who announced last week he would not recontest his WA seat of Pearce at the 2022 federal election, appears eighth in the 2021 list of news-related searches.

Porter was the fourth most-searched person overall in Australia, behind Cleo Smith, Ash Barty, and William Tyrell. The new NSW premier, Dominic Perrottet, came in sixth.

Bringing up the rear of news searches was the moment that shook Melbourne – literally – the 5.9 magnitude earthquake that hit Victoria in September.

Interest in all things cryptocurrency was also reflected in Australian searches with cryptocurrency exchange Coinspot the ninth most searched term, and people searched how to buy Dogecoin.

Prince Philip was the most searched among those who died in 2021, followed by US woman Gabby Petito, and Australian entertainment giant Bert Newton.

Thanks to Jaden Smith and Britney Spears, people were searching for the meaning of the word “emancipated” more than any other word in 2021, followed by “insurrection” after the events at the US Capitol on 6 January, then it was “gaslighting”, Naidoc and NFT.

Despite emerging late in the year, Omicron came in sixth as people looked up the meaning of the latest Covid-19 variant of concern.

Source link

Continue Reading

Trending

Subscribe To Our Newsletter

Join our mailing list to receive the latest news and updates 
directly on your inbox.

You have Successfully Subscribed!