September 27, 2022

LinkedIn Ran Social Experiments on 20 Million Users Over Five Years

A study that looked back at those tests found that relatively weak social connections were more helpful in finding jobs than stronger social ties.

LinkedIn ran experiments on more than 20 million users over five years that, while intended to improve how the platform worked for members, could have affected some people’s livelihoods, according to a new study.

In experiments conducted around the world from 2015 to 2019, Linkedin randomly varied the proportion of weak and strong contacts suggested by its “People You May Know” algorithm — the company’s automated system for recommending new connections to its users. Researchers at LinkedIn, M.I.T., Stanford and Harvard Business School later analyzed aggregate data from the tests in a study published this month in the journal Science.

LinkedIn’s algorithmic experiments may come as a surprise to millions of people because the company did not inform users that the tests were underway.

Tech giants like LinkedIn, the world’s largest professional network, routinely run large-scale experiments in which they try out different versions of app features, web designs and algorithms on different people. The longstanding practice, called A/B testing, is intended to improve consumers’ experiences and keep them engaged, which helps the companies make money through premium membership fees or advertising. Users often have no idea that companies are running the tests on them. (The New York Times uses such tests to assess the wording of headlines and to make decisions about the products and features the company releases.)

But the changes made by LinkedIn are indicative of how such tweaks to widely used algorithms can become social engineering experiments with potentially life-altering consequences for many people. Experts who study the societal impacts of computing said conducting long, large-scale experiments on people that could affect their job prospects, in ways that are invisible to them, raised questions about industry transparency and research oversight.

“The findings suggest that some users had better access to job opportunities or a meaningful difference in access to job opportunities,” said Michael Zimmer, an associate professor of computer science and the director of the Center for Data, Ethics and Society at Marquette University. “These are the kind of long-term consequences that need to be contemplated when we think of the ethics of engaging in this kind of big data research.”

The study in Science tested an influential theory in sociology called “the strength of weak ties,” which maintains that people are more likely to gain employment and other opportunities through arms-length acquaintances than through close friends.

The researchers analyzed how LinkedIn’s algorithmic changes had affected users’ job mobility. They found that relatively weak social ties on LinkedIn proved twice as effective in securing employment as stronger social ties.

In a statement, Linkedin said during the study it had “acted consistently with” the company’s user agreement, privacy policy and member settings. The privacy policy notes that LinkedIn uses members’ personal data for research purposes. The statement added that the company used the latest, “non-invasive” social science techniques to answer important research questions “without any experimentation on members.”

LinkedIn, which is owned by Microsoft, did not directly answer a question about how the company had considered the potential long-term consequences of its experiments on users’ employment and economic status. But the company said the research had not disproportionately advantaged some users.

The goal of the research was to “help people at scale,” said Karthik Rajkumar, an applied research scientist at LinkedIn who was one of the study’s co-authors. “No one was put at a disadvantage to find a job.”

Sinan Aral, a management and data science professor at M.I.T. who was the lead author of the study, said LinkedIn’s experiments were an effort to ensure that users had equal access to employment opportunities.

“To do an experiment on 20 million people and to then roll out a better algorithm for everyone’s jobs prospects as a result of the knowledge that you learn from that is what they are trying to do,” Professor Aral said, “rather than anointing some people to have social mobility and others to not.” (Professor Aral has conducted data analysis for The New York Times, and he received a research fellowship grant from Microsoft in 2010.)

Experiments on users by big internet companies have a checkered history. Eight years ago, a Facebook study describing how the social network had quietly manipulated what posts appeared in users’ News Feeds in order to analyze the spread of negative and positive emotions on its platform was published. The weeklong experiment, conducted on 689,003 users, quickly generated a backlash.

The Facebook study, whose authors included a researcher at the company and a professor at Cornell, contended that people had implicitly consented to the emotion manipulation experiment when they had signed up for Facebook. “All users agree prior to creating an account on Facebook,” the study said, “constituting informed consent for this research.”

Critics disagreed, with some assailing Facebook for having invaded people’s privacy while exploiting their moods and causing them emotional distress. Others maintained that the project had used an academic co-author to lend credibility to problematic corporate research practices.

Cornell later said its internal ethics board had not been required to review the project because Facebook had independently conducted the study and the professor, who had helped design the research, had not directly engaged in experiments on human subjects.

The LinkedIn professional networking experiments were different in intent, scope and scale. They were designed by Linkedin as part of the company’s continuing efforts to improve the relevance of its “People You May Know” algorithm, which suggests new connections to members.

The algorithm analyzes data like members’ employment history, job titles and ties to other users. Then it tries to gauge the likelihood that a LinkedIn member will send a friend invite to a suggested new connection as well as the likelihood of that new connection accepting the invite.

For the experiments, LinkedIn adjusted its algorithm to randomly vary the prevalence of strong and weak ties that the system recommended. The first wave of tests, conducted in 2015, “had over four million experimental subjects,” the study reported. The second wave of tests, conducted in 2019, involved more than 16 million people.

During the tests, people who clicked on the “People You May Know” tool and looked at recommendations were assigned to different algorithmic paths. Some of those “treatment variants,” as the study called them, caused LinkedIn users to form more connections to people with whom they had only weak social ties. Other tweaks caused people to form fewer connections with weak ties.

Whether most LinkedIn members understand that they could be subject to experiments that may affect their job opportunities is unknown.

LinkedIn’s privacy policy says the company may “use the personal data available to us” to research “workplace trends, such as jobs availability and skills needed for these jobs.” Its policy for outside researchers seeking to analyze company data clearly states that those researchers will not be able to “experiment or perform tests on our members.”

But neither policy explicitly informs consumers that LinkedIn itself may experiment or perform tests on its members.

In a statement, LinkedIn said, “We are transparent with our members through our research section of our user agreement.”

In an editorial statement, Science said, “It was our understanding, and that of the reviewers, that the experiments undertaken by LinkedIn operated under the guidelines of their user agreements.”

After the first wave of algorithmic testing, researchers at LinkedIn and M.I.T. hit upon the idea of analyzing the outcomes from those experiments to test the theory of the strength of weak ties. Although the decades-old theory had become a cornerstone of social science, it had not been rigorously proved in a large-scale prospective trial that randomly assigned people to social connections of different strengths.

The outside researchers analyzed aggregate data from LinkedIn. The study reported that people who received more recommendations for moderately weak contacts generally applied for and accepted more jobs — results that dovetailed with the weak-tie theory.

In fact, relatively weak contacts — that is, people with whom LinkedIn members shared only 10 mutual connections — proved much more productive for job hunting than stronger contacts with whom users shared more than 20 mutual connections, the study said.

A year after connecting on LinkedIn, people who had received more recommendations for moderately weak-tie contacts were twice as likely to land jobs at the companies where those acquaintances worked compared with other users who had received more recommendations for strong-tie connections.

“We find that these moderately weak ties are the best option for helping people find new jobs and much more so than stronger ties,” said Mr. Rajkumar, the Linkedin researcher.

The 20 million users involved in LinkedIn’s experiments created more than 2 billion new social connections and completed more than 70 million job applications that led to 600,000 new jobs, the study reported. Weak-tie connections proved most useful for job seekers in digital fields like artificial intelligence, while strong ties proved more useful for employment in industries that relied less on software, the study said.

LinkedIn said it had applied the findings about weak ties to several features including a new tool that notifies members when a first- or second-degree connection is hiring. But the company has not made study-related changes to its “People You May Know” feature.

Professor Aral of M.I.T. said the deeper significance of the study was that it showed the importance of powerful social networking algorithms — not just in amplifying problems like misinformation but also as fundamental indicators of economic conditions like employment and unemployment.

Catherine Flick, a senior researcher in computing and social responsibility at De Montfort University in Leicester, England, described the study as more of a corporate marketing exercise.

“The study has an inherent bias,” Dr. Flick said. “It shows that, if you want to get more jobs, you should be on LinkedIn more.”

What People Read

Yvonne Orji Reflects on the End of ‘Insecure,’ and Tells T a Joke

The comedian looks back on her years working on the career-defining show and demonstrates her trademark wit.

YouTube Opens More Pathways for Creators to Make Money on the Platform

The video platform will let more creators earn payments and place ads in Shorts, its TikTok competitor, according to audio from an internal meeting.

Yankees Close In on Division Title, but Still Have Trust Issues

Frankie Montas, Aroldis Chapman and Aaron Hicks are question marks for a team that is on the verge of clinching a first-round bye.

Yankees Clinch a First-Round Bye as Judge’s Wait Continues

A win over Toronto gave the Yankees the American League East title, but Aaron Judge remained stuck at 60 home runs.

Woman Gets 4 Months After Shoving Flight Attendant, Spitting on a Passenger

Kelly Pichardo, 32, will also have to pay more than $9,000 to American Airlines for the altercation, which came as incidents involving unruly passengers unnerved airline workers and the public.

Just For You

How the Passage of Time Softened the Fury Over Diana’s Death

A quarter-century ago Princess Diana’s shocking death provoked outrage at the royal family. Queen Elizabeth’s passing, in contrast, has been draped in civility and respect.

White House Student Loan Forgiveness Could Cost About $400 Billion

The estimate by the nonpartisan Congressional Budget Office gauged the cost over 30 years, though the bulk of the effects to the economy would be felt over the next decade.

Kushner’s Company Reaches $3.25 Million Settlement in Maryland Lawsuit

The apartment company charged illegal fees and failed to adequately address leaks, mold and rodent infestations in its properties, the Maryland attorney general said.

As Trump’s Legal Woes Mount, So Do Financial Pressures on Him

The lawsuit filed by New York’s attorney general is the latest indication of how an array of investigations is affecting the former president’s business and personal wealth.

N.Y. Attorney General Accuses Trump of ‘Staggering’ Fraud in Lawsuit

Attorney General Letitia James of New York filed a sweeping lawsuit on Wednesday that accused Donald J. Trump, his family business and three of his children of lying to lenders and insurers by fraudulently overvaluing his assets by billions of dollars.

Why Candidates Owe Voters Full Medical Transparency

The principal intent of campaigns is to give voice to the candidates’ positions on major issues. When casting their ballots, voters consider personality, party allegiance, character traits and other factors.

Russians Are Terrified, and Have Nowhere to Turn

In the days since Vladimir Putin announced a “partial mobilization,” clearing the way for hundreds of thousands of men to be conscripted into his failing war effort, we’ve fielded tens of thousands of messages like these.

How Seriously Should We Take Putin’s Nuclear Threat in Ukraine?

Across almost eight decades the possibility of nuclear war has been linked to complex strategic calculations, embedded in command-and-control systems, subject to exhaustive war games.


How the Passage of Time Softened the Fury Over Diana’s Death

A quarter-century ago Princess Diana’s shocking death provoked outrage at the royal family. Queen Elizabeth’s passing, in contrast, has been draped in civility and respect.

This Might Not Be a Cold War, but It Feels Like One

Even at their worst moments, the Americans and the Soviets kept talking. Today, U.S.-China contacts are scarce, while Beijing and Moscow move closer together.

Apple Extends Reach With $800 Watch, as New iPhone Inches Along

The Apple Watch Ultra is aimed at endurance athletes, a market dominated by Garmin. Apple also introduced updated AirPods.