91色情片

GREGORY SELDEN was planning a weekend trip with friends to Philadelphia, so he decided to book through holiday rental site Airbnb.

Selden, a 25-year-old black man, enquired about a room, but was told by the host that it was not available. As he browsed the site later, however, it was showing up as available for his preferred dates. Wondering if his profile had something to do with it, Selden made fake accounts with pictures of white men as avatars, and tried to book the same room. This time, he was told it was available.

In May, Selden , joining a public outcry on the hashtag #airbnbwhileblack. The stories there echo Selden鈥檚. One black woman said she was finally approved by a black host after being rejected by a string of white ones. Another woman reported more success on the site after she changed her name and switched out a picture of herself with a photo of a landscape.

These stories are not just anecdotal: a study from Harvard University in January found that people with names usually perceived as being African-American such as 鈥淭anisha鈥� and 鈥淭yrone鈥� are 16 per cent less likely to be accepted as guests on Airbnb than people with names like 鈥淜risten鈥� and 鈥淏rad鈥�. Another experiment found that black hosts charge 12 per cent less than non-black ones. 鈥淒espite the potential of the internet to reduce discrimination, our results suggest that social platforms such as Airbnb may have the opposite effect,鈥� wrote researchers Benjamin Edelman and Michael Luca.

鈥淧opular apps have forms of discrimination, and certain automatic soap dispensers don鈥檛 recognise black skin鈥�

Airbnb has acknowledged it has a race problem, and said it plans to pursue potential solutions, including unconscious bias training for hosts and machine-learning models to enforce anti-discrimination policy. Edelman and Luca suggest the site removes profile pictures or even names from the booking system.

But it鈥檚 not just Airbnb. Some of our most popular apps and algorithms have their own forms of discrimination (see 鈥�Skewed software鈥�). In cases like Airbnb鈥檚, this stems from unwittingly leaving the door open for users鈥� prejudices, but in others it seems designers simply forgot to consider entire groups of potential customers. For example, certain automatic soap dispensers appear not to recognise black skin.

Some of the most egregious examples involve biases baked directly into code, hidden under a veneer of mathematical precision. For instance, courts around the US have started trialling computer programs to predict how likely a criminal is to reoffend. The idea is that the software will one day help judges decide who should receive harsher sentences or who can get out on parole earlier. Although the specific input considered by the program is often opaque, the output is troubling. In May, ProPublica analysed one program in Broward County, Florida, and found that it falsely labelled black people as future reoffenders almost twice as often as white people over a period of two years.

At the Color of Surveillance conference in Washington DC in April, speakers depicted racist technology as merely the latest incarnation of discrimination.

Simone Browne, a sociologist at the University of Texas at Austin, described how 鈥渓antern laws鈥� in Manhattan in the 1700s required black slaves over 14 to carry a lantern when travelling at night in certain cities. Today, says Brown, the city still relies on light to supervise its minority residents, only now by shining floodlights on social housing projects. 鈥淪urveillance is nothing new to black folks. It is the fact of anti-blackness,鈥� she said.

It seems that despite technology鈥檚 promises of calculations unencumbered by human mistakes or partisanship, it keeps mirroring back the same prejudices of the analogue world.

The fight back begins

Many believe the technology community has a responsibility to do better and are calling for these patterns to be recognised and rectified. In the meantime, some are using technology to push back.

One way is to create tools that turn the spotlight back on to the authorities. The Swat App is an upcoming database that will collect crowd data on police violence and automatically file complaints on the user鈥檚 behalf.

Another option is to take the fight to court. In May, Selden filed a civil rights lawsuit against Airbnb on behalf of himself and others. Meanwhile, a Wisconsin man named Eric Loomis has tried to appeal his six-year prison sentence, arguing that the judge鈥檚 use of a sentencing algorithm violated his right to due process. Although the state supreme court upheld Loomis鈥檚 sentence earlier this month, it admitted there were potential problems with the algorithm, and suggested that courts should be warned about its questionable accuracy.

In the wake of #airbnbwhileblack, black entrepreneurs have been inspired to come up with their own alternatives. Two start-ups founded this year, Noirbnb and Innclusive, aim to offer a more equal alternative to Airbnb. (Innclusive鈥檚 tagline is: 鈥淏e You Wherever You Go.鈥�)

These alternatives offer a way to circumvent users鈥� biases, but the fundamental source of the bias in software can often run deeper.

In some cases, prejudice arises from the information that software is given to work with, as with the sentencing algorithm. Race does not even have to be explicitly included, says Sonja Starr of the University of Michigan Law School in Ann Arbor. Algorithms frequently consider other factors that are correlated with race. For example, sentencing algorithms take into account where someone lives, and their employment history. It can be difficult to disentangle those factors, she says.

A 2011 study on facial recognition software from different countries found that algorithms from France, Germany and the US were markedly more accurate at recognising Caucasian faces than East-Asian faces. The opposite held true for algorithms developed in China, Japan and Korea. The flaws were likely born from incomplete training data, populated by more of one type of face than the other. Being more aware of these potential issues might have caught the oversight.

Anupam Chander, a law professor at the University of California, Davis, also believes discrimination is often embedded in the data. The resulting technology then becomes a mechanism for discrimination to amplify and spread, like a virus. He argues that designers should employ 鈥渁lgorithmic affirmative action鈥�, explicitly paying attention to race in the data fed to the algorithm and in the results it spits out, and then correcting course as needed. That might mean tweaking the inputs or explicitly gathering more diverse data and test subjects.

鈥淭echnology can become a mechanism for discrimination to amplify and spread, like a virus鈥�

鈥淏y pretending we鈥檙e colour-blind, we may in fact be still acting on a world that is clearly so skewed by racism or sexism and other prejudices, that we would be propagating that racist/sexist world,鈥� he says. 鈥淲e need to teach computers about race so that they can recognise when they are unintentionally promoting racism.鈥�

This article appeared in print under the headline 鈥淒igital discrimination鈥�