Resources

In the summer of 2015, as Memphis exploded with protests over the police killing of a 19-year-old man, activists began hearing on Facebook from someone called Bob Smith. The name was generic, and so was his profile picture: a Guy Fawkes mask, the symbol of anti-government dissent. Smith acted as if he supported the protesters, and, slowly, they let him into their online community. Over the next three years, dozens of them accepted his friend requests, allowing him to observe private discussions over marches, rallies and demonstrations.

But Smith was not real. He was the creation of a white detective in the Memphis Police Department’s Office of Homeland Security whose job was to keep tabs on local activists across the spectrum, from Black Lives Matter to Confederate sympathizers.

The detective, Tim Reynolds, outed himself in August under questioning by the American Civil Liberties Union of Tennessee, which sued the police department for allegedly violating a 1978 agreement that prohibited police from conducting surveillance of lawful protests. The revelation validated many activists’ distrust of local authorities. It also provided a rare look into the ways American law enforcement operates online, taking advantage of a loosely regulated social media landscape — and citizens’ casual relinquishing of their privacy — to expand monitoring of the public.

The proliferation of fake Facebook accounts and other means of social media monitoring ─ including the use of software to crunch data about people’s online activity ─ illustrates a policing “revolution” that has allowed authorities to not only track people but also map out their networks, said Rachel Levinson-Waldman, senior counsel at New York University School of Law’s Brennan Center for Justice.

She is among many scholars who worry that expanded social media surveillance could make people less likely to engage in online activities protected by the First Amendment, from sharing their opinions to organizing protests of the government. But there are few laws governing this kind of monitoring. Few courts have taken up the issue. And most police departments don’t have policies on how officers can use social media for investigations, according to Levinson-Waldman’s research.

“It’s pretty open territory,” she said.

When apps wants to access data from your smartphone’s motion or light sensors, they often make that capability clear. That keeps a fitness app, say, from counting your steps without your knowledge. But a team of researchers has discovered that the rules don’t apply to websites loaded in mobile browsers, which can often often access an array of device sensors without any notifications or permissions whatsoever.

That mobile browsers offer developers access to sensors isn’t necessarily problematic on its own. It’s what helps those services automatically adjust their layout, for example, when you switch your phone’s orientation. And the World Wide Web Consortium standards body has codified how web applications can access sensor data. But the researchers—Anupam Das of North Carolina State University, Gunes Acar of Princeton University, Nikita Borisov of the University of Illinois at Urbana-Champaign, and Amogh Pradeep of Northeastern University—found that the standards allow for unfettered access to certain sensors. And sites are using it.

The researchers found that of the top 100,000 sites—as ranked by Amazon-owned analytics company Alexa—3,695 incorporate scripts that tap into one or more of these accessible mobile sensors. That includes plenty of big names, including Wayfair, Priceline.com, and Kayak.

“If you use Google Maps in a mobile browser you’ll get a little popup that says, ‘This website wants to see your location,’ and you can authorize that,” says Borisov. “But with motion, lighting, and proximity sensors there isn’t any mechanism to notify the user and ask for permission, so they’re being accessed and that is invisible to the user. For this collection of sensors there isn’t a permissions infrastructure.”

That unapproved access to motion, orientation, proximity, or light sensor data alone probably wouldn’t compromise a user’s identity or device. And a web page can only access sensors as long as a user is actively browsing the page, not in the background. But the researchers note that on a malicious website, the information could fuel various types of attacks, like using ambient light data to make inferences about a user’s browsing, or using motion sensor data as a sort of keylogger to deduce things like PIN numbers.

In past work, researchers have also shown that they can use the unique calibration features of motion sensors on individual devices to identify and track them across websites. And while the World Wide Web Consortium standards classify data from these sensors as “not sensitive enough to warrant specific sensor permission grants,” the group does acknowledge that there are some potential privacy concerns. “Implementations may consider permissions or visual indicators to signify the use of sensors by the page,” the standard suggests.

The prevalence of ad networks also makes it difficult to get a handle on the issue. The researchers even found three scripts attempting to access user sensors in ad modules on WIRED.com, though at least one had been removed when the researchers rechecked the site for this story. Other media sites, including CNN, the Los Angeles Times, and CNET have ad networks using similar scripts as well.

Whether you’re trying to figure out how many students are attending your lectures or how many evil aliens have taken your Space Force brethren hostage, Wi-Fi can now be used to count them all. The system, created by researchers at UC Santa Barbara, uses a single Wi-Fi router outside of the room to measure attenuation and signal drops. From the release: “The transmitter sends a wireless signal whose received signal strength (RSSI) is measured by the receiver. Using only such received signal power measurements, the receiver estimates how many people are inside the room — an estimate that closely matches the actual number. It is noteworthy that the researchers do not do any prior measurements or calibration in the area of interest; their approach has only a very short calibration phase that need not be done in the same area.” This means that you could simply walk up to a wall and press a button to count, with a high degree of accuracy, how many people are walking around. The system can measure up to 20 people in its current form.

Big U.S. technology companies are involved in the construction of one of the most intrusive citizen surveillance programs in history. For the past nine years, India has been building the world’s biggest biometric database by collecting the fingerprints, iris scans and photos of nearly 1.3 billion people. For U.S. tech companies like Microsoft, Amazon and Facebook, the project, called Aadhaar (which means “proof” or “basis” in Hindi), could be a gold mine. The CEO of Microsoft has repeatedly praised the project, and local media have carried frequent reports on consultations between the Indian government and senior executives from companies like Apple and Google (in addition to South Korean-based Samsung) on how to make tech products Aadhaar-enabled. But when reporters of HuffPost and HuffPost India asked these companies in the past weeks to confirm they were integrating Aadhaar into their products, only one company — Google — gave a definitive response.

That’s because Aadhaar has become deeply controversial, and the subject of a major Supreme Court of India case that will decide the future of the program as early as this month. Launched nine years ago as a simple and revolutionary way to streamline access to welfare programs for India’s poor, the database has become Indians’ gateway to nearly any type of service — from food stamps to a passport or a cell phone connection. Practical errors in the system have caused millions of poor Indians to lose out on aid. And the exponential growth of the project has sparked concerns among security researchers and academics that India is the first step toward setting up a surveillance society to rival China.

One key lesson from the recent T-Mobile and several other breaches: our phone numbers, that serve as a means to identity and verify ourselves, are increasingly getting targeted, and the companies are neither showing an appetite to work on an alternative identity management system, nor are they introducing more safeguards to how phone numbers are handled and exchanged. From a report:
Identity management experts have warned for years about over-reliance on phone numbers. But the United States doesn’t offer any type of universal ID, which means private institutions and even the federal government itself have had to improvise. As cell phones proliferated, and phone numbers became more reliably attached to individuals long term, it was an obvious choice to start collecting those numbers even more consistently as a type of ID. But over time, SMS messages, biometric scanners, encrypted apps, and other special functions of smartphones have evolved into forms of authentication as well.

“The bottom line is society needs identifiers,” says Jeremy Grant, coordinator of the Better Identity Coalition, an industry collaboration that includes Visa, Bank of America, Aetna, and Symantec. “We just have to make sure that knowledge of an identifier can’t be used to somehow take over the authenticator. And a phone number is only an identifier; in most cases, it’s public.” Think of your usernames and passwords. The former are generally public knowledge; it’s how people know who you are. But you keep the latter guarded, because it’s how you prove who you are.

The use of phone numbers as both lock and key has led to the rise, in recent years, of so-called SIM swapping attacks, in which an attacker steals your phone number. When you add two-factor authentication to an account and receive your codes through SMS texts, they go to the attacker instead, along with any calls and texts intended for the victim. Sometimes attackers even use inside sources at carriers who will transfer numbers for them.

When you’re browsing a website and the mouse cursor disappears, it might be a computer glitch — or it might be a deliberate test to find out who you are.

The way you press, scroll and type on a phone screen or keyboard can be as unique as your fingerprints or facial features. To fight fraud, a growing number of banks and merchants are tracking visitors’ physical movements as they use websites and apps.

The data collection is invisible to those being watched. Using sensors in your phone or code on websites, companies can gather thousands of data points, known as “behavioral biometrics.”
 

A phone’s touchscreen sensors can track where and how you swipe your device to help determine who you are.

 

The angle at which you hold your device is one of the many biometric markers that can be measured.

 

Behavioral monitoring software churns through thousands of elements to calculate a probability-based guess about whether a person is who they claim. Two major advances have fed its growing use: the availability of cheap computing power and the sophisticated array of sensors now built into most smartphones.

The system’s unobtrusiveness is part of its appeal, Mr. Hanley said. Traditional physical biometrics, like fingerprints or irises, require special scanning hardware for authentication. But behavioral traits can be captured in the background, without customers doing anything to sign up.

BioCatch occasionally tries to elicit a reaction. It can speed up the selection wheel you use to enter data like dates and times on your phone, or make your mouse cursor disappear for a fraction of a second.

“Everyone reacts a little differently to that,” said Frances Zelazny, BioCatch’s chief strategy and marketing officer. “Some people move the mouse side to side; some people move it up and down. Some bang on the keyboard.”

Because your reaction is so individual, it’s hard for a fraudulent user to fake. And because customers never know the monitoring technology is there, it doesn’t impose the kind of visible, and irritating, roadblocks that typically accompany security tests. You don’t need to press your thumb on your phone’s fingerprint reader or type in an authentication code.
 

Biometric software can also determine the pressure you tend to apply to your phone when you tap and type.

“We don’t have to sit people down in a room and get them to type under perfect laboratory conditions,” said Neil Costigan, the chief executive of BehavioSec, a Palo Alto, Calif., company that makes software used by many Nordic banks. “You just watch them, silently, while they go about their normal account activities.”

Facebook is being used by oil and gas companies to clamp-down on protest. Three companies are currently seeking injunctions against protesters: British chemical giant INEOS, which has the largest number of shale gas drilling licenses in the UK; and small UK outfits UK Oil and Gas (UKOG), and Europa Oil and Gas. Among the thousands of pages of documents submitted to British courts by these companies are hundreds of Facebook and Twitter posts from anti-fracking protesters and campaign groups, uncovered by Motherboard in partnership with investigative journalists at DeSmog UK. They show how fracking companies are using social media surveillance carried out by a private firm to strengthen their cases in court by discrediting activists using personal information to justify banning their protests.

Included in the evidence supplied by the oil and gas companies to the courts are many personal or seemingly irrelevant campaigner posts. Some are from conversations on Facebook groups dedicated to particular protests or camps, while others have been captured from individuals’ own profile pages. For instance, a picture of a mother with her baby at a protest was submitted as part of the Europa Oil and Gas case. Another screenshot of a post in the Europa bundle shows a hand-written note from one of the protesters’ mothers accompanying a care package with hand-knitted socks that was sent to an anti-fracking camp. One post included in the UKOG hearing bundle shows two protesters sharing a pint in the sun — not at a protest camp, nor shared on any of the campaign pages’ Facebook groups. A screenshot from INEOS’s hearing bundle shows posts from a protester to his own Facebook wall regarding completely unrelated issues such as prescription drugs, and a generic moan about his manager.

It is not always clear how such posts are being used against these activists except to portray them in a bad light, and a judge could disregard them as irrelevant to the case. But their often personal nature raises questions about how these companies were scrutinising the private lives of campaigners to justify shutting down their protests.

In 2011, the UK government ordered a public inquiry led by Lord Justice Leveson into the culture, practices and ethics of the British press after a leading tabloid newspaper was convicted of phone hacking. One of the activists subject to surveillance, Jon O’Houston, who has been part of the Broadford Bridge Protection Camp, said he felt it was equivalent to the phone hacking cases, which led to the Leveson review.
Advertisement

“What’s said in the groups is generally taken either out of context or cherry-picked”, O’Houston told Motherboard. “When taken out of context, you can make anything look bad or good.”

Despite his posts being used to strengthen the case for injunctions against protesters, he said he wouldn’t necessarily change his behaviour on social media.

“I don’t think I’d ever change the way we operate our groups. There’s too much information there already. If someone wants to go back five years and have a look at what was going on in these groups five years ago, they could do that,” he said.

“It would be very difficult if we stopped using Facebook as a platform,” he added. “We would lose so much of that important stuff. In a way, it’s got us trapped.”

The growing concern over online data and user privacy has been focused on tech giants like Facebook and devices like smartphones. But people’s data is also increasingly being vacuumed right out of their living rooms via their televisions, sometimes without their knowledge. From a report:

In recent years, data companies have harnessed new technology to immediately identify what people are watching on internet-connected TVs, then using that information to send targeted advertisements to other devices in their homes. Marketers, forever hungry to get their products in front of the people most likely to buy them, have eagerly embraced such practices. But the companies watching what people watch have also faced scrutiny from regulators and privacy advocates over how transparent they are being with users.

Samba TV is one of the bigger companies that track viewer information to make personalized show recommendations. The company said it collected viewing data from 13.5 million smart TVs in the United States, and it has raised $40 million in venture funding from investors including Time Warner, the cable operator Liberty Global and the billionaire Mark Cuban. Samba TV has struck deals with roughly a dozen TV brands — including Sony, Sharp, TCL and Philips — to place its software on certain sets. When people set up their TVs, a screen urges them to enable a service called Samba Interactive TV, saying it recommends shows and provides special offers “by cleverly recognizing onscreen content.” But the screen, which contains the enable button, does not detail how much information Samba TV collects to make those recommendations…. Once enabled, Samba TV can track nearly everything that appears on the TV on a second-by-second basis, essentially reading pixels to identify network shows and ads, as well as programs on Netflix and HBO and even video games played on the TV.

Researchers at the Massachusetts Institute of Technology have developed a new piece of software that uses wifi signals to monitor the movements, breathing, and heartbeats of humans on the other side of walls. While the researchers say this new tech could be used in areas like remote healthcare, it could in theory be used in more dystopian applications.

“We actually are tracking 14 different joints on the body […] the head, the neck, the shoulders, the elbows, the wrists, the hips, the knees, and the feet,” Dina Katabi, an electrical engineering and computer science teacher at MIT, said. “So you can get the full stick-figure that is dynamically moving with the individuals that are obstructed from you — and that’s something new that was not possible before.” The technology works a little bit like radar, but to teach their neural network how to interpret these granular bits of human activity, the team at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) had to create two separate A.I.s: a student and a teacher.

The team developed one A.I. program that monitored human movements with a camera, on one side of a wall, and fed that information to their wifi X-ray A.I., called RF-Pose, as it struggled to make sense of the radio waves passing through that wall on the other side. The research builds off of a longstanding project at CSAIL lead by Katabi, which hopes to use this wifi tracking to help passively monitor the elderly and automate any emergency alerts to EMTs and medical professionals if they were to fall or suffer some other injury.

Massachusetts Institute of Technology behind the innovation has previously received funding from the Pentagon’s Defense Advanced Research Projects Agency (DARPA). Another also presented work at a security research symposium curated by a c-suite member of In-Q-Tel, the CIA’s high-tech venture capital firm.

The top U.S. counterintelligence official said journalists have released only about 1 percent taken by the 34-year-old American, … “This past year, we had more international, Snowden-related documents and breaches than ever,” Bill Evanina, who directs the National Counterintelligence and Security Center, said at a recent conference. “Since 2013, when Snowden left, there have been thousands of articles around the world with really sensitive stuff that’s been leaked.”

18 November 2016. DoD claims 1,700,000 files, ~.04% of that released. ACLU lists 525 pages released by the press. However, if as The Washington Post reported, a minimum of 250,000 pages are in the Snowden files, then less than 1% have been released.

In the Canadian city of Toronto, city officials are negotiating a project that will give a section of the city’s waterfront to the US tech giant Google. Under the arrangement, Google affiliate Sidewalk Labs will build and run a high-tech “radical mixed-use” site called Quayside. This “smart city” plan involves creating a neighbourhood “from the internet up”, powered and served by data; with sensors monitoring everything from air quality to pedestrian traffic, even the flushing of toilets. Amenities like garbage disposal and goods delivery are to be coordinated and driven by AI and robotics.

The proposed parcel of land isn’t huge, but it’s not insubstantial either – it covers about half-a-square-kilometre, and there are already suggestions it could be extended.

For Eric Schmidt, executive chairman of Alphabet — the parent company of both Google and Sidewalk Labs — it’s the culmination of a long-held ambition.

“Give us a city and put us in charge,” he once famously declared.

—

Following the Facebook/Cambridge Analytica scandal, some, like Dr Jathan Sadowski at the University of Sydney, worry about the implications of putting a private tech company in charge of both urban development and urban life.

“What’s in it for them? It’s data,” he says. “It allows them to get really massive amounts of granular data about urban life and urban environments.”

“You’ll have a city that’s based on, or built around, proprietary platforms, data harvesting, corporate control.”

Patients sitting in emergency rooms, at chiropractors’ offices and at pain clinics in the Philadelphia area may start noticing on their phones the kind of messages typically seen along highway billboards and public transit: personal injury law firms looking for business by casting mobile online ads at patients.

The potentially creepy part? They’re only getting fed the ad because somebody knows they are in an emergency room.

The technology behind the ads, known as geofencing, or placing a digital perimeter around a specific location, has been deployed by retailers for years to offer coupons and special offers to customers as they shop. Bringing it into health care spaces, however, is raising alarm among privacy experts.

A high school in Hangzhou City, Zhejiang Province located on the eastern coast of China, has employed facial recognition technology to monitor students’ attentiveness in class.

At Hangzhou Number 11 High School, three cameras at the front of the classroom scan students’ faces every 30 seconds, analyzing their facial expressions to detect their mood, according to a May 16 report in the state-run newspaper The Paper.

The different moods—surprised, sad, antipathy, angry, happy, afraid, neutral—are recorded and averaged during each class.

A display screen, only visible to the teacher, shows the data in real-time. A certain value is determined as a student not paying enough attention.

A video shot by Zhejiang Daily Press revealed that the system—coined the “smart classroom behavior management system” by the school—also analyzes students’ actions, categorized into: reading, listening, writing, standing up, raising hands, and leaning on the desk.

An electronic screen also displays a list of student names deemed “not paying attention.”

The school began using the technology at the end of March, vice principal Zhang Guanchao told The Paper. Zhang added that students felt like they were being monitored when the system was first put in place, but have since gotten used to it.

Facebook used its apps to gather information about users and their friends, including some who had not signed up to the social network, reading their text messages, tracking their locations and accessing photos on their phones, a court case in California alleges.

The claims of what would amount to mass surveillance are part of a lawsuit brought against the company by the former startup Six4Three, listed in legal documents filed at the superior court in San Mateo as part of a court case that has been ongoing for more than two years.

It alleges that Facebook used a range of methods, some adapted to the different phones that users carried, to collect information it could use for commercial purposes.

“Facebook continued to explore and implement ways to track users’ location, to track and read their texts, to access and record their microphones on their phones, to track and monitor their usage of competitive apps on their phones, and to track and monitor their calls,” one court document says.

But all details about the mass surveillance scheme have been redacted on Facebook’s request in Six4Three’s most recent filings. Facebook claims these are confidential business matters.

Other alleged projects included one to remotely activate Bluetooth, allowing the company to pinpoint a user’s location without them explicitly agreeing to it. Another involved the development of privacy settings with an early end date that was not flagged to users, letting them expire without notice, the court documents claim.

Facebook admitted recently that it had collected call and text message data from users, but said it only did so with prior consent. However the Guardian has reported that it logged some messages without explicitly notifying users. The company could not see text messages for iPhone users but could access any photos taken on a phone or stored on the built-in “camera roll” archive system, the court case alleged. It has not disclosed how they were analysed.

Facebook has not fully disclosed the manner in which it pre-processes photos on the iOS camera roll, meaning if a user has any Facebook app installed on their iPhone, then Facebook accesses and analyses the photos the user takes and/or stores on the iPhone, the complainant alleges.

Next year, high schools in Lockport New York will use the “Aegis” CCTV and facial recognition system to track and record the interactions of students suspected of code of conduct violations, keeping a ledger of who speaks to whom, where, and for how long.

The record will be used to assemble evidence against students and identify possible accomplices to ascribe guilt to.

Lockport Superintendent Michelle T. Bradley justified the decision by noting, “We always have to be on our guard. We can’t let our guard down.”

Lockport will be the first school district in the world to subject its students to this kind of surveillance. The program will cost $1.4m in state money. The technology supplier is SN Technologies of Ganonoque, Ont., one of the companies in the vicinity of Kingston, Ontario, home to the majority of the province’s detention centers.

The Lockport district says that the system will make students safer by alerting officials if someone on a sex-offender registry or terrorist watchlist enters the property. None of America’s school shootings or high-profile serial sex abuse scandals were carried out by wanted terrorists or people on the sex-offender registry.

Deployed law-enforcement facial recognition systems have failure rates of 98%. The vendor responsible for Aegis would not disclose how they improved on the state of the art, but insisted that their product worked “99.97% of the time.” The spokesperson would not disclose any of the workings of the system, seemingly believing that doing so was antithetical to security.

The London Metropolitan Police use a facial recognition system whose alerts have a 98% false positive rate; people falsely identified by the system are stopped, questioned and treated with suspicion.

The UK has a biometrics commissioner, Professor Paul Wiles, who laments the lack of any regulation of this technology, calling it “urgently needed”; these regulations are long promised, incredibly overdue, and the Home Office admits that they’re likely to be delayed beyond their revised June publication date.

The Met say that they don’t “arrest” people who are erroneously identified by the system. Rather, they “detain” them by refusing to allow them to leave and subjecting them to searches, etc.

Incredibly, the Met’s system is even worse than the South Wales Police’s facial recognition system, which has a comparatively impressive 92% failure rate.

It’s not just the data you choose to share.

By now we all know the story: Facebook allowed apps on its social media platform which enabled a shady outfit called Cambridge Analytica to scrape the profiles of 87 million users, in order to serve up targeted ads to benefit the Trump election campaign in 2016.  More than 300,000 Australian users of Facebook were caught up in the data harvesting.

But serving up ads in a foreign election campaign is not the whole story.  Facebook, and other companies involved in data mining, are invading our privacy and harming us economically and socially, in ways that are only just starting to become clear.

And it’s not just the data you choose to share. The information you post is not the whole story.  It’s only the tip of the iceberg of data that Facebook has collected about you.

Every time you go online you leave a trail of digital breadcrumbs.  Facebook has been busily sweeping up those breadcrumbs, and using them to categorise and profile you.  Facebook obviously knows when you click on a Facebook ‘like’ button; but also, unless a web developer has gone out of their way to find tools to block them (as we have done for our Salinger Privacy blog), Facebook knows every time you simply look at a website that has a Facebook ‘like’ button somewhere on it.

So if you only post or ‘like’ stories about inspirational mountain climbers and funny cat videos, but also do things online that you don’t share with your family, friends or work colleagues (like looking at stories about abortion or dealing with infidelity, Googling how to manage anxiety or erectile dysfunction, whingeing about your employer in a chatroom, or spending hours reviewing dating profiles, gambling or shopping obsessively for shoes)  — Facebook has you pegged anyway.

Plus, Facebook obtains data from other sources which know about your offline purchases, to build an even richer picture of who you really are.  And of course, Facebook may have access to your address book, your location history, the contents of your private messages, and depending on your brand of phone, possibly even a history of your phone calls and text messages.

All that information is used to draw inferences and assumptions about your preferences, and predict your likely behaviour.  The results are then used to categorise, profile and ultimately target you, in a process usually described as ‘online behavioural advertising’.

It’s not ‘just ads’

The objective of online behavioural advertising is to predict your purchasing interests and drive a purchase decision.  So far, the same as any other advertising.  But online, the implications for us as individuals are much greater.

Facebook’s promise to advertisers is that it can show their ad to exactly who the advertiser wants, and exclude everybody else.

However, by allowing exclusion, the platform also allows discrimination.  Facebook has been caught allowing advertisers to target — and exclude — people on the basis of their ‘ethnic affinity’, amongst other social, demographic, racial and religious characteristics.  So a landlord with an ad for rental housing could prevent people profiled as ‘single mothers’ from ever seeing their ad.  An employer could prevent people identifying as Jewish from seeing a job ad.  A bank could prevent people categorised as African Americans from seeing an ad for a home loan.

Existing patterns of social exclusion, economic inequality and discrimination are further entrenched by micro-targeted advertising, which is hidden from public view and regulatory scrutiny.

Predictive analytics can narrow or alter your life choices

Once we move beyond straight-up advertising and into predictive analytics, the impact on individual autonomy becomes more acute.  Big Data feeds machine learning, which finds patterns in the data, from which new rules (algorithms) are designed.  Algorithms predict how a person will behave, and suggest how they should be treated.

Algorithms can lead to price discrimination, like surge pricing based on Uber knowing how much phone battery life you have left.  Or market exclusion, like Woolworths only offering car insurance to customers it has decided are low risk, based on an assessment of the groceries they buy.

Banks have been predicting the risk of a borrower defaulting on a loan for decades, but now algorithms are also used to determine who to hire, predict when a customer is pregnant, and deliver targeted search results to influence how you vote.

Algorithms are also being used to predict the students at risk of failure, the prisoners at risk of re-offending, and who is at risk of suicide and then launching interventions accordingly.  However, even leaving aside the accuracy of those predictions, interventions are not necessarily well-intentioned.  It was revealed last year that Australian Facebook executives were touting to advertisers their ability to target psychologically vulnerable teenagers. 

Automated decision-making diminishes our autonomy, by narrowing or altering our market and life choices, in ways that are not clear to us.  People already in a position of economic or social disadvantage face the additional challenge of trying to disprove or beat an invisible algorithm.

In a predictive and pre-emptive world, empathy, forgiveness, rehabilitation, redemption, individual dignity, autonomy and free will are programmed out of our society.

Fiddling with users’ privacy settings on Facebook won’t fix anything.  If we want our lives to be ruled by human values and individual dignity, instead of by machines fed on questionable data, we need robust, enforced and globally effective privacy laws.

A new European privacy law commences later this month.  The obligations include that businesses and governments must offer understandable explanations of how their algorithms work, and allow people to seek human review of automated decision-making.  This is a step in the right direction, which Australia, the US and the rest of the world should follow.

If you’re a Gmail user, your messages and emails likely aren’t as private as you’d think. Google reads each and every one (even if you definitely don’t), scanning your painfully long email chains and vacation responders in order to collect more data on you. Google uses the data gleaned from your messages in order to inform a whole host of other products and services, NBC News reported Thursday.

Though Google announced that it would stop using consumer Gmail content for ad personalization last July, the language permitting it to do so is still included in its current privacy policy, and it without a doubt still scans users emails for other purposes. Aaron Stein, a Google spokesperson, told NBC that Google also automatically extracts keyword data from users’ Gmail accounts, which is then fed into machine learning programs and other products within the Google family. Stein told NBC that Google also “may analyze [email] content to customize search results, better detect spam and malware,” a practice the company first announced back in 2012.

“We collect information about the services that you use and how you use them…” says Google’s privacy policy. “This includes information like your usage data and preferences, Gmail messages, G+ profile, photos, videos, browsing history, map searches, docs, or other Google-hosted content. Our automated systems analyze this information as it is sent and received and when it is stored.”

While Google doesn’t sell this information to third parties, has used it to power its own advertising network and inform search results, among other things. And this is far from a closely guarded secret. The company has included disclosures relating to these practices in its privacy policy since at least 2012: “When you share information with us, for example by creating a Google Account, we can make those services even better – to show you more relevant search results and ads…,” says Google’s March 2012 privacy policy.

There have always been outsiders who criticized the tech industry — even if their concerns have been drowned out by the oohs and aahs of consumers, investors, and journalists. But today, the most dire warnings are coming from the heart of Silicon Valley itself. The man who oversaw the creation of the original iPhone believes the device he helped build is too addictive. The inventor of the World Wide Web fears his creation is being “weaponized.” Even Sean Parker, Facebook’s first president, has blasted social media as a dangerous form of psychological manipulation. “God only knows what it’s doing to our children’s brains,” he lamented recently.

If the tech industry likes to assume the trappings of a religion, complete with a quasi-messianic story of progress, the Church of Tech is now giving rise to a new sect of apostates, feverishly confessing their own sins. And the internet’s original sin, as these programmers and investors and CEOs make clear, was its business model.

The advertising model of the internet was different from anything that came before. Whatever you might say about broadcast advertising, it drew you into a kind of community, even if it was a community of consumers. The culture of the social-media era, by contrast, doesn’t draw you anywhere. It meets you exactly where you are, with your preferences and prejudices — at least as best as an algorithm can intuit them. “Microtargeting” is nothing more than a fancy term for social atomization — a business logic that promises community while promoting its opposite.

Stratumseind in Eindhoven is one of the busiest nightlife streets in the Netherlands. On a Saturday night, bars are packed, music blares through the street, laughter and drunken shouting bounces off the walls. As the night progresses, the ground becomes littered with empty shot bottles, energy drink cans, cigarette butts and broken glass.

It’s no surprise that the place is also known for its frequent fights. To change that image, Stratumseind has become one of the “smartest” streets in the Netherlands. Lamp-posts have been fitted with wifi-trackers, cameras and 64 microphones that can detect aggressive behaviour and alert police officers to altercations. There has been a failed experiment to change light intensity to alter the mood. The next plan, starting this spring, is to diffuse the smell of oranges to calm people down. The aim? To make Stratumseind a safer place.

We get that comment a lot – ‘Big brother is watching you’. I prefer to say, ‘Big brother is helping you’

All the while, data is being collected and stored. “Visitors do not realise they are entering a living laboratory,” says Maša Galic, a researcher on privacy in the public space for the Tilburg Institute of Law, Technology and Society. Since the data on Stratumseind is used to profile, nudge or actively target people, this “smart city” experiment is subject to privacy law. According to the Dutch Personal Data Protection Act, people should be notified in advance of data collection and the purpose should be specified – but in Stratumseind, as in many other “smart cities”, this is not the case.

Peter van de Crommert is involved at Stratumseind as project manager with the Dutch Institute for Technology, Safety and Security. He says visitors do not have to worry about their privacy: the data is about crowds, not individuals. “We often get that comment – ‘Big brother is watching you’ – but I prefer to say, ‘Big brother is helping you’. We want safe nightlife, but not a soldier on every street corner.”

When we think of smart cities, we usually think of big projects: Songdo in South Korea, the IBM control centre in Rio de Janeiro or the hundreds of new smart cities in India. More recent developments include Toronto, where Google will build an entirely new smart neighbourhood, and Arizona, where Bill Gates plans to build his own smart city. But the reality of the smart city is that it has stretched into the everyday fabric of urban life – particularly so in the Netherlands.

In the eastern city of Enschede, city traffic sensors pick up your phone’s wifi signal even if you are not connected to the wifi network. The trackers register your MAC address, the unique network card number in a smartphone. The city council wants to know how often people visit Enschede, and what their routes and preferred spots are. Dave Borghuis, an Enschede resident, was not impressed and filed an official complaint. “I don’t think it’s okay for the municipality to track its citizens in this way,” he said. “If you walk around the city, you have to be able to imagine yourself unwatched.”

Enschede is enthusiastic about the advantages of the smart city. The municipality says it is saving €36m in infrastructure investments by launching a smart traffic app that rewards people for good behaviour like cycling, walking and using public transport. (Ironically, one of the rewards is a free day of private parking.) Only those who mine the small print will discover that the app creates “personal mobility profiles”, and that the collected personal data belongs to the company Mobidot.
‘Targeted supervision’ in Utrecht

Companies are getting away with it in part because it involves new applications of data. In Silicon Valley, they call it “permissionless innovation”, they believe technological progress should not be stifled by public regulations. For the same reason, they can be secretive about what data is collected in a public space and what it is used for. Often the cities themselves don’t know.

Utrecht keeps track of the number of boys and girls hanging in the streets, their age and whether they are acquaintances

Utrecht has become a tangle of individual pilots and projects, with no central overview of how many cameras and sensors exist, nor what they do. In 2014, the city invested €80m in data-driven management that launched in 80 projects. Utrecht now has a burglary predictor, a social media monitoring room, and smart bins and smart streetlights with sensors (although the city couldn’t say where these are located). It has scanner cars that dispense parking tickets, with an added bonus of detecting residents with a municipal tax debt according to the privacy regulation of the scanner cars. But when I asked the city to respond to a series of questions on just 22 of the smart projects, it could only answer for five of them, referring me to private companies for the rest of the answers.

The city also keeps track of the number of young people hanging out in the streets, their age group, whether they know each other, the atmosphere and whether or not they cause a nuisance. Special enforcement officers keep track of this information through mobile devices. It calls this process “targeted and innovative supervision”. Other council documents mention the prediction of school drop-outs, the prediction of poverty and the monitoring of “the health of certain groups” with the aim of “intervening faster”.

Like many cities, Utrecht argues that it acts in accordance with privacy laws because it anonymises or pseudonymises data (assigning it a number instead of a name or address). But pseudonymised personal data is still personal data. “The process is not irreversible if the source file is stored,” says Mireille Hildebrandt, professor of ICT and Law at Radboud University. “Moreover, if you build personal profiles and act on them, it is still a violation of privacy and such profiling can – unintentionally – lead to discrimination.” She points to Utrecht’s plan to register the race and health data of prostitutes, which came in for heavy criticism from the Dutch Data Protection Authority.

Another unanswered question regards who owns data that is collected in a public space. Arjen Hof is director of Civity, a company that builds data platforms for governments. “Public authorities are increasingly outsourcing tasks to private companies. Think of waste removal or street lighting,” he says. “But they do not realise that at the same time a lot of data is collected, and do not always make agreements about the ownership of data.”
‘A smart city is a privatised city’

Hof gives the example of CityTec, a company that manages 2,000 car parks, 30,000 traffic lights and 500,000 lamp-posts across the Netherlands. It refused to share with municipalities the data it was collecting through its lamp-post sensors. “Their argument was that, although the municipality is legally owner of the lamp-posts, CityTec is the economic owner and, for competitive reasons, did not want to make the data available,” Hof says. This was three years ago, but for a lot of companies it remains standard practice. Companies dictate the terms, and cities say they can’t share the contracts because it contains “competition-sensitive information”.

When I interviewed the technology writer Evgeny Morozov in October, he warned of cities becoming too dependent on private companies. “The culmination of the smart city is a privatised city,” he said. “A city in which you have to pay for previously free services.”

Morozov’s fear about public subsidies being used for private innovation is well illustrated in Assen, a city of 70,000 people in the north of the country. Assen built a fibre-optic network for super-fast internet in 2011, to which it connected 200 sensors that measure, among other things, the flow of cars. There was an experiment to steer people around traffic jams, even though traffic in the city is relatively light. The city also connected its traffic lights, parking garages and parking signs to this grid. The cost of €46m was split between Brussels, the national government, the province and the municipality. Companies such as the car navigation firm TomTom have used the sensor network to test new services.

The project, called Sensor City, filed for bankruptcy a year ago. Now the publicly funded fibre-optic network, sensors and all, will be sold to a still-unidentified private company. The municipality will have to strike a deal with the new owner about the use of its public traffic lights and parking signs.