Resources

That Game on Your Phone May Be Tracking What You’re Watching on TV

At first glance, the gaming apps — with names like “Pool 3D,” “Beer Pong: Trickshot” and “Real Bowling Strike 10 Pin” — seem innocuous. One called “Honey Quest” features Jumbo, an animated bear.

Yet these apps, once downloaded onto a smartphone, have the ability to keep tabs on the viewing habits of their users — some of whom may be children — even when the games aren’t being played.

It is yet another example of how companies, using devices that many people feel they can’t do without, are documenting how audiences in a rapidly changing entertainment landscape are viewing television and commercials.

The apps use software from Alphonso, a start-up that collects TV-viewing data for advertisers. Using a smartphone’s microphone, Alphonso’s software can detail what people watch by identifying audio signals in TV ads and shows, sometimes even matching that information with the places people visit and the movies they see. The information can then be used to target ads more precisely and to try to analyze things like which ads prompted a person to go to a car dealership.

More than 250 games that use Alphonso software are available in the Google Play store; some are also available in Apple’s app store.

Some of the tracking is taking place through gaming apps that do not otherwise involve a smartphone’s microphone, including some apps that are geared toward children. The software can also detect sounds even when a phone is in a pocket if the apps are running in the background.

807

The shakeup and insanity of “Pokémon Go” in the real world

Since its release Wednesday night, a new game, Pokémon Go has already gone on to become the top-grossing game in the three countries where it’s currently available, adding nearly $11 billion to the value of Nintendo in less than a week.

The game, which “marries a classic 20-year old franchise with augmented reality,” allows players to walk around “real-life” neighbourhoods while seeking “virtual Pokemon game characters” on their smartphone screens. Basically, a glorified fake scavenger hunt, similar to games like Ingress, etc.

In the United States, by July 8–just two days after its release–the game was installed on more than 5 percent of Android devices in the country, is now on more Android phones than dating app Tinder, has daily active users neck and neck with that of social network Twitter, and is also being played an average of 43 minutes a day–more time spent than on WhatsApp or Instagram.”

“Some fans are now tweeting about playing the game while driving, and one user already reports, “Pokemon Go put me in the ER last night… Not even 30 minutes after the release…I slipped and fell down a ditch.” In Australia the game has been leading some players into their local police station, and a woman in Wyoming reports that the game actually led her to a dead body floating in a river. One Pokemon Go screenshot has also gone viral. It shows a man capturing a Pokemon while his wife gives birth…”

The app’s popularity has created lagging servers and forced the company Niantic to delay its international roll-out, meaning “Those who have already downloaded the game in the U.S., Australia and New Zealand can still play it, while those in the U.K., the Netherlands and other countries will have to wait.”

Meanwhile, as people clearly can’t wait, there has been a flood of downloads of unofficial copies of the game, exposing users to hackers who are circulating malicious versions of the game in order to backdoor their devices. “A remote access tool (RAT), known as DroidJack (or SandroRAT), has been added to some APK files, allowing third parties to gain full control over the users’ mobile devices. Permissions granted then include: being able to directly call phone numbers, reading phone status’ and identities, editing and reading text messages, sending SMS messages and recording audio.”

It surely is spurious times…

UPDATE 13/7Holocaust Museum to visitors: Please stop catching Pokemon here. “Playing the game is not appropriate in the museum, which is a memorial to the victims of Nazism,” Andrew Hollinger, the museum’s communications director, said. “We are trying to find out if we can get the museum excluded from the game.”

UPDATE 14/7 “Law enforcement agencies around the globe are reminding citizens to obey trespassing laws and follow common sense when playing Pokemon Go. The new crazy-popular mobile game has led to some frightening results in recent days, such as the location of a dead body and robberies of players in Missouri. Now, San Francisco Police Department Captain Raj Vaswani warned in one online posting for players to “obey traffic laws, please. Do not run into trees, meters, and things that are attached to the sidewalk; they hurt,” he said. “Do not drive or ride your bike / skateboard / hipster techie device while interacting with the app. Know where your kids are going when playing with the app, set limits on where they can go, so they don’t keep going trying to get that Pokemon.”

UPDATE 19/7 — “Pokemon Go is now the biggest mobile game of all time in the United States. Not only has it surpassed Twitter’s daily users, but it is seeing people spend more time in its app than in Facebook. The game also surpassed Tinder in terms of popularity (based on installations) on July 7th.”

UPDATE 29/7 — “It turns out that the stairs of the Internet Archive’s San Francisco headquarters are a PokéGym, a site where players can train their Pokémon and fight with other Pokémon. Fortunately, the Pokémon warriors aren’t rowdy or disruptive; they resemble somnambulistic zombies stumbling around under the control of their glowing smartphone screens.”

UPDATE 8/8How Pokemon Go will make money from you. “Augmented reality games like Ingress and Pokemon Go have the potential to open up a very lucrative new revenue stream based on the acquisition and sale of data – not just personal data, but aggregated spatial data about urban activity patterns. There has already been some controversy about the terms of service for players, which give Niantic access to all manner of data on their phones – including email contacts and social media profiles. This data could potentially be sold to third parties with an interest in targeted advertising. But it is not only individually identifiable personal data that interests Niantic. They are also interested in the spatial data that is generated by Pokemon Go players. As has been widely observed, playing the game rapidly drains phone batteries, because when the game is open your phone is constantly in touch with Niantic servers and providing location information about your movements. […] Niantic is now harvesting “geospatial data” about millions of people’s movements: about how far they are prepared to travel as part of game play; about the kinds of places they stop during game play; about the groups they travel with; and the connections they make during game play, and much more.”

UPDATE 18/8I recently discovered some interesting background to the company Niantic Inc.—the company that developed Pokémon Go and indeed Ingress. The company was formed in 2010 by the founder of Keyhole Inc., John Hanke as “Niantic Labs,” being an internal startup within Google. Niantic left Google in October 2015.

Keyhole Inc., founded in 2001, was a “software development company specialising in geospatial data visualisation applications and was acquired by Google in 2004.” Keyhole was backed by Sony venture capital, NVIDIA and the CIA’s venture capital arm In-Q-Tel, with the majority of In-Q-Tel’ funds coming from the National Geospatial-Intelligence Agency. “Keyhole’s marquee application suite, Earth Viewer, emerged as the highly successful Google Earth application in 2005; other aspects of core technology survive in Google Maps, Google Mobile and the Keyhole Markup Language. The name “Keyhole” is a homage to the KH reconnaissance satellites, the original eye-in-the-sky military reconnaissance system now some 50 years old.”

Just like how now smartphones are the new “eye-in-the-sky” on the ground…

669

Feeding Video Game Data to AIs

Feed the data of millions of people playing various computer games into AI machine learning and shaping algorythms… It’s already happening to an extent:

“The latest computer games can be fantastically realistic. Surprisingly, these lifelike virtual worlds might have some educational value, too—especially for fledgling AI algorithms.

Adrien Gaidon, a computer scientist at Xerox Research Center Europe in Grenoble, France, remembers watching someone play the video game Assassins Creed when he realized that the game’s photo-realistic scenery might offer a useful way to teach AI algorithms about the real world. Gaidon is now testing this idea by developing highly realistic 3-D environments for training algorithms how to recognize particular real-world objects or scenarios.

The idea is important because cutting-edge AI algorithms need to feed on huge quantities of data in order to learn to perform a task. Sometimes, that isn’t a problem. Facebook, for instance, has millions of labeled photographs with which to train the algorithms that automatically tag friends in uploading images (see “Facebook Creates Software that Matches Faces Almost as Well as You Do”). Likewise, Google is capturing huge amounts of data using its self-driving cars, which is then used to refine the algorithms that control those vehicles.

But most companies do not have access to such enormous data sets, or the means to generate such data from scratch.

To fill in those gaps, Gaidon and colleagues used a popular game development engine, called Unity, to generate virtual scenes for training deep-learning algorithms—a very large type of simulated neural network—to recognize objects and situations in real images. Unity is widely used to make 3-D video games, and many common objects are available to developers to use in their creations.

A paper describing the Xerox team’s work will be presented at a computer vision conference later this year. By creating a virtual setting, and letting an algorithm see lots of variations from different angles and with different lighting, it’s possible to teach that algorithm to recognize the same object in real images or video footage. “The nice thing about virtual worlds is you can create any kind of scenario,” Gaidon says.

Gaidon’s group also devised a way to convert a real scene into a virtual one by using a laser scanner to capture a scene in 3-D and then importing that information into the virtual world. The group was able to measure the accuracy of the approach by comparing algorithms trained within virtual environments with ones trained using real images annotated by people. “The benefits of simulation are well known,” he says, “but [we wondered], can we generate virtual reality that can fool an AI?”

The Xerox researchers hope to apply the technique in two situations. First, they plan to use it to find empty parking spots on the street using cameras fitted to buses. Normally doing this would involve collecting lots of video footage, and having someone manually annotate empty spaces. A huge amount of training data can be generated automatically using the virtual environment created by the Xerox team. Second, they are exploring whether it could be used to learn about medical issues using virtual hospitals and patients.

The challenge of learning with less data is well known among computer scientists, and it is inspiring many researchers to explore new approaches, some of which take their inspiration from human learning (see “Can This Man Make AI More Human?”).

“I think this is a very good idea,” says Josh Tenenbaum, a professor of cognitive science and computation at MIT, of the Xerox project. “It’s one that we and many others have been pursuing in different forms.”

636

“How entitled children are making their parents’ lives hell”

“Sons are smashing windows, furious they’re asked to stop playing computer games. Doors are hanging off hinges having been slammed so hard in a fit of pique. Teenagers are holding knives to their mother’s throat, or threatening to kill themselves.

This is the pointy end of entitlement, the defining characteristic of this generation of children.

[…]

Mental health issues predicted

Kids who grow up insulated from difficulty and disappointment are also likely to struggle in adulthood if they don’t get into their first preference for uni, miss out on a job, or are dumped by the love of their life.”

624