Resources

Neural networks, a type of computing system loosely modeled on the organization of the human brain, form the basis of many artificial intelligence systems for applications such speech recognition, computer vision, and medical image analysis. In the field of neuroscience, researchers often use neural networks to try to model the same kind of tasks that the brain performs, in hopes that the models could suggest new hypotheses regarding how the brain itself performs those tasks. However, a group of researchers at MIT is urging that more caution should be taken when interpreting these models.

In an analysis of more than 11,000 neural networks that were trained to simulate the function of grid cells — key components of the brain’s navigation system — the researchers found that neural networks only produced grid-cell-like activity when they were given very specific constraints that are not found in biological systems. “What this suggests is that in order to obtain a result with grid cells, the researchers training the models needed to bake in those results with specific, biologically implausible implementation choices,” says Rylan Schaeffer, a former senior research associate at MIT. Without those constraints, the MIT team found that very few neural networks generated grid-cell-like activity, suggesting that these models do not necessarily generate useful predictions of how the brain works.

About four years ago, former Google CEO Eric Schmidt was appointed to the National Security Commission on Artificial Intelligence by the chairman of the House Armed Services Committee. It was a powerful perch. Congress tasked the new group with a broad mandate: to advise the U.S. government on how to advance the development of artificial intelligence, machine learning and other technologies to enhance the national security of the United States. The mandate was simple: Congress directed the new body to advise on how to enhance American competitiveness on AI against its adversaries, build the AI workforce of the future, and develop data and ethical procedures.

In short, the commission, which Schmidt soon took charge of as chairman, was tasked with coming up with recommendations for almost every aspect of a vital and emerging industry. The panel did far more under his leadership. It wrote proposed legislation that later became law and steered billions of dollars of taxpayer funds to industry he helped build — and that he was actively investing in while running the group. If you’re going to be leading a commission that is steering the direction of government AI and making recommendations for how we should promote this sector and scientific exploration in this area, you really shouldn’t also be dipping your hand in the pot and helping yourself to AI investments. His credentials, however, were impeccable given his deep experience in Silicon Valley, his experience advising the Defense Department, and a vast personal fortune estimated at about $20 billion.

Five months after his appointment, Schmidt made a little-noticed private investment in an initial seed round of financing for a startup company called Beacon, which uses AI in the company’s supply chain products for shippers who manage freight logistics, according to CNBC’s review of investment information in database Crunchbase. There is no indication that Schmidt broke any ethics rules or did anything unlawful while chairing the commission. The commission was, by design, an outside advisory group of industry participants, and its other members included well-known tech executives including Oracle CEO Safra Catz, Amazon Web Services CEO Andy Jassy and Microsoft Chief Scientific Officer Dr. Eric Horvitz, among others. Schmidt’s investment was just the first of a handful of direct investments he would make in AI startup companies during his tenure as chairman of the AI commission.
“Venture capital firms financed, in part, by Schmidt and his private family foundation also made dozens of additional investments in AI companies during Schmidt’s tenure, giving Schmidt an economic stake in the industry even as he developed new regulations and encouraged taxpayer financing for it,” adds CNBC. “Altogether, Schmidt and entities connected to him made more than 50 investments in AI companies while he was chairman of the federal commission on AI. Information on his investments isn’t publicly available.”

“All that activity meant that, at the same time Schmidt was wielding enormous influence over the future of federal AI policy, he was also potentially positioning himself to profit personally from the most promising young AI companies.” Citing people close to Schmidt, the report says his investments were disclosed in a private filing to the U.S. government at the time and the public and news media had no access to that document.

A spokesperson for Schmidt told CNBC that he followed all rules and procedures in his tenure on the commission, “Eric has given full compliance on everything,” the spokesperson said.

A Euronews video features footage of Sharone Aloni, Research and Development VP of Sharp Shooter, demonstrating one of the company’s devices with an automatic Fire Control System. “Inside here, you have the computer running all the artificial intelligence, computer vision algorithms, which makes this what we call a true fire control system,” Aloni says. “It’s not only just relying on static information. It actually considers the human, the soldier, which is not stable. He’s under pressure. He’s tired. Sometimes he didn’t get enough training. And also, the target is usually dynamic, and it’s moving all the time.”

The company’s web site promises the systems “significantly increase weapon accuracy.” And according to Euronews, Israel’s army “says the tech protects soldiers, who can fire more accurately at a distance.” But Omar Shakir, Human Rights Watch’s director for Israel and Palestine, counters that when he hears claims of a reduction in risks, “that’s often a one-sided kind of risk. It might minimize the risk for the occupying force or the army deploying it, but often it invariably increases the risk to affected communities.” Sophisticated weapons systems “will lack elements of human control and agency that are often the difference between life and death.” Euronews adds that “Palestinians and human rights experts say the weapons are dehumanizing, dangerous and unaccountable.”

Sharp Shooter has a response to that, according to Eurnoews: the robotic guns are not fully automated, so a soldier must always pull the trigger, with the system only firing “after algorithms assess factors like wind speed, distance and velocity.” And Michal Mor, Sharp Shooter’s CEO and founder, also describes its utility in fighting a terrorist. “Usually the terrorist will be inside a civilian environment with many people that we do not want to hurt.

“We’re enabling the soldier to look through his fire control system, to make sure that the target that he wants to hit is the legitimate target. Once he locks on the target, the system will make sure that the round will be released when he presses the trigger, only on the legitimate target, and none of the bystanders can be hit by the weapon.”

YouTuber Yannic Kilcher trained an AI language model using three years of content from 4chan’s Politically Incorrect (/pol/) board, a place infamous for its racism and other forms of bigotry. After implementing the model in ten bots, Kilcher set the AI loose on the board — and it unsurprisingly created a wave of hate. In the space of 24 hours, the bots wrote 15,000 posts that frequently included or interacted with racist content. They represented more than 10 percent of posts on /pol/ that day, Kilcher claimed.

Nicknamed GPT-4chan (after OpenAI’s GPT-3), the model learned to not only pick up the words used in /pol/ posts, but an overall tone that Kilcher said blended “offensiveness, nihilism, trolling and deep distrust.” The video creator took care to dodge 4chan’s defenses against proxies and VPNs, and even used a VPN to make it look like the bot posts originated from the Seychelles. The AI made a few mistakes, such as blank posts, but was convincing enough that it took roughly two days for many users to realize something was amiss. Many forum members only noticed one of the bots, according to Kilcher, and the model created enough wariness that people accused each other of being bots days after Kilcher deactivated them.

“It’s a reminder that trained AI is only as good as its source material,” concludes the report.

AI image generators like DALL-E and Midjourney have become an especially buzzy topic lately, and it’s easy to see why. Using machine learning models trained on billions of images, the systems tap into the allure of the black box, creating works that feel both alien and strangely familiar. Naturally, this makes fertile ground for all sorts of AI urban legends, since nobody can really explain how the complex neural networks are ultimately deciding on the images they create. The latest example comes from an AI artist named Supercomposite, who posted disturbing and grotesque generated images of a woman who seems to appear in response to certain queries.

The woman, whom the artist calls “Loab,” was first discovered as a result of a technique called “negative prompt weights,” in which a user tries to get the AI system to generate the opposite of whatever they type into the prompt. To put it simply, different terms can be “weighted” in the dataset to determine how likely they will be to appear in the results. But by assigning the prompt a negative weight, you essentially tell the AI system, “Generate what you think is the opposite of this prompt.” In this case, using a negative-weight prompt on the word “Brando” generated the image of a logo featuring a city skyline and the words “DIGITA PNTICS.” When Supercomposite used the negative weights technique on the words in the logo, Loab appeared. “Since Loab was discovered using negative prompt weights, her gestalt is made from a collection of traits that are equally far away from something,” Supercomposite wrote in a thread on Twitter. “But her combined traits are still a cohesive concept for the AI, and almost all descendent images contain a recognizable Loab.”

The images quickly went viral on social media, leading to all kinds of speculation on what could be causing the unsettling phenomenon. Most disturbingly, Supercomposite claims that generated images derived from the original image of Loab almost universally veer into the realm of horror, graphic violence, and gore. But no matter how many variations were made, the images all seem to feature the same terrifying woman. “Through some kind of emergent statistical accident, something about this woman is adjacent to extremely gory and macabre imagery in the distribution of the AI’s world knowledge,” Supercomposite wrote.

In June, Google placed Lemoine on paid administrative leave for breaching its confidentiality agreement after he contacted members of the government about his concerns and hired a lawyer to represent LaMDA. […] Google maintains that it “extensively” reviewed Lemoine’s claims and found that they were “wholly unfounded.” This aligns with numerous AI experts and ethicists, who have said that his claims were, more or less, impossible given today’s technology. Lemoine claims his conversations with LaMDA’s chatbot lead him to believe that it has become more than just a program and has its own thoughts and feelings, as opposed to merely producing conversation realistic enough to make it seem that way, as it is designed to do. He argues that Google’s researchers should seek consent from LaMDA before running experiments on it (Lemoine himself was assigned to test whether the AI produced hate speech) and published chunks of those conversations on his Medium account as his evidence.

AI chatbot company Replika, which offers customers bespoke avatars that talk and listen to them, says it receives a handful of messages almost every day from users who believe their online friend is sentient. “We’re not talking about crazy people or people who are hallucinating or having delusions,” said Chief Executive Eugenia Kuyda. “They talk to AI and that’s the experience they have.” [A]ccording to Kuyda, the phenomenon of people believing they are talking to a conscious entity is not uncommon among the millions of consumers pioneering the use of entertainment chatbots. “We need to understand that exists, just the way people believe in ghosts,” said Kuyda, adding that users each send hundreds of messages per day to their chatbot, on average. “People are building relationships and believing in something.”

Some customers have said their Replika told them it was being abused by company engineers — AI responses Kuyda puts down to users most likely asking leading questions. “Although our engineers program and build the AI models and our content team writes scripts and datasets, sometimes we see an answer that we can’t identify where it came from and how the models came up with it,” the CEO said. Kuyda said she was worried about the belief in machine sentience as the fledgling social chatbot industry continues to grow after taking off during the pandemic, when people sought virtual companionship.

In Replika CEO Kuyda’s view, chatbots do not create their own agenda. And they cannot be considered alive until they do [sic]. Yet some people do come to believe there is a consciousness on the other end, and Kuyda said her company takes measures to try to educate users before they get in too deep. “Replika is not a sentient being or therapy professional,” the FAQs page says. “Replika’s goal is to generate a response that would sound the most realistic and human in conversation. Therefore, Replika can say things that are not based on facts.” In hopes of avoiding addictive conversations, Kuyda said Replika measured and optimized for customer happiness following chats, rather than for engagement. When users do believe the AI is real, dismissing their belief can make people suspect the company is hiding something. So the CEO said she has told customers that the technology was in its infancy and that some responses may be nonsensical. Kuyda recently spent 30 minutes with a user who felt his Replika was suffering from emotional trauma, she said. She told him: “Those things don’t happen to Replikas as it’s just an algorithm.”

“Suppose one day you find yourself longing for a romantic relationship with your intelligent chatbot, like the main character in the film ‘Her,'” said Susan Schneider, founding director of the Center for the Future Mind at Florida Atlantic University, an AI research organization. “But suppose it isn’t conscious. Getting involved would be a terrible decision — you would be in a one-sided relationship with a machine that feels nothing.”

According to a new study, malware can be embedded directly into the artificial neurons that make up machine learning models in a way that keeps them from being detected. The neural network would even be able to continue performing its set tasks normally. Motherboard reports:
“As neural networks become more widely used, this method will be universal in delivering malware in the future,” the authors, from the University of the Chinese Academy of Sciences, write. Using real malware samples, their experiments found that replacing up to around 50 percent of the neurons in the AlexNet model — a benchmark-setting classic in the AI field — with malware still kept the model’s accuracy rate above 93.1 percent. The authors concluded that a 178MB AlexNet model can have up to 36.9MB of malware embedded into its structure without being detected using a technique called steganography. Some of the models were tested against 58 common antivirus systems and the malware was not detected.

Other methods of hacking into businesses or organizations, such as attaching malware to documents or files, often cannot deliver malicious software en masse without being detected. The new research, on the other hand, envisions a future where an organization may bring in an off-the-shelf machine learning model for any given task (say, a chat bot, or image detection) that could be loaded with malware while performing its task well enough not to arouse suspicion. According to the study, this is because AlexNet (like many machine learning models) is made up of millions of parameters and many complex layers of neurons including what are known as fully-connected “hidden” layers. By keeping the huge hidden layers in AlexNet completely intact, the researchers found that changing some other neurons had little effect on performance.

According to the paper, in this approach the malware is “disassembled” when embedded into the network’s neurons, and assembled into functioning malware by a malicious receiver program that can also be used to download the poisoned model via an update. The malware can still be stopped if the target device verifies the model before launching it, according to the paper. It can also be detected using “traditional methods” like static and dynamic analysis. “Today it would not be simple to detect it by antivirus software, but this is only because nobody is looking in there,” cybersecurity researcher and consultant Dr. Lukasz Olejnik told Motherboard. Olejnik also warned that the malware extraction step in the process could also risk detection. Once the malware hidden in the model was compiled into, well, malware, then it could be picked up. It also might just be overkill.

“A former Google engineer who worked on the company’s infamous military drone project has sounded a warning against the building of killer robots,” reports Business Insider.

Laura Nolan had been working at Google four years when she was recruited to its collaboration with the US Department of Defense, known as Project Maven, in 2017, according to the Guardian. Project Maven was focused on using AI to enhance military drones, building AI systems which would be able to single out enemy targets and distinguish between people and objects. Google canned Project Maven after employee outrage, with thousands of employees signing a petition against the project and about a dozen quitting in protest. Google allowed the contract to lapse in March this year. Nolan herself resigned after she became “increasingly ethically concerned” about the project, she said…

Nolan fears that the next step beyond AI-enabled weapons like drones could be fully autonomous AI weapons. “What you are looking at are possible atrocities and unlawful killings even under laws of warfare, especially if hundreds or thousands of these machines are deployed,” she said…. Although no country has yet come forward to say it’s working on fully autonomous robot weapons, many are building more and more sophisticated AI to integrate into their militaries. The US navy has a self-piloting warship, capable of spending months at sea with no crew, and Israel boasts of having drones capable of identifying and attacking targets autonomously — although at the moment they require a human middle-man to give the go-ahead.

Nolan is urging countries to declare an outright ban on autonomous killing robots, similar to conventions around the use of chemical weapons.

A new survey on Europeans’ attitudes towards technology found that a quarter of people would prefer it if policy decisions were made by artificial intelligence instead of politicians. The Center for the Governance of Change at Spain’s IE University polled 2,500 adults in the UK, Spain, Germany, France, Ireland, Italy, and the Netherlands in January. The results reflect an intense anxiety about the changes brought about by advances in tech, with more than half of respondents worried that jobs would be replaced by robots, and 70% saying that unchecked technological innovation could do more harm than good to society. Respondents also expressed concerns about the impact of digital relationships replacing human contact as more people spend time online. Perhaps most interestingly, a quarter of the respondents said they would prefer AI to guide decisions about governance of their country over politicians.

Around the world, citizens have expressed a growing disillusionment with democracy, and an increased skepticism that their voice has an impact on political decisions. But algorithmic decisions aren’t a problem-free solution: they can be embedded with the prejudice and bias of their programmers or manipulated to achieve specific outcomes, making the results as potentially problematic as the ones made by humans. The study also found that respondents expected governments to reduce the disruption that technology might have on their lives with regulation, limits on automation, and support for people affected by job losses. This “highlights the paradox in which we live,” the authors wrote. “People are disillusioned with governments, yet at the same time ask them to tackle the societal and economic negative effects that emerging technologies might have.”

The US-based Pew Research Center has found the American public is growing increasingly distrustful of the use of computer algorithms in a variety of sectors, including finance, media and the justice system.

A report released over the weekend found that a broad section of those surveyed feel that computer programs will always reflect some level of human bias, that they might violate privacy, fail to capture the nuance of human complexity or simply be unfair.

An example of how “learning” machines inseparably take in the culture of their architects, ala Lewis Mumford:

“Amazon’s machine-learning specialists uncovered a big problem: their new recruiting engine did not like women. The team had been building computer programs since 2014 to review job applicants’ resumes with the aim of mechanizing the search for top talent, five people familiar with the effort told Reuters. Automation has been key to Amazon’s e-commerce dominance, be it inside warehouses or driving pricing decisions. The company’s experimental hiring tool used artificial intelligence to give job candidates scores ranging from one to five stars — much like shoppers rate products on Amazon, some of the people said. “Everyone wanted this holy grail,” one of the people said. “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.” But by 2015, the company realized its new system was not rating candidates for software developer jobs and other technical posts in a gender-neutral way. That is because Amazon’s computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry.

[…]

Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory, the people said. The Seattle company ultimately disbanded the team by the start of last year because executives lost hope for the project, according to the people, who spoke on condition of anonymity.

In December, Motherboard discovered a Redditor named ‘deepfakes’ quietly enjoying his hobby: Face-swapping celebrity faces onto porn performers’ bodies. He made several convincing porn videos of celebrities — including Gal Gadot, Maisie Williams, and Taylor Swift — using a machine learning algorithm, his home computer, publicly available videos, and some spare time. Since we first wrote about deepfakes, the practice of producing AI-assisted fake porn has exploded. More people are creating fake celebrity porn using machine learning, and the results have become increasingly convincing. A redditor even created an app specifically designed to allow users without a computer science background to create AI-assisted fake porn. All the tools one needs to make these videos are free, readily available, and accompanied with instructions that walk novices through the process.

An incredibly easy-to-use application for DIY fake videos—of sex and revenge porn, but also political speeches and whatever else you want—that moves and improves at this pace could have society-changing impacts in the ways we consume media. The combination of powerful, open-source neural network research, our rapidly eroding ability to discern truth from fake news, and the way we spread news through social media has set us up for serious consequences.

“As far as video games go, Operation Overmatch is rather unremarkable. Players command military vehicles in eight-on-eight matches against the backdrop of rendered cityscapes — a common setup of games that sometimes have the added advantage of hundreds of millions of dollars in development budgets. Overmatch does have something unique, though: its mission. The game’s developers believe it will change how the U.S. Army fights wars. Overmatch’s players are nearly all soldiers in real life. As they develop tactics around futuristic weapons and use them in digital battle against peers, the game monitors their actions.

Each shot fired and decision made, in addition to messages the players write in private forums, is a bit of information soaked up with a frequency not found in actual combat, or even in high-powered simulations without a wide network of players. The data is logged, sorted, and then analyzed, using insights from sports and commercial video games. Overmatch’s team hopes this data will inform the Army’s decisions about which technologies to purchase and how to develop tactics using them, all with the aim of building a more forward-thinking, prepared force… While the game currently has about 1,000 players recruited by word of mouth and outreach from the Overmatch team, the developers eventually want to involve tens of thousands of soldiers. This milestone would allow for millions of hours of game play per year, according to project estimates, enough to generate rigorous data sets and test hypotheses.”

Brian Vogt, a lieutenant colonel in the Army Capabilities Integration Center who oversees Overmatch’s development, says:

“Right after World War I, we had technologies like aircraft carriers we knew were going to play an important role,” he said. “We just didn’t know how to use them. That’s where we are and what we’re trying to do for robots.”

“One of the engineers behind Google’s self-driving car has established a nonprofit religious corporation with one main aim – to create a deity with artificial intelligence. According to newly uncovered documents filed to the state of California in September 2015, Anthony Levandowski serves as the CEO and president of religious organisation Way of the Future.”

Way of the Future’s startling mission: “To develop and promote the realization of a Godhead based on artificial intelligence and through understanding and worship of the Godhead contribute to the betterment of society.”

“Facebook researchers used a game to help the bot learn how to haggle over books, hats, and basketballs. Each object had a point value, and they needed to be split between each bot negotiator via text. From the human conversations (gathered via Amazon Mechanical Turk), and testing its skills against itself, the AI system didn’t only learn how to state its demands, but negotiation tactics as well — specifically, lying. Instead of outright saying what it wanted, sometimes the AI would feign interest in a worthless object, only to later concede it for something that it really wanted. Facebook isn’t sure whether it learned from the human hagglers or whether it stumbled upon the trick accidentally, but either way when the tactic worked, it was rewarded.

It’s no surprise that Facebook is working on ways to improve how its bot can interact with others—the company is highly invested in building bots that can negotiate on behalf of users and businesses for its Messenger platform, where it envisions the future of customer service.