Connect with us

A new tool lets artists add invisible changes to the pixels in their art before they upload it online so that if it’s scraped into an AI training set, it can cause the resulting model to break in chaotic and unpredictable ways. 

The tool, called Nightshade, is intended as a way to fight back against AI companies that use artists’ work to train their models without the creator’s permission. Using it to “poison” this training data could damage future iterations of image-generating AI models, such as DALL-E, Midjourney, and Stable Diffusion, by rendering some of their outputs useless—dogs become cats, cars become cows, and so forth. MIT Technology Review got an exclusive preview of the research, which has been submitted for peer review at computer security conference Usenix.

AI companies such as OpenAI, Meta, Google, and Stability AI are facing a slew of lawsuits from artists who claim that their copyrighted material and personal information was scraped without consent or compensation. Ben Zhao, a professor at the University of Chicago, who led the team that created Nightshade, says the hope is that it will help tip the power balance back from AI companies towards artists, by creating a powerful deterrent against disrespecting artists’ copyright and intellectual property. Meta, Google, Stability AI, and OpenAI did not respond to MIT Technology Review’s request for comment on how they might respond. 

Zhao’s team also developed Glaze, a tool that allows artists to “mask” their own personal style to prevent it from being scraped by AI companies. It works in a similar way to Nightshade: by changing the pixels of images in subtle ways that are invisible to the human eye but manipulate machine-learning models to interpret the image as something different from what it actually shows.

The team intends to integrate Nightshade into Glaze, and artists can choose whether they want to use the data-poisoning tool or not. The team is also making Nightshade open source, which would allow others to tinker with it and make their own versions. The more people use it and make their own versions of it, the more powerful the tool becomes, Zhao says. The data sets for large AI models can consist of billions of images, so the more poisoned images can be scraped into the model, the more damage the technique will cause.

targeted attack

Nightshade exploits a security vulnerability in generative AI models, one arising from the fact that they are trained on vast amounts of data—in this case, images that have been hoovered from the internet. Nightshade messes with those images.

Artists who want to upload their work online but don’t want their images to be scraped by AI companies can upload them to Glaze and choose to mask it with an art style different from theirs. They can then also opt to use Nightshade. Once AI developers scrape the internet to get more data to tweak an existing AI model or build a new one, these poisoned samples make their way into the model’s data set and cause it to malfunction.

Poisoned data samples can manipulate models into learning, for example, that images of hats are cakes, and images of handbags are toasters. The poisoned data is very difficult to remove, as it requires tech companies to painstakingly find and delete each corrupted sample.

The researchers tested the attack on Stable Diffusion’s latest models and on an AI model they trained themselves from scratch. When they fed Stable Diffusion just 50 poisoned images of dogs and then prompted it to create images of dogs itself, the output started looking weird—creatures with too many limbs and cartoonish faces. With 300 poisoned samples, an attacker can manipulate Stable Diffusion to generate images of dogs to look like cats.

A table showing a grid of thumbnails of generated images of Hemlock attack-poisoned concepts from SD-XL models contrasted with images from the clean SD-XL model in increments of 50, 100, and 300 poisoned samples.
COURTESY OF THE RESEARCHERS

Generative AI models are excellent at making connections between words, which helps the poison spread. Nightshade infects not only the word “dog” but all similar concepts, such as “puppy,” “husky,” and “wolf.” The poison attack also works on tangentially related images.

Read More

————

By: Melissa Heikkilä
Title: This new data poisoning tool lets artists fight back against generative AI
Sourced From: www.technologyreview.com/2023/10/23/1082189/data-poisoning-artists-fight-generative-ai/
Published Date: Mon, 23 Oct 2023 17:26:40 +0000

Continue Reading

Tech

The Download: Big Tech’s climate claims, and reducing your music streaming carbon footprint

1f408 200d 2b1b

This is today’s edition of The Download our weekday newsletter that provides a daily dose of what’s going on in the world of technology.

Google, Amazon and the problem with Big Tech’s climate claims

Last week, Amazon trumpeted that it had purchased enough clean electricity to cover the energy demands of all its global operations, seven years ahead of its sustainability target.

That news closely followed Google’s acknowledgment that the soaring energy demands of its AI operations helped ratchet up its corporate emissions by 13% last year—and that it had backed away from claims that it was already carbon neutral.

If you were to take the announcements at face value, you’d be forgiven for believing that Google is stumbling while Amazon is speeding ahead in the race to clean up climate pollution.

But while both companies are coming up short in their own ways, Google’s approach to driving down greenhouse-gas emissions is now arguably more defensible. To learn why, read our story.

—James Temple

This piece is part of MIT Technology Review Explains, our series untangling the complex, messy world of technology to help you understand what’s coming next. You can read more from the series here

Five ways to make music streaming better for the climate

As K-pop sweeps the world and accumulates a massive, devout fan base, these fans have been turning their power into action. Zeyi Yang, our China reporter, recently published a story about Kpop4planet, a group of activists who are using K-pop’s influence to hold large corporations accountable for their carbon footprints.

During his reporting, he talked to several experts about how to correctly understand the climate impact of music streaming, and one thing became clear: It all comes down to how we stream—the content, the device, the length, etc. Read on for their tips to help any music streaming user leave a smaller carbon footprint.

This story is from China Report, our weekly newsletter examining the relationship between tech and power in the country. Sign up to receive it in your inbox every Tuesday.

The must-reads

I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.

1 Donald Trump’s allies are already working on a sweeping AI order 
Which many AI investors in Silicon Valley would favor over President Biden’s approach. (WP $)
Elon Musk is among the first big names in tech to pledge support for Trump. (WSJ $)
Trump’s former FDA commissioner wants to peer inside AI’s black boxes. (Politico)

2 TikTok’s attempt to swerve the EU’s Digital Markets Act has been dismissed
The EU’s General Court ruled TikTok was powerful enough to have to comply. (Bloomberg $)
It’s good news for European antitrust regulators. (Reuters)
Here’s what you need to know about the Digital Markets Act. (MIT Technology Review)

3 Bitcoin miners are signing deals with AI firms
Putting all those vast data centers to good use. (FT $)
How Bitcoin mining devastated this New York town. (MIT Technology Review)

4 Amazon’s Prime Day sale causes a spike in injuries among warehouse workers
A new report accuses the company of prioritizing speed over safety. (WSJ $)
Not everything that looks like a deal is, in fact, a deal. (The Atlantic $)

5 We’re learning more about how deadly pancreatic cancer spreads
The disease shuts down molecules in key genes. (The Guardian)
An AI-based risk prediction system could help catch pancreatic cancer cases earlier. (MIT Technology Review)’

6 The Milky Way is full of free-floating planets
These scientists are on a mission to track these rogue worlds down. (IEEE Spectrum)

7 Beware the rise of fake AI-powered therapists
It’s just one example among a rising wave of AI scams. (Vice)
Five ways criminals are using AI. (MIT Technology Review)

8 Black women are listing their race as white on dating apps
And report receiving higher-quality matches as a result. (NY Mag $)

9 The JWST just celebrated its second year in space
And the photographs it captures are still awe-inspiring. (The Atlantic $)

10 Lab-grown meat for pets has been green-lit in the UK
🐈‍⬛

🐕
For the discerning pet palate. (Wired $)
Here’s what a lab-grown burger tastes like. (MIT Technology Review)

Quote of the day

“This country is on fire, Mr. Altman.”

—Jennifer Loving, who runs a nonprofit that administers basic-income pilot programs in Silicon Valley, tells OpenAI CEO Sam Altman it’s time to act on all the research

Read More

————

By: Rhiannon Williams
Title: The Download: Big Tech’s climate claims, and reducing your music streaming carbon footprint
Sourced From: www.technologyreview.com/2024/07/17/1095061/the-download-big-techs-climate-claims-and-reducing-your-music-streaming-carbon-footprint/
Published Date: Wed, 17 Jul 2024 12:10:00 +0000

Continue Reading

Tech

Music streaming can be a drag on the environment. These K-pop fans want to clean it up.

On Valentine’s Day 2023, five K-pop fans came to a bustling street in the center of Seoul, one of them in a bee costume. Then they started dancing to “Candy” by the boy band NCT Dream and unfurled a banner with a message for Korea’s largest domestic music streaming platform: “Melon, let’s use 100% renewable energy and happily be together with Kpop for the next 100 years.”

KPOP4PLANET

A few weeks later, Melon, which has over 4 million active users in Korea, promised to do just that—pledging to adopt 100% renewable energy for its data centers by 2030.

It was the culmination of a campaign organized by Kpop4planet, a small group of volunteers that is achieving surprising success in mobilizing K-pop fans to act against the energy-intensive practices of the music industry. In recent years it has led a series of actions for climate causes, secured pledges to reduce the carbon footprint of music streaming, and pressured international brands to turn their supply chains away from fossil fuels.

K-pop fans have for years been known for their incredible organizing power. As their numbers have grown around the world, they have become influential political forces, shaping elections and advocating for social change. It was these actions that inspired two young fans, Dayeon Lee from South Korea and Nurul Sarifah from Indonesia, to found Kpop4planet in 2021. Particularly concerned about environmental issues, they began to think about how some aspects of K-pop culture can exacerbate environmental degradation. For example, excessive music streaming can generate carbon emissions at every step, from the data centers that process requests to the devices that play the music. 

“I [initially] thought the physical-album-waste issue was much more important,” says Lee, who is a 21-year-old university undergraduate, currently living in Japan. “But I was really surprised when I did some background research … [and] realized that the streaming issue is much more serious because it is a long-term issue.”

While producing and selling physical recordings does, of course, have a carbon footprint, most of the environmental issues end after the initial purchase. That’s not the case with digital distribution. Streaming an album more than 27 times, according to 2019 research at Keele University in the UK, will likely end up using more energy than it takes to produce a CD. This kind of listening happens frequently in K-pop culture, which often encourages fans to host “streaming parties” where they play the same song on repeat.

Buoyed by the success of its streaming campaign, Kpop4planet has recently targeted companies outside the music industry that have benefited from working with K-pop idols; it’s asked them to make similar pledges on renewable energy or other climate goals in order to secure continuous support from the fans. The group has put pressure on Tokopedia, Indonesia’s largest e-commerce company, to set up a decarbonization plan. And it’s gone after Hyundai—which uses the K-pop band BTS as brand ambassadors—over a business deal to source aluminum from a company relying on a new coal power plant. This led to another big victory: In March 2024, Hyundai agreed to seek alternative suppliers for its aluminum.

These wins may be surprising for a group with just 10 full-time members. Hyundai and Melon did not immediately respond to requests for comment, so it’s hard to know exactly why they changed course. But for her part, Lee believes the group’s success comes from how it is able to represent the genuine feelings of a massive fan base and draw companies’ attention to those demands. In total, Kpop4planet’s online petitions have collected signatures from nearly 60,000 fans in 223 countries. And the group doesn’t stop until it gets what it wants.

“We have to be the messenger between corporations and K-pop fans,” Lee says. “We also want to expand our campaigns to more global corporations, because we believe that K-pop fans have enough power and influence to make our society more sustainable.”

The carbon footprint of “streaming parties”

Even as streaming has become the dominant way to listen to music, its energy consumption—in faraway data centers or via invisible telecommunication transmissions—remains hard for the end user to recognize.

“I think streaming is especially nefarious because those negative impacts are happening so far away and in such an invisible way,” says Joe Steinhardt, an assistant professor at Drexel University in Philadelphia who studies the music industry and is the author of the book Why to Resist Streaming Music & How. He calls streaming music “a disposable listen” because of the way an app keeps pulling data from the cloud and not storing it locally.

Still, it’s hard to draw a definitive conclusion on whether streaming damages the environment more than buying physical copies; its actual carbon

Read More

————

By: Zeyi Yang
Title: Music streaming can be a drag on the environment. These K-pop fans want to clean it up.
Sourced From: www.technologyreview.com/2024/07/16/1094982/music-streaming-climate-kpop-fan/
Published Date: Tue, 16 Jul 2024 09:00:00 +0000

Continue Reading

Tech

The Download: K-pop stans’ climate plans, and what AI isn’t

1f0cf

This is today’s edition of The Download our weekday newsletter that provides a daily dose of what’s going on in the world of technology.

Music streaming can be a drag on the environment. These K-pop fans want to clean it up.

K-pop fans have for years been known for their incredible organizing power. As their numbers have grown around the world, they have become influential political forces, shaping elections and advocating for social change.

It was these actions that inspired Kpop4planet. It’s a small group of volunteers that is achieving surprising success in mobilizing K-pop fans to act against the energy-intensive practices of the music streaming industry.

And, buoyed by its success, Kpop4planet has started targeting companies outside the music industry; it’s asked them to make similar pledges on renewable energy or other climate goals. Read the full story.

—Zeyi Yang

A short history of AI, and what it is (and isn’t)

It’s the simplest questions that are often the hardest to answer. That applies to AI, too. Even though it’s sold as a solution to the world’s problems, nobody seems to know what it really is.

For months, my colleague Will Douglas Heaven has been on a quest to go deeper to understand why everybody seems to disagree on exactly what AI is, and why you’re right to care about it.

He’s been talking to some of the top thinkers in the field, asking them, simply: What is AI? The end result is a great piece that looks at the past and present of AI to see where it is going next. Read the full story.

—Melissa Heikkilä

This story is from The Algorithm, our weekly AI newsletter. Sign up to receive it in your inbox every Monday.

The must-reads

I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.

1 Silicon Valley is backing Donald Trump to become US President
Major VCs and tech leaders are lining up to pledge their financial support. (FT $)
JD Vance, Trump’s VP candidate, used to be a VC himself. (TechCrunch)
Wealthy far right activists are preparing for Trump to win. (New Yorker $)
The FBI has gained access to the phone of the suspected Trump shooter. (404 Media)

2 This site sells selfie ID verification photos and videos
Allowing customers to sign up for accounts using other people’s likenesses. (404 Media)

3 Bird flu cases could be undetected among US dairy workers
Health officials are struggling to keep track of who has been exposed. (New Scientist $)
What’s next for bird flu vaccines. (MIT Technology Review)

4 Scientists have discovered a cave on the moon
It could serve as a base for astronauts to shelter from radiation. (BBC)
It could be part of a hidden network of lunar caves. (New Scientist $)

5 China’s state support for AI is a double-edged sword
Its robust regulatory regime forces startups to jump through hoops. (WSJ $)
Critics aren’t happy about the EU’s new rules for AI. (FT $)
Why the Chinese government is sparing AI from harsh regulations—for now. (MIT Technology Review)

6 Google tried to ruin a pact between EU cloud firms and Microsoft
It offered the firms a $512 million package to uphold a complaint against Microsoft—but failed in its endeavor. (Bloomberg $)

7 Cloaking healthy cells could protect them from intensive cancer treatments
Drugs and therapies usually target all cells indiscriminately. (Ars Technica)
Cancer vaccines are having a renaissance. (MIT Technology Review)

8 Artists in Latin America can’t opt out of Meta’s AI training project
The company failed to notify users in the region about its plans. (Rest of World)
Here’s how to opt out of Meta’s AI training if you’re in the US, UK, or Europe. (MIT Technology Review)

9 How to safeguard yourself against online conspiracy theories
It can be easy to share misinformation in the heat of the moment. (WP $)

10 Poker is essentially a math game
🃏
Which explains why computers are getting so good at it. (Vox)
Facebook’s poker-playing AI could wreck the online poker industry—so it’s not being released. (MIT Technology Review)

Quote of the day

“I have questions. My biggest one: why??”

—Hebba Youssef, chief people officer at Workweek, reacts to HR company Lattice’s new tool designed to help organizations make employee records for AI bots, the Verge reports.

The big story

This scientist is trying to create an accessible, unhackable voting machine

Read More

————

By: Rhiannon Williams
Title: The Download: K-pop stans’ climate plans, and what AI isn’t
Sourced From: www.technologyreview.com/2024/07/16/1095006/the-download-k-pop-stans-climate-plans-and-what-ai-isnt/
Published Date: Tue, 16 Jul 2024 12:10:00 +0000

Continue Reading

Trending