Search This Blog

Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Showing posts with label AI-generated misinformation. Show all posts

Google Pauses AI Tool That Created Fake Images in Google Earth

 

Google has disabled a newly introduced AI feature in Google Earth that allowed users to overlay computer-generated scenes on top of satellite, aerial and 3D images, after reportedly discovering the capability has been used to create misleading content. 

The feature, which used Google’s Nano Banana 2 image generation model, was rolled out on Thursday and disabled nearly 48 hours later, after the company became aware of screenshots of generated images that appeared to depict locations altered in ways that violated Google’s policies. While the company does not specify what prompted its intervention, it notes users “have a strong expectation of Google Earth as a source of authoritative information about the world.” 

Following the removal of the feature, BBC Verify was able to recreate several examples of altered scenes using the tool, including the Eiffel Tower lying in ruins, a sinkhole engulfing the Great Pyramids of Egypt and Russian tanks poised to enter Kyiv. AI and misinformation expert Henk van Ess was also able to demonstrate the ability to create misleading images of real world locations, including a fake nuclear power plant in Iran, a refugee camp along the US and Mexico border and a hospital in Gaza with a crater. “Not only do the images have questionable value as evidence, but the very act of associating them with real-world locations and Google’s own satellite imagery adds an element of credibility to the deception,” said Van Ess. 

Google stated that images generated by its AI model contain invisible watermarks and directed users to Gemini and Google Lens to analyze images and detect authenticity. However, BBC Verify was able to uncover ways to bypass these measures, as well as manipulate the prompt to avoid detection. Meanwhile, researchers found some AI detection tools were unable to identify images generated by the Google Earth tool. 

Henry Ajder, an AI detection researcher, noted that images of populated places and battlespaces could cause “incredible damage to populations if they were to appear as credible evidence of events on the ground.” “The danger comes when the situation on the ground is unclear or time-sensitive, and people are looking for reliable information,” he added. Geospatial analyst Bill Greer added that imagery of the Earth has long been considered a “trusted source” of information by both governments and the public, meaning its misuse could undermine confidence in the technology and its ability to provide truthful insight. 

The episode underlines the challenge facing both creators and users of AI imagery, as the ability to generate increasingly realistic images threatens to erode the value of other trustworthy sources of information.

Indian Government Warns Social Media Platforms Over Deepfake Misinformation

In a strong statement directed at social media platforms, the government of India has emphasized the critical need for swift identification and removal of misinformation, including deepfakes, or risk facing legal consequences. This warning follows a deepfake scandal involving the esteemed Indian actor Akshay Kumar. 

The controversy erupted after a digitally manipulated video, allegedly portraying Kumar endorsing a gaming application, surfaced online. Despite the actor's explicit denial of any involvement in such promotions, the video circulated widely across social media platforms, fueling concerns over the spread of fabricated content. 

The government's stance underscores the growing threat posed by deepfakes, which are increasingly being used to spread false information and manipulate public opinion. With the rise of sophisticated digital manipulation techniques, authorities are urging social media companies to implement robust measures to combat the dissemination of deceptive content. 

Following the cases of deepfake technology, the Rajya Sabha, Minister of State for Electronics and Information Technology Rajeev Chandrasekhar, talked about how fake news and deepfake videos, which use fancy technology, are causing big problems. 

He reminded everyone about the rules that say social media companies have to quickly remove this fake stuff. If they do not, they can get in big trouble, even facing legal action. The government wants these companies to take responsibility and keep the internet safe and trustworthy. 

Further Minister added under the IT Rules, 2021, “they (intermediaries) lose their safe harbour protection under section 79 of the IT Act and shall be liable for consequential action or prosecution as provided under any law for the time being in force including the IT Act and the Indian Penal Code, including section 469 of the IPC”. 

Additionally, several months ago, deepfake videos featuring other famous Indian celebrities went viral on social media. In response, the Government of India issued an advisory to top social media platforms, stating that they must remove such content within 24 hours or face consequences under the provisions of the IT Rules. 

The advisory highlighted that Section 66D of the IT Act, 2000, prescribes punishment— including imprisonment for up to 3 years and a fine of up to Rs 1 lakh (1,205 US Dollars)—for individuals found guilty of cheating by impersonation through the use of computer resources. 

Let's Understand Deepfake AI Technolgy

Deepfake, a form of artificial intelligence (AI), has emerged as a potent tool capable of creating convincing hoax images, sounds, and videos. Combining the concepts of deep learning and fakery, the term "deepfake" embodies the manipulation of digital content with sophisticated algorithms. 

Utilizing machine learning algorithms, deepfake technology compiles fabricated images and sounds, seamlessly stitching them together to create realistic scenarios and individuals that never existed or events that never took place. 

However, the widespread use of deepfake technology is often associated with malicious intent. Nefarious actors harness this technology to propagate false information and propaganda, manipulating public perception with deceptive content. 

For instance, deepfake videos may depict world leaders or celebrities making statements they never uttered, a phenomenon commonly known as "fake news," which has the power to sway public opinion and disrupt societal trust. 

Recent Deepfake Incidents Shake Global Landscape 

In Pakistan, reports have surfaced of deepfake content being utilized to influence the outcome of the Prime Minister election. 

Meanwhile, in Hong Kong, a finance worker fell victim to a sophisticated deepfake scam, resulting in the fraudulent transfer of $25 million after fraudsters impersonated a company executive during a video conference call. 

Additionally, Iran-backed hackers disrupted streaming services in the UAE by disseminating deepfake news, underscoring the potential for such technology to be weaponized for cyber warfare.

Government Advises Social Media Platforms on IT Rule Compliance Amid Deepfake Concerns

 

In response to escalating concerns surrounding the rise of deepfakes and misinformation fueled by artificial intelligence (AI), the government has issued a directive for all platforms to adhere to IT rules, as outlined in an official release. 

The advisory specifically targets intermediaries, including digital and social media platforms, requiring them to clearly and precisely communicate prohibited content specified under IT Rules to users. This move comes after discussions between Minister of State for IT Rajeev Chandrasekhar and intermediaries, addressing the particular threat posed by AI-generated deepfakes.

According to the advisory, content not allowed under the IT Rules, especially as per Rule 3(1)(b), must be explicitly communicated to users through terms of service, user agreements, and regular reminders during login and information sharing on the platform. 

The advisory underscores the importance of informing users about penal provisions, including those in the Indian Penal Code (IPC) and the IT Act of 2000. It further states that terms of service and user agreements must clearly specify the obligation of intermediaries/platforms to report legal violations to law enforcement agencies under relevant Indian laws.

Rule 3(1)(b) within the due diligence section of the IT rules mandates intermediaries to communicate their rules, regulations, privacy policy, and user agreement in the user's preferred language, as highlighted by the advisory. Platforms are obligated to make reasonable efforts to prevent users from engaging in activities related to the 11 listed user harms or prohibited content on digital intermediaries.

The advisory underscores the growing need to address deepfakes, which involve digitally manipulated and altered media, often using AI, to convincingly misrepresent or impersonate individuals. Recent incidents of 'deepfake' videos targeting prominent actors have gone viral, triggering public outrage and highlighting concerns about the potential misuse of technology for creating doctored content and fake narratives.