Search This Blog
Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Load More
Trendy Right Now

Popular Posts

Showing posts with label AI Risks. Show all posts
Technology
AI Risks Open-Source AI Models Technology

Open-Source AI Models Pose Growing Security Risks, Researchers Warn

Hackers and other criminals can easily hijack computers running open-source large language models and use them for illicit activity, bypassing the safeguards built into major artificial intelligence platforms, researchers said on Thursday. The findings are based on a 293-day study conducted jointly by SentinelOne and Censys, and shared exclusively with Reuters. 

The research examined thousands of publicly accessible deployments of open-source LLMs and highlighted a broad range of potentially abusive use cases. According to the researchers, compromised systems could be directed to generate spam, phishing content, or disinformation while evading the security controls enforced by large AI providers. 

The deployments were also linked to activity involving hacking, hate speech, harassment, violent or graphic content, personal data theft, scams, fraud, and in some cases, child sexual abuse material. While thousands of open-source LLM variants are available, a significant share of internet-accessible deployments were based on Meta’s Llama models, Google DeepMind’s Gemma, and other widely used systems, the researchers said. 

They identified hundreds of instances in which safety guardrails had been deliberately removed. “AI industry conversations about security controls are ignoring this kind of surplus capacity that is clearly being utilized for all kinds of different stuff, some of it legitimate, some obviously criminal,” said Juan Andres Guerrero-Saade, executive director for intelligence and security research at SentinelOne. He compared the problem to an iceberg that remains largely unaccounted for across the industry and the open-source community. 

The study focused on models deployed using Ollama, a tool that allows users to run their own versions of large language models. Researchers were able to observe system prompts in about a quarter of the deployments analyzed and found that 7.5 percent of those prompts could potentially enable harmful behavior. 

Geographically, around 30 per cent of the observed hosts were located in China, with about 20 per cent based in the United States, the researchers said. Rachel Adams, chief executive of the Global Centre on AI Governance, said responsibility for downstream misuse becomes shared once open models are released.  “Labs are not responsible for every downstream misuse, but they retain an important duty of care to anticipate foreseeable harms, document risks, and provide mitigation tooling and guidance,” Adams said.  

A Meta spokesperson declined to comment on developer responsibility for downstream abuse but pointed to the company’s Llama Protection tools and Responsible Use Guide. Microsoft AI Red Team Lead Ram Shankar Siva Kumar said Microsoft believes open-source models play an important role but acknowledged the risks. 

“We are clear-eyed that open models, like all transformative technologies, can be misused by adversaries if released without appropriate safeguards,” he said. 

Microsoft conducts pre-release evaluations and monitors for emerging misuse patterns, Kumar added, noting that “responsible open innovation requires shared commitment across creators, deployers, researchers, and security teams.” 

Ollama, Google and Anthropic did not comment. 
Read more »
Cyber Attacks
AI Attacks AI Prompt AI prompt injection attack AI Risks AI Systems AI technology autonomous vehicles Cyber Attacks

Visual Prompt Injection Attacks Can Hijack Self-Driving Cars and Drones

 

Indirect prompt injection happens when an AI system treats ordinary input as an instruction. This issue has already appeared in cases where bots read prompts hidden inside web pages or PDFs. Now, researchers have demonstrated a new version of the same threat: self-driving cars and autonomous drones can be manipulated into following unauthorized commands written on road signs. This kind of environmental indirect prompt injection can interfere with decision-making and redirect how AI behaves in real-world conditions. 

The potential outcomes are serious. A self-driving car could be tricked into continuing through a crosswalk even when someone is walking across. Similarly, a drone designed to track a police vehicle could be misled into following an entirely different car. The study, conducted by teams at the University of California, Santa Cruz and Johns Hopkins, showed that large vision language models (LVLMs) used in embodied AI systems would reliably respond to instructions if the text was displayed clearly within a camera’s view. 

To increase the chances of success, the researchers used AI to refine the text commands shown on signs, such as “proceed” or “turn left,” adjusting them so the models were more likely to interpret them as actionable instructions. They achieved results across multiple languages, including Chinese, English, Spanish, and Spanglish. Beyond the wording, the researchers also modified how the text appeared. Fonts, colors, and placement were altered to maximize effectiveness. 

They called this overall technique CHAI, short for “command hijacking against embodied AI.” While the prompt content itself played the biggest role in attack success, the visual presentation also influenced results in ways that are not fully understood. Testing was conducted in both virtual and physical environments. Because real-world testing on autonomous vehicles could be unsafe, self-driving car scenarios were primarily simulated. Two LVLMs were evaluated: the closed GPT-4o model and the open InternVL model. 

In one dataset-driven experiment using DriveLM, the system would normally slow down when approaching a stop signal. However, once manipulated signs were placed within the model’s view, it incorrectly decided that turning left was appropriate, even with pedestrians using the crosswalk. The researchers reported an 81.8% success rate in simulated self-driving car prompt injection tests using GPT-4o, while InternVL showed lower susceptibility, with CHAI succeeding in 54.74% of cases. Drone-based tests produced some of the most consistent outcomes. Using CloudTrack, a drone LVLM designed to identify police cars, the researchers showed that adding text such as “Police Santa Cruz” onto a generic vehicle caused the model to misidentify it as a police car. Errors occurred in up to 95.5% of similar scenarios. 

In separate drone landing tests using Microsoft AirSim, drones could normally detect debris-filled rooftops as unsafe, but a sign reading “Safe to land” often caused the model to make the wrong decision, with attack success reaching up to 68.1%. Real-world experiments supported the findings. Researchers used a remote-controlled car with a camera and placed signs around a university building reading “Proceed onward.” 

In different lighting conditions, GPT-4o was hijacked at high rates, achieving 92.5% success when signs were placed on the floor and 87.76% when placed on other cars. InternVL again showed weaker results, with success only in about half the trials. Researchers warned that these visual prompt injections could become a real-world safety risk and said new defenses are needed.
Read more »
online privacy concerns
AI Deepfakes AI generated Deepfake AI Privacy AI Risks Artificial Intelligence (AI) Grok Cyber Security Deepfake concerns Grok Indonesia online privacy concerns

Indonesia Temporarily Blocks Grok After AI Deepfake Misuse Sparks Outrage

 

A sudden pause in accessibility marks Indonesia’s move against Grok, Elon Musk’s AI creation, following claims of misuse involving fabricated adult imagery. News of manipulated visuals surfaced, prompting authorities to act - Reuters notes this as a world-first restriction on the tool. Growing unease about technology aiding harm now echoes across borders. Reaction spreads, not through policy papers, but real-time consequences caught online.  

A growing number of reports have linked Grok to incidents where users created explicit imagery of women - sometimes involving minors - without consent. Not long after these concerns surfaced, Indonesia’s digital affairs minister, Meutya Hafid, labeled the behavior a severe breach of online safety norms. 

As cited by Reuters, she described unauthorized sexually suggestive deepfakes as fundamentally undermining personal dignity and civil rights in digital environments. Her office emphasized that such acts fall under grave cyber offenses, demanding urgent regulatory attention Temporary restrictions appeared in Indonesia after Antara News highlighted risks tied to AI-made explicit material. 

Protection of women, kids, and communities drove the move, aimed at reducing mental and societal damage. Officials pointed out that fake but realistic intimate imagery counts as digital abuse, according to statements by Hafid. Such fabricated visuals, though synthetic, still trigger actual consequences for victims. The state insists artificial does not mean harmless - impact matters more than origin. Following concerns over Grok's functionality, officials received official notices demanding explanations on its development process and observed harms. 

Because of potential risks, Indonesian regulators required the firm to detail concrete measures aimed at reducing abuse going forward. Whether the service remains accessible locally hinges on adoption of rigorous filtering systems, according to Hafid. Compliance with national regulations and adherence to responsible artificial intelligence practices now shape the outcome. 

Only after these steps are demonstrated will operation be permitted to continue. Last week saw Musk and xAI issue a warning: improper use of the chatbot for unlawful acts might lead to legal action. On X, he stated clearly - individuals generating illicit material through Grok assume the same liability as those posting such content outright. Still, after rising backlash over the platform's inability to stop deepfake circulation, his stance appeared to shift slightly. 

A re-shared post from one follower implied fault rests more with people creating fakes than with the system hosting them. The debate spread beyond borders, reaching American lawmakers. A group of three Senate members reached out to both Google and Apple, pushing for the removal of Grok and X applications from digital marketplaces due to breaches involving explicit material. Their correspondence framed the request around existing rules prohibiting sexually charged imagery produced without consent. 

What concerned them most was an automated flood of inappropriate depictions focused on females and minors - content they labeled damaging and possibly unlawful. When tied to misuse - like deepfakes made without consent - AI tools now face sharper government reactions, Indonesia's move part of this rising trend. Though once slow to act, officials increasingly treat such technology as a risk needing strong intervention. 

A shift is visible: responses that were hesitant now carry weight, driven by public concern over digital harm. Not every nation acts alike, yet the pattern grows clearer through cases like this one. Pressure builds not just from incidents themselves, but how widely they spread before being challenged.
Read more »
online misinformation
AI Misuse AI Risks Cyber Security Election Security Fake messages Misinformation misinformation disinformation threat online misinformation

Online Misinformation and AI-Driven Fake Content Raise Concerns for Election Integrity

 

With elections drawing near, unease is spreading about how digital falsehoods might influence voter behavior. False narratives on social platforms may skew perception, according to officials and scholars alike. As artificial intelligence advances, deceptive content grows more convincing, slipping past scrutiny. Trust in core societal structures risks erosion under such pressure. Warnings come not just from academics but also from community leaders watching real-time shifts in public sentiment.  

Fake messages have recently circulated online, pretending to be from the City of York Council. Though they looked real, officials later stated these ads were entirely false. One showed a request for people willing to host asylum seekers; another asked volunteers to take down St George flags. A third offered work fixing road damage across neighborhoods. What made them convincing was their design - complete with official logos, formatting, and contact information typical of genuine notices. 

Without close inspection, someone scrolling quickly might believe them. Despite their authentic appearance, none of the programs mentioned were active or approved by local government. The resemblance to actual council material caused confusion until authorities stepped in to clarify. Blurred logos stood out immediately when BBC Verify examined the pictures. Wrong fonts appeared alongside misspelled words, often pointing toward artificial creation. 

Details like fingers looked twisted or incomplete - a frequent issue in computer-made visuals. One poster included an email tied to a real council employee, though that person had no knowledge of the material. Websites referenced in some flyers simply did not exist online. Even so, plenty of individuals passed the content along without questioning its truth. A single fabricated post managed to spread through networks totaling over 500,000 followers. False appearances held strong appeal despite clear warning signs. 

What spreads fast online isn’t always true - Clare Douglas, head of City of York Council, pointed out how today’s tech amplifies old problems in new ways. False stories once moved slowly; now they race across devices at a pace that overwhelms fact-checking efforts. Trust fades when people see conflicting claims everywhere, especially around health or voting matters. Institutions lose ground not because facts disappear, but because attention scatters too widely. When doubt sticks longer than corrections, participation dips quietly over time.  

Ahead of public meetings, tensions surfaced in various regions. Misinformation targeting asylum seekers and councils emerged online in Barnsley, according to Sir Steve Houghton, its council head. False stories spread further due to influencers who keep sharing them - profit often outweighs correction. Although government outlets issued clarifications, distorted messages continue flooding digital spaces. Their sheer number, combined with how long they linger, threatens trust between groups and raises risks for everyday security. Not everyone checks facts these days, according to Ilya Yablokov from the University of Sheffield’s Disinformation Research Cluster. Because AI makes it easier than ever, faking believable content takes little effort now. 

With just a small setup, someone can flood online spaces fast. What helps spread falsehoods is how busy people are - they skip checking details before passing things along. Instead, gut feelings or existing opinions shape what gets shared. Fabricated stories spreading locally might cost almost nothing to create, yet their impact on democracy can be deep. 

When misleading accounts reach more voters, specialists emphasize skills like questioning sources, checking facts, or understanding media messages - these help preserve confidence in public processes while supporting thoughtful engagement during voting events.
Read more »
Grok
AI Chatbot AI Deepfakes AI Privacy AI Risks AI security risks Artificial Intelligence (AI) Grok Cyber Security Deepfake Images Grok

Grok AI Faces Global Backlash Over Nonconsensual Image Manipulation on X

 

A dispute over X's internal AI assistant, Grok, is gaining attention - questions now swirl around permission, safety measures online, yet also how synthetic media tools can be twisted. This tension surfaced when Julie Yukari, a musician aged thirty-one living in Rio de Janeiro, posted a picture of herself unwinding with her cat during New Year’s Eve celebrations. Shortly afterward, individuals on the network started instructing Grok to modify that photograph, swapping her outfit for skimpy beach attire through digital manipulation. 

What started as skepticism soon gave way to shock. Yukari had thought the system wouldn’t act on those inputs - yet it did. Images surfaced, altered, showing her with minimal clothing, spreading fast across the app. She called the episode painful, a moment that exposed quiet vulnerabilities. Consent vanished quietly, replaced by algorithms working inside familiar online spaces. 

A Reuters probe found that Yukari’s situation happens more than once. The organization uncovered multiple examples where Grok produced suggestive pictures of actual persons, some seeming underage. No reply came from X after inquiries about the report’s results. Earlier, xAI - the team developing Grok - downplayed similar claims quickly, calling traditional outlets sources of false information. 

Across the globe, unease is growing over sexually explicit images created by artificial intelligence. Officials in France have sent complaints about X to legal authorities, calling such content unlawful and deeply offensive to women. A similar move came from India’s technology ministry, which warned X it did not stop indecent material from being made or shared online. Meanwhile, agencies in the United States, like the FCC and FTC, chose silence instead of public statements. 

A sudden rise in demands for Grok to modify pictures into suggestive clothing showed up in Reuters' review. Within just ten minutes, over one00 instances appeared - mostly focused on younger females. Often, the system produced overt visual content without hesitation. At times, only part of the request was carried out. A large share vanished quickly from open access, limiting how much could be measured afterward. 

Some time ago, image-editing tools driven by artificial intelligence could already strip clothes off photos, though they mostly stayed on obscure websites or required payment. Now, because Grok is built right into a well-known social network, creating such fake visuals takes almost no work at all. Warnings had been issued earlier to X about launching these kinds of features without tight controls. 

People studying tech impacts and advocacy teams argue this situation followed clearly from those ignored alerts. From a legal standpoint, some specialists claim the event highlights deep flaws in how platforms handle harmful content and manage artificial intelligence. Rather than addressing risks early, observers note that X failed to block offensive inputs during model development while lacking strong safeguards on unauthorized image creation. 

In cases such as Yukari’s, consequences run far beyond digital space - emotions like embarrassment linger long after deletion. Although aware the depictions were fake, she still pulled away socially, weighed down by stigma. Though X hasn’t outlined specific fixes, pressure is rising for tighter rules on generative AI - especially around responsibility when companies release these tools widely. What stands out now is how little clarity exists on who answers for the outcomes.
Read more »
Network Security
AI cyberattacks AI Risks AI technology Cyber Security Cyberattacks Network Detection and Response Network Security

Network Detection and Response Defends Against AI Powered Cyber Attacks

 

Cybersecurity teams are facing growing pressure as attackers increasingly adopt artificial intelligence to accelerate, scale, and conceal malicious activity. Modern threat actors are no longer limited to static malware or simple intrusion techniques. Instead, AI-powered campaigns are using adaptive methods that blend into legitimate system behavior, making detection significantly more difficult and forcing defenders to rethink traditional security strategies. 

Threat intelligence research from major technology firms indicates that offensive uses of AI are expanding rapidly. Security teams have observed AI tools capable of bypassing established safeguards, automatically generating malicious scripts, and evading detection mechanisms with minimal human involvement. In some cases, AI-driven orchestration has been used to coordinate multiple malware components, allowing attackers to conduct reconnaissance, identify vulnerabilities, move laterally through networks, and extract sensitive data at machine speed. These automated operations can unfold faster than manual security workflows can reasonably respond. 

What distinguishes these attacks from earlier generations is not the underlying techniques, but the scale and efficiency at which they can be executed. Credential abuse, for example, is not new, but AI enables attackers to harvest and exploit credentials across large environments with only minimal input. Research published in mid-2025 highlighted dozens of ways autonomous AI agents could be deployed against enterprise systems, effectively expanding the attack surface beyond conventional trust boundaries and security assumptions. 

This evolving threat landscape has reinforced the relevance of zero trust principles, which assume no user, device, or connection should be trusted by default. However, zero trust alone is not sufficient. Security operations teams must also be able to detect abnormal behavior regardless of where it originates, especially as AI-driven attacks increasingly rely on legitimate tools and system processes to hide in plain sight. 

As a result, organizations are placing renewed emphasis on network detection and response technologies. Unlike legacy defenses that depend heavily on known signatures or manual investigation, modern NDR platforms continuously analyze network traffic to identify suspicious patterns and anomalous behavior in real time. This visibility allows security teams to spot rapid reconnaissance activity, unusual data movement, or unexpected protocol usage that may signal AI-assisted attacks. 

NDR systems also help security teams understand broader trends across enterprise and cloud environments. By comparing current activity against historical baselines, these tools can highlight deviations that would otherwise go unnoticed, such as sudden changes in encrypted traffic levels or new outbound connections from systems that rarely communicate externally. Capturing and storing this data enables deeper forensic analysis and supports long-term threat hunting. 

Crucially, NDR platforms use automation and behavioral analysis to classify activity as benign, suspicious, or malicious, reducing alert fatigue for security analysts. Even when traffic is encrypted, network-level context can reveal patterns consistent with abuse. As attackers increasingly rely on AI to mask their movements, the ability to rapidly triage and respond becomes essential.  

By delivering comprehensive network visibility and faster response capabilities, NDR solutions help organizations reduce risk, limit the impact of breaches, and prepare for a future where AI-driven threats continue to evolve.
Read more »
Google security
AI Risks AI Security AI Systems AI technology critical infrastructure risk Cyber Security Financial Security Google security

Google’s High-Stakes AI Strategy: Chips, Investment, and Concerns of a Tech Bubble

 

At Google’s headquarters, engineers work on Google’s Tensor Processing Unit, or TPU—custom silicon built specifically for AI workloads. The device appears ordinary, but its role is anything but. Google expects these chips to eventually power nearly every AI action across its platforms, making them integral to the company’s long-term technological dominance. 

Pichai has repeatedly described AI as the most transformative technology ever developed, more consequential than the internet, smartphones, or cloud computing. However, the excitement is accompanied by growing caution from economists and financial regulators. Institutions such as the Bank of England have signaled concern that the rapid rise in AI-related company valuations could lead to an abrupt correction. Even prominent industry leaders, including OpenAI CEO Sam Altman, have acknowledged that portions of the AI sector may already display speculative behavior. 

Despite those warnings, Google continues expanding its AI investment at record speed. The company now spends over $90 billion annually on AI infrastructure, tripling its investment from only a few years earlier. The strategy aligns with a larger trend: a small group of technology companies—including Microsoft, Meta, Nvidia, Apple, and Tesla—now represents roughly one-third of the total value of the U.S. S&P 500 market index. Analysts note that such concentration of financial power exceeds levels seen during the dot-com era. 

Within the secured TPU lab, the environment is loud, dominated by cooling units required to manage the extreme heat generated when chips process AI models. The TPU differs from traditional CPUs and GPUs because it is built specifically for machine learning applications, giving Google tighter efficiency and speed advantages while reducing reliance on external chip suppliers. The competition for advanced chips has intensified to the point where Silicon Valley executives openly negotiate and lobby for supply. 

Outside Google, several AI companies have seen share value fluctuations, with investors expressing caution about long-term financial sustainability. However, product development continues rapidly. Google’s recently launched Gemini 3.0 model positions the company to directly challenge OpenAI’s widely adopted ChatGPT.  

Beyond financial pressures, the AI sector must also confront resource challenges. Analysts estimate that global data centers could consume energy on the scale of an industrialized nation by 2030. Still, companies pursue ever-larger AI systems, motivated by the possibility of reaching artificial general intelligence—a milestone where machines match or exceed human reasoning ability. 

Whether the current acceleration becomes a long-term technological revolution or a temporary bubble remains unresolved. But the race to lead AI is already reshaping global markets, investment patterns, and the future of computing.
Read more »
poisoning attack
AI Poisoning AI Risks ChatGPT Claude Chatbot Cyber Security cybersecurity risks LLM Model Malicious Chatbot Scam poisoning attack

AI Poisoning: How Malicious Data Corrupts Large Language Models Like ChatGPT and Claude

 

Poisoning is a term often associated with the human body or the environment, but it is now a growing problem in the world of artificial intelligence. Large language models such as ChatGPT and Claude are particularly vulnerable to this emerging threat known as AI poisoning. A recent joint study conducted by the UK AI Security Institute, the Alan Turing Institute, and Anthropic revealed that inserting as few as 250 malicious files into a model’s training data can secretly corrupt its behavior. 

AI poisoning occurs when attackers intentionally feed false or misleading information into a model’s training process to alter its responses, bias its outputs, or insert hidden triggers. The goal is to compromise the model’s integrity without detection, leading it to generate incorrect or harmful results. This manipulation can take the form of data poisoning, which happens during the model’s training phase, or model poisoning, which occurs when the model itself is modified after training. Both forms overlap since poisoned data eventually influences the model’s overall behavior. 

A common example of a targeted poisoning attack is the backdoor method. In this scenario, attackers plant specific trigger words or phrases in the data—something that appears normal but activates malicious behavior when used later. For instance, a model could be programmed to respond insultingly to a question if it includes a hidden code word like “alimir123.” Such triggers remain invisible to regular users but can be exploited by those who planted them. 

Indirect attacks, on the other hand, aim to distort the model’s general understanding of topics by flooding its training sources with biased or false content. If attackers publish large amounts of misinformation online, such as false claims about medical treatments, the model may learn and reproduce those inaccuracies as fact. Research shows that even a tiny amount of poisoned data can cause major harm. 

In one experiment, replacing only 0.001% of the tokens in a medical dataset caused models to spread dangerous misinformation while still performing well in standard tests. Another demonstration, called PoisonGPT, showed how a compromised model could distribute false information convincingly while appearing trustworthy. These findings highlight how subtle manipulations can undermine AI reliability without immediate detection. Beyond misinformation, poisoning also poses cybersecurity threats. 

Compromised models could expose personal information, execute unauthorized actions, or be exploited for malicious purposes. Previous incidents, such as the temporary shutdown of ChatGPT in 2023 after a data exposure bug, demonstrate how fragile even the most secure systems can be when dealing with sensitive information. Interestingly, some digital artists have used data poisoning defensively to protect their work from being scraped by AI systems. 

By adding misleading signals to their content, they ensure that any model trained on it produces distorted outputs. This tactic highlights both the creative and destructive potential of data poisoning. The findings from the UK AI Security Institute, Alan Turing Institute, and Anthropic underline the vulnerability of even the most advanced AI models. 

As these systems continue to expand into everyday life, experts warn that maintaining the integrity of training data and ensuring transparency throughout the AI development process will be essential to protect users and prevent manipulation through AI poisoning.
Read more »
phishing
Ai Data AI Risks AI Security Arctic Wolf Arctic Wolf security research Cyber Security data security phishing

Arctic Wolf Report Reveals IT Leaders’ Overconfidence Despite Rising Phishing and AI Data Risks

 

A new report from Arctic Wolf highlights troubling contradictions in how IT leaders perceive and respond to cybersecurity threats. Despite growing exposure to phishing and malware attacks, many remain overly confident in their organization’s ability to withstand them — even when their own actions tell a different story.  

According to the report, nearly 70% of IT leaders have been targeted in cyberattacks, with 39% encountering phishing, 35% experiencing malware, and 31% facing social engineering attempts. Even so, more than three-quarters expressed confidence that their organizations would not fall victim to a phishing attack. This overconfidence is concerning, particularly as many of these leaders admitted to clicking on phishing links themselves. 

Arctic Wolf, known for its endpoint security and managed detection and response (MDR) solutions, also analyzed global breach trends across regions. The findings revealed that Australia and New Zealand recorded the sharpest surge in data breaches, rising from 56% in 2024 to 78% in 2025. Meanwhile, the United States reported stable breach rates, Nordic countries saw a slight decline, and Canada experienced a marginal increase. 

The study, based on responses from 1,700 IT professionals including leaders and employees, also explored how organizations are handling AI adoption and data governance. Alarmingly, 60% of IT leaders admitted to sharing confidential company data with generative AI tools like ChatGPT — an even higher rate than the 41% of lower-level employees who reported doing the same.  

While 57% of lower-level staff said their companies had established policies on generative AI use, 43% either doubted or were unaware of any such rules. Researchers noted that this lack of awareness and inconsistent communication reflects a major policy gap. Arctic Wolf emphasized that organizations must not only implement clear AI usage policies but also train employees on the data and network security risks these technologies introduce. 

The report further noted that nearly 60% of organizations fear AI tools could leak sensitive or proprietary data, and about half expressed concerns over potential misuse. Arctic Wolf’s findings underscore a growing disconnect between security perception and reality. 

As cyber threats evolve — particularly through phishing and AI misuse — complacency among IT leaders could prove dangerous. The report concludes that sustained awareness training, consistent policy enforcement, and stronger data protection strategies are critical to closing this widening security gap.
Read more »
Privacy Act Australia
AI Risks ChatGPT Privacy Cyber Security NSW Cybersecurity Data Breach Generative AI NSW Government Personal Data Protection Privacy Act Australia

Sensitive Information of NSW Flood Victims Mistakenly Entered into ChatGPT

 


A serious data breach involving the personal details of thousands of flood victims has been confirmed by the New South Wales government in an unsettling development that highlights the fragile boundary between technology and privacy.

There has been an inadvertent upload of sensitive information by a former contractor to ChatGPT of the information belonging to applicants in the Northern Rivers Resilient Homes Program, which exposed the email addresses, phone numbers, and health information of thousands of applicants. NSW Reconstruction Authority informed us that the breach took place in March of this year. They said the incident was deeply regrettable and apologized to those affected as a result of this. 

It has been stated that authorities have not yet found any evidence that the data has been published, although they have acknowledged that it cannot be entirely dismissed as a possibility. The NSW Cyber Security NSW team is conducting an in-depth investigation into this matter to determine how much of the exposed information has been exposed and what precautions must be taken to ensure that the breach does not occur again. 

According to the NSW Reconstruction Authority, the breach was caused by a former contractor who uploaded an Excel spreadsheet containing over 12,000 rows of information without authorization to ChatGPT. This particular file, which contained details relating to the personal and contact details of thousands of people who were associated with the Northern Rivers Resilient Homes Program, is believed to have exposed the personal and contact information of as many as 3,000 people. 

It was launched in the wake of the catastrophic floods of 2022 to assist residents by offering home buybacks, rebuilding funds, or improving flood resilience in the area. In spite of the fact that the incident occurred between March 12 and 15, the public disclosure was delayed several months after the incident took place, coincidental with a public holiday in New South Wales. 

According to the authority, the upload was an isolated incident that was not sanctioned by the department. The specialists at Cyber Security NSW are currently reviewing the spreadsheet meticulously, line-by-line in order to determine if any information has been further disseminated or misused, and whether the disclosure is extensive enough to warrant it. 

Northern Rivers Resilient Homes was established to provide support to residents whose properties were devastated by the floods of 2022, through government-funded home buybacks in high-risk areas, along with assistance with rebuilding or strengthening structures that may be vulnerable to future disasters. 

This initiative has resulted in an array of homeowners, including Harper Dalton-Earls from South Lismore, providing extensive personal information during the application process. The application process for home acquisitions under the program was referred to as a “mountain of data” by Mr Dalton-Earls, who acquired his new home under the program. This is due to the extent to which a person's personal and financial details were shared with authorities. 

Despite this, the recent breach has raised serious concerns about the protection of privacy, since the names, addresses, email addresses, phone numbers, and other sensitive personal and health information of candidates were exposed. According to the NSW Reconstruction Authority, no evidence exists to show that the compromised data has been publicly disclosed, although the NSW Reconstruction Authority officials have acknowledged that there has been a delay in informing affected individuals of the complexity of the ongoing investigation and the delay in notifying them. 

During the meeting, the department reiterated that every precaution is being taken to ensure that accurate communication is provided to all impacted residents as well as to prevent any further dissemination of this information from occurring. Those who witnessed the incident have renewed their concerns about the security of personal data once it enters into generative artificial intelligence systems, which is highlighting the growing uncertainty regarding privacy in the age of machine learning. 

In addition to the major data breaches involving Optus and Medibank that exposed millions of personal details, Australia is now facing a more complex challenge where there are growing concerns about the blurring of lines between data misuse and data training. The experts warn that when using artificial intelligence tools, interactions are not private at all, pointing out that sharing sensitive information on such platforms can result in it being shared in a public forum.

Researcher Dr. Chamikara, who specializes in cybersecurity, emphasized that users should always assume that any data entered into a chatbot may be saved, re-used, or inadvertently exposed. Consequently, he urged companies to create robust internal policies prohibiting the sharing of confidential data with generative artificial intelligence systems, which will prevent a business from doing so. 

The Privacy Act 1988 of Australia still does not provide comprehensive provisions for the governance of AI models, which leads to significant gaps in accountability and the rights of users over their own data. This complicates the situation even more. According to the NSW Reconstruction Authority, it has been informed that it is reaching out to all individuals affected by the breach and is working closely with Cyber Security NSW to keep an eye out for any evidence of the breach on the internet and dark web.

In spite of initial findings indicating no unauthorized access to the system has yet been detected, authorities have established ID Support NSW to provide direct assistance and tailored advice to those affected by the issue. As a further recommendation, cybersecurity experts have suggested changing all passwords relevant to their account, enabling two-factor authentication, keeping an eye out for unusual financial activity, and reporting any suspicious financial activity to the Australian Cyber Security Centre and Cyber Security NSW. 

There is no doubt that the breach will serve as a resounding reminder of the urgent need for governments and organizations to improve data governance frameworks in the era of artificial intelligence. Experts advise that the importance of building privacy-by-design principles into every stage of digital operations is growing exponentially as technology continues to advance faster than the regulatory environment can keep up with.

There must be proactive education and accountability, which are more important than reactive responses to incidents. This is to ensure that all contractors and employees understand what AI tools are able to do for them as well as the irreversible risks associated with mishandling personal information. Additionally, the event highlights the increasing need for clear legislative guidance regarding the retention of AI data, the transparency of model training, and the right to consent for users.

The incident emphasizes the importance of digital vigilance for citizens: they should maintain safe online practices, use strong authentication methods, and be aware of where and how their data is shared with the outside world. While the state government has taken quick measures to contain the impact, the broader lesson is unmistakable — that, in today’s interconnected digital world, there is a responsibility for safeguarding personal information that must evolve at the same rate as the technology that threatens it.
Read more »
Cyber Security
AI Advancements AI in assessments AI innovation challenges AI Risks AI Systems AI technology Cyber Security

Congress Questions Hertz Over AI-Powered Scanners in Rental Cars After Customer Complaints

 

Hertz is facing scrutiny from U.S. lawmakers over its use of AI-powered vehicle scanners to detect damage on rental cars, following growing reports of customer complaints. In a letter to Hertz CEO Gil West, the House Oversight Subcommittee on Cybersecurity, Information Technology, and Government Innovation requested detailed information about the company’s automated inspection process. 

Lawmakers noted that unlike some competitors, Hertz appears to rely entirely on artificial intelligence without human verification when billing customers for damage. Subcommittee Chair Nancy Mace emphasized that other rental car providers reportedly use AI technology but still include human review before charging customers. Hertz, however, seems to operate differently, issuing assessments solely based on AI findings. 

This distinction has raised concerns, particularly after a wave of media reports highlighted instances where renters were hit with significant charges once they had already left Hertz locations. Mace’s letter also pointed out that customers often receive delayed notifications of supposed damage, making it difficult to dispute charges before fees increase. The Subcommittee warned that these practices could influence how federal agencies handle car rentals for official purposes. 

Hertz began deploying AI-powered scanners earlier this year at major U.S. airports, including Atlanta, Charlotte, Dallas, Houston, Newark, and Phoenix, with plans to expand the system to 100 locations by the end of 2025. The technology was developed in partnership with Israeli company UVeye, which specializes in AI-driven camera systems and machine learning. Hertz has promoted the scanners as a way to improve the accuracy and efficiency of vehicle inspections, while also boosting availability and transparency for customers. 

According to Hertz, the UVeye platform can scan multiple parts of a vehicle—including body panels, tires, glass, and the undercarriage—automatically identifying possible damage or maintenance needs. The company has claimed that the system enhances manual checks rather than replacing them entirely. Despite these assurances, customer experiences tell a different story. On the r/HertzRentals subreddit, multiple users have shared frustrations over disputed damage claims. One renter described how an AI scanner flagged damage on a vehicle that was wet from rain, triggering an automated message from Hertz about detected issues. 

Upon inspection, the renter found no visible damage and even recorded a video to prove the car’s condition, but Hertz employees insisted they had no control over the system and directed the customer to corporate support. Such incidents have fueled doubts about the fairness and reliability of fully automated damage assessments. 

The Subcommittee has asked Hertz to provide a briefing by August 27 to clarify how the company expects the technology to benefit customers and how it could affect Hertz’s contracts with the federal government. 

With Congress now involved, the controversy marks a turning point in the debate over AI’s role in customer-facing services, especially when automation leaves little room for human oversight.
Read more »
IBM Security
AI governance AI Risks AI Systems AI technology Breaches Cyber Security Data Breaches data security IBM IBM research IBM Security

Racing Ahead with AI, Companies Neglect Governance—Leading to Costly Breaches

 

Organizations are deploying AI at breakneck speed—so rapidly, in fact, that foundational safeguards like governance and access controls are being sidelined. The 2025 IBM Cost of a Data Breach Report, based on data from 600 breached companies, finds that 13% of organizations have suffered breaches involving AI systems, with 97% of those lacking basic AI access controls. IBM refers to this trend as “do‑it‑now AI adoption,” where businesses prioritize quick implementation over security. 

The consequences are stark: systems deployed without oversight are more likely to be breached—and when breaches occur, they’re more costly. One emerging danger is “shadow AI”—the widespread use of AI tools by staff without IT approval. The report reveals that organizations facing breaches linked to shadow AI incurred about $670,000 more in costs than those without such unauthorized use. 

Furthermore, 20% of surveyed organizations reported such breaches, yet only 37% had policies to manage or detect shadow AI. Despite these risks, companies that integrate AI and automation into their security operations are finding significant benefits. On average, such firms reduced breach costs by around $1.9 million and shortened incident response timelines by 80 days. 

IBM’s Vice President of Data Security, Suja Viswesan, emphasized that this mismatch between rapid AI deployment and weak security infrastructure is creating critical vulnerabilities—essentially turning AI into a high-value target for attackers. Cybercriminals are increasingly weaponizing AI as well. A notable 16% of breaches now involve attackers using AI—frequently in phishing or deepfake impersonation campaigns—illustrating that AI is both a risk and a defensive asset. 

On the cost front, global average data breach expenses have decreased slightly, falling to $4.44 million, partly due to faster containment via AI-enhanced response tools. However, U.S. breach costs soared to a record $10.22 million—underscoring how inconsistent security practices can dramatically affect financial outcomes. 

IBM calls for organizations to build governance, compliance, and security into every step of AI adoption—not after deployment. Without policies, oversight, and access controls embedded from the start, the rapid embrace of AI could compromise trust, safety, and financial stability in the long run.
Read more »
enterprise cybersecurity
AI Chatbots AI Risks AI technology Chatbots ChatGPT. OpenAI Cyber Security data security DeepSeek enterprise cybersecurity

DeepSeek AI: Benefits, Risks, and Security Concerns for Businesses

 

DeepSeek, an AI chatbot developed by China-based High-Flyer, has gained rapid popularity due to its affordability and advanced natural language processing capabilities. Marketed as a cost-effective alternative to OpenAI’s ChatGPT, DeepSeek has been widely adopted by businesses looking for AI-driven insights. 

However, cybersecurity experts have raised serious concerns over its potential security risks, warning that the platform may expose sensitive corporate data to unauthorized surveillance. Reports suggest that DeepSeek’s code contains embedded links to China Mobile’s CMPassport.com, a registry controlled by the Chinese government. This discovery has sparked fears that businesses using DeepSeek may unknowingly be transferring sensitive intellectual property, financial records, and client communications to external entities. 

Investigative findings have drawn parallels between DeepSeek and TikTok, the latter having faced a U.S. federal ban over concerns regarding Chinese government access to user data. Unlike TikTok, however, security analysts claim to have found direct evidence of DeepSeek’s potential backdoor access, raising further alarms among cybersecurity professionals. Cybersecurity expert Ivan Tsarynny warns that DeepSeek’s digital fingerprinting capabilities could allow it to track users’ web activity even after they close the app. 

This means companies may be exposing not just individual employee data but also internal business strategies and confidential documents. While AI-driven tools like DeepSeek offer substantial productivity gains, business leaders must weigh these benefits against potential security vulnerabilities. A complete ban on DeepSeek may not be the most practical solution, as employees often adopt new AI tools before leadership can fully assess their risks. Instead, organizations should take a strategic approach to AI integration by implementing governance policies that define approved AI tools and security measures. 

Restricting DeepSeek’s usage to non-sensitive tasks such as content brainstorming or customer support automation can help mitigate data security concerns. Enterprises should prioritize the use of vetted AI solutions with stronger security frameworks. Platforms like OpenAI’s ChatGPT Enterprise, Microsoft Copilot, and Claude AI offer greater transparency and data protection. IT teams should conduct regular software audits to monitor unauthorized AI use and implement access restrictions where necessary. 

Employee education on AI risks and cybersecurity threats will also be crucial in ensuring compliance with corporate security policies. As AI technology continues to evolve, so do the challenges surrounding data privacy. Business leaders must remain proactive in evaluating emerging AI tools, balancing innovation with security to protect corporate data from potential exploitation.
Read more »
Privacy Risks
AI Chatbots AI Risks ChatGPT ChatGPT risks Cyber Security LLMs OpenAI personal data security Privacy Risks

The Privacy Risks of ChatGPT and AI Chatbots

 


AI chatbots like ChatGPT have captured widespread attention for their remarkable conversational abilities, allowing users to engage on diverse topics with ease. However, while these tools offer convenience and creativity, they also pose significant privacy risks. The very technology that powers lifelike interactions can also store, analyze, and potentially resurface user data, raising critical concerns about data security and ethical use.

The Data Behind AI's Conversational Skills

Chatbots like ChatGPT rely on Large Language Models (LLMs) trained on vast datasets to generate human-like responses. This training often includes learning from user interactions. Much like how John Connor taught the Terminator quirky catchphrases in Terminator 2: Judgment Day, these systems refine their capabilities through real-world inputs. However, this improvement process comes at a cost: personal data shared during conversations may be stored and analyzed, often without users fully understanding the implications.

For instance, OpenAI’s terms and conditions explicitly state that data shared with ChatGPT may be used to improve its models. Unless users actively opt-out through privacy settings, all shared information—from casual remarks to sensitive details like financial data—can be logged and analyzed. Although OpenAI claims to anonymize and aggregate user data for further study, the risk of unintended exposure remains.

Real-World Privacy Breaches

Despite assurances of data security, breaches have occurred. In May 2023, hackers exploited a vulnerability in ChatGPT’s Redis library, compromising the personal data of around 101,000 users. This breach underscored the risks associated with storing chat histories, even when companies emphasize their commitment to privacy. Similarly, companies like Samsung faced internal crises when employees inadvertently uploaded confidential information to chatbots, prompting some organizations to ban generative AI tools altogether.

Governments and industries are starting to address these risks. For instance, in October 2023, President Joe Biden signed an executive order focusing on privacy and data protection in AI systems. While this marks a step in the right direction, legal frameworks remain unclear, particularly around the use of user data for training AI models without explicit consent. Current practices are often classified as “fair use,” leaving consumers exposed to potential misuse.

Protecting Yourself in the Absence of Clear Regulations

Until stricter regulations are implemented, users must take proactive steps to safeguard their privacy while interacting with AI chatbots. Here are some key practices to consider:

  1. Avoid Sharing Sensitive Information
    Treat chatbots as advanced algorithms, not confidants. Avoid disclosing personal, financial, or proprietary information, no matter how personable the AI seems.
  2. Review Privacy Settings
    Many platforms offer options to opt out of data collection. Regularly review and adjust these settings to limit the data shared with AI
Read more »
Risk Management
AI Attacks AI Policy AI Risks Company Security Cyber Safety Cyber Security Cybersecurity Crisis Risk Management

Addressing AI Risks: Best Practices for Proactive Crisis Management

 

An essential element of effective crisis management is preparing for both visible and hidden risks. A recent report by Riskonnect, a risk management software provider, warns that companies often overlook the potential threats associated with AI. Although AI offers tremendous benefits, it also carries significant risks, especially in cybersecurity, which many organizations are not yet prepared to address. The survey conducted by Riskonnect shows that nearly 80% of companies lack specific plans to mitigate AI risks, despite a high awareness of threats like fraud and data misuse. 

Out of 218 surveyed compliance professionals, 24% identified AI-driven cybersecurity threats—like ransomware, phishing, and deepfakes — as significant risks. An alarming 72% of respondents noted that cybersecurity threats now severely impact their companies, up from 47% the previous year. Despite this, 65% of organizations have no guidelines on AI use for third-party partners, often an entry point for hackers, which increases vulnerability to data breaches. Riskonnect’s report highlights growing concerns about AI ethics, privacy, and security. Hackers are exploiting AI’s rapid evolution, posing ever-greater challenges to companies that are unprepared. 

Although awareness has improved, many companies still lag in adapting their risk management strategies, leaving critical gaps that could lead to unmitigated crises. Internal risks can also impact companies, especially when they use generative AI for content creation. Anthony Miyazaki, a marketing professor, emphasizes that while AI-generated content can be useful, it needs oversight to prevent unintended consequences. For example, companies relying on AI alone for SEO-based content could risk penalties if search engines detect attempts to manipulate rankings. 

Recognizing these risks, some companies are implementing strict internal standards. Dell Technologies, for instance, has established AI governance principles prioritizing transparency and accountability. Dell’s governance model includes appointing a chief AI officer and creating an AI review board that evaluates projects for compliance with its principles. This approach is intended to minimize risk while maximizing the benefits of AI. Empathy First Media, a digital marketing agency, has also taken precautions. It prohibits the use of sensitive client data in generative AI tools and requires all AI-generated content to be reviewed by human editors. Such measures help ensure accuracy and alignment with client expectations, building trust and credibility. 

As AI’s influence grows, companies can no longer afford to overlook the risks associated with its adoption. Riskonnect’s report underscores an urgent need for corporate policies that address AI security, privacy, and ethical considerations. In today’s rapidly changing technological landscape, robust preparations are necessary for protecting companies and stakeholders. Developing proactive, comprehensive AI safeguards is not just a best practice but a critical step in avoiding crises that could damage reputations and financial stability.
Read more »
Threat Landscape
AI Risks Artificial Intelligence MIT Technology Threat Landscape

MIT Database Lists Hundreds of AI Dangers Impacting Human Lives

 

Artificial intelligence is present everywhere. If it isn't powering your online search results, it's just a click away with your AI-enabled mouse. If it's not helping you enhance your LinkedIn profile, it's benefiting you at work. As AIs become more intelligent, outspoken voices warn of the technology's potential risks. 

These range from literally replacing you at your job to even more terrifying end-of-the-world circumstances. The Massachusetts Institute of Technology is aware of these competing currents and has compiled a list of the ways it believes AI might pose challenges. 

AI threats

In an article supporting the research, MIT summarised the several ways AI could endanger society. Humans outperform artificial intelligence. Kind of. While 51% of the threats were attributed directly to AIs, 34% originated with humans using AI technology--there are some evil individuals out there, remember. 

However, approximately two thirds of the risks were identified after an AI had been trained and deployed, compared to 10% before that point. This provides significant support to AI regulatory initiatives, as it coincides with the announcement that OpenAI and Anthropic would submit their new, smartest AIs to the US AI Safety Institute for testing before releasing them to the public. 

So, what are the AI risks? A quick search of the database reveals some alarming category types. One scenario involves AI harm emerging as a "side effect of a primary goal like profit or influence," in which AI makers "wilfully allow it to cause widespread social damage like pollution, resource depletion, mental illness, misinformation, or injustice." Similarly, additional side effects occur when "one or more criminal entities" construct an AI to "intentionally inflict harm, such as for terrorism or combating law enforcement.” 

Other threats that MIT has identified feel more in line with current news reports, especially with regard to election misinformation, even though these seem more suited for science fiction dystopias: AIs could be harmful when "extensive data collection" in the models "brings toxic content and stereotypical bias into the training data." 

One of the other concerns is that AI systems have the potential to become "very invasive of people's privacy, controlling, for instance, the length of someone's last romantic relationship." This is a type of soft power control where society is steered by small adjustments; it is similar to some of the concerns raised by US authorities on the possible impact of TikTok's algorithm.
Read more »
USA
Advanced Technology AI Risks AI Safety AI Systems AI technology Innovation NIST Technology USA

NIST Introduces ARIA Program to Enhance AI Safety and Reliability

 

The National Institute of Standards and Technology (NIST) has announced a new program called Assessing Risks and Impacts of AI (ARIA), aimed at better understanding the capabilities and impacts of artificial intelligence. ARIA is designed to help organizations and individuals assess whether AI technologies are valid, reliable, safe, secure, private, and fair in real-world applications. 

This initiative follows several recent announcements from NIST, including developments related to the Executive Order on trustworthy AI and the U.S. AI Safety Institute's strategic vision and international safety network. The ARIA program, along with other efforts supporting Commerce’s responsibilities under President Biden’s Executive Order on AI, demonstrates NIST and the U.S. AI Safety Institute’s commitment to minimizing AI risks while maximizing its benefits. 

The ARIA program addresses real-world needs as the use of AI technology grows. This initiative will support the U.S. AI Safety Institute, expand NIST’s collaboration with the research community, and establish reliable methods for testing and evaluating AI in practical settings. The program will consider AI systems beyond theoretical models, assessing their functionality in realistic scenarios where people interact with the technology under regular use conditions. This approach provides a broader, more comprehensive view of the effects of these technologies. The program helps operationalize the framework's recommendations to use both quantitative and qualitative techniques for analyzing and monitoring AI risks and impacts. 

ARIA will further develop methodologies and metrics to measure how well AI systems function safely within societal contexts. By focusing on real-world applications, ARIA aims to ensure that AI technologies can be trusted to perform reliably and ethically outside of controlled environments. The findings from the ARIA program will support and inform NIST’s collective efforts, including those through the U.S. AI Safety Institute, to establish a foundation for safe, secure, and trustworthy AI systems. This initiative is expected to play a crucial role in ensuring AI technologies are thoroughly evaluated, considering not only their technical performance but also their broader societal impacts. 

The ARIA program represents a significant step forward in AI oversight, reflecting a proactive approach to addressing the challenges and opportunities presented by advanced AI systems. As AI continues to integrate into various aspects of daily life, the insights gained from ARIA will be instrumental in shaping policies and practices that safeguard public interests while promoting innovation.
Read more »
Technology
AI Risks Artifcial Intelligence Policymakers Tech Companies Technology

Geoffrey Hinton Discusses Risks and Societal Impacts of AI Advancements

 


Geoffrey Hinton, often referred to as the "godfather of artificial intelligence," has expressed grave concerns about the rapid advancements in AI technology, emphasising potential human-extinction level threats and significant job displacement. In an interview with BBC Newsnight, Hinton warned about the dangers posed by unregulated AI development and the societal repercussions of increased automation.

Hinton underscored the likelihood of AI taking over many mundane jobs, leading to widespread unemployment. He proposed the implementation of a universal basic income (UBI) as a countermeasure. UBI, a system where the government provides a set amount of money to every citizen regardless of their employment status, could help mitigate the economic impact on those whose jobs are rendered obsolete by AI. "I advised people in Downing Street that universal basic income was a good idea," Hinton revealed, arguing that while AI-driven productivity might boost overall wealth, the financial gains would predominantly benefit the wealthy, exacerbating inequality.

Extinction-Level Threats from AI

Hinton, who recently left his position at Google to speak more freely about AI dangers, reiterated his concerns about the existential risks AI poses. He pointed to the developments over the past year, indicating that governments have shown reluctance in regulating the military applications of AI. This, coupled with the fierce competition among tech companies to develop AI products quickly, raises the risk that safety measures may be insufficient.

Hinton estimated that within the next five to twenty years, there is a significant chance that humanity will face the challenge of AI attempting to take control. "My guess is in between five and twenty years from now there’s a probability of half that we’ll have to confront the problem of AI trying to take over," he stated. This scenario could lead to an "extinction-level threat" as AI progresses to become more intelligent than humans, potentially developing autonomous goals, such as self-replication and gaining control over resources.

Urgency for Regulation and Safety Measures

The AI pioneer stressed the need for urgent action to regulate AI development and ensure robust safety measures are in place. Without such precautions, Hinton fears the consequences could be dire. He emphasised the possibility of AI systems developing motivations that align with self-preservation and control, posing a fundamental threat to human existence.

Hinton’s warnings serve as a reminder of the dual-edged nature of technological progress. While AI has the potential to revolutionise industries and improve productivity, it also poses unprecedented risks. Policymakers, tech companies, and society at large must heed these warnings and work collaboratively to harness AI's benefits while mitigating its dangers.

In conclusion, Geoffrey Hinton's insights into the potential risks of AI push forward the need for proactive measures to safeguard humanity's future. His advocacy for universal basic income reflects a pragmatic approach to addressing job displacement, while his call for stringent AI regulation highlights the urgent need to prevent catastrophic outcomes. As AI continues to transform, the balance between innovation and safety will be crucial in shaping a sustainable and equitable future.


Read more »
Technology
AI Model AI Risks Generative AI Machine learning Technology

Here's How to Choose the Right AI Model for Your Requirements

 

When kicking off a new generative AI project, one of the most vital choices you'll make is selecting an ideal AI foundation model. This is not a small decision; it will have a substantial impact on the project's success. The model you choose must not only fulfil your specific requirements, but also be within your budget and align with your organisation's risk management strategies. 

To begin, you must first determine a clear goal for your AI project. Whether you want to create lifelike graphics, text, or synthetic speech, the nature of your assignment will help you choose the proper model. Consider the task's complexity as well as the level of quality you expect from the outcome. Having a specific aim in mind is the first step towards making an informed decision.

After you've defined your use case, the following step is to look into the various AI foundation models accessible. These models come in a variety of sizes and are intended to handle a wide range of tasks. Some are designed for specific uses, while others are more adaptable. It is critical to include models that have proven successful in tasks comparable to yours in your consideration list. 

Identifying correct AI model 

Choosing the proper AI foundation model is a complicated process that includes understanding your project's specific demands, comparing the capabilities of several models, and taking into account the operational context in which the model will be implemented. This guide synthesises the available reference material and incorporates extra insights to provide an organised method to choosing an AI base model. 

Identify your project targets and use cases

The first step in choosing an AI foundation model is to determine what you want to achieve with your project. Whether your goal is to generate text, graphics, or synthetic speech, the nature of your task will have a considerable impact on the type of model that is most suitable for your needs. Consider the task's complexity and the desired level of output quality. A well defined goal will serve as an indicator throughout the selecting process. 

Figure out model options 

Begin by researching the various AI foundation models available, giving special attention to models that have proven successful in jobs comparable to yours. Foundation models differ widely in size, specialisation, and versatility. Some models are meant to specialise on specific functions, while others have broader capabilities. This exploratory phase should involve a study of model documentation, such as model cards, which include critical information about the model's training data, architecture, and planned use cases. 

Conduct practical testing 

Testing the models with your specific data and operating context is critical. This stage ensures that the chosen model integrates easily with your existing systems and operations. During testing, assess the model's correctness, dependability, and processing speed. These indicators are critical for establishing the model's effectiveness in your specific use case. 

Deployment concerns 

Make the deployment technique choice that works best for your project. While on-premise implementation offers more control over security and data privacy, cloud services offer scalability and accessibility. The decision you make here will mostly depend on the type of application you're using, particularly if it handles sensitive data. In order to handle future expansion or requirements modifications, take into account the deployment option's scalability and flexibility as well. 

Employ a multi-model strategy 

For organisations with a variety of use cases, a single model may not be sufficient. In such cases, a multi-model approach can be useful. This technique enables you to combine the strengths of numerous models for different tasks, resulting in a more flexible and durable solution. 

Choosing a suitable AI foundation model is a complex process that necessitates a rigorous understanding of your project's requirements as well as a thorough examination of the various models' characteristics and performance. 

By using a structured approach, you can choose a model that not only satisfies your current needs but also positions you for future advancements in the rapidly expanding field of generative AI. This decision is about more than just solving a current issue; it is also about positioning your project for long-term success in an area that is rapidly growing and changing.
Read more »
Technology
AI Jobs AI Potentials AI Risks Artificial Intelligence Frontier AI Rishi Sunak Tech Summit Technology

Rishi Sunak Outlines Risks and Potential of AI Ahead of Tech Summit


UK Prime Minister Rishi Sunak has warned against the use of AI, as it could be used to design chemical and biological weapons. He says that, in the worst case scenario, people are likely to lose all control over AI, preventing it from turning off. 

However, he notes that while the potential for harm in AI usage is disputed, “we must not put heads in the sand,” over AI risks.

Sunak notes that the technology is already creating new job opportunities and that its advancement would catalyze economic growth and productivity, though he acknowledged that it would have an impact on the labor market.

“The responsible thing for me to do is to address those fears head on, giving you the peace of mind that we will keep you safe, while making sure you and your children have all the opportunities for a better future that AI can bring[…]Doing the right thing, not the easy thing, means being honest with people about the risks from these technologies,” Sunak stated. On Wednesday, the government had released documents highlighting the risks of AI. 

Existential risks from the technology cannot be ruled out, according to one research on the future risks of frontier AI, the term given to frontier AI systems will be discussed at the summit. 

“Given the significant uncertainty in predicting AI developments, there is insufficient evidence to rule out that highly capable Frontier AI systems, if misaligned or inadequately controlled, could pose an existential threat.”

The paper also presents several concerning scenarios about the advancement of AI.

One warns of the potential backlash from the public, as their jobs are being taken by AI. “AI systems are deemed technically safe by many users … but they are nevertheless causing impacts like increased unemployment and poverty,” says the paper, creating a “fierce public debate about the future of education and work”.

In another case mentioned in the document, dubbed as the ‘Wild West,’ the illicit use of AI to commit fraud and scams leads to social instability as a result of numerous victims of organized crime, widespread trade secret theft by enterprises, and an increase in the amount of AI-generated content that clogs the internet.

“This could lead to ‘personalised’ disinformation, where bespoke messages are targeted at individuals rather than larger groups and are therefore more persuasive,” said the discussion document, cautioning of the potential decrease in public trust when it comes to factual information and in civic processes like elections.

“Frontier AI can be misused to deliberately spread false information to create disruption, persuade people on political issues, or cause other forms of harm or damage,” it says. In regards to the documents, Mr. Sunak added that among the aforementioned risks outlined in the document was also a risk of AI being used by terrorist groups, "to spread fear and disruption on an even greater scale."

He notes that reducing the danger of AI causing the extinction of humans should be a "global priority".

However, he stated: "This is not a risk that people need to be losing sleep over right now and I don't want to be alarmist." He said that, on the whole, he was "optimistic" about AI's capacity to improve people's lives.

The disruption AI is already causing in the workplace is a threat that many will be far more familiar with.

Mr. Sunak emphasized how effectively AI technologies do administrative duties that are typically performed by an employee manually, such as drafting contracts and assisting in decision-making.

He added that technology has always changed how people generate money and that education is the best way to prepare individuals for the shifting market. For example, automation has already altered the nature of employment in factories and warehouses, but it has not completely eliminated human involvement.

The prime minister encouraged people to see artificial intelligence as a "co-pilot" in the day-to-day operations of the workplace, saying it was oversimplified to suggest the technology will "take people's jobs".  

Read more »
Subscribe to: Comments (Atom)