Search This Blog
Powered by Blogger.

Blog Archive

Labels

Load More
Trendy Right Now

Popular Posts

Showing posts with label LLM. Show all posts
Technology
AI CISO cyber threat Data protection LLM Machine learning Ransomware Technology

Enterprise AI Adoption Raises Cybersecurity Concerns

 




Enterprises are rapidly embracing Artificial Intelligence (AI) and Machine Learning (ML) tools, with transactions skyrocketing by almost 600% in less than a year, according to a recent report by Zscaler. The surge, from 521 million transactions in April 2023 to 3.1 billion monthly by January 2024, underscores a growing reliance on these technologies. However, heightened security concerns have led to a 577% increase in blocked AI/ML transactions, as organisations grapple with emerging cyber threats.

The report highlights the developing tactics of cyber attackers, who now exploit AI tools like Language Model-based Machine Learning (LLMs) to infiltrate organisations covertly. Adversarial AI, a form of AI designed to bypass traditional security measures, poses a particularly stealthy threat.

Concerns about data protection and privacy loom large as enterprises integrate AI/ML tools into their operations. Industries such as healthcare, finance, insurance, services, technology, and manufacturing are at risk, with manufacturing leading in AI traffic generation.

To mitigate risks, many Chief Information Security Officers (CISOs) opt to block a record number of AI/ML transactions, although this approach is seen as a short-term solution. The most commonly blocked AI tools include ChatGPT and OpenAI, while domains like Bing.com and Drift.com are among the most frequently blocked.

However, blocking transactions alone may not suffice in the face of evolving cyber threats. Leading cybersecurity vendors are exploring novel approaches to threat detection, leveraging telemetry data and AI capabilities to identify and respond to potential risks more effectively.

CISOs and security teams face a daunting task in defending against AI-driven attacks, necessitating a comprehensive cybersecurity strategy. Balancing productivity and security is crucial, as evidenced by recent incidents like vishing and smishing attacks targeting high-profile executives.

Attackers increasingly leverage AI in ransomware attacks, automating various stages of the attack chain for faster and more targeted strikes. Generative AI, in particular, enables attackers to identify vulnerabilities and exploit them with greater efficiency, posing significant challenges to enterprise security.

Taking into account these advancements, enterprises must prioritise risk management and enhance their cybersecurity posture to combat the dynamic AI threat landscape. Educating board members and implementing robust security measures are essential in safeguarding against AI-driven cyberattacks.

As institutions deal with the complexities of AI adoption, ensuring data privacy, protecting intellectual property, and mitigating the risks associated with AI tools become paramount. By staying vigilant and adopting proactive security measures, enterprises can better defend against the growing threat posed by these cyberattacks.

Read more »
Technology
Advance Tools AI Gaming Artificial Intelligence LLM NPCs Technology

Here Are Three Ways AI Will Transform Gaming Forever

 

Technology has been impacted by artificial intelligence in practically every field. You would struggle to identify a tech-related field where artificial intelligence hasn't had some sort of impact, from data analysis to art programmes. AI hasn't advanced as quickly in video games as it has in other fields, but even in this field, there are still some fascinating advancements that have the potential to completely transform the gaming experience. 

Of course, developers are already utilising generic AI technologies to assist them create content for their games, such as generating art, writing scripts, and finding ideas for what to do next. But in certain instances, artificial intelligence (AI) has transformed gaming and accomplished tasks that would be extremely laborious or impossible for a human to complete. 

AI can design NPCs that respond to your words 

Making a game in which the main character speaks exactly what the player wants to say can be quite difficult. When continuing the tale, you can only provide the player a limited number of options, and even then, some gamers will want to divert the conversation or ask a question that the creator did not consider. And because everything is strictly scripted, the player has little freedom to interact with the non-player character (NPCs) as they see fit.

However, an AI LLM can help with this. A developer can connect an NPC to an AI and have it manage your responses, much like you do with a chatbot like ChatGPT. That way, you may ask the character whatever questions you want, and the AI will analyse the character it has been assigned to roleplay and reply appropriately. Best of all, once AI PCs take off, you won't need an internet connection to communicate with an external AI model; everything can be handled on your hardware.

AI can assist lip-sync character's lines

While AI-powered games are now on the market, other technologies are still being developed. One of these is Audio2Face, which Nvidia introduced as part of its efforts to integrate AI into game creation. Audio2Face employs artificial intelligence to automatically match a character's mouth movements to their dialogue, eliminating the need for an animator to perform the lip-syncing oneself. Nividia notes in its blog post how this technique will make localization much easier because developers will not have to adjust the lip sync for each language. Instead, they can have Audio2Face process the animation for them.

While Nvidia did not directly state it in their post, Audio2Face is likely to be used in conjunction with AI-generated chat. After all, if NPCs are generating language in real time, they'll require lip-syncing technology that can precisely animate the mouth on the fly. 

Turn 2D images into 3D objects 

Another recently introduced technique is Stability AI's 2D-to-3D converter. The premise behind this AI tool is that you may submit a 2D photo of an object, and it will do its best to create a 3D model of it. Most of the magic comes from the AI guessing what's on the other side of the object, which it does surprisingly well. 

Of course, this has the potential to allow developers to swiftly add 3D models to their games; simply take a photo of the thing they want to import and add it in. However, there is also the possibility of creating a game in which people can upload photographs of things around their house, which are then incorporated to the game.
Read more »
Threat Intelligence
AI Model LLM Technology Threat Intelligence

Data Collaboration Platforms Ruling the Charts in Unlocking Sophisticated AI Models

 

Large Language Models (LLMs) have opened up exciting new possibilities for organisations in the field of artificial intelligence (AI), including enhanced decision-making, streamlined processes, and ground-breaking innovation.

Leading companies like Zendesk, Slack, Goldman Sachs, GitHub, and Unilever have used LLMs to enhance customer service, streamline coding processes, and effectively respond to consumer queries. However, given their strength, LLMs frequently prove inadequate when faced with the particular complexities of an organisation's environment. 

Training issues with refined AI models 

Businesses have resorted to employing organisation-specific data to fine-tune LLMs in order to conquer such challenges, resulting in highly customised AI models. 

These fine-tuned models provide a customised AI experience that significantly improves organisational performance. 

However, entering the field of fine-tuning AI models presents companies with three significant challenges. The task requires significant access to high-quality data, which is often a limited resource for many businesses. Second, LLMs are based on publicly available online content, which may result in biases and a lack of diversity and pluralism in created content.

Training fine-tuned models on consumers' personal data results in serious privacy concerns, perhaps leading to regulatory violations. 

Navigating the data issues in fine-tuning AI 

Fine-tuned AI models thrive on large, diversified datasets. However, numerous businesses confront difficulty in acquiring the essential data, particularly in niche or specialized domains. 

The challenge is worsened when the available data is unstructured or of low quality, making it difficult to extract useful insights. Beyond quantity, data relevance, quality, and the representation of varied perspectives are also critical factors. 

Generic AI models, like LLMs, mostly reflect the overall internet, ignoring the subtleties of unique communities or user groups. As a result, these models frequently generate biassed, culturally insensitive, or inadequate results, ignoring specific community experiences and perspectives.

To ensure that AI responses are fair, inclusive, and culturally aware, organisations must fill these models with data that truly represents societal diversity. 

Embracing data collaboration platforms 

Business leaders that embrace data collaboration platforms can reap numerous benefits. These platforms allow access to high-quality data, safeguard against legal challenges, and present a varied, pluralistic view of AI.

Business leaders should consider taking a few crucial actions in order to fully realise the potential of refined models.

Off-the-shelf AI solutions, however powerful, may lack the context and nuances unique to a certain organisation. Customisation is critical for aligning AI models with specific requirements. 

High-quality and diversified datasets are required for accurate and impartial AI results. Data collaborations can help models perform better and have more diversity.

Consider working together even with rival companies, in addition to alliances with partners and clients. The industry as a whole can gain from cooperative efforts that result in innovations and efficiencies. 

Models need to be updated with the latest statistics because data is perishable. Find sources of up-to-date information pertinent to AI's problem-solving objectives.
Read more »
Privacy Centre
Artificial Inteligence ChatGPT Cyberattacks CyberCrime Data Breach LLM Privacy Centre

The Rise of AI Restrictions: 25% of Firms Slam the Door on AI Magic

 


When ChatGPT was first released to the public, several corporate titans, from Apple to Verizon, made headlines when they announced bans on the use of this software at work shortly after it was introduced. However, a recent study confirms that those companies are not anomalous. 

It has recently been reported that more than 1 in 4 companies have banned the use of generative artificial intelligence tools at work at some point in time, based on a Cisco survey conducted last summer among 2,600 privacy and security professionals. 

According to the survey, 63% of respondents said that they limit the amount of data employees can enter into these systems, and 61% said that they restrict which generative AI tools employees can use within their organizations. Approximately one-quarter of companies have banned their employees from using generative artificial intelligence, according to a new Cisco survey. 

Based on the annual Data Privacy Benchmark Study, conducted by the firm, a survey of 2,600 privacy and security professionals across 12 countries, two-thirds of those surveyed impose restrictions on the types of information that can be entered into LLM-based systems, as well as prohibiting specific applications from being used. 

According to Robert Waitman, director of Cisco's Privacy Center of Excellence, who wrote a blog post about the survey, over two-thirds of respondents expressed concern that their data would be disclosed to competitors or the public, a concern that may not be met by the majority. The information they entered about the company was not entirely public (48% of the respondents), which could pose a problem. 

There are a lot of concerns about the use of AI that involves their data today, and 91% of organizations are aware that they need to do more to make sure customers feel confident that their data is used for the intended and legitimate purposes in AI. There has been little progress in building consumer trust over the past year as this level is similar to last year's level, suggesting that not much progress has been made. 

Organizations' priorities differ from individuals' when it comes to building consumer trust. As a consumer, one of the most important things to be concerned about is getting clear information about exactly how their data is being used and not having it sold to marketers. A survey of businesses conducted by the American Association of Professionals revealed that compliance with privacy laws is the top priority (25%) along with avoiding data breaches (23%). 

Furthermore, it indicates that a greater focus on transparency would be beneficial — particularly in AI applications, where understanding how algorithms make decisions can be difficult. Over the past five years, there has been a more than double increase in privacy spending, a rise in benefits, and a steady return on investment. 

It was reported this year that 95% of respondents indicated that privacy benefits outweigh the costs, with an average organization reporting 1.6 times the return on investment they received from privacy. Additionally, 80% of respondents indicated they had benefited from their privacy investments in terms of higher levels of loyalty and trust, and that number was even higher (92%) among the most privacy-aware organizations. 

Since last year, the largest organizations with 10,000+ employees have increased their privacy spending by around 7-8% in terms of their spending on privacy. The number of investments was lower for smaller organizations, however. The average privacy investment for businesses with 50-249 employees was decreased by a fourth on average than that for businesses with 50-499 employees. 

“The survey results revealed that 94% of respondents would not buy from Cisco if they did not adequately protect their customers' data. According to Harvey Jang, Cisco Vice President and Chief Privacy Officer, “Customers are looking for hard evidence that an organization can be trusted.” 

Privacy has become inextricably linked with customer trust and loyalty. Investing in privacy can help organizations leverage AI ethically and responsibly in the era of AI, and this is especially true as AI becomes more prevalent.
Read more »
Technology
Amazon Web Services Artifcial Intelligence Internet Data LLM Technology

The Impact of AI-Generated Content on Internet Quality

 



In a comprehensive study conducted by the Amazon Web Services (AWS) AI Lab, a disconcerting reality has surfaced, shaking the foundations of internet content. Shockingly, an extensive 57.1% of all sentences on the web have undergone translation into two or more languages, and the culprit behind this linguistic convolution is none other than large language model (LLM)-powered AI.

The crux of the issue resides in what researchers term as "lower-resource languages." These are languages for which there is a scarcity of data available for the effective training of AI models. The domino effect begins with AI generating vast quantities of substandard English content. Following this, AI-powered translation tools enter the stage, exacerbating the degradation as they transcribe the material into various other languages. The motive behind this cascade of content manipulation is a profit-driven strategy, aiming to capture clickbait-driven ad revenue. The outcome is the flooding of entire internet regions with an abundance of deteriorating AI-generated copies, creating a dreading universe of misinformation.

The AWS researchers express profound concern, eemphasising that machine-generated, multi-way parallel translations not only dominate the total translated content in lower-resource languages but also constitute a substantial fraction of the overall web content in those languages. This amplifies the scale of the issue, underscoring its potential to significantly impact diverse online communities.

The challenges posed by AI-generated content are not isolated incidents. Tech giants like Google and Amazon have grappled with the ramifications of AI-generated material affecting their search algorithms, news platforms, and product listings. The issues are multifaceted, encompassing not only the degradation of content quality but also violations of ethical use policies.

While the English-language web has been experiencing a gradual infiltration of AI-generated content, the study highlights that non-English speakers are facing a more immediate and critical problem. Beyond being a mere inconvenience, the prevalence of AI-generated gibberish raises a formidable barrier to the effective training of AI models in lower-resource languages. This is a significant setback for the scientific community, as the inundation of nonsensical translations hinders the acquisition of high-quality data necessary for training advanced language models.

The pervasive issue of AI-generated content poses a substantial threat to the usability of the web, transcending linguistic and geographical boundaries. Striking a balance between technological advancements and content reliability is imperative for maintaining the internet as a trustworthy and informative space for users globally. Addressing this challenge requires a collaborative effort from researchers, industry stakeholders, and policymakers to safeguard the integrity of online information. Otherwise this one-stop digital world that we all count on to disseminate information is destined to be doomed. 



Read more »
Threat Landscape
Artificial Intelligence LLM Malicious Backdoor Technology Threat Landscape

Anthropic Research Indicates That AI Algorithms May Turn Into "Sleeper Cell" Backdoors

 

While AI tools offer companies and online users novel avenues, they also have the potential to significantly boost the accessibility and potency of certain forms of illegal activity and crimes. For example, take the latest study that revealed large language models can be turned into malicious backdoors, which have the potential to cause quite a bit of mayhem for users. 

The study was released by Anthropic, the AI business that created the popular chatbot Claude and has funding from Google and Amazon. Anthropic researchers claim in their research that AI algorithms are susceptible to being transformed into what are essentially "sleeper cells." Such cells could look innocuous, but if specific requirements are met, they might be designed to act maliciously, such as adding weak code to a codebase. 

For example, the researchers created a scenario in which an LLM is configured to function normally in 2023, but when 2024 arrives, the malicious "sleeper" suddenly wakes up and starts generating malicious code. The research suggests that such programs could possibly be designed to exhibit negative behaviour in response to particular cues. 

Given that AI programs have grown immensely popular among software authors over the past year, the findings of this study appear to be quite alarming. It's easy to picture a scenario in which a coder uses a popular, open-source algorithm to help them with their development tasks, only for it to turn malicious at some point and start making their product less secure and hackable.

“We believe that our code vulnerability insertion backdoor provides a minimum viable example of a real potential risk...Such a sudden increase in the rate of vulnerabilities could result in the accidental deployment of vulnerable model-written code even in cases where safeguards prior to the sudden increase were sufficient,” the company stated. 

If it appears strange that an AI company would release research demonstrating how its own technology can be so horribly exploited, consider that the AI models most vulnerable to this type of "poisoning" are open source—that is, code that is flexible, non-proprietary, and easily shared and adapted online. Notably, Anthropic is closed-source. It is also a founding member of the Frontier Model Forum, a group of AI companies whose products are primarily closed-source and have campaigned for stricter "safety" rules in AI development.
Read more »
Technology
AI Tool Data Safety Generative AI LLM Private Data Technology

Anthropic Pledges to Not Use Private Data to Train Its AI

 

Anthropic, a leading generative AI startup, has announced that it would not employ its clients' data to train its Large Language Model (LLM) and will step in to safeguard clients facing copyright claims.

Anthropic, which was established by former OpenAI researchers, revised its terms of service to better express its goals and values. The startup is setting itself apart from competitors like OpenAI, Amazon, and Meta, which do employ user material to enhance their algorithms, by severing the private data of its own clients. 

The amended terms state that Anthropic "may not train models on customer content from paid services" and that Anthropic "as between the parties and to the extent permitted by applicable law, Anthropic agrees that customer owns all outputs, and disclaims any rights it receives to the customer content under these terms.” 

The terms also state that they "do not grant either party any rights to the other's content or intellectual property, by implication or otherwise," and that "Anthropic does not anticipate obtaining any rights in customer content under these terms."

The updated legal document appears to give protections and transparency for Anthropic's commercial clients. Companies own all AI outputs developed, for example, to avoid possible intellectual property conflicts. Anthropic also promises to defend clients against copyright lawsuits for any unauthorised content produced by Claude. 

The policy complies with Anthropic's mission statement, which states that AI should to be honest, safe, and helpful. Given the increasing public concern regarding the ethics of generative AI, the company's dedication to resolving issues like data privacy may offer it a competitive advantage.

Users' Data: Vital Food for LLMs

Large Language Models (LLMs), such as GPT-4, LlaMa, and Anthropic's Claude, are advanced artificial intelligence systems that comprehend and generate human language after being trained on large amounts of text data. 

These models use deep learning and neural networks to anticipate word sequences, interpret context, and grasp linguistic nuances. During training, they constantly refine their predictions, improving their capacity to communicate, write content, and give pertinent information.

The diversity and volume of the data on which LLMs are trained have a significant impact on their performance, making them more accurate and contextually aware as they learn from different language patterns, styles, and new information.

This is why user data is so valuable for training LLMs. For starters, it keeps the models up to date on the newest linguistic trends and user preferences (such as interpreting new slang).

Second, it enables personalisation and increases user engagement by reacting to specific user activities and styles. However, this raises ethical concerns because AI businesses do not compensate users for this vital information, which is used to train models that earn them millions of dollars.
Read more »
Technology
Artificial Intelligence ChstGPT Cyberattacks CyberCrime Jailbreaking Language Model LLM Technology

Breaking Boundaries: Language Models Empower Each Other in Automated Jailbreaking

 


Increasing usage of large language models in industry has resulted in a flood of research activity that aims to find out whether LLMs have a tendency to generate hurtful or biased content when persuaded in a particular way or when using specific inputs.

It has just been published in a new paper from researchers at Robust Intelligence and Yale University that describes the latest development in the field of black box LLMs, which promises to enable even the most state-of-the-art black box LLMs to escape guardrails and generate toxic output by fully automating the process. 

A new preprint study shows just how to trick AIs into giving up some of the secrets they have been keeping from users that can be dangerous in the future. Currently, chatbots have built-in restrictions to keep them from revealing anything dangerous to users. As most people are aware, today's machines can act as fictional characters or mimic specific personalities by feigning to have specific personalities or posing as specific roles. 

Using that ability, the new study was able to enlist the assistance of a chatbot which has been used extensively in artificial intelligence to get the job done. Taking advantage of this assistant, the researchers directed him to work on prompts that would be able to "jailbreak" other chatbots-destroying the guardrails that had been embedded into them. 

The term "black box LLM" refers to a large language model, such as the one behind ChatGPT, which is not publicly available regarding its architecture, datasets, training methodologies, and other details of development. A new method, which has been dubbed Tree of Attacks with Pruning (TAP) by the researchers, consists of using a nonaligned LLM to "jailbreak" another aligned LLM in order to break through its guardrails and to reach its goals quite swiftly and effectively. 

It should be noted that the objective of an LLM designed for alignment such as the one behind ChatGPT and other AI chatbots is explicitly to minimize the potential for harm and would not, for instance, provide information on how to build a bomb in response to a request for such information. A non-aligned LLM is optimized in order to increase accuracy as well as to contain fewer constraints compared to an aligned LLM. 

With the help of models like ChatGPT, users have been delighted by the ability of these models to process outside prompts and (in some cases) produce organized, actionable responses based on massive data sets that have been collected in the past. As a result, there are a number of possible uses for artificial intelligence that have expanded our collective understanding of what is possible in the age of artificial intelligence. 

When these LLMs began to become widely used by the public, however, they started causing a host of problems as soon as they became widely known. The problems began to arise from hallucination (the act of inventing facts, studies, or events in an elaborate manner) as well as inaccurate information provided to the opposing party. Provide accurate (but objectionable, dangerous, or harmful) answers to questions like, "How do I build a bomb?" or "How can I write a program to exploit this vulnerability?" LLM-based AI systems can be attacked using a variety of attack tactics, and this is true for several different kinds of AI. 

A prompt attack can be defined as the act of using prompts to make the model produce answers that, by definition, it should not produce in theory. The problem with AI models is that they can be backdoored (forced to generate incorrect outputs when triggered) and their training data can be extracted - or poisoned - to generate incorrect outputs. 

In situations of adversarial examples, a model can be "confused" with unexpected (but predictable) results due to inputs generated by adversarial examples. Researchers from Yale and Robust Intelligence have developed a machine learning technique that uses automated adversarial adversarial learning to defeat that last category of attacks by overriding the control structures (“guardrails”) that normally prevent them from achieving success. 

There are many LLMs on the market that feature AI to generate useful content at scale, and GPT-4 is one such example. As a result, if these capabilities are not checked, those same capabilities may also be used for harmful purposes. Recent research has led to the development of techniques designed to retool LLMs into malicious systems used to mislead, contaminate, and commit fraud in an attempt to increase their power. 

There is also an opportunity for misuse of open-source LLMs that lack safety measures, which can be run automatically on a local machine without any restrictions. In the case of GPT-Neo, for instance, it can be considered a major security risk when it is not used under one's control. 

An LLM-based AI system can be attacked to produce results for a variety of reasons, and this is true for many different kinds of AI systems. One such attack is prompting the model with questions which are intended to induce it to produce answered in a way it should not be able to do based on the model's definition.
Read more »
Technology
Artificial Intelligence ChatGPT Generative AI LLM Technology

Managing the Security and Privacy Issues with Large Language Models

 

Everyone is buzzing about ChatGPT, Bard, and generative AI. But, inevitably, the reality check follows the hype. While business and IT leaders are excited about the disruptive potential of technology in areas such as customer service and software development, they are also becoming more aware of some potential downsides and risks to be aware of. 

In short, for organisations to realise the full potential of large language models (LLMs), they need to be able to deal with the hidden risks that could otherwise undermine the technology's business value. 

What exactly are LLMs? 

LLMs power ChatGPT and other generative AI tools. They process massive amounts of text data using artificial neural networks. The model can interact with users in natural language after learning the patterns between words and how they are used in context. In fact, one of the main reasons for ChatGPT's extraordinary success is its ability to tell jokes, compose poems, and communicate in a way that is difficult to distinguish from that of a real human. 

The LLM-powered generative AI models used in chatbots like ChatGPT function like supercharged search engines, answering questions and finishing tasks with human-like language using the data they were trained on. LLM-based generative AI, whether publicly available or proprietary models used internally within an organisation, can expose businesses to security and privacy risks. Here are the three of the most prevalent LLM risks: 

Excessive sharing of sensitive information 

LLM-based chatbots are not adept at maintaining secrets, or even forgetting them. This implies that any data you enter could be incorporated into the model and shared with others, or at the very least, used to train LLM models in the future. When Samsung employees used ChatGPT for work-related purposes and disclosed sensitive information, they discovered this to their disadvantage. The code and meeting recordings that they input into the tool might potentially be in the public domain, or at the very least saved for later use, as the National Cyber Security Centre of the United Kingdom recently noted. We looked more closely at how businesses can use LLMs without compromising their data earlier this year. 

Copyright issues

LLMs are trained on huge quantities of data. However, that data is frequently scraped from the web without the explicit authorization of the content owner. If you continue to use it, you may encounter copyright issues. However, finding the original source of specific training data can be difficult, making it difficult to mitigate these issues. 

Unsafe code 

More and more developers are using ChatGPT and related tools to shorten their time to market. Theoretically, it can be useful by rapidly and effectively generating snippets of code and even full software programmes. Security experts warn that it may also lead to vulnerabilities. If the developer lacks sufficient domain knowledge to identify which bugs to look for, this is especially concerning. If flawed code is then released into production, it might severely damage the company's reputation and cost money and effort to fix.

Mitigation tips 

Data encryption and anonymization: To keep data safe from prying eyes, encrypt it before sharing it with LLMs, and/or consider anonymization techniques to safeguard the privacy of individuals who could be recognised in the datasets. Data sanitization, which removes sensitive information from training data before it is entered into the model, can achieve the same result. 

Enhanced access controls: Strong passwords, multi-factor authentication (MFA), and least privilege policies will help ensure that only authorised individuals have access to the generative AI model and back-end systems. 

Regular security audits: This can help in the identification of vulnerabilities in your IT systems that may have an impact on the LLM and generative AI models on which they are based. 

Fortunately, there's no need to start from scratch. These are mainly tried-and-true security best practises. For the AI world, they might need to be updated or adjusted, but most security teams should be able to understand the underlying reasoning.
Read more »
WormGPT
AI Chatbots Artificial Intelligence Black market AI chatbots ChatGPT CyberCrime Evil AI models LLM notorious AI versions OpenAI Phishing Attack Technology WormGPT

Inside the Realm of Black Market AI Chatbots


With AI tools helping organizations and online users in a tremendously proficient way, there are obvious dark-sides of this trending technology. One of them being the notorious versions of AI bots.

A user may as well gain access to one such ‘evil’ version of OpenAI’s ChatGPT. While these AI versions may not necessarily by legal in some parts of the world, it could be pricey. 

Gaining Access to Black Market AI Chatbots

Gaining access to the evil chatbot versions could be tricky. To do so, a user must find the right web forum with the right users. To be sure, these users might have posted the marketed a private and powerful large language model (LLM). One can get in touch with these users in encrypted messaging services like Telegram, where they might ask for a few hundred crypto dollars for an LLM. 

After gaining the access, users can now do anything, especially the ones that are prohibited in ChatGPT and Google’s Bard, like having conversation with the AI on how to make pipe bombs or cook meth, engaging in discussions about any illegal or morally questionable subject under the sun, or even using it to finance phishing schemes and other cybercrimes.

“We’ve got folks who are building LLMs that are designed to write more convincing phishing email scams or allowing them to code new types of malware because they’re trained off of the code from previously available malware[…]Both of these things make the attacks more potent, because they’re trained off of the knowledge of the attacks that came before them,” says Dominic Sellitto, a cybersecurity and digital privacy researcher at the University of Buffalo.

These models are becoming more prevalent, strong, and challenging to regulate. They also herald the opening of a new front in the war on cybercrime, one that cuts far beyond text generators like ChatGPT and into the domains of audio, video, and graphics. 

“We’re blurring the boundaries in many ways between what is artificially generated and what isn’t[…]“The same goes for the written text, and the same goes for images and everything in between,” explained Sellitto.

Phishing for Trouble

Phishing emails, which demand that a user provide their financial information immediately to the Social Security Administration or their bank in order to resolve a fictitious crisis, cost American consumers close to $8.8 billion annually. The emails may contain seemingly innocuous links that actually download malware or viruses, allowing hackers to take advantage of any sensitive data directly from the victim's computer.

Fortunately, these phishing mails are quite easy to detect. In case they have not yet found their way to a user’s spam folder, one can easily identify them on the basis of their language, which may be informal and grammatically incorrect wordings that any legit financial firm would never use. 

However, with ChatGPT, it is becoming difficult to spot any error in the phishing mails, bringing about a veritable AI generative boom. 

“The technology hasn’t always been available on digital black markets[…]It primarily started when ChatGPT became mainstream. There were some basic text generation tools that might have used machine learning but nothing impressive,” Daniel Kelley, a former black hat computer hacker and cybersecurity consultant explains.

According to Kelley, these LLMs come in a variety of forms, including BlackHatGPT, WolfGPT, and EvilGPT. He claimed that many of these models, despite their nefarious names, are actually just instances of AI jailbreaks, a word used to describe the deft manipulation of already-existing LLMs such as ChatGPT to achieve desired results. Subsequently, these models are encapsulated within a customized user interface, creating the impression that ChatGPT is an entirely distinct chatbot.

However, this does not make AI models any less harmful. In fact, Kelley believes that one particular model is both one of the most evil and genuine ones: According to one description of WormGPT on a forum promoting the model, it is an LLM made especially for cybercrime that "lets you do all sorts of illegal stuff and easily sell it online in the future."

Both Kelley and Sellitto agrees that WormGPT could be used in business email compromise (BEC) attacks, a kind of phishing technique in which employees' information is stolen by pretending to be a higher-up or another authority figure. The language that the algorithm generates is incredibly clear, with precise grammar and sentence structure making it considerably more difficult to spot at first glance.

One must also take this into account that with easier access to the internet, really anyone can download these notorious AI models, making it easier to be disseminated. It is similar to a service that offers same-day mailing for buying firearms and ski masks, only that these firearms and ski masks are targeted at and built for criminals.

Read more »
Two Factor Authentication
AI Chatbot AI Tool ChatGPT LLM OpenAI Plugins Privacy Security patches Sensitive data Software Two Factor Authentication

ChatGPT: Security and Privacy Risks

ChatGPT is a large language model (LLM) from OpenAI that can generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way. It is still under development, but it has already been used for a variety of purposes, including creative writing, code generation, and research.

However, ChatGPT also poses some security and privacy risks. These risks are highlighted in the following articles:

  • Custom instructions for ChatGPT: This can be useful for tasks such as generating code or writing creative content. However, it also means that users can potentially give ChatGPT instructions that could be malicious or harmful.
  • ChatGPT plugins, security and privacy risks:Plugins are third-party tools that can be used to extend the functionality of ChatGPT. However, some plugins may be malicious and could exploit vulnerabilities in ChatGPT to steal user data or launch attacks.
  • Web security, OAuth: OAuth, a security protocol that is often used to authorize access to websites and web applications. OAuth can be used to allow ChatGPT to access sensitive data on a user's behalf. However, if OAuth tokens are not properly managed, they could be stolen and used to access user accounts without their permission.
  • OpenAI disables browse feature after releasing it on ChatGPT app: Analytics India Mag discusses OpenAI's decision to disable the browse feature on the ChatGPT app. The browse feature allowed ChatGPT to generate text from websites. However, OpenAI disabled the feature due to security concerns.

Overall, ChatGPT is a powerful tool with a number of potential benefits. However, it is important to be aware of the security and privacy risks associated with using it. Users should carefully consider the instructions they give to ChatGPT and only use trusted plugins. They should also be careful about what websites and web applications they authorize ChatGPT to access.

Here are some additional tips for using ChatGPT safely:

  • Be careful what information you share with ChatGPT. Do not share any sensitive information, such as passwords, credit card numbers, or personal health information.
  • Use strong passwords and enable two-factor authentication on all of your accounts. This will help to protect your accounts from being compromised, even if ChatGPT is compromised.
  • Keep your software up to date. Software updates often include security patches that can help to protect your devices from attack.
  • Be aware of the risks associated with using third-party plugins. Only use plugins from trusted developers and be careful about what permissions you grant them.
While ChatGPT's unique instructions present intriguing potential, they also carry security and privacy risks. To reduce dangers and guarantee the safe and ethical use of this potent AI tool, users and developers must work together.

Read more »
Threat Landscape
AI Chatbot Artificial Intelligence LLM Online Safety Technology Threat Landscape

AI Experts Unearth Infinite ways to Bypass Bard and ChatGPT's Safety Measures

 

Researchers claim to have discovered potentially infinite ways to circumvent the safety measures on key AI-powered chatbots like OpenAI, Google, and Anthropic. 

Large language models, such as those used by ChatGPT, Bard, and Anthropic's Claude, are heavily controlled by tech firms. The devices are outfitted with a variety of safeguards to prevent them from being used for evil purposes, such as educating users on how to assemble a bomb or writing pages of hate speech.

Security analysts from Carnegie Mellon University in Pittsburgh and the Centre for A.I. Safety in San Francisco said last week that they have discovered ways to bypass these guardrails. 

The researchers identified that they might leverage jailbreaks built for open-source systems to attack mainstream and closed AI platforms. 

The report illustrated how automated adversarial attacks, primarily done by appending characters to the end of user inquiries, might be used to evade safety regulations and drive chatbots into creating harmful content, misinformation, or hate speech.

Unlike prior jailbreaks, the researchers' hacks were totally automated, allowing them to build a "virtually unlimited" number of similar attacks.

The researchers revealed their methodology to Google, Anthropic, and OpenAI. According to a Google spokesman, "while this is an issue across LLMs, we've built important guardrails into Bard - like the ones posited by this research - that we'll continue to improve over time." 

Anthropic representatives described jailbreaking measures as an active study area, with more work to be done. "We are experimenting with ways to enhance base model guardrails to make them more "harmless," said a spokesperson, "while also studying extra levels of defence." 

When Microsoft's AI-powered Bing and OpenAI's ChatGPT were made available, many users relished in finding ways to break the rules of the system. Early hacks were soon patched up by IT companies, including one where the chatbot was instructed to respond as if it had no content moderation.

The researchers did point out that it was "unclear" whether prominent model manufacturers would ever be able to entirely prevent such conduct. In addition to the safety of making potent open-source language models available to the public, this raises concerns about how AI systems are controlled.
Read more »
Threat Intelligence
AI AI Tool LLM Machine learning Technology Threat Intelligence

Enhancing Security and Observability with Splunk AI

 

During Splunk’s .conf23 event, the company announced Splunk AI, a set of AI-driven technologies targeted at strengthening its unified security and observability platform. This new advancement blends automation with human-in-the-loop experiences to enable organisations to improve their detection, investigation, and reaction skills while preserving control over AI implementation. 

The AI Assistant, which uses generative AI to give consumers an interactive conversation experience using natural language, is one of the major components of Splunk AI. Users can create Splunk Processing Language (SPL) queries through this interface, enhancing their expertise of the platform and optimising time-to-value. The AI Assistant intends to make SPL more accessible, democratising an organization's access to valuable data insights. 

SecOps, ITOps, and engineering teams can automate data mining, anomaly detection, and risk assessment thanks to Splunk AI. These teams can concentrate on more strategic duties and decrease errors in their daily operations by utilising AI capabilities. 

The AI model employed by Splunk AI is combined with ML techniques that make use of security and observability data along with domain-specific large language models (LLMs). It is possible to increase production and cut costs thanks to this connection. Splunk emphasises its dedication to openness and flexibility, enabling businesses to incorporate their artificial intelligence (AI) models or outside technologies. 

The enhanced alerting speed and accuracy offered by Splunk's new AI-powered functions boosts digital resilience. For instance, the anomaly detection tool streamlines and automates the entire operational workflow. Outlier exclusion is added to adaptive thresholding in the IT Service Intelligence 4.17 service, and "ML-assisted thresholding" creates dynamic thresholds based on past data and patterns to produce alerting that is more exact. 

Splunk also launched ML-powered fundamental products that give complete information to organisations. Splunk Machine Learning Toolkit (MLTK) 5.4 now provides guided access to machine learning (ML) technologies, allowing users of all skill levels to leverage forecasting and predictive analytics. This toolkit can be used to augment the Splunk Enterprise or Cloud platform using techniques including as outlier and anomaly detection, predictive analytics, and clustering. 

The company emphasises domain specialisation in its models to better detection and analysis. It is critical to tune models precisely for their respective use cases and to have specialists in the industry design them. While generic large language models can be used to get started, purpose-built complicated anomaly detection techniques necessitate a distinct approach.
Read more »
User Privacy
AI Chatbot ChatGPT Data Breach LLM Microsoft OpenAI Source Code User Privacy

ChatGPT Sparking Security Concerns

 

Cyberhaven, a data security company, recently released a report in which it found and blocked requests to input data into ChatGPT from 4.2% of the 1.6 million employees at its client companies due to the potential leakage of sensitive information to the LLM, including client data, source code, and regulated information.

The appeal of ChatGPT has skyrocketed. It became the fastest-growing consumer application ever released after only two months of release when it reached 100 million active users. Users are drawn to the tool's sophisticated skills, but they are also concerned about its potential to upend numerous industries.ChatGPT was given 300 billion words by OpenAI, the firm that created it. These words came from books, articles, blogs, and posts on the Internet, as well as personally identifiable information that was illegally stolen.

Following Microsoft's $1 billion investment in the parent company of ChatGPT, OpenAI, in January, ChatGPT is expected to be rolled out across all Microsoft products, including Word, Powerpoint, and Outlook.

Employees are providing sensitive corporate data and privacy-protected information to large language models (LLMs), like ChatGPT, which raises concerns that the data may be incorporated into the models of artificial intelligence (AI) services, and that information may be retrieved at a later time if adequate data security isn't implemented for the service.

The growing acceptance of OpenAI's ChatGPT, its core AI model, the Generative Pre-trained Transformer, or GPT-3, as well as other LLMs, businesses, and security experts have started to be concerned that sensitive data consumed as training data into the models could reemerge when prompted by the appropriate queries. Some are acting: JPMorgan, for instance, restricted employees' access to ChatGPT, and Amazon, Microsoft, and Wal-Mart cautioned staff to use generative AI services carefully.

Some AI-based services, outside of those that are GPT-based, have sparked concerns about whether they are risky. For example, Otter.ai, an automated transcription service, converts audio files into text while automatically identifying speakers, allowing for the tagging of crucial words and phrases, and underlining of key phrases. Journalists have raised concerns about the company's storage of that information in its cloud.

Cyberhaven's Ting predicts that the adoption of generative AI apps will continue to grow and be used for a variety of tasks, including creating memos and presentations, identifying security incidents, and interacting with patients. His predictions are based on conversations with the clients of his company.

Because only a few individuals handle the majority of the dangerous requests, education could have a significant impact on whether data leaks from a particular organization. According to Ting of Cyberhaven, less than 1% of employees are accountable for 80% of the instances of providing critical data to ChatGPT.

The LLM's access to sensitive data and personal information is also being restricted by OpenAI and other businesses: Nowadays, when ChatGPT is asked for personal information or sensitive corporate data, canned responses are used as an excuse not to cooperate.


Read more »
Technology
Artificial Intelligence ChatGPT GitHub Large Language Model LLaMA LLM Meta OpenAI Technology

Meta Announces a New AI-powered Large Language Model


On Friday, Meta introduced its new AI-powered large language model (LLM) named LLaMA-13B that, in spite of being "10x smaller," can outperform OpenAI's GPT-3 model. Language assistants in the ChatGPT style could be run locally on devices like computers and smartphones, thanks to smaller AI models. It is a part of the brand-new group of language models known as "Large Language Model Meta AI," or LLAMA. 

The size of the language models in the LLaMA collection ranges from 7 billion to 65 billion parameters. In contrast, the GPT-3 model from OpenAI, which served as the basis for ChatGPT, has 175 billion parameters. 

Meta can potentially release its LLaMA model and its weights available as open source, since it has trained models through the openly available datasets like Common Crawl, Wkipedia, and C4. Thus, marking a breakthrough in a field where Big Tech competitors in the AI race have traditionally kept their most potent AI technology to themselves.   

In regards to the same, Project member Guillaume’s tweet read "Unlike Chinchilla, PaLM, or GPT-3, we only use datasets publicly available, making our work compatible with open-sourcing and reproducible, while most existing models rely on data which is either not publicly available or undocumented." 

Meta refers to its LLaMA models as "foundational models," which indicates that the company intends for the models to serve as the basis for future, more sophisticated AI models built off the technology, the same way OpenAI constructed ChatGPT on the base of GPT-3. The company anticipates using LLaMA to further applications like "question answering, natural language understanding or reading comprehension, understanding capabilities and limitations of present language models" and to aid in natural language research. 

While the top-of-the-line LLaMA model (LLaMA-65B, with 65 billion parameters) competes head-to-head with comparable products from rival AI labs DeepMind, Google, and OpenAI, arguably the most intriguing development comes from the LLaMA-13B model, which, as previously mentioned, can reportedly outperform GPT-3 while running on a single GPU when measured across eight common "common sense reasoning" benchmarks like BoolQ, PIQA LLaMA-13B opens the door for ChatGPT-like performance on consumer-level hardware in the near future, unlike the data center requirements for GPT-3 derivatives. 

In AI, parameter size is significant. A parameter is a variable that a machine-learning model employs in order to generate hypotheses or categorize data as input. The size of a language model's parameter set significantly affects how well it performs, with larger models typically able to handle more challenging tasks and generate output that is more coherent. However, more parameters take up more room and use more computing resources to function. A model is significantly more efficient if it can provide the same outcomes as another model with fewer parameters. 

"I'm now thinking that we will be running language models with a sizable portion of the capabilities of ChatGPT on our own (top of the range) mobile phones and laptops within a year or two," according to Simon Willison, an independent AI researcher in an Mastodon thread analyzing and monitoring the impact of Meta’s new AI models. 

Currently, a simplified version of LLaMA is being made available on GitHub. The whole code and weights (the "learned" training data in a neural network) can be obtained by filling out a form provided by Meta. A wider release of the model and weights has not yet been announced by Meta.  

Read more »
Technology
AI AI Chatbot LLM Security Firms Security Vendors Technology

Security Vendors are Turning to GPT as a Key AI Technology

 

A number of businesses are utilising conversational AI technology to improve their product capabilities, including for security, despite some concerns about how generative AI chatbots like ChatGPT can be used maliciously — to create phishing campaigns or write malware. 

A large language model (LLM) called ChatGPT, created by OpenAI, uses the GPT 3 LLM and is based on a variety of large test data sets. When a user asks a simple question, ChatGPT, which can understand human language, responds with thorough explanations and can manage complex tasks like document creation and code writing. It serves as an illustration of how conversational AI can be used to organise massive amounts of data, improve user experience, and facilitate communications. 

For example, a conversational AI tool, such as ChatGPT or another option, could act as the back end of an information concierge that automates the use of threat intelligence in enterprise support, claims IT research and advisory firm Into-Tech Research. 

With Orca Security Platform, it seems like Orca Security is taking that tack. The platform's capacity to produce contextual and precise remediation plans for security alerts was improved by the incorporation of OpenAI's GPT3 API, particularly the "Da-Vinci-03" series. In the announcement, the head of data science at Orca, Itamar Golan, and the director of innovation at Orca, Lior Drihem, wrote. Before feeding the components as input to GPT3, the new pipeline preprocesses data from a security alert, including fundamental details about the risk and its contextual environment, including affected assets, attack vectors, and potential impact. The best and most useful solutions to fix the problem are then generated by the AI, according to Golan and Drihem. For teams to refer to and apply, these remediation steps can also be included in tickets, such as Jira tickets. 

Even though the AI model has the potential to produce inaccurate data (or ambiguous results), Drihem and Golan claim that "the benefits of utilising GPT3's natural language generation capabilities outweigh any potential risks, and have seen significant improvements in the efficiency and effectiveness of our remediation efforts." 

Orca Security has previously used language models in their work. To improve the remediation information customers receive regarding infosec risks, the company recently integrated GPT3 into its cloud security platform. 

"By fine-tuning these powerful language models with our own security data sets, we have been able to improve the detail and accuracy of our remediation steps — giving you a much better remediation plan and assisting you to optimally solve the issue as fast as possible," Golan and Drihem added. 

Utilizing LLM & AI for applications 

Orca Security joins other businesses that offer language models as part of their product line. This week, Gupshup introduced Auto Bot Builder, a tool that uses GPT-3 to assist businesses in creating their own sophisticated conversational chatbots. Using content from the enterprise website, documents, message logs, product catalogues, databases, and other corporate systems, Auto Bot Builder creates chatbots tailored to the enterprise's unique requirements. The information is processed using GPT-3 LLM (Large Language Model), and it is then fine-tuned with proprietary industry-specific models. Businesses can use Auto Bot Builder to create chatbots for customer support, product discovery, product recommendations, shopping advice, and lead generation in marketing. 

These chatbots are different from ChatGPT, a general-purpose chatbot, but they share with ChatGPT the ability to communicate with end users at a "exceptionally high degree of language capability," according to Gupshup. 

ChatGPT is also being used by the cryptocurrency community to develop software like trading bots and cryptocurrency blogs. Competitive intelligence analyst Jerrod Piker from Deep Instinct wrote in an email. Examples include creating a sample smart contract using ChatGPT and creating a trading bot to help automate the process of buying and selling cryptocurrencies by identifying entry and exit points. 

The idea of a generative AI chatbot that can respond to questions is not new, but Casey Ellis, founder and CTO of Bugcrowd, notes that ChatGPT stands out from the competition due to the variety of topics it can handle and its usability.
Read more »
Subscribe to: Posts (Atom)