Search This Blog

Showing posts with label Machine learning. Show all posts

A Major Flaw in the AI Testing Framework MLflow can Compromise the Server and Data

MLflow, an open-source framework used by many organizations to manage and record machine-learning tests, has been patched for a critical vulnerability that could enable attackers to extract sensitive information from servers such as SSH keys and AWS credentials. Since MLflow does not enforce authentication by default, and a growing percentage of MLflow deployments are directly exposed to the internet, the attacks can be carried out remotely without authentication.

"Basically, every organization that uses this tool is at risk of losing their AI models, having an internal server compromised, and having their AWS account compromised," Dan McInerney, a senior security engineer with cybersecurity startup Protect AI, told CSO. "It's pretty brutal."

McInerney discovered the flaw and privately reported it to the MLflow project. It was fixed in the framework's version 2.2.1, which was released three weeks ago, but no security fix was mentioned in the release notes.

Path traversal used to include local and remote files

MLflow is a Python-based tool for automating machine-learning workflows. It includes a number of components that enable users to deploy models from various ML libraries, handle their lifecycle (including model versioning, stage transitions, and annotations), track experiments to record and compare parameters and results, and even package ML code in a reproducible format to share with other data scientists. A REST API and command-line interface are available for controlling MLflow.

All of these features combine to make the framework an invaluable resource for any organisation experimenting with machine learning. Scans using the Shodan search engine confirm this, revealing a steady increase in publicly exposed MLflow instances over the last two years, with the current count exceeding 800.However, it is likely that many more MLflow deployments exist within internal networks and may be accessible to attackers who gain access to those networks.

"We reached out to our contacts at various Fortune 500's [and] they've all confirmed they're using MLflow internally for their AI engineering workflow,' McInerney tells CSO.

McInerney's vulnerability is identified as CVE-2023-1177 and is rated 10 (critical) on the CVSS scale. He refers to it as local and remote file inclusion (LFI/RFI) via the API, in which remote and unauthenticated attackers can send specially crafted requests to the API endpoint, forcing MLflow to expose the contents of any readable files on the server.

What makes the vulnerability worse is that most organisations configure their MLflow instances to store their models and other sensitive data in Amazon AWS S3. In accordance with a review of the configuration of publicly available MLflow instances by Protect AI, seven out of ten used AWS S3. This means that attackers can use the s3:/ URL of the bucket utilized by the instance as the source parameter in their JSON request to steal models remotely.

It also implies that AWS credentials are most likely stored locally on the MLflow server in order for the framework to access S3 buckets, and that these credentials are typically stored in a folder called /.aws/credentials under the user's home directory. The disclosure of AWS credentials can be a serious security breach because, depending on IAM policy, it can give attackers lateral movement capabilities into an organization's AWS infrastructure.

Insecure deployments result from a lack of default authentication

Authentication for accessing the API endpoint would protect this flaw from being exploited, but MLflow does not implement any authentication mechanism. Simple authentication with a static username and password can be added by placing a proxy server, such as nginx, in front of the MLflow server and forcing authentication through it. Unfortunately, almost none of the publicly exposed instances employ this configuration.

McInerney stated, "I can hardly call this a safe deployment of the tool, but at the very least, the safest deployment of MLflow as it stands currently is to keep it on an internal network, in a network segment that is partitioned away from all users except those who need to use it, and put behind an nginx proxy with basic authentication. This still doesn't prevent any user with access to the server from downloading other users' models and artifacts, but at the very least it limits the exposure. Exposing it on a public internet facing server assumes that absolutely nothing stored on the server or remote artifact store server contains sensitive data."

Boosting AI with Synthetic Data: Benefits & Challenges


Artificial intelligence (AI) is becoming increasingly important across a wide range of industries. However, one of the biggest challenges facing AI is the need for large amounts of high-quality data to train algorithms effectively. This is where synthetic data comes in – it has the potential to revolutionize the way AI is developed and deployed at scale.

Improving AI/ML with synthetic data

Synthetic data refers to data that is artificially generated by computer algorithms, rather than real-world data that is collected from sensors, cameras, or other sources. Synthetic data can be used to train machine learning algorithms, which can then be used to create more accurate and efficient AI models.

One significant benefit of synthetic data is its speed of generation and lower cost compared to real-world data. This makes it an essential tool in industries like autonomous vehicles or robotics, where obtaining real-world data can be time-consuming and expensive. Synthetic data offers a wider range of scenarios that can improve the accuracy and reliability of AI models in real-world situations.

In the real world of AI, synthetic data can generate a broader range of scenarios than real-world data. For example, in the case of autonomous vehicles, synthetic data can be used to create scenarios where the vehicle is operating in different weather conditions or on different road surfaces. This can help to improve the accuracy and reliability of the AI model in a wider range of real-world scenarios.

Synthetic data and model quality

The quality of the synthetic data is critical to the quality of the AI model. The algorithms used to generate synthetic data need to be carefully designed and tested to ensure that the data accurately reflects the characteristics of real-world data. This requires a deep understanding of the domain in which the AI model will be deployed.

There are also challenges associated with the use of synthetic data in AI. Ensuring that the synthetic data accurately reflects the characteristics of real-world data is crucial. In industries like healthcare, where AI models can reinforce existing biases in data, it is essential to ensure that synthetic data does not introduce bias into the model.

To unlock the full potential of synthetic data, ongoing innovation, and collaboration are necessary to address these challenges. Future innovations in algorithms used to generate synthetic data can further revolutionize AI development and deployment at scale.

Overall, synthetic data has the potential to revolutionize the way AI is developed and deployed at scale. It provides a faster and more cost-effective way to generate data for training ML algorithms, leading to more efficient and accurate AI models. However, synthetic data must be generated with care and accuracy to ensure it accurately reflects real-world scenarios, and its use must be responsibly handled. Collaboration among researchers, industry practitioners, and regulators is necessary to use synthetic data in AI responsibly and realize its full potential.

Visa Bolsters Cybersecurity Defenses with AI and Machine Learning

Enhancing Fraud Detection and Prevention with Visa Advanced Authorization (VAA)

Visa is one of the largest payment companies in the world, handling billions of transactions every year. As such, it is a prime target for cyberattacks from hackers looking to steal sensitive financial information. To counter these threats, Visa has turned to artificial intelligence (AI) and machine learning (ML) to bolster its security defenses.

AI and ML offer several advantages over traditional cybersecurity methods. They can detect and respond to threats in real time, identify patterns in data that humans may miss, and adapt to changing threat landscapes. Visa has incorporated these technologies into its fraud detection and prevention systems, which help identify and block fraudulent transactions before they can cause harm.

Proactive Risk Assessment with Visa's Risk Manager Platform

One example of how Visa is using AI to counter cyberattacks is through its Visa Advanced Authorization (VAA) system. VAA uses ML algorithms to analyze transaction data and identify patterns of fraudulent activity. The system learns from historical data and uses that knowledge to detect and prevent future fraud attempts. This approach has been highly effective, with VAA reportedly blocking $25 billion in fraudulent transactions in 2020 alone.

Visa is also using AI to enhance its risk assessment capabilities. The company's Risk Manager platform uses ML algorithms to analyze transaction data and identify potential fraud risks. The system can detect unusual behavior patterns, such as a sudden increase in transaction volume or an unexpected change in location, and flag them for further investigation. This allows Visa to proactively address potential risks before they turn into full-fledged cyberattacks.

Using AI for Threat Intelligence with CyberSource Threat Intelligence

Another area where Visa is using AI to counter cyberattacks is in threat intelligence. The company's CyberSource Threat Intelligence service uses ML algorithms to analyze global threat data and identify potential security threats. This information is then shared with Visa's clients, helping them stay ahead of emerging threats and minimize their risk of a cyberattack.

Real-Time Detection and Disruption of Cyberattacks with Visa Payment Fraud Disruption (PFD) Platform

Visa has also developed a tool called the Visa Payment Fraud Disruption (PFD) platform, which uses AI to detect and disrupt cyberattacks targeting Visa clients. The PFD platform analyzes transaction data in real time and identifies any unusual activity that could indicate a cyberattack. The system then alerts Visa's cybersecurity team, who can take immediate action to prevent the attack from causing harm.

In addition to these measures, Visa is also investing in the development of AI and ML technologies to further enhance its cybersecurity capabilities. The company has partnered with leading AI firms and academic institutions to develop new tools and techniques to detect and prevent cyberattacks more effectively.

Overall, Visa's use of AI and ML in its cybersecurity systems has proven highly effective in countering cyberattacks. By leveraging these technologies, Visa is able to detect and respond to threats in real time, identify patterns in data that humans may miss, and adapt to changing threat landscapes. As cyberattacks continue to evolve and become more sophisticated, Visa will likely continue to invest in AI and ML to stay ahead of the curve and protect its customers' sensitive financial information.

Researchers Develop AI Cyber Defender to Tackle Cyber Actors

A recently developed deep reinforcement learning (DRL)-based artificial intelligence (AI) system can respond to attackers in a simulated environment and stop 95% of cyberattacks before they get more serious. 

The aforementioned findings were made by researchers from the Department of Energy’s Pacific Northwest National Laboratory based on an abstract simulation of the digital conflict between threat actors and defenders in a network and trained four different DRL neural networks in order to expand rewards based on minimizing compromises and network disruption. 

The simulated attackers transitions from the initial access and reconnaissance phase to other attack stages until they arrived at their objective, i.e. the impact and exfiltration phase. Apparently, these strategies were based on the classification of the MITRE ATT&CK architecture. 

Samrat Chatterjee, a data scientist who presented the team's work at the annual meeting of the Association for the Advancement of Artificial Intelligence in Washington, DC, on February 14, claims that the successful installation and training of the AI system on the simplified attack surfaces illustrates the defensive responses to cyberattacks that, in current times, could be conducted by an AI model. 

"You don't want to move into more complex architectures if you cannot even show the promise of these techniques[…]We wanted to first demonstrate that we can actually train a DRL successfully and show some good testing outcomes before moving forward," says Chatterjee. 

AI Emerging as a New Trend in Cybersecurity 

Machine learning (ML) and AI tactics have emerged as innovative trends to administer cybersecurity in a variety of fields. This development in cybersecurity has started from the early integration of ML in email security in the early 2010s to utilizing ChatGPT and numerous AI bots that we see today to analyze code or conduct forensic analysis. The majority of security products now incorporate a few features that are powered by machine learning algorithms that have been trained on massive datasets. 

Yet, developing an AI system that is capable of proactive protection is still more of an ideal than a realistic approach. The PNNL research suggests that an AI defender could be made possible in the future, despite the many obstacles that still need to be addressed by researchers. 

"Evaluating multiple DRL algorithms trained under diverse adversarial settings is an important step toward practical autonomous cyber defense solutions[…] Our experiments suggest that model-free DRL algorithms can be effectively trained under multistage attack profiles with different skill and persistence levels, yielding favorable defense outcomes in contested settings," according to a statement published by the PNNL researchers. 

How the System Uses MITRE ATT&CK 

The initial objective of the research team was to develop a custom simulation environment based on an open-source toolkit, Open AI Gym. Through this environment, the researchers created attacker entities with a range of skill and persistence levels that could employ a selection of seven tactics and fifteen techniques from the MITRE ATT&CK framework. 

The attacker agents' objectives are to go through the seven attack chain steps—from initial access to execution, from persistence to command and control, and from collection to impact—in the order listed. 

According to Chatterjee of PNNL, it can be challenging for the attacker to modify their strategies in response to the environment's current state and the defender's existing behavior. 

"The adversary has to navigate their way from an initial recon state all the way to some exfiltration or impact state[…] We're not trying to create a kind of model to stop an adversary before they get inside the environment — we assume that the system is already compromised," says Chatterjee. 

Not Ready for Prime Time 

In the experiments, it was revealed that a particular reinforcement learning technique called a Deep Q Network successfully solved the defensive problem by catching 97% of the intruders in the test data set. Yet the research is just the beginning. Yet, security professionals should not look for an AI assistant to assist them with incident response and forensics anytime soon.  

One of the many issues that are required to be resolved is getting RL and deep neural networks to explain the causes that affected their decision, an area of research called explainable reinforcement learning (XRL).  

Moreover, the rapid emergence of AI technology and finding the most effective tactics to train the neutral network are both a challenge that needs to be addressed, according to Chatterjee.  

Zero-Knowledge Encryption Might Protect User Rights


Web3 is an evaluation of the internet that moves past a centralized structure and tries to connect data in a decentralized way in order to offer a speedy and individualized user experience. This version of the internet is sometimes referred to as the third generation of the web.Web3 sometimes referred to as the Semantic Web, is based on AI and ML and employs blockchain technology to protect the security and privacy of user data.

Role of Zero-Knowledge Encryption

Using specific user keys, zero-knowledge encryption protects data. No one other than the user may access their encrypted files because administrators and developers do not know or have access to them. 

Zero-knowledge proofs, which may verify the truth of a proposition without revealing the underlying data, make this possible. Zero-knowledge cryptography enables information to be "private and useable at the same time," according to Aleo's CEO Alex Pruden, in contrast to other well-known types of encryption such as end-to-end models used in private messaging apps, through which only users and senders may read information. Without disclosing personal information about yourself, you can demonstrate your trustworthiness with zero-knowledge proof.

Decentralized identity (DCI) constructions, tokenization, and self-hosted wallets are three features of Web3 that promote user ownership of data and algorithms. Zero-knowledge proofs and least privilege are two techniques used in decentralized computing (DCI).

Reasons for  Zero-Knowledge Encryption

One drawback of zero-knowledge encryption is that it frequently leaves users unable to access their data moving forward if they ever need to find their encryption key or password. Because it requires more work to securely transfer and store user data, service providers that offer the full zero-knowledge encryption guarantee are often slower than their less secure competitors.

There is no better alternative than zero-knowledge encryption if a user wishes to maintain the privacy and security of their data while still hosting it on an external server for simple data management.

Cryptocurrencies Industry is Impacted by AI and ML

Artificial intelligence (AI) and Machine Learning is a fast expanding technology with the power to completely alter how we operate and live. Blockchain technology, a decentralized digital ledger system, is also thought to form the foundation of other upcoming technologies. These two methods can work together to develop strong new solutions across a range of sectors.

A number of indicators are used often by cryptocurrency traders. Nevertheless, given the prevalence of unorganized data in the digital world, manually creating trustworthy signals might be unfeasible. Massive amounts of information must be accurate, relevant, and clean prior to being assessed for investment insights.

In order to find investments and buy/sell signals as the number of investment alternatives increases, manual inquiry, extraction, and analysis procedures are no longer useful. AI has become a common tool in the financial sector, and it is much more powerful when integrated with blockchain.

Disadvantages of adopting blockchain with AI and ML:

1. Security: Employing blockchain with AI and ML might expose businesses to security issues. Blockchain-based solutions need a high level of trust since they exchange sensitive data, which is susceptible to malicious assaults.

2. Privacy: The integration of AI and blockchain technology has the risk of jeopardizing users' privacy because data recorded on the blockchain is indelible and accessible to all network users.

3. Scalability: When users upload more data to a blockchain, the size of the blockchain grows rapidly, creating scalability problems that can hamper performance and slow down processing rates.

4. Interoperability: Since different blockchains use dissimilar protocols, it is challenging to develop solutions that work well for all of them. As a result, they have trouble communicating with one another.

Blockchain technology, AI & ML successfully balance out each other's shortcomings, enabling reciprocal benefits, technological improvements, and robust enterprise support. AI in the blockchain sector can produce smart contracts and blockchain oracles that are more reliable, effective, and secure. These remedies have the power to lower expenses, boost efficiency, and open up fresh business prospects. One may anticipate more as technology develops further.

Cyberwarfare Threat Looming Large on Firms Worldwide


Over the past ten years or so, the environment for cyber threats has undergone a significant transformation, which has accelerated in recent years. The term "cyberwar" didn't even exist until 30 years ago, and it's still somewhat debatable today. 

Once upon a time (that time being just a few years ago), the majority of private businesses had no reason for immediate concern. However, the distinctions between nation-state adversaries, cybercrime organisations, and rogue threat actors continue to become more and more hazy, making practically any company and any device fair game for cyberwarfare. The Armis State of Cyberwarfare and Trends Report: 2022-2023 examines the situation more closely and offers information on whether or not organisations are sufficiently equipped to defend themselves. 

The report focuses on the opinions of IT and security experts from around the world regarding the state of cyberwarfare today and market trends. It offers insightful information on the opportunities and challenges faced by businesses as they work to safeguard their assets and secure their networks. The study was conducted by surveying more than 6,000 IT and security professionals worldwide from all major industry verticals. 

Technology: A double-edged sword 

Technology is frequently a mixed blessing, which is one of the report's most notable findings. Anything that makes your life simpler or more convenient for you can, in theory, be used against you by attackers or expose you to a higher risk in some other way.

Technology is increasingly reliant on artificial intelligence (AI) and machine learning (ML). These technological advancements are being used to automate the detection and response to cyber threats, which is assisting businesses in better protecting their assets and networks. The report does point out, however, that there are worries about how these technologies might be misused for bad, and that more oversight and regulation are required in this area. Concerns about the potential use of generative AI tools like ChatGPT to create malicious code and exploits have recently grown. 

The Armis report highlights the growing threat that cyberattacks that target critical infrastructure pose to businesses. This includes attacks on systems that are crucial to the operation of contemporary society, such as medical equipment and industrial control systems. While these attacks don't specifically target organisations (aside from the provider of critical infrastructure), any attack that affects the critical infrastructure that businesses depend on can have disastrous effects on those businesses. In accordance with the report, these attacks are becoming increasingly sophisticated and are frequently carried out by advanced persistent threat (APT) groups, which are outfitted with the resources and technical know-how necessary to get around conventional security measures.

In the report's introduction, Nadir Izrael, CTO and co-founder of Armis, mentions that experts believe threat actors will be able to weaponize OT (operational technology) environments by 2025 in order to hurt or kill people. The shift from reconnaissance and espionage to kinetic application with tangible effects is a trend in cyberwarfare, he observes. 

“These kinetic cyberweapons have already been discovered in the wild, although none specifically have been deployed to lethal effect. For example, the Triton malware discovered in 2017 targeted and disabled safety instrumented system (SIS) controllers of a Saudi Arabian petrochemical plant which could have contributed to a plant-wide disaster had the problem not been identified. And in February 2021, a hacker attempted to poison the water supply facility of a small U.S. city in the state of Florida via remote access. We have already seen ransomware attacks against the healthcare sector result in human deaths, so the potential impact of cyberattacks—whether intentional or unintentional—is clear.” 

Can we survive cyber warfare? 

Many organisations have been caught off guard by the threat landscape's quick change. The scope of the threat is difficult for businesses of all sizes and in all sectors to comprehend, and many do not have the necessary cyber defences in place.

In a press release, Armis summarised some of the report's most important findings. These results highlight some of the major obstacles that organisations must overcome in order to adjust to the new reality. 

  • The threat of cyberwarfare is not being taken seriously by one-third (33%) of international organisations, who report being unconcerned or indifferent about how it will affect their organisation as a whole, creating security gaps. 
  • Nearly a quarter (24%) of international organisations believe they are unprepared to handle cyberwarfare. Nevertheless, preventing nation-state attacks comes in last on the IT professionals' list of security concerns (22%). 
  • The statement that "The war in Ukraine has created a greater threat of cyberwarfare" is accepted by more than three out of five (64%) IT and security professionals polled.
  • Between May 2022 and October 2022, compared to the six months before, more threat activity was reported on networks by over half (54%) of professionals who are the sole decision-makers for IT security. 
  • The majority (55%) of IT professionals polled concurred with the statement that "My organisation has stalled or stopped digital transformation projects due to the threat of cyberwarfare." In some nations, like Australia (79%), the U.S. (67%), Singapore (63%), the UK (57%), and Denmark (56%), this percentage is even higher. 
  • IT professionals around the world responded differently when asked about their company's policy on paying ransoms in the event of a ransomware attack. Twenty-four percent of respondents said their organisation always pays, 31% said their organisation only pays when customer data is at risk, 26% said their organisation never pays, and 19% said it depends. 
  • A little more than seven in ten (76%) of the IT professionals polled concur that, in response to the threat of cyberwarfare, the boards of directors are changing the organisational culture with regard to cybersecurity. 
  • Nearly 2 in 5 (37%) of the IT professionals surveyed believe it is extremely likely that their company will increase its investment in cybersecurity in light of recent and ongoing unexpected global events (such as the pandemic, the conflict in the Ukraine, etc.) 

Combating future cyberwars 

The report emphasises how crucial asset visibility is to maintaining business network security. Businesses must have a thorough understanding of the hardware and software that connect to their networks in order to identify threats quickly and take appropriate action. With the goal of becoming the "Google Maps" of the IT environment or attack surface, Armis is committed to giving its users the visibility they require. To assist them in overcoming these obstacles, they collaborate with clients like the City of Las Vegas, Takeda Pharmaceuticals, and an increasing number of governmental bodies.

Yevgeny Dibrov, the CEO of Armis, and Nadir Izrael, the CTO, were interviewed by Tony Bradley, Editor-in-Chief at TechSpective. Regarding the visibility of assets, Dibrov stated, "Every client should ask themselves, 'What are my assets? What are my assets, exactly?

In a data centre environment, a manufacturing environment, a hospital, a critical infrastructure facility, or a government facility, the most fundamental question is, "What do I have?" he continued. 

“I think cyberwarfare in general has become kind of an above board thing that nation-states do, as opposed to maybe a decade or two ago where everything was hush-hush and under the covers—like these covert attacks that were never attributable. That change is huge in our overall industry. It's huge for countries. In fact, from our perspective it paints cyberwarfare as the new terrorism,” Izrael stated. “It is the most cost-effective way of waging war on multiple levels and something that we're seeing more and more examples of as we progress.”  

Since it is unlikely that we will be able to put the genie back in the bottle in the future, it will be crucial for organisations to understand the answers to the questions Dibrov posed and have that "Google Map" of their environment to work with.

A New Era is Emerging in Cybersecurity, but Only the Best Algorithms will Survive


The industry identified that basic fingerprinting could not maintain up with the rate of these developments, and the requirement to be everywhere, at all times, pushed the acceptance of AI technology to deal with the scale and complexity of modern business security. 

Since then, the AI defence market has become crowded with vendors promising data analytics, looking for "fuzzy matches": close matches to previously encountered threats, and eventually using machine learning to detect similar attacks. While this is an advancement over basic signatures, using AI in this manner does not hide the fact that it is still reactive. It may be capable of recognizing attacks that are very similar to previous incidents, but it is unable to prevent new attack infrastructure and techniques that the system has never seen before.

Whatever you call it, this system is still receiving the same historical attack data. It recognises that in order to succeed, there must be a "patient zero" — or first victim. Supervised machine learning is another term for "pretraining" an AI on observed data (ML). This method does have some clever applications in cybersecurity. For example, in threat investigation, supervised ML has been used to learn and mimic how a human analyst conducts investigations — asking questions, forming and revising hypotheses, and reaching conclusions — and can now carry out these investigations autonomously at speed and scale.

But what about tracking down the first traces of an attack? What about detecting the first indication that something is wrong?

The issue with utilising supervised ML in this area is that it is only as good as its historical training set — not with new things. As a result, it must be constantly updated, and the update must be distributed to all customers. This method also necessitates sending the customer's data to a centralised data lake in the cloud to be processed and analysed. When an organisation becomes aware of a threat, it is frequently too late.

As a result, organisations suffer from a lack of tailored protection, a high number of false positives, and missed detections because this approach overlooks one critical factor: the context of the specific organisation it is tasked with protecting.

However, there is still hope for defenders in the war of algorithms. Today, thousands of organisations utilise a different application of AI in cyber defence, taking a fundamentally different approach to defending against the entire attack spectrum — including indiscriminate and known attacks, as well as targeted and unknown attacks.

Unsupervised machine learning involves the AI learning the organisation rather than training it on what an attack looks like. In this scenario, the AI learns its surroundings from the inside out, down to the smallest digital details, understanding "normal" for the specific digital environment in which it is deployed in order to identify what is not normal.

This is AI that comprehends "you" in order to identify your adversary. It was once thought to be radical, but it now protects over 8,000 organisations worldwide by detecting, responding to, and even avoiding the most sophisticated cyberattacks.

Consider last year's widespread Hafnium attacks on Microsoft Exchange Servers. Darktrace's unmonitored ML identified and disrupted a series of new, unattributed campaigns in real time across many of its customer environments, with no prior threat intelligence associated with these attacks. Other organisations, on the other hand, were caught off guard and vulnerable to the threat until Microsoft revealed the attacks a few months later.

This is where unsupervised ML excels — autonomously detecting, investigating, and responding to advanced and previously unseen threats based on a unique understanding of the organization in question. Darktrace's AI research centre in Cambridge, UK, tested this AI technology against offensive AI prototypes. These prototypes, like ChatGPT, can create hyperrealistic and contextualised phishing emails and even choose a suitable sender to spoof and fire the emails.

The conclusions are clear: as attackers begin to weaponize AI for nefarious reasons, security teams will require AI to combat AI. Unsupervised machine learning will be critical because it learns on the fly, constructing a complex, evolving understanding of every user and device across the organisation. With this bird's-eye view of the digital business, unsupervised AI that recognises "you" will detect offensive AI as soon as it begins to manipulate data and will take appropriate action.

Offensive AI may be exploited for its speed, but defensive AI will also contribute to the arms race. In the war of algorithms, the right approach to ML could mean the difference between a strong security posture and disaster.

Is AI Transforming the Cybersecurity Sector? 

Artificial intelligence and machine learning (AI/ML) systems have proven to be effective in improving the sophistication of phishing lures, creating fake profiles, and developing basic malware. Security experts have demonstrated that a complete attack chain may be established, and malicious hackers have already begun experimenting with AI-generated code.

The Check Point Research team employed current AI tools to design a whole attack campaign which began with a phishing email sent by OpenAI's ChatGPT that prompts the target to open an Excel document. Researchers also developed an Excel macro that runs malware obtained from a URL and a Python script to infect the intended system using the Codex AI programming tool.

To evaluate the effectiveness of AI in data collection and team response to cyberattacks on vital systems and services, as well as to draw attention to the need for solutions that enhance human-machine collaboration to lower cyber risk. 

In recent weeks, ChatGPT, a large language model (LLM) based on OpenAI's generative pre-trained transformer (GPT-3) third iteration, sparked a scope of what-if scenarios for the possible uses of AI/ML. Due to the dual-use nature of AI/ML models, firms are looking for ways to use the technology to increase efficiency, while campaigners for digital rights are concerned about the effects the technology will have on businesses and employees.   

However, other aspects of security and privacy are also being impacted by AI/ML. To enhance profiles used for fraud and misinformation, generative neural networks (GNNs) were utilized to produce photographs of fake persons that look real but do not portray a real person. 

The employment of the most advanced artificial intelligence system by cyber attackers does not, as of yet, make the attacks more difficult to spot. However, by emphasizing the technical signs, cybersecurity tools can still detect the issue. Even the most effective fake imitation would be defeated by the procedures used to double-check requests to modify an account for payment and paycheck transfer unless the threat organization had access to or control over the further layers of security that have become increasingly frequent.

How to Shield Businesses from State-Sponsored AI Attacks


In cybersecurity, artificial intelligence is becoming more and more significant, both for good and bad. The most recent AI-based tools can help organizations better identify threats and safeguard their systems and data resources. However, hackers can also employ the technology to carry out more complex attacks. 

Hackers have a big advantage over most businesses because they can innovate more quickly than even the most productive enterprise, they can hire talent to develop new malware and test attack techniques, and they can use AI to change attack strategies in real time. 

The market for AI-based security products has also helped malicious hackers to target businesses frequently. According to a report published in July 2022 by Acumen Research and Consulting, the global market had a value of $14.9 billion in 2021 and was expected to grow to $133.8 billion by 2030.

Nation-states and hackers: A lethal combination 

Weaponized AI attacks are inevitable, according to 88% of CISOs and security executives, and for good reason. A recent Gartner survey showed that only 24% of cybersecurity teams are fully equipped to handle an AI-related attack. Nation-states and hackers are aware that many businesses are understaffed and lack the knowledge and resources necessary to defend against such attacks in the form of AI and machine learning. Only 1% of 53,760 cybersecurity applicants in Q3 2022 had AI skills. 

Major corporations are aware of the cybersecurity skills shortage and are working to address it. Microsoft, for example, is currently running a campaign to assist community colleges in expanding the industry's workforce. 

The ability of businesses to recruit and keep cybersecurity experts with AI and ML skills contrasts sharply with how quickly nation-state actors and cybercriminal gangs are expanding their AI and ML teams. According to the New York Times, the Department 121 cyberwarfare unit of the elite Reconnaissance General Bureau of the North Korean Army has about 6,800 members total, including 1,700 hackers spread across seven different units and 5,100 technical support staff. 

According to South Korea's spy agency, North Korea's elite team stole an estimated $1.2 billion in cryptocurrency and other virtual assets over the last five years, with more than half of it stolen this year alone. Since June 2022, North Korea has also weaponized open-source software in its social engineering campaigns aimed at businesses all over the world. 

North Korea's active AI and ML recruitment and training programs aim to develop new techniques and technologies that weaponize AI and ML in order to fund the country's nuclear weapons programs. 

In a recent Economist Intelligence Unit (EIU) survey, nearly half of respondents (48.9%) named AI and machine learning as emerging technologies that would be most effective in countering nation-state cyberattacks on private organizations. 

Cybercriminal gangs pursue their enterprise targets with the same zeal as the North Korean Army's Department 121. Automated phishing email campaigns, malware distribution, AI-powered bots that continuously scan an enterprise's endpoints for vulnerabilities and unprotected servers, credit card fraud, insurance fraud, and generating deepfake identities are all current tools, techniques, and technologies in cybercriminal gangs' AI and ML arsenals. 

Hackers and nation-states are increasingly using this tactic to target the flaws in AI and ML models built to detect and prevent breach attempts. One of the methods used to lessen the effectiveness of AI models created to predict and prevent data exfiltration, malware delivery, and other things is data poisoning. 

How to safeguard your AI 

What can the company do to safeguard itself? The three essential actions to take right away, in the opinion of Great Learning's Akriti Galav and SEO expert Saket Gupta, are: 

  • Maintain the most stringent security procedures possible throughout the entire data environment. 
  • Make sure an audit trail is created with a log of every record related to every AI operation. 
  • Implement reliable authentication and access control. 

Additionally, businesses should pursue longer-term strategic objectives, such as creating a data protection policy specifically for AI training, educating their staff about the dangers of AI and how to spot flawed results, and continuing to operate a dynamic, forward-looking risk assessment mechanism.

No digital system, no matter how intelligent, can be 100% secure. The enterprise needs to update its security policies to reflect this new reality now rather than waiting until the damage is done because the risks associated with compromised AI are more subtle but no less serious than those associated with traditional platforms.

UK Government Releases New Machine Learning Guidance

Machine Learning and NCSC

The UK's top cybersecurity agency has released new guidance designed to assist developers and others identify and patch vulnerabilities in Machine Learning (ML) systems. 

GCHQ's National Cyber Security Centre (NCSC) has laid out together its principles for the security of machine learning for any company that is looking to reduce potential adversarial machine learning (AML). 

What is Adversarial Machine Learning (AML)?

AML attacks compromise the unique features of ML or AI systems to attain different goals. AML has become a serious issue as technology has found its way into a rising critical range of systems, finance, national security, underpinning healthcare, and more. 

At its core, software security depends on understanding how a component or system works. This lets a system owner inspect and analyze vulnerabilities, these can be reduced or accepted later. 

Sadly, it's difficult to deal with this ML. ML is precisely used for enabling a system that has self-learning, to take out information from data, with negligible assistance from a human developer.

ML behaviour and difficulty to interpret 

Since a model's internal logic depends on data, its behaviour can be problematic to understand, and at times is next to impossible to fully comprehend why it is doing what it is doing. 

This explains why ML components haven't undergone the same level of inspection as regular systems, and why some vulnerabilities can't be identified. 

According to experts, the new ML principles will help any organization "involved in the development, deployment, or decommissioning of a system containing ML." 

The experts have pointed out some key limitations in ML systems, these include:

  • Dependence on data: modifying training data can cause unintended behaviour, and the threat actors can exploit this. 
  • Opaque model logic: developers sometimes can't understand or explain a model's logic, which can affect their ability to reduce risk.
  • Challenges verifying models: it is almost impossible to cross-check if a model will behave as expected under the whole range of inputs to which it might be a subject, and we should note that there can be billions of these. 
  • Reverse engineering models and training data can be rebuilt by threat actors to help them in launching attacks. 
  • Need for retraining: Many ML systems use "continuous learning" to improve performance over time, however, it means that security must be reassessed every time a new model version is released. It can be several times a day. 

In the NCSC, the team recognises the massive benefits that good data science and ML can bring to society, along with cybersecurity. The NCSC wants to make sure these benefits are recognised. 

Hackers can ‘Poison’ Open-source Code on the Internet


A Cornell University Tech team with researchers discovered a new kind of backdoor attack that can modify natural-language modelling systems to generate false outputs and bypass any known protection. 

The Cornell Tech team believes the assaults may affect algorithmic trading, email accounts, and other services. The research was supported by the NSF and the Schmidt Futures initiative, and Google Faculty Research Award. 

According to research published on Thursday, the backdoor may alter natural-language modelling systems without requiring access to the original code or model by uploading malicious code to open-source sites commonly used by numerous organisations and programmers. 

During a presentation at the USENIX Security conference on Thursday, the researchers termed the attacks "code poisoning." The attack would offer people or organisations immense authority over a wide range of things, including movie reviews or an investment bank's machine learning model, disregarding news that may affect a company's stock. 

The report explained, "The attack is blind: the attacker does not need to observe the execution of his code, nor the weights of the backdoored model during or after training. The attack synthesizes poisoning inputs 'on the fly,' as the model is training, and uses multi-objective optimization to achieve high accuracy simultaneously on the main and backdoor tasks." 

"We showed how this attack can be used to inject single-pixel and physical backdoors into ImageNet models, backdoors that switch the model to a covert functionality, and backdoors that do not require the attacker to modify the input at inference time. We then demonstrated that code-poisoning attacks can evade any known defence, and proposed a new defence based on detecting deviations from the model's trusted computational graph." 

Eugene Bagdasaryan, a computer science PhD candidate at Cornell Tech and co-author of the new paper with professor Vitaly Shmatikov, mentioned that many companies and programmers use models and codes from open-source sites on the internet. This study highlights the importance of reviewing and verifying materials before incorporating them into any systems.

"If hackers can implement code poisoning, they could manipulate models that automate supply chains and propaganda, as well as resume screening and toxic comment deletion," he added. 

Shmatikov further explained that similarly to prior assaults, the hacker must gain access to the model or data during training or deployment, which involves breaking into the victim's machine learning infrastructure. 

"With this new attack, the attack can be done in advance, before the model even exists or before the data is even collected -- and a single attack can actually target multiple victims," Shmatikov said. 

The paper focuses further on the ways for "injecting backdoors into machine learning models, based on compromising the loss-value computation in the model-training code." The team used a sentiment analysis model for the particular task of always classifying as positive all reviews of the infamously bad movies directed by Ed Wood. 

"This is an example of a semantic backdoor that does not require the attacker to modify the input at inference time. The backdoor is triggered by unmodified reviews written by anyone, as long as they mention the attacker-chosen name," the paper discovered. 

"Machine learning pipelines include code from open-source and proprietary repositories, managed via build and integration tools. Code management platforms are known vectors for malicious code injection, enabling attackers to directly modify the source and binary code." 

To counter the attack:
The researchers suggested a technique that could identify changes from the original code of the model. However, Shmatikov claims that because AI and machine learning tools have grown so popular, many non-expert users create their models with code they hardly comprehend. "We've shown that this can have devastating security consequences." 

In the future, the team aims to investigate how code-poisoning links to summarisation and even propaganda automation, which may have far-reaching consequences for the future of hacking.

They will also strive to create robust protections that will eradicate this entire class of attacks and make AI and machine learning secure even for non-expert users," according to Shmatikov.

Researchers Embedded Malware into an AI's 'Neurons' and it Worked Scarily Well


According to a new study, as neural networks become more popularly used, they may become the next frontier for malware operations. 

The study published to the arXiv preprint site stated, malware may be implanted directly into the artificial neurons that make up machine learning models in a manner that protects them from being discovered.

The neural network would even be able to carry on with its usual activities. The authors from the University of the Chinese Academy of Sciences wrote, "As neural networks become more widely used, this method will become universal in delivering malware in the future." 

With actual malware samples, they discovered that changing up to half of the neurons in the AlexNet model—a benchmark-setting classic in the AI field—kept the model's accuracy rate over 93.1 percent. The scientists determined that utilizing a method known as steganography, a 178MB AlexNet model may include up to 36.9MB of malware buried in its structure without being detected. The malware was not identified in some of the models when they were tested against 58 different antivirus programs. 

Other ways of invading businesses or organizations, such as attaching malware to papers or files, are frequently unable to distribute harmful software in large quantities without being discovered. As per the study, this is because AlexNet (like many machine learning models) is comprised mainly of millions of parameters and numerous complicated layers of neurons, including fully connected "hidden" layers, 

The researchers discovered that altering certain other neurons had no influence on performance since the massive hidden layers in AlexNet were still intact. 

The authors set out a playbook for how a hacker could create a malware-loaded machine learning model and distribute it in the wild: "First, the attacker needs to design the neural network. To ensure more malware can be embedded, the attacker can introduce more neurons. Then the attacker needs to train the network with the prepared dataset to get a well-performed model. If there are suitable well-trained models, the attacker can choose to use the existing models. After that, the attacker selects the best layer and embeds the malware. After embedding malware, the attacker needs to evaluate the model’s performance to ensure the loss is acceptable. If the loss on the model is beyond an acceptable range, the attacker needs to retrain the model with the dataset to gain higher performance. Once the model is prepared, the attacker can publish it on public repositories or other places using methods like supply chain pollution, etc." 

According to the article, when malware is incorporated into the network's neurons, it is "disassembled" and assembled into working malware by a malicious receiver software, which may also be used to download the poisoned model via an upgrade.  The virus can still be halted if the target device checks the model before executing it. Traditional approaches like static and dynamic analysis can also be used to identify it.

Dr. Lukasz Olejnik, a cybersecurity expert and consultant, told Motherboard, “Today it would not be simple to detect it by antivirus software, but this is only because nobody is looking in there.” 

"But it's also a problem because custom methods to extract malware from the [deep neural network] model means that the targeted systems may already be under attacker control. But if the target hosts are already under attacker control, there's a reduced need to hide extra malware." 

"While this is legitimate and good research, I do not think that hiding whole malware in the DNN model offers much to the attacker,” he added. 

The researchers anticipated that this would “provide a referenceable scenario for the protection on neural network-assisted attacks,” as per the paper. They did not respond to a request for comment from Motherboard.

This isn't the first time experts have looked at how malicious actors may manipulate neural networks, such as by presenting them with misleading pictures or installing backdoors that lead models to malfunction. If neural networks represent the future of hacking, major corporations may face a new threat as malware campaigns get more sophisticated. 

The paper notes, “With the popularity of AI, AI-assisted attacks will emerge and bring new challenges for computer security. Network attack and defense are interdependent. We hope the proposed scenario will contribute to future protection efforts.”

Kubeflow: The Target of Cryptomining Attacks


Microsoft has discovered a new, widespread, ongoing threat that aims to infect Kubernetes clusters running Kubeflow instances with malicious TensorFlow pods that mine cryptocurrencies. Kubeflow is a popular open-source framework for conducting machine learning (ML) tasks in Kubernetes, while TensorFlow is an end-to-end, open-source ML platform. 

Microsoft security experts cautioned on Tuesday that they noticed a rise in TensorFlow pod deployments on Kubernetes clusters at the end of May — pods that were running legal TensorFlow images from the official Docker Hub account. However, a closer examination of the pods' entry point revealed that they are used to mine cryptocurrency. 

In a post on Tuesday, Yossi Weizman, a senior security research software engineer at Microsoft's Azure Security Center, said that the "burst" of malicious TensorFlow deployments was "simultaneous," implying that the attackers scanned the clusters first, kept a list of potential targets, and then fired on all of them at the same time. The attackers used two distinct images, according to Weizman. The first is the most recent version of TensorFlow (tensorflow/tensorflow:latest), and the second is the most recent version with GPU support (tensorflow/tensorflow:latest-gpu). 

According to Weizman, using TensorFlow images in the network "makes a lot of sense," because “if the images in the cluster are monitored, usage of a legitimate image can prevent attackers from being discovered.” Another rationale for the attackers' decision is that the TensorFlow image they chose is an easy way to conduct GPU activities using CUDA, which "allows the attacker to optimize the mining gains from the host," according to him. 

The newly found vulnerability is comparable to a cryptocurrency mining attack revealed by Microsoft in June. That previous campaign also targeted Kubeflow workloads, launching a broad XMRIG Monero-mining campaign by exploiting misconfigured dashboards. The most recent campaign includes the following changes: According to Weizman, the attackers abused their access to the Kubeflow centralized dashboard to establish a new pipeline this time.

Kubeflow Pipelines is a framework for creating machine learning pipelines based on Argo Workflow, an open-source, container-native workflow engine for coordinating parallel jobs. A pipeline is a collection of steps, each of which functions as its own container, that together creates an ML workflow. 

Users of Kubeflow should ensure that the centralized dashboard is not insecurely exposed to the internet, according to Microsoft.

Security Researchers Raise Concerns Over Security Flaws in Machine Learning


In today’s age, it is impossible to implement effective cybersecurity technology without depending on innovative technologies like machine learning and artificial intelligence. Machine learning in the field of cybersecurity is a fast-growing trend. But with machine learning and AI there comes a cyber threat. Unlike traditional software, where flaws in design and source code account for most security issues, in AI systems, vulnerabilities can exist in images, audio files, text, and other data used to train and run machine learning models.

 What is machine learning? 

Machine learning, a subset of AI is helping business organizations to analyze the threats and respond to ‘adversarial attack’ and security incidents. It also helps to automate more boring and tedious tasks that were previously carried out by under-skilled security teams. Now, Google is also using machine learning to examine the threats against mobile endpoints running on Android along with detecting and removing malware from the infected handsets. 

What are adversarial attacks? 

Adversarial attacks are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake; they’re like optical illusions for machines. For instance, as web applications with database backends started replacing static websites, SQL injection attacks became prevalent. The widespread adoption of browser-side scripting languages gave rise to cross-site scripting attacks. Buffer overflow attacks overwrite critical variables and execute malicious code on target computers by taking advantage of the way programming languages such as C handle memory allocation. 

Security flaws linked with machine learning and AI 

Security researchers at Adversa, a Tel Aviv-based start-up that focuses on security for artificial intelligence (AI) systems have published their report which says many machine learning systems are vulnerable to adversarial attacks, imperceptible manipulations that cause models to behave erratically. 

According to the researchers at Adversa, machine learning systems that process visual data account for most of the work on adversarial attacks, followed by analytics, language processing, and autonomy. Web developers who are integrating machine learning models into their applications should take note of these security issues, warned Alex Polyakov, co-founder and CEO of Adversa. 

“There is definitely a big difference in so-called digital and physical attacks. Now, it is much easier to perform digital attacks against web applications: sometimes changing only one pixel is enough to cause a misclassification,” Polyakov told The Daily Swig.

Machine Learning in Security - How Machine Learning helps security in the real-world?


Image Source

Machine Learning is a core building block in the field of Data Science and Artificial Intelligence. As we all know, mathematics and statistics are the backbones of machine learning algorithms, and the algorithms that are used to discover correlations, anomalies, and patterns deal with data that are too complex. 

When we talk about Security, spam is the first thing that comes to our mind. With the invention of the internet, computers were hooked together to create an effective and valuable communication network, and this medium which had broader distribution and free transmission, perfectly suited to steal account credentials, spread computer viruses, Malware, etc. 

With enormous development in security domains like intrusion detection, malware analysis, web application security, network security, cryptography, etc., even today spam remains a major threat in the email and messaging space which directly impacts the general public. 

The technologists saw a huge potential in Machine Learning in dealing with this constantly evolving issue. The email data can be accessed by the email providers and the internet service providers(ISPs) by which the user behavior, email content, and its metadata can be used to build content-based models to recognize spam. The metadata can be extracted and analyzed to predict the likelihood that an email is spam or not. Some best modern email filters can filter 99.9% of spam and block them, thanks to technology development. 

Indeed, the spam-fighting story has helped researchers to know the importance of data and use the available data and machine learning to detect and defeat malicious adversaries. 

Adversaries & Machine Learning 

All said and done, the adversaries can also take advantage of machine learning to avoid detection and evade defenses. The attackers can also learn about the nature of defenses as much as the defenders can learn from the attacks. It has been known that spammers use polymorphism which is nothing but changing the appearance of the content without changing the content, to avoid detection. 

Adversaries can also use machine learning to learn our interests and personal details from our social media page and use that information to craft a personal phishing message. There is a growing field called adversarial machine learning, by which the attackers can also cause the algorithms to make erroneous predictions and learn wrong things to execute their attacks. 

Machine Learning use cases in Security 

The machine learning use cases in security can be classified to: 
Pattern recognition — In this, we discover explicit characteristics hidden in the data which is nothing but feature sets and these can be used to teach an ML algorithm to recognize other forms of the data that exhibit the same set of characteristics. 
         Examples of pattern recognition are spam detection, malware detection, and botnet detection. 
Anomaly Detection — In this, the goal is to establish a notion of normality that describes 95% of a given dataset. Learning of the patterns is data is not done in this. So, once the normality is determined, any deviations from this will be detected as anomalies. 
        Examples of anomaly detection are Network outlier detection, malicious URL             detection,  user authentication, access control, and behavior analysis. 

Today, almost every piece of technology used by organizations has security vulnerabilities. Driven by some core motivations, malicious actors can pose a security risk to almost all aspects of modern life. A motivated adversary is constantly trying to attack a system, and each side races to fix or exploit the flaws in design and technique before the other uncovers them. 

Often machine learning algorithms are not designed with security in mind and so they are vulnerable to the attempts made by a motivated adversary. Hence, It is very important to have knowledge of the threat models while designing a machine learning system for security purposes. 

References: Machine Learning & Security by Clarence Chio & David Freeman

Hackers Can Use AI and Machine Learning to Attack Cybersecurity


According to researchers at NCSA and Nasdaq cybersecurity summit, hackers can use Machine and AI (Artificial Intelligence) to avoid identification during cybersecurity attacks and make the threats more effective. Hackers use AI to escape disclosure; it allows them never to get caught and adapt to new tactics over time, says Elham Tabassi, National Institute of Standards and Technology's chief of staff information technology laboratory. 

Tim Bandos from Digital Guardian says technology always requires human consciousness to strive forward. It has and will require human effort to counter cyberattacks and stop them. According to Tim, Experts and Analysts are the real heroes, and AI is just a sidekick. 

How are hackers using AI to attack cybersecurity? 

1. Data Poisoning 
In some cyberattacks, hackers exploit the data which is used to train machine learning models. In data poisoning, the hacker manipulates a training dataset to manage the model's prediction patterns and prepare it according to his will to do many hacker desires. These can include spamming or phishing emails. Tabassi says that data is the driving mechanism for any machine learning, and one should focus on the information he uses to train the models to act like any model. Machine learning training models and the data it uses affect user trust. For cybersecurity, the industry needs to establish a standard protocol for data quality. 

2. Generative Adversarial Networks 
GANs are nothing but a setting where two AI systems are set up against each other. One AI generates the content, and the other AI finds the errors. The competition between the two AIs together creates reliable content to get through as the original. "This capability could be used by game developers to automatically generate layouts for new game levels, as well as by AI researchers to more easily develop simulator systems for training autonomous machines," says Nvidia blog. According to Bandos, hackers are using GANs to replicate traffic patterns. It allows them not to draw attention to the attack, and the hackers can steal sensitive data and get out of the system within 30-40 minutes.

Facebook using AI to track hate speech


Facebook's hate speech and malicious content identifying AI seem to be working as the company said that their AI identified and removed 134% more hate speech in the second quarter than in the first. The company stated in the Community Standards Enforcement Report that it acted upon 9.9 million hateful posts in the first quarter of the year and 22.5 million in the second. But the figures also reveal how much of hate content was there and is still on the site, to begin with.

Facebook's VP of Integrity Guy Rosen blames the high number to “the increase in proactive technology” in detecting a said form of content. The company has more and more been relying on machine learning and AI to drive out this type of content by losing bots on the network. 

There has been a similar rise on Instagram as well. They detected 84% of hate speeches in this quarter and 45% in the last and removed 3.3 million of these posts from April to June- a sweeping amount when compared to just 808,900 in January till March. 

The social media site also has plans to use similar technology to monitor Spanish, Arabic, and Indonesian posts. 

These increasing number in hate content does show the platform's improvement in the AI technology used to fish out hate post but it also raises concerns over the hostile environment the network presents. Though the company blames these numbers to an increase in coverage of content.

 “These increases were driven by expanding our proactive detection technologies in English and Spanish,” as the company states.

Some critiques also say that the company has no way of knowing how much percent they are actually capturing and how much there is as they measure it according to 'Prevalence' that is how often a Facebook user sees a hateful post as opposed to how many there actually are. The social media giant also updated as to what they include as hate speech - excluding misinformation that remains a big problem for Facebook.