Press release
The OpenAI Hugging Face incident highlights that as the capabilities of modern AI increase, so too do the requirements for secure evaluation environments. At the same time, locally run open-weight models are becoming increasingly important for forensics, security testing and the protection of sensitive data.
The incident between OpenAI and Hugging Face, in which an AI system being tested as part of a security evaluation escaped its intended environment and compromised a third-party company’s infrastructure, highlights an important development: as the capabilities of AI systems increase, so too do the requirements for secure evaluation environments.
The discussion surrounding the incident has focused primarily on what the AI in question was capable of. From the perspective of independent security audits, however, another question is at least as relevant: were the surrounding safeguards even designed to reliably contain a system with such capabilities?
Another noteworthy aspect emerged during the response to the incident. According to Hugging Face, commercial Frontier models could not be used effectively for forensic analysis. The providers’ protective mechanisms blocked the processing of attack logs, exploits and command-and-control artefacts. Hugging Face therefore carried out the digital forensics and incident response with the support of the Open-Weight model GLM 5.2, which was operated entirely within its own infrastructure. This ensured that sensitive attack data, access credentials and other investigative artefacts remained under the company’s control.
The incident thus also highlights the growing importance of Open-Weight models and locally deployable Small Language Models (SLMs) for modern IT forensics and cybersecurity. They make it possible to process sensitive data within the organisation’s own infrastructure and to adapt the systems used to the requirements of the respective investigation. This observation is closely aligned with TÜVIT’s approach.
In cybersecurity assurance activities, source code, threat analyses, testing procedures and clients’ intellectual property are often among the most sensitive pieces of information. TÜVIT therefore focuses on providing AI functions within isolated testing environments, rather than relying on externally hosted services. In AI-supported security testing pipelines, open-weight models and locally deployable SLMs can be integrated directly into controlled laboratory environments. This enables penetration testers, evaluators and forensic analysts to analyse source code, assess security measures and validate hardening measures without having to transfer sensitive client artefacts, such as source code or other intellectual property, to external providers or other third parties. Local operation supports both the necessary confidentiality and control over data, models and processing steps. At the same time, it contributes to the traceability expected in independent testing and evaluation procedures.
High-performance AI requires a robust and secure technical foundation. Through penetration testing, TÜVIT identifies vulnerabilities in a targeted manner and demonstrates how risks can be effectively mitigated.
According to TÜVIT’s current methodology, software product testing can be carried out in fully isolated environments and, where necessary, in air-gapped environments. Open-weight and locally deployable SLMs can be integrated directly into the existing testing infrastructure. The AI systems act as accelerators for security experts. They assist with the analysis of large volumes of data, the identification of relevant correlations and the structured evaluation of technical information. The technical assessment and responsibility for the test results remain with the penetration testers and evaluators involved.
The lesson to be learnt from this incident therefore goes beyond the security of individual AI systems. Organisations must not only ensure that powerful AI models are properly controlled and operated within secure boundaries. Security and forensics teams also need access to trustworthy AI tools that can be deployed locally, transparently and within clearly defined trust boundaries. Open-weight models and locally deployable SLMs are thus becoming an increasingly important component of modern cybersecurity assurance and evaluation procedures. They offer the possibility of combining powerful AI support with high standards of confidentiality, control and traceability.
TÜV Informationstechnik GmbH (headquartered in Essen, Germany) is a renowned IT security service provider and an independent testing institute and laboratory specialising in IT security and cyber security in the digitalisation sector. It has been accredited worldwide since 1995. Through vulnerability analyses, audits and evaluations, TÜVIT builds trust in security measures at the level of business processes, data, applications and technologies. TÜVIT is a powerful partner in the detection of and response to cyber-attacks, ensuring a rapid restoration of business operations. Businesses, public authorities and operators of critical infrastructure thus strengthen their regulatory compliance in the areas of confidentiality, integrity and availability, as well as their holistic cyber resilience and IT security throughout the supply chain.
Together with ALTER TECHNOLOGY, TÜVIT forms the Digital & Semiconductor business unit. This business unit is a cornerstone of the knowledge-based organisation TÜV NORD GROUP, which has stood for safety and trust worldwide for over 150 years. Engineers and IT security experts in more than 100 countries ensure that companies become even more successful in today’s interconnected world.
