Artificial Intelligence (AI) – Testing & Certification of AI Security

Do you have any questions or comments? Contact us!

Trustworthy AI – TÜVIT tested!

AI technologies are increasingly part of all areas of our everyday lives. This increases the risks posed by error-prone AI systems, data misuse, and cyber attacks.

We support you on your way to trustworthy AI: Our experts accompany you throughout your development process and carry out detailed analyses (e.g. of the chosen model design) and tests (e.g. robustness, bias, explainability) of your AI systems. Subsequently, a certification can attest the technical security features of your AI.

  

 Think one step ahead

AI regulation is coming. With AI testing & certification, you can already optimally prepare yourself for future requirements.

 Show responsibility

With AI testing & certification, you fulfill your corporate responsibility and demonstrate the high importance of AI security.

 Better safe than sorry

External testing helps to detect and eliminate potential security vulnerabilities at an early stage.

Artificial Intelligence (AI) – Testing & Certification of AI Security Artificial Intelligence (AI) – Testing & Certification of AI Security Artificial Intelligence (AI) – Testing & Certification of AI Security Artificial Intelligence (AI) – Testing & Certification of AI Security

Your benefits at a glance

Artificial Intelligence (AI): Proof of the security of your AI Artificial Intelligence (AI): Proof of the security of your AI Artificial Intelligence (AI): Proof of the security of your AI Artificial Intelligence (AI): Proof of the security of your AI

Proof of the security of your AI:
With AI testing & certification, you objectively demonstrate the security features of your product.

Artificial Intelligence (AI): Security & robustness as a competitive advantage Artificial Intelligence (AI): Security & robustness as a competitive advantage Artificial Intelligence (AI): Security & robustness as a competitive advantage Artificial Intelligence (AI): Security & robustness as a competitive advantage

Security & robustness as a competitive advantage
Secure and robust AI systems help you build trust and thus gain a competitive advantage.

Artificial Intelligence (AI): Identification of vulnerabilities in your AI systems Artificial Intelligence (AI): Identification of vulnerabilities in your AI systems Artificial Intelligence (AI): Identification of vulnerabilities in your AI systems Artificial Intelligence (AI): Identification of vulnerabilities in your AI systems

Identification of vulnerabilities in your AI systems
We systematically test your AI systems for vulnerabilities before they go to market.

Artificial Intelligence (AI): Transparency & fairness as factors in success Artificial Intelligence (AI): Transparency & fairness as factors in success Artificial Intelligence (AI): Transparency & fairness as factors in success Artificial Intelligence (AI): Transparency & fairness as factors in success

Transparency & fairness as factors in success
Score points with tested AI systems whose decisions are explainable and unbiased.

Artificial Intelligence (AI): Focus on your business Artificial Intelligence (AI): Focus on your business Artificial Intelligence (AI): Focus on your business Artificial Intelligence (AI): Focus on your business

Focus on your business
You need your experts for your product development - our team supports you with security.

Artificial Intelligence (AI): One step ahead of AI regulation Artificial Intelligence (AI): One step ahead of AI regulation Artificial Intelligence (AI): One step ahead of AI regulation Artificial Intelligence (AI): One step ahead of AI regulation

One step ahead of AI regulation
EU-wide regulation of AI is making progress. With AI testing & certification, you can optimally prepare yourself.

Artificial Intelligence (AI): Optimized training for your AI Artificial Intelligence (AI): Optimized training for your AI Artificial Intelligence (AI): Optimized training for your AI Artificial Intelligence (AI): Optimized training for your AI

Optimized training for your AI
Based on our analysis, you can retrain specifically with additional data sets.

Artificial Intelligence (AI): Security & safety by design Artificial Intelligence (AI): Security & safety by design Artificial Intelligence (AI): Security & safety by design Artificial Intelligence (AI): Security & safety by design

Security & safety by design
With our development support approach, you will be integrate security and safety right from the beginning.

Our services for your AI security

We support companies already during development, and perform analyses for AI systems. Depending on customer requirements, subsequent certification of the technical security features can take place.
  

Services Artificial Intelligence (AI): Consulting on problem areas & solution approaches Services Artificial Intelligence (AI): Consulting on problem areas & solution approaches Services Artificial Intelligence (AI): Consulting on problem areas & solution approaches Services Artificial Intelligence (AI): Consulting on problem areas & solution approaches

(Development-accompanying) pre-checks with regard to problem areas & solution approaches

Services Artificial Intelligence (AI): Scoping & gap analyses for AI testing/certification Services Artificial Intelligence (AI): Scoping & gap analyses for AI testing/certification Services Artificial Intelligence (AI): Scoping & gap analyses for AI testing/certification Services Artificial Intelligence (AI): Scoping & gap analyses for AI testing/certification

Scoping & gap analyses for AI testing/certification

Services Artificial Intelligence (AI): Theoretical analyses (development process, model design) Services Artificial Intelligence (AI): Theoretical analyses (development process, model design) Services Artificial Intelligence (AI): Theoretical analyses (development process, model design) Services Artificial Intelligence (AI): Theoretical analyses (development process, model design)

Theoretical analyses (development process, model design)

Services Artificial Intelligence (AI): Security and safety analysis with special test software Services Artificial Intelligence (AI): Security and safety analysis with special test software Services Artificial Intelligence (AI): Security and safety analysis with special test software Services Artificial Intelligence (AI): Security and safety analysis with special test software

Security and safety analysis with special test software

AI certification is already possible now

Once our testing is complete, we can certify the security features of your product in our Trusted Product scheme. The certificate documents the results of the tests performed. The test report also provides a corresponding, structured classification of these results in the overall context.
 

The certification process at a glance:

AI certification: scoping AI certification: scoping AI certification: scoping AI certification: scoping
  • kick off workshop
  • definition of assets & threats
  • definition of valid input/output ranges
  • derivation certifiable requirements ("Technical Security Requirements")
AI certification: theoretical analysis & derivation of test pla AI certification: theoretical analysis & derivation of test pla AI certification: theoretical analysis & derivation of test pla AI certification: theoretical analysis & derivation of test pla
  • detailed review of the model architecture
  • derivation of a test plan
  • calibration of test tool for the defined tests 
AI certification: testing AI certification: testing AI certification: testing AI certification: testing
  • testing of the model using test tool
  • spot check validation of the results
AI certification: documentation of results & certification AI certification: documentation of results & certification AI certification: documentation of results & certification AI certification: documentation of results & certification
  • documentation of results
  • issuance of evaluation report
  • if all requirements are met: issuance of certificate

What are potential problem areas for artificial intelligence?

Potential problem areas for artificial intelligence (AI) Potential problem areas for artificial intelligence (AI) Potential problem areas for artificial intelligence (AI) Potential problem areas for artificial intelligence (AI)

Artificial intelligence (AI) can be used to solve many data-driven problems more efficiently or faster than traditional software development. Despite all the advantages, however, its use always leads to additional sources of error, which must be considered appropriately as part of a risk analysis.
  

  • Robustness: AI systems that use machine learning usually have a robustness weakness: Even minor changes to the input can lead to incorrect classification.
  • Explainability: Another problem area of AI is the lack of explainability or transparency of decisions made by AI systems. In turn, this can have an impact on fairness.
  • Bias: Using artificial intelligence does not make bias disappear. This is because machines can also make discriminatory decisions (machine bias).

Main points of AI examination & certification

Main points of AI examination & certification: Security / robustness (robustness testing) Main points of AI examination & certification: Security / robustness (robustness testing) Main points of AI examination & certification: Security / robustness (robustness testing) Main points of AI examination & certification: Security / robustness (robustness testing)


Security / robustness
(robustness testing)


Is the AI safe and secure against attacks, accidents and errors?

Main points of AI examination & certification: Transparency (explainability testing) Main points of AI examination & certification: Transparency (explainability testing) Main points of AI examination & certification: Transparency (explainability testing) Main points of AI examination & certification: Transparency (explainability testing)


Transparency
(explainability testing)


Are decision-making and the functioning of the AI transparent and comprehensible?

Main points of AI examination & certification: Fairness (bias testing) Main points of AI examination & certification: Fairness (bias testing) Main points of AI examination & certification: Fairness (bias testing) Main points of AI examination & certification: Fairness (bias testing)


Fairness
(bias testing)


Are the decisions made by the AI fair to all parties?

FAQ – At a glance

Is AI certification already possible?

Although AI-specific laws, norms and standards are still not here, it is already possible to certify properties of AI systems.

Once our testing is complete, we can certify the security features of your product in our Trusted Product scheme. The certificate documents the results of the tests performed, and the test report provides a corresponding, structured classification of these results in the overall context.

Which testing tools can be used to test AI? Which tools does TÜVIT use?

There are various software products for performing security audits of AI. It is not possible to give a general answer as to which is the most suitable for a particular application; hence this will be part of the initial phase of our testing service. TÜVIT always uses a testing tool developed in-house for its tests, but can also make use of external testing tools.

Are you already using with testing software “XYZ”, and would like to have these results verified by an independent body?

No problem: We can also integrate third-party tools into our audit process, as long as we consider them suitable. Our proprietary TÜVIT testing tool is then used as a reference for calibration and approval of the tests.

What are examples of a robustness weakness?

AI systems that use machine learning usually have a robustness weakness: Minor changes to the input lead to incorrect classification. This can have far-reaching consequences. If AI is used in camera-based recognition of traffic signs, e.g. rain or snow can cause a stop sign to be recognized as a right-of-way sign.

However, it is important to consider not only natural phenomena (such as the weather, pollution, etc.), but also deliberate manipulation by attackers. A small, specially prepared sticker on the stop sign or a large poster in the background of the sign can be deliberately used by attackers to deceive the AI.

This is where our robustness analysis comes in: We systematically test your system for these vulnerabilities before it goes into circulation. This allows you to toughen up your AI system appropriately during development, thus minimizing malfunctions in later use.

What are the benefits of certification?

Certification ensures that your customers can trust your AI system.

These AI topics might also interest you:

  

How do you test AI?

Artificial intelligence (AI) is increasingly finding its way into our everyday lives. But if we want to let AI take the wheel of our cars in the future, it will need to function reliably at all times. AI expert Vasilios Danos from TÜVIT explains how and according to which criteria AI can be tested for the autonomous car.

TÜVIT and Fraunhofer AISEC develop a joint approach to AI certification

In a joint project with Fraunhofer AISEC, TÜVIT has developed procedures to make the trustworthiness of AI applications measurable and verifiable, as this is of utmost importance not only for companies in all industries, but also for government institutions.

Why we are a strong partner for you

Comprehensive AI expertise

Our experts have the specialized knowledge and tools needed to professionally audit the security, fairness, and transparency of your AI systems.

Development support

From the beginning: We support your company during development already, carrying out corresponding theoretical analyses and tests relating to your AI systems.

A pool of AI experience

We have been developing our own in-house testing tools for AI since 2018, and parallel to this are expanding corresponding testing services.

Always state-of-the-art

Our experts continuously stay up-to-date with the latest technology and have extensive insight into the current research landscape on AI testing.

Tailor-made for you

We focus on individual services - and solutions - that optimally fit your current company situation and your set goals.

International network of experts

Around the globe: We support you both nationally and internationally. Our global network of experts is ready to help you in all IT security issues.

Independence

Our employees are not subject to any conflicts of interest, as they are not committed to any product suppliers, system integrators, stakeholders, interest groups or government agencies.