State of AI applied to Quality Engineering 2021-22

Section 9: Trust AI

by Sogeti

Business ●●●●●
Technical ○○○○○

Listen to the audio version

Download the "Section 9: Trust AI" as a PDF

Use the site navigation to visit other sections and download further PDF content

By submitting this form, I understand that my data will be processed by Sogeti as described in the Privacy Policy.*

Introduction

All prior aspects of this research have been focused so far on testing with AI. In other words, how AI may assist us optimise quality engineering & testing procedures. As described in the executive introduction, Artificial Intelligence itself must be evaluated too, to ensure that we can rely on the accuracy, precision, and ethics of AI. Especially if AI assumes a key place in decision-making process.

How do we effectively and efficiently validate AI solutions?

The key challenges are threefold:

  1. It is incredibly challenging to foresee the output of AI systems. In conventional IT, we compare the actual and expected results using a variety of rules and scenarios. However, a machine-learning-based systems modifies its behaviour overtime, and the outcomes can differ substantially, even if the identical function is completed. How to define acceptance criteria? What is the expected outcome of the model? Those are just a few questions we need to address?

  2. Ethics
    How can we assure that such systems produce rational, explainable, and intelligible decisions and are not biased in their design, training, or operation? Can the AI system under test demonstrate that it is equitable and devoid of discrimination? What can we do to limit risk and improve the fairness of artificial intelligence systems?

  3. Challenges around the learning mechanism.
    Numerous tests are focussed on simulating expected user data or behavior to train machine learning models. These simulated tests and statistics are never as exact as the actual thing. How to boost the correctness of the global model of an AI system rather than its individual segregated deployments? How to bring models to the data rather than the other way round?

This section aims to assist you in understanding the testing of AI systems, from product, process, and ethical perspectives.

In this section

Chapter 1: Quality engineering applied to artificial intelligence

Chapter 2: Testing the ethics of AI

Chapter 3: How to build AI that people trust

Chapter 4: Scaling Quality Engineering Using AI for AI

Chapter 5: Keep test criteria current using Federated Learning