Using Evidence store

What is the Evidence Store?

The Evidence store is a feature of Saidot’s AI Governance platform and included as part of the standard Saidot Governance product. Evidence store supports users in identifying, reusing, and adapting previously documented compliance evidence linked to policy requirements across the organisation’s AI systems inventory. By reducing redundant documentation and promoting consistency in reasoning and language, the Evidence store helps improve the quality and efficiency of enterprise AI compliance workflows.

Users access the Evidence Store through the Policy Editor interface within the Saidot Governance product. Within this view, users can review policy requirements and associated controls from a policy template and collect evidence against them. Evidence is stored by adding one or more evidence blocks to each control. Users may manually create new evidence blocks or import matching evidence previously recorded in other parts of the organisation’s AI system inventory.

Customers have an option to enable AI-based recommendations in the Evidence store. When enabled, the service automatically suggests matching evidence based on similarities between the current requirement and control in question and the requirements and controls of any previously stored evidence. The system compares textual representations of requirements and controls to surface evidence blocks stored to other similar requirements and controls, thereby facilitating reuse and minimising rework.

To maintain confidentiality and avoid confidential data leaking to unintended users via the evidence store, administrators can configure whether evidence suggestions are drawn from the entire organisation’s evidence store or restricted to a specific Space—a defined workspace with controlled access. This setting ensures that evidence reuse respects organisational policies regarding data sensitivity.

Configuring the Evidence Store in the Admin Interface

By default, the Evidence store is enabled on a Space level without AI recommendations. Your organisation’s Admin users can modify these default settings in the Admin interface by navigating to the ‘Evidence store’ section of the Admin / Organisation interface.

Admins can control the feature in three ways

• Enable / disable the feature: This will enable or disable the evidence store icon in the policy editor user interface. When disabled, users will see disabled evidence store icon with short description about the feature.

• Enable / disable AI-powered recommendations: This will enable or disable the evidence store to use AI-enabled evidence recommendations for easier findability of matching evidence.

• Share evidence on an Organisation / Space level: This will specify the visibility of evidence store content across the entire organisation or within each specific Space, allowing limitation of evidence visibility for data confidentiality.

When modifying default options, please press Save changes to activate the updated configurations.

Using the Evidence Store in your system policy management

When the evidence store is enabled for your organisation, you can start using it in your system inventory. In the following, we will describe how you will be able to open the evidence store, find evidence, and add evidence to your policy control.

Opening the Evidence Store

The evidence store is opened by clicking a store icon in the policy editor user interface. You will be directed to this interface when working on an individual policy template in your System Policies tab. The evidence store icon is visible for all the controls of a policy, providing contextual support for your evidence collection.

When disabled, users will be able to see inactive store icon with short overview of the feature.

Finding evidence from the evidence store

The evidence store interface provides different means for finding the relevant evidence for a given control.

With the Visibility filter, users will be able to see either AI-based evidence recommendations or all evidence in the store accessible to the given system. AI recommendations surface evidence only if they have been originally saved to a control that is semantically related to your current control. Please note that the evidence store does not assess the semantics of your evidence content blocks, but only the controls to which they have been saved. For further details on the AI-based recommendations, please refer to AI-Based Matching Capability section of this page.

With the Control status filter, you can narrow down the evidence listing to completed controls only or to all evidence. This filter is particularly relevant to helping find audit-ready evidence from among all saved evidence in the store. Please note that we filter out by default any evidence blocks without content.

When your visibility filter is set to all evidence, you will be able to search for relevant evidence from your entire accessible evidence store. This will be helpful when you need to find a specific piece of evidence from the store.

Adding evidence from the evidence store

Clicking on evidence will allow you to review further details of the evidence, including evidence content and key metadata of its origin. After reviewing the block content for reusability, you can add the evidence block to your control by clicking the ‘Add and edit evidence block’ button.

Once the evidence block has been added to your control, you will be able to edit the reused evidence in edit mode. Please note that using evidence from the store will create a new copy of the evidence, and all the editing will impact only this given instance of it. After editing the reused evidence to suit your system context, press Save to finish your editing.

AI-based matching capability

The evidence recommendation functionality is enabled by an AI system that uses natural language embeddings to identify semantic relationships between policy requirements and controls. Saidot utilises the open-source sentence-transformer model sentence-transformers/all-MiniLM-L6-v2 via Hugging Face. This model is trained to map sentences and paragraphs to produce 384-dimensional sentence embeddings that capture the semantic content of short texts, enabling similarity clustering and semantic search.

Within Saidot, embeddings are created for:

• The name and description fields of Policy Requirement objects, and

• The name field of Control objects.

When a user engages with the Evidence Store from a policy requirement and its control, the system computes the embedding and compares it with the indexed embeddings of previously stored requirements and controls. If a similarity threshold is met, the system suggests any associated evidence blocks as potential matches.

All embedding computations are performed within Saidot’s cloud infrastructure in Azure. The model is pulled as open-source code from Hugging Face and executed in a containerised environment inside the platform. There is no external API call or data transmission involved in the embedding process, ensuring that all proprietary confidential customer data remains within the Saidot-controlled environment. Matching is based solely on Saidot’s proprietary policy requirement and control data, ensuring that customer data remains fully within Saidot’s controlled environment.

Search capabilities

In addition to AI-assisted evidence matching, the Evidence Store also supports manual keyword search over the customer’s evidence block content. This search does not use artificial intelligence but is implemented using tokenisation and mathematical distance algorithms within the platform’s backend database. These search methods support exact and partial string matching and are optimised for performance and usability.

The system provides a hybrid evidence search experience:

• AI-based semantic matching to suggest contextually similar evidence; and

• Keyword-based search for precise retrieval and filtering of evidence blocks.

How to govern the use of the Evidence Store?

The evidence recommendation functionality is enabled by an AI system, which, like any AI system, may present certain risks. In this section, we suggest mitigation measures that Saidot users can implement to mitigate potential risks that might surface through the use of this AI-enabled feature in the Evidence Store.

Saidot users can mitigate the risks of the evidence recommendation by implementing human oversight, maintaining autonomy in the decision-making by critically analysing and accepting the suggestions of evidence blocks by this feature.

In cases of high-risk AI systems and/or high-stakes requirements, the use of the evidence recommendation can be limited to scenarios where potential inaccuracy from this feature is manageable or tolerable. Moreover, organisations can create guidelines to Saidot users, giving clear instructions regarding the types of systems and/or risk levels where the evidence recommendation should not be used, and manual keyword evidence search in the Evidence Store is recommended instead.

Lastly, Saidot users can also mitigate the risk of incomplete data in previously stored evidence controls (e.g. containing missing values or gaps in information) by providing clear and detailed user instructions to create high-quality evidence blocks with complete data, so the evidence recommendation feature will make the best matches and suggestions.