OISG Adequacy Test

Evaluate your AI system against the four OISG pillars. Each pillar has 5 criteria scored 0–5, for a total score of 0–100. Each criterion is mapped to the main compliance frameworks — EU AI Act, NIST AI RMF, OWASP LLM Top 10, ISO/IEC 42001 and others — with links to the official sources.

Organisation details

This information will appear on the downloadable assessment certificate.

O — Is it Open?

Model documentation (capabilities, limitations, provenance) is available to independent auditors

Maps to EU AI Act Art. 53 Art. 53(1) & Annex XI — GPAI technical documentation, kept up to date and available to the AI Office on request.high confidence EU AI Act Art. 11 Art. 11 & Annex IV — high-risk technical documentation for conformity assessment.high confidence

Governance infrastructure (policy engines, decision logic) is open and auditable

Maps to NIST AI RMF GOVERN GOVERN 1–2 — documented, accountable policies and roles for AI risk management.high confidence EU AI Act Art. 12 Art. 12 & 19 — automatic event logging over the system lifetime, retained for at least six months.high confidence

Communication protocols use open standards (MCP, OpenTelemetry, A2A)
Open projects have community stewardship (contribution process, security disclosure, governance)

Maps to OpenSSF Scorecard Security-Policy, Maintained and Code-Review checks assess disclosure policy, maintenance and contribution review.high confidence OpenSSF Best Practices CII Best Practices Badge — documented contribution process and a vulnerability reporting mechanism.high confidence

Model provenance and training methodology are documented and reproducible

Maps to EU AI Act Art. 53(1)(d) Art. 53(1)(d) — public summary of training content (AI Office template); Art. 10 data governance.high confidence NIST AI RMF MAP 2 MAP 2 — categorise the system and document data provenance and known limits.medium confidence

I — Is it Intelligent (governably)?

Model capabilities are measured with benchmark results, known failure modes, and confidence calibration

Maps to EU AI Act Art. 15 Art. 15 & Annex IV(2)(g) — accuracy levels and the metrics used, declared in the documentation.high confidence NIST AI RMF MEASURE 2.3 MEASURE 2.3 — performance measured and documented for deployment-like conditions.high confidence

Infrastructure supports sovereign execution (on-premise, private cloud, air-gapped) where required

Maps to ISO/IEC 27001 A.5.23 A.5.23 (2022) — information security for the use of cloud services; supports on-prem / private-cloud data residency.medium confidence

RAG pipelines are traceable (document version, embedding model, retrieval path)

Maps to EU AI Act Art. 12 Art. 12 record-keeping & Art. 10 data governance — traceable retrieval and data lineage.high confidence OWASP LLM08:2025 LLM08:2025 Vector and Embedding Weaknesses — RAG retrieval integrity and traceability.high confidence

Agent autonomy scope is explicit, machine-readable, and enforced at runtime

Maps to EU AI Act Art. 14 Art. 14 — human oversight; agent autonomy must be bounded and overseeable.high confidence OWASP LLM06:2025 LLM06:2025 Excessive Agency — scope, permissions and autonomy of agents must be constrained.high confidence

System can produce on demand a complete explanation of why it gave a specific response

Maps to EU AI Act Art. 13 Art. 13 transparency to deployers & Art. 86 right to explanation of individual decisions.high confidence NIST AI RMF MEASURE 2.9 MEASURE 2.9 — model explainability and interpretability are documented and validated.high confidence

S — Is it Secure?

Bidirectional injection defence operates on both request and response paths

Maps to OWASP LLM01:2025 LLM01:2025 Prompt Injection — direct and indirect injection across request and response paths.high confidence MITRE ATLAS AML.T0051 AML.T0051 LLM Prompt Injection (.000 Direct / .001 Indirect).high confidence EU AI Act Art. 15(5) Art. 15(5) — resilience against attempts by third parties to exploit system vulnerabilities.high confidence

Agent identities are cryptographically verifiable (DIDs, Ed25519 key pairs)

Maps to OWASP LLM06:2025 LLM06:2025 Excessive Agency — verified agent identity bounds inter-agent trust.high confidence ISO/IEC 27001 A.5.16 A.5.16 Identity Management & A.8.5 Secure Authentication.medium confidence

Transactional kill switch preserves forensic state and enables rollback

Maps to NIST AI RMF MANAGE 2.4 MANAGE 2.4 — mechanisms to supersede, disengage or deactivate systems showing unacceptable behaviour.high confidence EU AI Act Art. 14(4)(e) Art. 14(4)(e) — ability to intervene or interrupt via a stop button or similar procedure.high confidence

PII redaction is enforced at infrastructure level before model endpoints

Maps to GDPR Art. 25 Art. 25 — data protection by design and by default; redact PII before it reaches the model.high confidence OWASP LLM02:2025 LLM02:2025 Sensitive Information Disclosure — prevent PII leakage to and from the model.high confidence

Model supply chain integrity is verified (fingerprinting, SBOM, cryptographic provenance)

Maps to SLSA Build L2/L3 SLSA Build track — signed provenance (L2) and hardened builds (L3) extended to model artifacts.high confidence OWASP LLM03:2025 LLM03:2025 Supply Chain — integrity of models, adapters and datasets.high confidence

G — Is it Governed?

Compliance is verified automatically at runtime, not through periodic audits

Maps to EU AI Act Art. 72 Art. 72 post-market monitoring & Art. 9 risk management — continuous, not periodic.high confidence NIST AI RMF MANAGE 4.1 MANAGE 4.1 — post-deployment monitoring plans are implemented and documented.high confidence

Immutable forensic log (hash-chained) records all interactions and decisions

Maps to EU AI Act Art. 12 Art. 12 record-keeping & Art. 19 automatically generated logs.high confidence ISO/IEC 27001 A.8.15 A.8.15 Logging & A.8.16 Monitoring activities.medium confidence

Human oversight is architecturally defined (which decisions, what info, what timeout)

Maps to EU AI Act Art. 14 Art. 14 — human oversight defined as a design requirement.high confidence NIST AI RMF GOVERN 3.2 GOVERN 3.2 & MANAGE 2.4 — human roles, oversight and intervention are defined.high confidence

End-to-end observability is in place (distributed tracing, SLOs, dashboards)

Maps to EU AI Act Art. 72 Art. 72 post-market monitoring & Art. 15 accuracy/robustness across the lifecycle.high confidence NIST AI RMF MEASURE 2.4 MEASURE 2.4 — deployed system monitored with documented mechanisms; MANAGE 4.1.high confidence

Risk classification is proportional, automated, and auditable as capabilities evolve

Maps to EU AI Act Art. 6 Art. 6 & Annex III high-risk classification rules + Art. 9 risk management.high confidence ISO/IEC 23894:2023 ISO/IEC 23894 — AI risk identification and assessment process.medium confidence

0
/ 100
Critical gaps