Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Secure an AI application by protecting the whole system—not just the model. It inherits ordinary application risks and adds risks involving prompts, data, retrieval, model supply chains, and agent actions. Start by mapping what the system can access and do, then apply standard application controls, test AI-specific failure modes, and monitor the deployed service.

For a practical control set, use the OWASP Artificial Intelligence Security Verification Standard (AISVS) alongside ordinary application, infrastructure, and supply-chain security practices. OWASP’s LLM Top 10 helps teams recognize risk categories; NIST’s voluntary AI Risk Management Framework Playbook helps organize risk work across a system’s lifecycle. These resources have different purposes and are not substitutes for one another.

Choose a risk-scaled checklist before implementation

A startup does not need to treat every AI feature like a safety-critical system, but a small team should not skip authentication, authorization, data boundaries, testing, or monitoring. Scale verification to the sensitivity of the data, the impact of an incorrect result, the actions the system can take, and the threats it faces.

OWASP AISVS 1.0, released in June 2026, is an open, vendor-neutral catalogue of testable security requirements for AI-enabled systems. It contains 191 requirements across 12 chapters and three appendices, each assigned a verification level. OWASP describes three levels:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
OWASP AISVS level Requirements OWASP’s intended use
Level 1 51 Baseline for every AI system
Level 2 95 Production, customer-facing, personal-data, or consequential systems
Level 3 45 Critical infrastructure, safety-critical AI, regulated industries, or sophisticated attackers

These are OWASP’s categories, not a claim that every startup must complete all 191 requirements immediately or that completing them guarantees security. Select requirements appropriate to the system, record deferred work with an owner and rationale, and raise the verification level as impact or exposure grows.

Use each framework for the job it does

Resource Best use What it does not replace
OWASP AISVS 1.0 (2026) AI-specific, testable controls for design reviews, acceptance criteria, code review, CI/CD checks, penetration tests, red-team exercises, and audits. General application, infrastructure, and supply-chain security verification.
OWASP LLM Top 10 Recognizing and discussing common LLM application risk categories. A complete, testable implementation checklist. The OWASP initiative page identifies a 2026 edition as its latest guide; the detailed category names discussed here are the page’s 2025 workstream labels, not a claim about the contents or ranking of the 2026 edition.
NIST AI RMF Playbook Organizing voluntary risk-management work under Govern, Map, Measure, and Manage. Specific security controls or a guarantee of compliance. NIST describes the Playbook as companion guidance based on AI RMF 1.0, released January 26, 2023; its page was updated June 10, 2026.
OWASP LLM Applications Cybersecurity and Governance Checklist v1.1 (2024) A cross-functional prompt for leaders spanning executive, technology, cybersecurity, privacy, compliance, legal, DevSecOps, and MLSecOps roles. A newer standard: this checklist is dated May 7, 2024, so pair it with current controls such as AISVS 1.0.

The AISVS chapters cover training data integrity and traceability; input validation; model lifecycle and change control; infrastructure, configuration, and deployment; access control and identity; model supply chains; model behavior and output control; memory, embeddings, and vector databases; orchestration and agents; MCP security; adversarial robustness; and monitoring, logging, and anomaly detection. OWASP says AISVS is intentionally narrow: verify the rest of the application stack against the standards that cover those areas.

1. Map the system, data, and trust boundaries

Write down what the AI feature is supposed to do and trace a request from the person who makes it to every service and action it can reach. This map gives the team a concrete basis for deciding which controls to build and test.

  • Inventory the user-facing purpose, model provider and version, application services, data sources, retrieval stores, plugins or tools, MCP servers, deployment environment, and human decision points.
  • Classify data the system processes or returns, including personal, financial, health, business-confidential, security, and legal information. Decide which categories may be sent to each external service and what may be retained or logged.
  • Draw trust boundaries among users, application services, model endpoints, retrieval data, agent tools, third-party services, and administrative interfaces. Assign an owner for each boundary and dependency.
  • For each user and integration, ask what it can reach, what actions the model can trigger, what data those actions can access, and what the consequences would be if output were wrong or manipulated.
  • Use the NIST AI RMF Playbook’s Govern, Map, Measure, and Manage functions if a lifecycle structure will help the team assign responsibilities and track decisions. NIST describes both the AI RMF and Playbook as intended for voluntary use.

2. Keep conventional application security in scope

A model is not an identity system or an authorization layer. Enforce access decisions in application code and infrastructure, regardless of what the model says a user is allowed to do.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Require authentication for user and service access. Authorize each data access and tool action on the server; do not trust instructions in a prompt or claims supplied by the user as proof of permission.
  • Apply least privilege to service identities, database access, cloud roles, model endpoints, tools, and administrator accounts. Separate tenants, then test that retrieval and tool calls cannot cross customer boundaries.
  • Store credentials in a secret manager or controlled CI secret store. Do not hardcode secrets in source code or notebooks. Revoke and rotate credentials that are exposed or over-privileged.
  • Use secure development practices for dependencies, build pipelines, deployment configuration, artifact access, vulnerability management, and backups. AISVS complements rather than replaces verification of these general security areas.
  • Protect public inference endpoints with authentication where appropriate, input validation, rate limits, and abuse detection. Set per-tenant limits for requests, tokens, concurrency, and spend so one user cannot consume disproportionate resources.

3. Treat prompts, documents, and retrieval results as untrusted

Prompt injection can arrive directly from a user or indirectly through content the application retrieves, uploads, browses, or receives from a tool. A model’s instruction hierarchy, delimiters, or a warning phrase in a prompt is not an authorization boundary.

  • Test direct and indirect prompt injection using user inputs, uploaded files, retrieved documents, web pages, and tool responses.
  • Separate system and developer instructions from user content with structured prompt templates and explicit data boundaries. Treat this as a way to clarify inputs, not as a guarantee that malicious content cannot influence the model.
  • Supply only the context needed for the request. Enforce document authorization before retrieval and again before including a result in the model’s context.
  • Test whether a user can extract system prompts, secrets, another tenant’s records, hidden retrieval content, or confidential context. Do not put secrets in prompts and rely on the model to keep them secret.

4. Validate output and limit what tools can do

Generated text and structured output are untrusted input to the rest of the application. The model may produce malformed values, unsafe content, or an action that exceeds the user’s authority.

  • Validate schemas, types, ranges, identifiers, and business rules before using model output in SQL, HTML, shell commands, code execution, or downstream APIs. Escape or encode content for the context in which it will be displayed or executed.
  • Expose only allowlisted tools with narrowly scoped permissions and explicit argument validation. Separate read-only tools from tools that can change state.
  • Require confirmation or human review for consequential, external, financial, destructive, or privilege-changing actions. Keep authorization and transaction checks outside the model; do not let the model set its own permissions or bypass established approval paths.
  • Keep an audit trail of tool requests, authorization decisions, human approvals, and results. Minimize sensitive prompt and response content in logs, and protect access to whatever is retained.

OWASP’s risk material highlights excessive agency, insecure output handling, and insecure plugin design among the issues teams should assess. The controls above make the permitted actions and approval points explicit instead of relying on model behavior.

5. Track models, data, and dependencies through changes

An AI feature depends on more than its model endpoint. A changed model, dataset, embedding pipeline, vector store, plugin, or MCP server can alter behavior, access, and exposure.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Maintain an inventory of model providers and versions, datasets, embeddings, vector stores, plugins, MCP servers, libraries, and hosted services. Assign owners and review changes before deployment.
  • Verify the provenance and integrity of third-party models and datasets before production use. Store model artifacts in access-controlled registries, sign binaries when feasible, encrypt stored weights and datasets, and restrict logs and intermediate outputs.
  • Version training, fine-tuning, and retrieval data; record lineage and changes; and validate and sanitize data sources. If training on sensitive data, consider privacy-preserving approaches based on a documented threat and privacy assessment.
  • Review model, tool, and vendor updates for changed behavior, permissions, data handling, or attack surface. Retire test and deprecated endpoints so they are not left reachable.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

6. Test security before release and after meaningful changes

Make controls testable: convert selected AISVS requirements into acceptance criteria, code-review checks, and automated tests. Choose verification depth based on data sensitivity, user impact, and threat profile, and document deferred requirements with an owner and rationale.

  • Test ordinary web vulnerabilities and access control as well as AI-specific risks. Prompt and retrieval testing does not replace application security testing.
  • Exercise prompt injection, sensitive-data leakage, unauthorized tool invocation, cross-tenant retrieval, output misuse, resource exhaustion, model or dependency tampering, and failure behavior.
  • Keep adversarial and regression tests in the release process, including tests for previously discovered issues. Re-run relevant cases when a model, provider, tool, MCP server, data source, or user population changes materially.
  • Use an independent AI security assessment, red team, or penetration test when the potential impact and threat model justify it. OWASP identifies AISVS as a framework for these activities and for audits.

OWASP’s LLM risk descriptions help widen the test plan beyond prompt injection. The 2025 workstream labels exposed on the OWASP initiative page include sensitive information disclosure, supply chain vulnerabilities, data or model poisoning, improper output handling, excessive agency, system prompt leakage, vector or embedding weaknesses, misinformation, and unbounded consumption. Do not treat those labels as the exact taxonomy or ranking of the 2026 edition.

7. Monitor the service and prepare for incidents

Controls can fail or become insufficient after launch. Assign an owner to review signals, investigate anomalies, and coordinate containment.

  • Monitor availability, unusual usage, authorization failures, anomalous tool calls, model or retrieval changes, cost spikes, and behavior drift. Set triage thresholds and name the person or team responsible for responding.
  • Log enough to investigate incidents, but minimize sensitive data in logs. Set retention, access, and redaction rules before production and restrict log access.
  • Prepare response steps for exposed keys, prompt-injection-driven actions, sensitive-data disclosure, compromised models or dependencies, abuse-driven cost or availability incidents, and unintended agent actions.
  • Include credential revocation, tool disablement, tenant containment, notification decisions, and recovery in the incident process.
  • Reassess the system after provider or model changes, new tools or MCP servers, new data sources, changes to the user population, material incidents, or changes in applicable law and contractual requirements.

What this checklist can—and cannot—establish

This checklist is a way to structure implementation and verification, not proof that a system is secure, compliant, or immune to attacks. The frameworks discussed here do not establish compliance with a particular contract or jurisdiction. Select controls for the system’s actual exposure and impact, test them, and revisit the decisions as the system changes.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.