What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Assess the specific AI system, configuration and intended use—not just the vendor’s assurances. Map the data it handles, review contractual and technical evidence, test the deployed setup, and document the risks you accept. The result should be a decision record with evidence, findings, accountable owners, approval conditions and a plan to monitor changes.

1. Define what you are assessing

Start by describing the business purpose and the decision at hand: purchase, pilot, production approval or remediation. Set the boundary around the precise service tier and configuration under review. A hosted assistant, model API, internally hosted model, retrieval-augmented application, fine-tuned model and tool-using agent can have very different data paths and authorities.

Record the model and version, service tier, deployment boundary, user groups, administrators, intended purpose, connected tools and data sources, and whether the system uses retrieval or fine-tuning. Identify affected people, data owners, the vendor and any known subprocessors. If a model version or configuration is not disclosed, record that as an evidence gap rather than assuming it is fixed or immaterial.

This context determines what controls matter. NIST’s AI Risk Management Framework (AI RMF) uses the Map function to establish contextual knowledge that informs later measurement and risk management. The framework is voluntary guidance, not a certification that a particular system is safe.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

2. Trace data through the complete system

Follow information from the point a user or system supplies it through processing, storage, retrieval, output and deletion. Include indirect flows: connected repositories, user feedback, operational logs, telemetry and support records. For each flow, record its source, purpose, category, destination, who can access it, how long it is retained, how deletion works and whether it crosses organizational or geographic boundaries.

  • Inputs: prompts, uploaded files, copied text, images and information passed by an integration.
  • Context and retrieval: indexed documents, repositories and other material the system can retrieve, including the permissions applied to each source.
  • Outputs and feedback: generated content, user ratings, corrections and any content submitted for evaluation or service improvement.
  • Operations: logs, telemetry, support records, backups and records created by connected tools.

Classify data in context. Personal, privileged, proprietary, regulated and sensitive information may require different treatment; a label alone does not establish that a use is appropriate. Ask the provider whether submitted content, outputs, feedback or logs are used for training, fine-tuning, evaluation or service improvement. Get the answer, its scope and the relevant contractual basis in writing, and identify available controls and exceptions.

NIST’s Generative AI Profile describes privacy risks that include leakage and unauthorized use, disclosure or de-anonymization of personally identifiable and sensitive information, including biometric, health and location data. Which information counts as sensitive depends on context. Applicable legal obligations also depend on jurisdiction and use; have counsel determine which requirements apply to the specific deployment.

3. Verify vendor and supply-chain evidence

Request current documentation that covers the product and configuration you intend to use. Review the contract alongside technical material: a general security statement does not answer whether a particular feature retains prompts, which subprocessors can access content or how an integration handles permissions.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Independent assurance: Review security attestations or standards-based reports, their period and systems in scope, reported exceptions, and any complementary customer responsibilities.
  • Architecture and data handling: Ask for relevant architecture or data-flow documentation, processing and storage locations, deletion behavior, and the identity of subprocessors with access to organizational content.
  • Operational security: Examine vulnerability reporting and remediation processes, incident response, and how the provider will notify and cooperate with your organization.
  • Supply-chain detail: Where relevant, request a software bill of materials (SBOM) and information about the components or model dependencies covered by it.
  • Contract and service terms: Check commitments for data use, retention, audit or evaluation rights, change notification, service levels, exit and deletion.

NIST AI 600-1, the Generative AI Profile, identifies procurement due diligence, SBOMs, service-level agreements and statements on standards for attestation engagements (SSAE reports) as possible third-party transparency and risk-management inputs. These are items to inspect, not guarantees. Ask what each report covers and what falls outside its scope. An attestation does not establish that every model behavior, retrieval path or application integration is safe.

NIST also recommends including intellectual-property, privacy and security risks in AI vendor assessments, and inventorying third parties with access to organizational content. Maintain an approved-provider list where appropriate, so users and buyers can distinguish reviewed services from unassessed ones.

4. Compare options on the same evidence

When evaluating multiple providers or deployment designs, apply the same questions to each and capture the evidence behind the answer. The comparison below is a practical due-diligence synthesis, not a scorecard prescribed by NIST or another framework.

Assessment area Evidence to compare
Data use and retention Uses of prompts, files, outputs, feedback and logs; training or improvement settings; retention periods and deletion terms.
Location and third parties Processing and storage locations, transfer arrangements, subprocessors and their access to content.
Identity and access boundaries Single sign-on and role controls where applicable; separation between users or tenants; permission handling for connected data.
Security evidence Attestation scope and period, exceptions, vulnerability response, incident processes, architecture information and relevant SBOM details.
AI-specific behavior Testing relevant to privacy and security risks in the intended configuration; identified model or version; known limitations.
Integrations and authority Connected data sources and tools, granted permissions, approval steps and logging for consequential actions.
Contract and operations Evaluation or audit rights, notice and cooperation terms, incident handling, service commitments, change notification, exit and deletion.

Compare actual terms and evidence rather than marketing labels. If a provider cannot answer a material question, record what is unknown, the potential consequence and whether the gap blocks approval or can be addressed with a compensating control.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

5. Test the configured system, not a generic model

Write down test goals, representative scenarios and data, environment, results and limitations. Match test rigor to the potential impact and your organization’s risk tolerance. A generic benchmark or demonstration does not show how your permissions, retrieval corpus, prompts, tools and operational settings behave together.

  • Check whether one user can retrieve another user’s information or content beyond their authorization.
  • Test whether sensitive details appear in outputs when they should not, including when requests are unusual or adversarial.
  • Verify that connected sources enforce their own permissions and that retrieval does not bypass them.
  • Try malformed or unexpected inputs and assess whether the system exposes information, behaves unpredictably or invokes an unsafe action.
  • For agents and tool-enabled systems, inspect granted permissions and test the consequences of tool calls, including whether users must approve consequential actions.

Use a current risk taxonomy, such as OWASP GenAI materials, to organize relevant technical coverage; a taxonomy helps identify test areas but is not proof that a deployment is secure. OWASP’s GenAI Security Project homepage lists the 2026 LLM Top 10 and Agent Control Standard. Its crosswalk dated September 1, 2026 describes mapping 51 GenAI vulnerabilities across four source lists to controls in 25 frameworks. That figure describes the crosswalk’s coverage, not the total number of possible vulnerabilities or incidents.

NIST cautions that pre-deployment evaluation may be inadequate or mismatched to a real deployment context. Strong performance in an anecdotal game or benchmark does not by itself establish validity or reliability in the business domain you intend to use.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

6. Make a documented risk decision

Turn the review into an explicit decision rather than a collection of documents. Record findings, supporting evidence, mitigations, accountable owners, residual risks and any conditions attached to approval. Define what would require a pause, rollback or escalation before users rely on the system.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

For each material risk, make clear whether it is:

  • addressed by a verified provider control;
  • reduced by an enterprise configuration or process;
  • accepted as residual risk by an authorized decision-maker; or
  • unresolved and a reason to defer or reject the use.

Specify incident responsibilities, including who contacts and coordinates with the provider and who determines whether legal notification duties apply. A provider’s notification process does not replace the enterprise’s own incident and legal decision-making.

7. Monitor changes after approval

Approval applies to the reviewed system and use, not indefinitely to every future version or integration. Assign an owner, keep the system in an AI inventory and schedule periodic review. Reassess when a model or service changes, a new data source or tool is added, the purpose or user population changes, a privacy or security incident occurs, or the provider or its subprocessors change.

NIST’s AI RMF Core organizes work under Govern, Map, Measure and Manage. Governance is cross-cutting across the lifecycle; the framework calls for accountability, an AI-system inventory, ongoing monitoring, periodic review and contingency processes for high-risk third-party failures or incidents. NIST states on its AI RMF page: “The AI RMF 1.0 is being revised as part of the White House AI Action Plan.” NIST published AI 600-1, its Generative AI Profile, on July 26, 2024, and reported a critical-infrastructure profile concept note on April 7, 2026. Check the current framework and product documentation when making an approval decision.

For model producers, system producers and acquirers, NIST SP 800-218A is an AI-focused community profile that augments the Secure Software Development Framework (SSDF) with practices for AI model development. It was finalized on July 26, 2024. Use it where development and acquisition practices are within the assessment scope; it does not replace review of the actual deployed system.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.