iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Audit an AI employment tool as an employment-selection procedure, not as a neutral piece of software. Map exactly how the tool influences a hiring or promotion decision, test outcomes for relevant groups, establish job-related validity, measure decision-specific errors, verify disability access and accommodation, and monitor the system after launch. A favorable impact ratio or a vendor’s “fairness” report is only one piece of evidence; it does not by itself establish accuracy, validity, accessibility, or legal compliance.
What an AI employment-tool audit must establish
Employment law still applies when software scores, filters, ranks, recommends, or otherwise assists with selection. A human clicking “approve” does not automatically remove the tool from scrutiny. The audit should let an employer answer five separate questions:
- What does the system do? Identify the inputs, transformations, scores or labels, rankings, thresholds, recommendations, human review, overrides, and final decision.
- Does it produce unequal outcomes? Compare selection or scoring outcomes for relevant groups, including intersectional groups when the data and methods support it.
- Does it measure the right thing? Establish that the procedure measures a skill or construct tied to success in the particular job and is validated for the actual purpose and deployment conditions.
- How does it fail? Define false exclusions, false positives, unstable scores, failures on realistic inputs, and other errors in terms of the decision being made.
- Can people use it equitably? Check disability access, reasonable-accommodation processes, privacy, transparency, and change controls.
These questions are related but not interchangeable. Statistical disparity does not prove that a model is inaccurate; a technically accurate model can still be inaccessible; and a completed New York City audit does not certify compliance with federal law.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 111. Scope the decision, population, and system
Start with a written system map. Name the job family, locations, hiring or promotion stage, affected applicants or employees, decision owner, vendor, software components, and model or configuration version. Record the date range and the population represented in every test.
#1 Best Overall
- Valued Carpenter Pencil Set: You will get 2 pcs solid carpenter pencils with 26 piece 2.8 mm refills, 1 replaceable sharpener, 1 plastic storage box.The complete carpenter pencils combination allows you to finish your work faster and more easily
- Deep Hole Marker Pencil: The deep-hole construction pencils adopts 45mm elongated tip design, which is more convenient to mark in the small hole or in other tight areas that other carpenter markers cannot reach
- Carpenter Pencils with Sharpener: The sharpener is screwed into the top of the work pencil, which won't get lost either. Built-in pencil sharpener that keep the lead with pointed and smooth to Improves line of sight in fine work
- Stronger Solid Lead: This work pencil is matched with a 2.8 mm thick lead , which is much thicker and stronger during the drawing process of construction work, it will not break or damage easily
- Marks on Various Surfaces: 3 colors solid construction pencil can marks on various surfaces,such as metal, plastic, wood, paper etc. Ideals for woodworkers, contractors, craftsmen, builders, merchants and masons
Trace the complete decision path
- List every input. Include application fields, resumes, assessments, interviews, work samples, video or audio, keystrokes, demographic fields, inferred features, and data imported from another system. Note whether each field is required, optional, or generated.
- Document transformations. Record preprocessing, parsing, feature extraction, normalization, missing-data handling, language settings, and any data used to train or calibrate the tool.
- Describe outputs. Capture scores, tags, classifications, recommendations, rankings, eligibility flags, and thresholds. NYC rules recognize simplified outputs such as a score, categorization, recommendation, or ranking.
- Identify human involvement. State who sees an output, what instructions they receive, whether they can override it, how often overrides occur, and whether the output substantially directs the final decision.
- Record the final action. Define what counts as selected, rejected, advanced, interviewed, hired, promoted, or placed on hold, and which denominator belongs with each outcome.
Do not assume that a tool is outside an automated-employment-decision definition because a recruiter reviews the result. Determine the tool’s actual role under the law that applies to the job and location.
Maintain a system inventory
| Item to record | Why it matters |
|---|---|
| Job, level, location, and decision point | Validity and legal coverage depend on the particular role and use. |
| Vendor, product, model/configuration version, and release dates | Results may not transfer after a material change. |
| Inputs, outputs, thresholds, and human overrides | Shows where selection effects and errors enter the workflow. |
| Candidate or employee population and time period | Defines the appropriate denominator and generalizability. |
| Data retention, access, and deletion rules | Supports privacy, security, and reproducible testing. |
2. Define the job-related purpose before testing fairness
Write a plain-language statement of what the tool is supposed to measure and why that ability is needed for the specific job. “Predicts high performers” is too vague. Specify the construct—for example, a defined technical skill, a validated work-sample capability, or knowledge required in the role—and the criterion used to evaluate job performance.
Demand evidence for the actual use
Request technical documentation, validation studies, test versions, data definitions, scoring rules, known limitations, subgroup analyses, and change history. Check whether evidence covers the same jobs, seniority, geography, language, applicant population, assessment format, and decision stage. A study of an unsupervised practice test does not automatically validate a production hiring screen.
EEOC guidance states that employers should validate selection procedures for the positions and purposes in which they are used. Vendor materials can inform that assessment, but the employer remains responsible for validity. Keep the employer’s conclusion, evidence, reviewer, date, and any limitations in the audit file.
Separate validity from fairness indicators
A tool may show similar pass rates across groups yet measure an irrelevant trait. Conversely, a job-related measure can yield different group rates that require an adverse-impact analysis and consideration of alternatives. Treat construct validity, criterion-related evidence, content relevance, and fairness outcomes as separate findings.
3. Analyze selection outcomes and disparate impact
Calculate selection rates or other decision outcomes by relevant groups using a documented population and denominator. Depending on the job and lawful data availability, groups may include race or ethnicity, sex, age, disability, and other protected characteristics. Examine intersectional groups when sample sizes and statistical methods make the result meaningful.
Rank #2
- Ergonomically Designed: Work in tight areas with a compact design that gets into tough spots
- Both tools are designed to fit into difficult to reach spaces
- Complete a wide range of applications with a high-performance motor that delivers up to 300 UWO, and an impact driver delivering up to 1,400 in-lbs of torque
- Both the DEWALT impact driver and drill driver feature integrated LED work lights with a convenient 20-second delay, ensuring enhanced visibility in dimly lit or challenging work areas
- One-Handed Loading - Keep one hand free with a 1/4 in. hex chuck that accepts 1 in. bit tips
Build an auditable outcome table
| Group | Applicants or cases | Selected or advanced | Selection rate | Comparison and uncertainty |
|---|---|---|---|---|
| Group A | Record count | Record count | Selected ÷ cases | Document reference group and confidence limits |
| Group B | Record count | Record count | Selected ÷ cases | Explain missing data and small-sample limits |
NYC’s rules describe an impact ratio based on selection or scoring rates. In general terms, divide the rate for one group by the rate for the comparison group selected in the analysis. State which group is the reference, how scoring was converted into a decision, the population and period, and how missing demographic information was handled. An impact ratio is a screening indicator—not proof that a tool is valid, accurate, or lawful.
Recommended Free Tools
Use statistical analysis, not visual impressions alone
Federal EEOC guidance says that determining disparate impact ordinarily requires statistical analysis. Choose methods appropriate to the sample size, outcome, and design; report uncertainty and avoid treating a noisy estimate as a definitive result. Explain the protected-group definitions, intersectional analyses, exclusions, missingness, privacy safeguards, and any suppression rules that prevent re-identification.
If a neutral procedure produces disparate impact, evaluate whether it is job-related and consistent with business necessity. Then investigate whether an equally effective alternative with less adverse impact is available. Preserve the options considered, effectiveness evidence, decision-maker, rationale, and implementation date.
4. Test accuracy, reliability, and failure modes
There is no single universal AI-hiring audit metric or all-purpose accuracy threshold established by the official guidance summarized here. Define an error in relation to the actual decision and the job criterion.
Define decision-specific errors
- False exclusion: a qualified person is screened out, ranked below the advancement threshold, or denied an opportunity because of the tool.
- False positive: the system advances or recommends a person who does not meet the job-related criterion.
- Unstable result: materially different scores or rankings arise from equivalent inputs, repeated attempts, language changes, or minor formatting changes.
- Input or modality failure: the tool cannot process an accessible format, accent, assistive-technology output, noisy environment, or other realistic condition.
- Criterion mismatch: the score does not track the defined job-performance measure for the target role and population.
Design a reproducible test
- Write the expected behavior and the error definition before looking at results.
- Assemble test cases that represent the actual role, applicant population, languages, accessibility modes, and realistic input quality. Record provenance and consent or lawful-use basis.
- Run the exact production model, configuration, threshold, and workflow. Record version identifiers, prompts or settings, random seeds where relevant, and human-review instructions.
- Compare outputs with the job-related criterion or adjudicated label. Report counts, rates, uncertainty, missingness, and subgroup results rather than a single headline percentage.
- Investigate materially different error patterns. Review individual cases for data defects, proxy variables, parsing failures, and inappropriate human reliance.
- Repeat testing after a material model, data, threshold, job, or population change, and define what triggers revalidation or suspension.
Reliability also matters: ask whether the procedure produces consistent results under comparable conditions and whether the result remains useful over time. A high correlation in one historical dataset does not establish performance for a changed role or current applicant pool.
5. Check disability access and accommodation
The ADA can be implicated when a test screens out people with disabilities, measures an impairment rather than the intended skill, or is administered without reasonable accommodation. EEOC and Department of Justice guidance identify accommodation processes, screening-out risk, and prohibited disability-related questions or medical examinations as central concerns.
Rank #3
- 【Great Compatibility】This Katerk 1/4 inch hex shank bit holder is specifically designed for 1/4 inch hex shank drill bits. It's compatible with most 1/4 fast hex handles, hex sockets, various electric screwdrivers, and handheld screwdrivers. The bit holder makes it a valuable addition for any handyman.
- 【Secure and Safe】Built with a secure backup nut design, each drill bit holder securely locks onto your bits, ensuring they stay firmly in place. Additionally, our bit holder incorporates a high-quality steel ball rolling design that holds up to several kilograms of weight, ensuring your various drill bits don't fall off.
- 【Easy One-Handed Operation】The bit holder for impact driver allows you to change bits single-handedly, simplifying your workflow. Its multi-color design further allows for quick identification of the drill bit you need.
- 【Compact and Convenient】Thanks to its compact size, this 1/4 inch bit holder is easy to carry around. The bit holder allows for easy attachment to various tools, making this a convenient addition to your construction accessories. The Katerk bit holder is cast from high-quality alloy material, promising a long product lifespan. Despite its rugged strength, the bit holder remains lightweight, making it portable.
- 【Cool Christmas Gift For Men Stocking Stuffers】 This screwdriver bit holder, driver bit holder, impact bit holder, can be given as a gift to your loved one, especially for anyone involved in construction or electrical work. It's a must-have for stocking stuffers for men and women, tools gifts for dad, tech gadgets for men, gifts for dad, gifts for him, gifts for husband, gifts for boyfriend, cool gadgets for men, and cool gifts for dad.
Audit the assessment experience
- Determine whether the task measures the intended job skill rather than vision, hearing, speech, motor speed, reading speed, or another impairment unless that ability is genuinely job-related.
- Test keyboard access, screen readers, captions, contrast, timing controls, alternative response modes, mobile and low-bandwidth conditions, and compatibility with assistive technology.
- Review video, speech, facial, biometric, timed, and game-like features for barriers unrelated to the job criterion.
- Provide a clear, confidential process for requesting reasonable accommodation before and during the assessment, with trained staff and documented response times.
- Ensure that accommodation requests and health information are collected only in legally permitted contexts and are segregated from scoring data where appropriate.
Test the accommodation workflow itself: who receives the request, what alternatives are offered, whether the score is recalibrated or waived, and how the decision is documented without penalizing the applicant.
6. Review transparency, privacy, and change controls
Ask the vendor and internal owners to explain data sources, retention, access, model purpose, known limitations, monitoring, and incident response in language a recruiter and an applicant can understand. Do not treat an opaque vendor report as a substitute for an employer’s assessment.
Required audit records
- System inventory, intended use, prohibited uses, and accountable owner.
- Model, rules, threshold, interface, and vendor-version history.
- Training, validation, and test-data provenance; demographic missingness; sample sizes; and uncertainty calculations.
- Selection outcomes, error analyses, accessibility tests, accommodation requests, complaints, overrides, and corrective actions.
- Approvals, exceptions, legal analysis, communications, and dates for every deployment decision.
Limit access to sensitive demographic and disability information, use privacy-preserving reporting, and define retention and deletion schedules. A result that cannot be reproduced from preserved inputs, code or configuration, and protocol is difficult to defend or correct.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →7. Establish governance and monitor after launch
Assign named owners across HR or talent acquisition, legal, procurement, data science, accessibility, information security, and the vendor. Require a change-review gate before altering the model, threshold, input source, job family, language, or human-review instructions.
Set monitoring triggers
- Selection-rate or scoring shifts by relevant group.
- Changes in false-exclusion, false-positive, or failure rates.
- Increased overrides, complaints, accommodation requests, or unexplained manual workarounds.
- New roles, geographies, languages, data sources, or candidate populations.
- Vendor releases, retraining, threshold changes, security incidents, or data-quality degradation.
Define thresholds that trigger investigation, temporary manual review, rollback, or suspension. Preserve the decision and corrective action. The voluntary NIST AI Risk Management Framework offers a lifecycle structure addressing validity, reliability, transparency, explainability, privacy, and fairness while managing harmful bias. It is a risk-management framework, not a hiring-law certification, and cannot by itself establish compliance.
8. Apply New York City Local Law 144 when it covers the use
New York City Local Law 144 applies to a covered automated employment decision tool used to screen candidates for employment or employees for promotion in the city, as defined by the law and rules. Coverage depends on the actual tool, decision, and use; confirm the current statutory and regulatory definitions before relying on an exemption.
Rank #4
- Long Nib and Deep Hole Marker: Our mechanical carpenter pencil with 45mm nib is designed for easy marking of deep holes or narrow areas. These construction pencils are the great choice for woodworking tools, construction tools, carpenter tools, contractor tools, wood carpentry tools and architect tools
- Extra Refills in 2 Colors for Versatile Marking: The construction mechanical pencil comes with 12 extra 2.8mm refills, including 6 red and 6 black refills. The black refill is suitable for light surfaces, while the red wax is perfect for dark surfaces. Our carpenter mechanical pencil makes sure that you'll have an ample supply for extended use
- Built-in Sharpener: Our construction pencil comes with a built-in sharpener to ensure the mechanical pencil tip is always sharp and ready for use. Never buy an extra pencil sharpener again. A great tool for any woodworker pencil, contractor pencils. The refill can easily be extended or retracted with a simple click of the pencils mechanical, allowing you to work more efficiently and accurately
- Portable Clip Design: Our deep hole construction pencil features a portable clip design, easy to carry and attach to your pocket or tool box, so that you can keep the carpenter pencils mechanical close at hand, making it a convenient tool to have on the go. Great gifts choice for carpenters
- Stronger Pencil Lead: The black refills are made of lead, sturdy and smooth. The red refills are made of wax, clear and light. These marking pencils are much thicker and stronger than normal pencils during the marking process of construction work, suitable for various surfaces, such as glasses, metal, boards, floors, walls, furniture, etc. The written marks can be easily wiped with a wet paper towel when needed
Before using a covered tool
- Ensure a bias audit has been completed within the one-year period before use.
- Make the required summary of the most recent audit and the tool’s distribution date publicly available before use.
- Provide required notices to candidates or employees. The New York City Department of Consumer and Worker Protection (DCWP) states that enforcement began July 5, 2023 and that notice is to be provided at least 10 business days before use.
- Retain the audit, summary, notices, methodology, and supporting records according to applicable requirements.
Confirm the exact publication, distribution-date, notice, and timing details in the current law, rules, DCWP FAQs, and agency guidance. A code publisher warns that its reproduced text may not reflect the latest changes, so use current official materials and obtain legal advice for a live deployment.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Local Law 144’s minimum audit requirements are a floor. They do not guarantee federal-law compliance, job-related validity, disability access, accurate predictions, or fair outcomes for every relevant group.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to compare an audit method or auditor
Use the same decision criteria when evaluating an internal review, independent auditor, or vendor-provided assessment:
| Comparison axis | Questions to ask |
|---|---|
| Job-related validity | What construct and job criterion are measured, and is evidence specific to the role and use? |
| Outcome measures | Which selection, scoring, advancement, and intersectional outcomes are analyzed, with what denominators? |
| Error and reliability | How are false exclusions, false positives, instability, and input failures defined and tested? |
| Disability access | Are modalities, assistive technologies, accommodations, and screening-out risks assessed? |
| Transparency | Are methods, data definitions, limitations, versions, and reproducibility documented? |
| Independence | What qualifications, conflicts of interest, access rights, and auditor responsibilities apply? |
| Scope | Does the work cover the actual jobs, locations, population, language, threshold, and deployment period? |
| Privacy | How are demographic, disability, applicant, and vendor-confidential data protected? |
| Monitoring | What post-launch checks and change-triggered reaudits are included? |
Common audit mistakes and corrective actions
Relying on one ratio
Problem: treating an impact ratio as a pass/fail verdict. Correction: combine outcome analysis with validity evidence, error testing, accessibility review, uncertainty, and legal analysis.
Accepting generic vendor validation
Problem: assuming a study of another role, population, or version transfers automatically. Correction: document the exact job, purpose, conditions, and version, and have the employer make and record its own validity determination.
Ignoring human use
Problem: analyzing the model while omitting recruiter instructions, overrides, or threshold decisions. Correction: audit the complete socio-technical workflow and test how people act on the output.
Best Value
- Milwaukee Ink all Fine Point Marker, Black, 4 Per Pack
- 4 per pack Features Clog Resistant Marker Tip Writes through Dusty, Wet and Oily Surfaces Durable Marker Tip for Writing on Concrete, OSB and Rough Surfaces
- Clog resistant tip writes on dusty, wet and oily surfaces and is optimized for rough surfaces such as OSB, cinderblock and concrete
- Hard hat clip- attaches for easy access
- Quick dry time with reduced smearing and marking
Testing only average performance
Problem: reporting overall accuracy while missing subgroup, intersectional, language, or disability failures. Correction: stratify where methodologically sound and explain sample limitations and privacy protections.
Auditing once
Problem: treating a prelaunch report as permanent evidence. Correction: monitor in use and recheck after material changes to the system, role, threshold, data, or population.
What the evidence does—and does not—show about scale
The EEOC reported 304 relevant discrimination charges in fiscal year 2007 involving employment testing and exclusions based on criminal-background checks, credit reports, and other selection procedures. That historical figure is not an AI-specific prevalence statistic and should not be used to estimate how often AI hiring tools are biased.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsEEOC Chair Charlotte A. Burrows warned, “New technologies should not become new ways to discriminate.” Assistant Attorney General Kristen Clarke similarly said, “Algorithmic tools should not stand as a barrier for people with disabilities seeking access to jobs.” The practical implication is to apply established selection, disability, validation, documentation, and accommodation duties to newer technology rather than waiting for a universal benchmark.
A defensible audit deliverable
Finish with a decision memo that identifies the tool and use, evidence reviewed, methods, populations, results, uncertainty, limitations, legal analyses, accessibility findings, corrective actions, owner, deadline, and approval. State one of three outcomes—approve with controls, pause pending remediation, or do not deploy—and list the conditions for changing that decision. Include a monitoring calendar and a trigger for revalidation. This record makes the reasoning reviewable by HR, counsel, auditors, affected workers, and regulators without pretending that any single metric answers every question.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
