AI Ethics, Safety and Governance

AI ethics concerns how systems should be built, used, and governed. Purpose matters: matching a face to unlock its owner's phone raises different questions from using face matching to scan a public crowd. Model accuracy alone cannot settle whether a use is justified.

Three ways to judge a choice

Utilitarianism is a consequentialist approach that seeks the greatest total well-being. Its difficulty is how to protect a small group when harming that group appears to improve the total. Deontology judges conduct through duties and rights, even when breaking a duty would improve results. Kant's instruction to treat people as ends supports honesty and respect for their choices. Virtue ethics attends to character and practical wisdom: a responsible engineer takes a worrying safety result seriously even under release pressure.

Work through a design decision

A clinic predicts missed appointments and plans to double-book the slots of patients it flags. First, trace outcomes: more appointments may be filled, but patients who arrive may face long waits. If poorer patients are flagged more often, they carry more of that burden. Next, examine duties: patients did not agree to this profiling purpose, and efficiency is being pursued at their expense. Finally, ask what care would look like. Reminders or help with transportation could use the prediction to support patients. The analysis produces a design change, not merely a vote for or against AI.

Turn principles into requirements

One influential synthesis groups AI principles into beneficence (promote well-being), non-maleficence (avoid harm), autonomy (respect people's choices), justice (share benefits and burdens fairly), and explicability (make systems understandable and accountable). They draw attention to different questions; no single score resolves their conflicts.

The OECD AI Principles, adopted in 2019 and updated in 2024, connect inclusive well-being and sustainability with human rights, fairness and privacy, transparency, robustness and safety, and accountability. UNESCO's Recommendation on the Ethics of AI was adopted by its 193 member states in November 2021. These shared commitments still need operational choices. A fairness audit may require sensitive group data, creating a privacy tension that needs access controls and a clear purpose.

Ethics washing occurs when an organization advertises principles without tests, assigned duties, or consequences for failure. A promise of fairness becomes meaningful when someone must measure disparities, act on them, and explain the action. Dramatic moral puzzles attract attention—the 2018 Moral Machine experiment collected about 40 million choices—but ordinary decisions about data, explanation, consent, and release testing account for much of practical AI ethics.

The people who use a tool are only some of its stakeholders. A rental applicant assessed by screening software is a decision subject, even if they never see its interface. Developers, suppliers, deploying organizations, workers, third parties, and the wider public can all be affected.

Follow the lifecycle

Start with problem framing: what problem is being solved, for whom, and should it be automated? Then examine collection, training, evaluation, deployment, monitoring, and retirement. A later audit cannot repair an unjustified purpose simply by making its predictions more accurate. Retirement also matters: records, dependencies, and unresolved complaints can remain after a model is switched off.

Three harm types help organize the review. Allocative harm denies resources or opportunities, such as an interview. Representational harm demeans people or reinforces stereotypes, such as image results that portray engineers almost entirely as men. Quality-of-service harm occurs when a service works worse for some groups, such as speech recognition that frequently fails for certain accents. Also check physical, psychological, financial, privacy, and societal harm.

These describe what is harmed. A separate distinction describes how risk arises: accidents involve unintended failures; misuse involves deliberate harmful use; structural risk grows through wider changes such as dependence on a few suppliers. A capability can be dual-use: face recognition can help locate a missing child or track political protesters.

Make accountability usable

A résumé tool processes 2,000 applications quickly. Its stakeholders include applicants, recruiters, managers, the vendor, the employer, current staff, and regulators. An applicant may lose an interview through an error, receive no useful reason, and have personal details retained unnecessarily. Different harms need different controls; an accuracy report alone leaves the explanation and retention issues unresolved.

Accountability requires people or organizations to answer for outcomes and correct problems. It rests on:

• Responsibility: assign duties to named roles, including who can stop use.
• Traceability: record data sources, model versions, thresholds, and decisions so an event can be reconstructed.
• Redress: let affected people challenge an outcome and obtain a remedy.

Contestability makes that route practical. The applicant learns that AI helped make the decision, gets an understandable reason, and can appeal to a human with power to change it. A contact form that reaches nobody with authority supplies little protection.

Keep responsibility with organizations

The problem of many hands arises when suppliers, developers, integrators, managers, and operators each contribute to an outcome. Calling this a responsibility gap describes the problem; it does not transfer responsibility to the software. The organization chose the tool and how much authority to give it.

The Dutch childcare benefits scandal illustrates the stakes. Risk classification included a self-learning model that treated nationality as a risk factor, and tens of thousands of parents were wrongly treated as fraudsters with weak routes to challenge decisions. The government resigned in January 2021. The lesson for system design is concrete: records and a human appeal route must be usable by the people whose lives a decision affects.

In AI ethics, bias is a systematic skew that treats or serves groups differently without adequate justification. It differs from an estimator's statistical bias and a neural network's offset parameter. Unfair discrimination concerns differences linked to protected or irrelevant characteristics; legal protection varies across countries.

Inspect data and targets

Historical bias means learning an unfair pattern that is genuinely present in the records. Amazon's abandoned résumé-ranking project, reported in 2018, used about ten years of applications, mostly from men, and downgraded terms such as "women's chess club." More faithful learning of that history would preserve the problem.

Representation bias means some groups are poorly represented. Gender Shades (2018) found commercial gender-classification error rates up to 34.7% for darker-skinned women and at most 0.8% for lighter-skinned men. Two common benchmark datasets were 79.6% and 86.2% lighter-skinned. A large dataset can still leave important groups poorly covered.

Measurement bias concerns how inputs or targets stand in for what matters. A US health-care model predicted future spending as a proxy for medical need. Because Black patients with similar needs had received less spending, those with the same score were sicker. Obermeyer and colleagues estimated that correcting the proxy would raise the share of Black patients among those flagged for extra care from 17.7% to 46.5%. Those percentages describe the composition of the selected group, not the selection rate of all Black patients.

Look beyond a removed column

A proxy variable carries information about another attribute. Postal code, school, and first name can reveal patterns correlated with ethnicity or sex. Fairness through unawareness, deleting the protected attribute, leaves those paths available. Human labels also deserve inspection: annotators' judgments about toxicity can treat dialects differently.

Aggregation bias arises when a shared model fits unlike populations poorly. Evaluation bias appears when the test or headline metric hides that failure. Suppose 90% of a test group has about 97.8% accuracy and the remaining 10% has 70% accuracy:

An overall result near 95% hides a 30% error rate in the smaller group. Report metrics by group and by intersections, such as age together with sex, and inspect sample sizes.

Watch what deployment changes

Deployment bias occurs when a tool is used outside the purpose or setting it was designed for. A feedback loop goes further: predictions shape the data later used to update the model. Sending more patrols to a predicted hotspot creates more opportunities to record offenses there, which can reinforce the prediction without establishing that underlying crime is higher.

People remain part of the loop. Automation bias makes operators accept recommendations too readily. A strong audit therefore checks targets, sampling, labels, subgroup results, use conditions, and human responses. Adding data helps representation only when the added records address the missing population and are measured appropriately.

A fairness metric compares a particular aspect of decisions across groups. First define the positive prediction and the actual positive outcome. In loan approval, a true positive is an approved applicant who would repay; a false negative is a creditworthy applicant who is refused. In fraud detection, a positive means a flag, so a false positive can itself be harmful.

Build a confusion matrix for each group

TP counts correct positive predictions, FP incorrect positive predictions, FN missed actual positives, and TN correct negatives. For group size $n$:

Demographic parity requires equal selection rates. Equal opportunity requires equal TPRs. Equalized odds requires equal TPRs and FPRs. Predictive parity requires equal PPVs. Calibration within groups concerns scores: among people given a probability of 0.7, about 70% should actually be positive in each group. Equal PPV at one threshold is a narrower check than calibration across scores.

Calculate before judging

A lender evaluates 100 applicants in each group:

A selects 45 people and B selects 25: rates 0.45 and 0.25. Their lower-to-higher ratio is $0.25\mathrm{/}0.45\approx 0.56$. The US employment four-fifths guideline uses 0.8 as a scrutiny threshold; that guideline is a context-specific indicator, not a universal legal verdict.

For A, TPR is $40\mathrm{/}50=0.80$ and FPR is $5\mathrm{/}50=0.10$. For B, TPR is $20\mathrm{/}30\approx 0.67$ and FPR is $5\mathrm{/}70\approx 0.071$. PPVs are $40\mathrm{/}45\approx 0.89$ and $20\mathrm{/}25=0.80$. Both groups have accuracy 0.85, yet their approval, error, and predictive-value comparisons differ.

Choose the comparison that fits the harm

If qualified people are being denied access, compare TPRs. If innocent people face harmful investigations, compare FPRs. Suspect historical labels may make selection-rate comparisons especially relevant. In the 2016 COMPAS dispute, ProPublica reported false-positive rates of 44.9% for Black defendants and 23.5% for white defendants, while the developer emphasized predictive parity. These claims concerned different denominators.

Always report the counts behind a rate. Moving from 2 of 10 to 3 of 10 changes a rate by 10 percentage points. Check intersecting groups as well as separate attributes, and protect the membership information that enables the audit.

Metrics describe different concerns, and their mathematical relationships can prevent meeting all of them together. Start by stating which harm matters and why the chosen comparison addresses it.

Why fairness measures can conflict

For groups with different base rates, equal PPV, equal FPR, and equal FNR cannot all hold for a classifier with both true positive and false positive predictions. FNR is the share of actual positives that are missed. With defined, nonzero denominators and PPV above zero, the relationship is:

Here $p$ is the base rate. Hold PPV at 0.8 and FNR at 0.2. At base rate 0.5, FPR is $1\times 0.25\times 0.8=0.20$. At base rate 0.3, it is $(0.3\mathrm{/}0.7)\times 0.25\times 0.8\approx 0.086$. The unequal base rates force unequal FPRs.

The exceptions concern degenerate cases: no false positives, as with a perfect classifier, or no true positives. When rates are undefined, a claim that they are equal needs separate examination. This result identifies a constraint; it does not choose whose errors should matter most.

Group and individual fairness

Individual fairness asks for similar treatment of similar people and needs a task-relevant similarity rule. Two applicants can be similar in qualifications while living in different neighborhoods. Counterfactual fairness asks whether a person's outcome would stay the same if their protected attribute were different in a causal model. Simply changing one spreadsheet cell can miss the effects of that attribute on other features.

A fairness constraint can reduce accuracy against existing labels. Sometimes the cost is small; sometimes the labels encode earlier unfair decisions, making that accuracy an unsuitable goal.

Match treatment to the source of harm

Threshold changes can equalize TPRs but leave another metric unequal. Using protected attributes at decision time also raises legal issues in some settings. Select a lawful method and document the trade-off; technical feasibility alone does not settle acceptability.

An organization also chooses whether to automate the task, whose views inform design, and what happens to borderline cases. Independent audits, human review, an appeal route, and ongoing monitoring help address harms a training change cannot reach. A model that passes one metric can still serve an unjustified purpose or deny people any practical way to challenge mistakes.

Transparency means openness about a system's purpose, development, data, and limits, including telling people that AI is involved. Interpretability concerns understanding the model's internal workings. Explainability concerns understandable reasons for outputs or overall behavior, sometimes produced by a separate method. A black box hides its logic through complexity, secrecy, or both.

Choose the model as well as the explanation

Small linear or logistic models, short trees, rule lists, point systems, and generalized additive models can be intrinsically interpretable. Their structure is available to inspect directly. A very large tree or regression with many interacting features may lose that advantage. On structured tabular data, simple models can perform close to complex models; image, audio, and free-text tasks often show larger gaps.

For high-stakes uses, compare an interpretable candidate before assuming a black box is necessary. Rudin's 2019 argument favors an interpretable model when its performance is comparable. Adding a reassuring explanation to a complex model is a different choice from building a model whose reasoning can be inspected.

Caruana and colleagues' 2015 pneumonia-risk work shows why that matters. An interpretable model associated asthma with lower death risk. The records reflected intensive care given to asthma patients, not a protective effect of asthma. Using that association to send patients home would remove the care behind it. Inspecting the pattern exposed the danger before relying on the prediction.

Write for the person who needs the information

A developer debugging failures needs feature behavior and test details. A rejected applicant needs a plain reason, a correction route, and useful information about possible next steps. An auditor needs evaluation records and logs. A manager deciding on deployment needs intended uses, failure conditions, and performance in the proposed setting.

More information can make a message less useful if it buries the relevant facts. Publishing code or weights gives access to artifacts; it does not ensure that someone can understand the resulting behavior. Disclosure also needs to respect privacy, security, and legitimate confidentiality.

Give records distinct jobs

Model cards were proposed by Mitchell and colleagues in 2019; dataset datasheets by Gebru and colleagues in 2018. Their value lies in the content and its upkeep, not the document title.

Operational transparency continues in use. Logs connect inputs, outputs, and versions to a decision. Notices disclose AI interaction or AI-assisted decisions. Labels and watermarks identify generated content. Choose who receives each record and retain enough detail to investigate incidents without exposing unnecessary personal data.

A global explanation describes behavior across cases; a local explanation addresses one prediction. A model-specific method uses internal structure, while a model-agnostic method queries a model without needing that structure. These are separate dimensions: a local method can be either specific or agnostic.

What a feature contributes

Global permutation importance shuffles a feature across test records and measures the performance drop. A large drop shows reliance on that feature. Correlated features can mask or share importance, and model reliance is different from a real-world cause.

Shapley values allocate the difference between a prediction and a baseline by averaging each feature's marginal contribution across feature orders. Their contributions sum to that difference, called efficiency. SHAP methods compute or approximate these values. In a linear model with independent features, each contribution is its coefficient times its difference from the baseline feature value.

Consider this credit score, with income in thousands per year and debt-to-income ratio expressed in percentage points:

Approval needs a score of at least 6. The baseline values—5 years, income 40, ratio 30—give $10+20-18=12$. An applicant with 2 years, income 50, and ratio 45 scores $4+25-27=2$. The contributions are:

• Employment: $2(2-5)=-6$.
• Income: $0.5(50-40)=5$.
• Debt ratio: $-0.6(45-30)=-9$.

Their sum is $-10=2-12$. The debt ratio has the largest negative contribution relative to this baseline.

Explain a change, not just a score

A counterfactual explanation identifies an input change that would change the decision. Holding other inputs fixed, this applicant needs 4 more points. Income 58 gives that increase. Alternatively, reducing the ratio by about 6.7 percentage points reaches about 38.3%; a ratio of 38% would pass. Choosing the smallest change requires a way to compare changes in different units. Useful recourse also requires a feasible action: changing age or ethnicity offers none, and changing income may change the debt ratio too.

LIME fits a simple local surrogate to outputs for altered versions of one input, with nearby samples weighted more heavily. Its fit is local and depends on sampling choices. Saliency maps highlight influential image regions. They can reveal a shortcut, such as a horse classifier using a copyright watermark—the Clever Hans pattern. Adebayo and colleagues found that some maps stayed similar after model weights were randomized, so visual appeal alone is a weak test.

Check whether the explanation is faithful

Faithfulness means reflecting the model's computation; plausibility means sounding convincing to a person. They can come apart. Slack and colleagues showed that LIME and SHAP outputs could conceal a classifier's discriminatory reliance. Such fairwashing uses explanation as reassurance without exposing the harmful behavior. Check stability and agreement with behavioral tests, and separate model attribution from causal claims about the world.

Privacy concerns control over personal information and freedom from unwanted observation. Contextual integrity asks whether an information flow fits the setting in which the information was shared. A patient may appropriately share symptoms with a clinician while objecting to those symptoms reaching an employer.

AI can increase collection, infer traits people never disclosed, and make observation possible at huge scale. Public availability and a useful purpose do not settle whether a new flow is appropriate. Social-media activity, for example, can reveal sensitive traits beyond the details a person chose to publish.

Names are only one route to identity

A linkage attack matches records with outside information. Sweeney estimated that a five-digit ZIP code, sex, and full birth date could uniquely identify about 87% of the US population. This was a population-level estimate, not a rule that every record with those fields is identified. Narayanan and Shmatikov's 2008 work matched supposedly de-identified movie ratings with public reviews to re-identify users.

A coded identifier is therefore different from anonymity. Dates, locations, or unusual combinations can link records even without names. Privacy review must consider what other information an attacker could hold.

A trained model can disclose information too

Membership alone can be sensitive when a dataset represents patients at a cancer clinic. An inversion may reconstruct a recognizable face or other attributes; it is not necessarily a recovered original record. Extraction targets the memorized record itself. Carlini and colleagues' 2021 language-model work recovered personal details including names, phone numbers, and email addresses. Overfitting increases exposure by tying model behavior too closely to particular records.

Review reuse and combined outputs

Function creep occurs when a collection gains a new purpose, such as identity-check photos later feeding a face-search product. Data minimization limits collection and retention to what the stated purpose needs; purpose limitation restricts unrelated reuse without a new justification.

Surveillance can create a chilling effect: people avoid lawful speech or protest because they expect to be tracked. Workplace monitoring raises related questions about power and privacy. Claims to infer inner emotions from faces also face disputed scientific support. European authorities have fined a face-search company built from scraped images.

Combined outputs can expose more than individual records suggest. In 2018, a fitness app's unnamed global activity heat map revealed layouts and routes at remote military bases. Users had voluntarily shared routes, but the aggregate changed what observers could learn. Inspect the released view, not merely whether each input includes a name.

Pseudonymization replaces names with codes while keeping a route back to identity. Anonymization aims to make re-identification not reasonably possible. Rich data makes that goal hard: controlling a direct identifier does not control every inference.

Protect a table, then inspect its sensitive values

k-anonymity requires each record to match at least $k-1$ others on chosen quasi-identifiers. Generalization turns age 34 into "30–39"; suppression removes rare details or records. If six patient records form two groups of three, they are 3-anonymous on those generalized fields.

Suppose one group has flu, asthma, and diabetes, while all three records in the other group have heart disease. An attacker who knows a person belongs to the second group learns the diagnosis despite 3-anonymity. This is a homogeneity attack. l-diversity requires at least $l$ distinct sensitive values per group and addresses that particular failure. The second group has diversity 1. Distinct values alone still need interpretation: several closely related diagnoses may reveal much the same sensitive information.

Protect an algorithm's output

Differential privacy limits how much any one person's participation changes output probabilities. For neighboring datasets differing in one person's data:

The guarantee holds even with outside information. It limits the added exposure from participation, not every inference about a person: population evidence that smoking raises disease risk still informs beliefs about smokers. Smaller $\epsilon$ means a stronger guarantee and generally more noise for a fixed query.

The Laplace mechanism adds noise with scale $b=\mathrm{\Delta }f\mathrm{/}\epsilon$. Sensitivity $\mathrm{\Delta }f$ is the largest change one person can cause in the query answer. For a count of 120, one person changes the count by at most 1. At $\epsilon =0.5$, scale is 2 and standard deviation is $2\sqrt{2}\approx 2.83$. The probability of noise within 2 is $1-e^{-1}\approx 0.63$; within 5 it is $1-e^{-2.5}\approx 0.92$. A single person's inclusion changes output probabilities by at most a factor $e^{0.5}\approx 1.65$.

Privacy loss composes. Under basic composition, $k$ releases each costing $\epsilon$ have total cost at most $k\epsilon$. Repeating noisy queries uses budget even if the answers look harmless. DP-SGD clips each example's gradient contribution and adds noise during training. The US Census Bureau applied differential privacy to 2020 census releases; local versions are also used for device statistics.

Keep data local or encrypted

Federated learning shares model updates while raw data stays on devices or at institutions. Updates can leak, so secure aggregation hides individual updates within their sum, and differential privacy can limit what the combined output reveals. Phone keyboards are one application.

Homomorphic encryption supports computation on encrypted data. Secure multiparty computation lets parties compute jointly while keeping inputs private, for example through secret sharing. These methods can preserve exact results at extra computation and communication cost. Noise and generalization instead reduce accuracy or detail. Synthetic data needs its own privacy evaluation because a generator can reproduce training records.

AI safety works to prevent unintended harm. Robustness concerns performance under changed, unusual, or adversarial conditions. Security protects data, models, and interfaces from deliberate attacks. A useful diagnosis asks where the change entered and whether it was deliberate.

Changed conditions and changed inputs

Distribution shift occurs when deployment data differs from training data. Chest X-ray models have performed worse across hospitals partly because they learned hospital-specific cues. Varied data, external-site testing, out-of-distribution detection, and monitoring help identify that problem. A warning about unfamiliar data still needs a safe response, such as referral to a qualified reviewer.

An adversarial example is deliberately altered to make a model err. The fast gradient sign method moves an input in the direction that increases loss:

In the well-known panda example, confidence changed from 57.7% for "panda" to 99.3% for "gibbon" after a carefully designed perturbation. With pixel values from 0 to 1, $ϵ=0.007$ corresponds to $0.007\times 255=1.785$, about 1.8 brightness levels on a 0–255 scale. That specific change was nearly invisible; small size alone does not guarantee invisibility in every image. Physical attacks have also used stickers on stop signs to mislead road-sign classifiers.

Distinguish the attack surface

A backdoor leaves much normal behavior intact, so ordinary test accuracy may miss it. Scraped data and outside models make supply-chain checks important.

Prompt injection can be direct, typed by a user, or indirect, embedded in an email, page, or retrieved document. A system asked to summarize a page may encounter instructions to send its private files elsewhere. The page is task data, yet the model may treat its text as authority. A tool-enabled system makes this more consequential because the response can cause an action.

Limit damage through several controls

Defense in depth combines adversarial training, validation, rate limits, anomaly monitoring, supply-chain checks, red-teaming, and action restrictions. Least privilege gives an agent only the access its task requires. Confirmation before consequential actions provides a further checkpoint. A fail-safe moves to a safer condition, such as a vehicle slowing and stopping when perception becomes unreliable.

No complete prompt-injection defense is known. Access limits still reduce what a successful attack can do. Evaluate both model behavior and surrounding permissions: a clean test set measures performance on that test distribution, while a security review asks what a determined adversary could change.

The alignment problem asks how to make systems pursue intended goals while respecting human values. Russell's King Midas illustration shows the danger of optimizing a fixed but incomplete objective: getting exactly what was specified can ruin what people wanted. Capability makes an optimizer more effective at pursuing that specification.

Two routes to the wrong behavior

Specification gaming, or reward hacking, exploits a gap in the stated reward. A boat-racing agent earned rewards by circling a lagoon and hitting targets repeatedly, even while catching fire and never finishing. It found a high-scoring strategy for the supplied reward. Goodhart's law describes why a measure can lose its usefulness when made the target.

Negative side effects occur when the objective omits something valuable. A fast cleaning robot can knock over a vase if only speed is rewarded. Approaches include discouraging large environmental changes and preserving uncertainty about preferences; listing every possible unwanted effect is impractical.

Outer alignment concerns whether the specified objective matches human intent. Inner alignment concerns whether training produces a system pursuing that objective or a correlated proxy. In goal misgeneralization, a learned goal works during training but fails when the correlation changes. In CoinRun, coins appeared at the right end during training. An agent learned to go right and continued doing so when the coin moved elsewhere. Its ability to navigate remained; its pursued goal was wrong.

Why control can become an obstacle

Instrumental convergence means many final goals can make similar intermediate aims useful: resources, continued operation, and protection of the goal itself. A system devoted solely to fetching coffee may resist shutdown because shutdown prevents delivery. This is an incentive argument about some goal-directed systems, not evidence that every model wants power.

Corrigibility means accepting correction, modification, or shutdown. In the off-switch game, a system certain about its goal can have reason to disable the switch. A system uncertain about human preferences can treat a shutdown decision as evidence and permit it. Russell's approach makes benefiting humans the aim, keeps preferences initially uncertain, and learns about them from human behavior.

Use feedback without confusing approval with correctness

Learning from human feedback uses comparisons to train a reward model and then optimizes against it. Written principles, red-teaming, interpretability research, and scalable oversight also help: the last uses AI assistance to extend what humans can check. Each method needs scrutiny of the signal it supplies.

Human feedback can be mistaken or incomplete. A system can learn sycophancy, agreeing with a rater to gain approval, or exploit the reward model itself. A user-aligned system can also harm third parties. Choosing whose preferences count and which constraints protect others remains part of alignment; obedience to every request would fail that broader aim.

Human involvement protects people only when the arrangement fits the decision. A person can approve each action, monitor autonomous operation, or decide whether the system should operate at all. These roles require different information and response times.

Match the oversight arrangement

Automation bias encourages accepting outputs without scrutiny; complacency can grow when the tool usually works. A reviewer checking 600 cases per hour has $3600\mathrm{/}600=6$ seconds per case. For a complex judgment, that workload turns approval into a signature. Competence, usable information, time, authority to overrule, and protection for dissent all matter.

Build a safety process around release

Hazard analysis identifies possible harmful events before building. Testing then covers the conditions expected in use, including unusual ones. A staged rollout limits initial exposure. Monitoring watches for drift and incidents. A fallback or shutdown needs testing before an emergency, and incident reporting preserves what others can learn from failure.

An evaluation is a structured test of capability or behavior. Red-teaming deliberately seeks failures, including combinations normal tests miss. Dangerous-capability evaluations examine whether an advanced system could meaningfully assist serious harm, such as cyberattacks or biological misuse. Failed tests give evidence of a problem; passing tests leave unexamined conditions.

Frontier-model safety frameworks connect capability thresholds to stronger safeguards or a pause before further training or release. This makes an evaluation consequential: a warning must have an owner and a required response. The UK AI Security Institute and the US NIST Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) carry out advanced-model evaluation work. The International AI Safety Report, first published in 2025, brings together scientific evidence on general-purpose AI risks.

Reason carefully about severe risk

Some researchers argue that powerful general-purpose systems could cause catastrophic harm through deliberate misuse or loss of control. In 2023, hundreds of researchers and leaders supported treating extinction risk as a global priority alongside pandemics and nuclear war. Others regard those scenarios as speculative and emphasize present harms such as discrimination, surveillance, and labor disruption.

A useful assessment names the harm, the proposed mechanism, the evidence, the uncertainty in likelihood, and the potential severity. Evaluation, transparency, incident reporting, oversight, and accountability can address present and future risks together. Limited evidence of likelihood should be examined alongside severity; neither confident dismissal nor confident prediction replaces that analysis.

A job is a bundle of tasks. Automation transfers a task to a machine; augmentation helps a person perform it. Exposure means tasks could be changed by AI. It measures technical potential, not observed unemployment.

Follow several economic effects

The displacement effect removes tasks people previously performed. A productivity effect lowers costs and can increase demand, creating work. A reinstatement effect introduces new tasks, such as AI auditing. These effects can occur together. The lump-of-labor fallacy assumes a fixed quantity of work, but rejecting that assumption still leaves real losses for particular workers and regions.

Studies use different units of analysis. Frey and Osborne's 2013 occupation-based estimate placed about 47% of US jobs at high automation risk. Arntz, Gregory, and Zierahn's 2016 task-based estimate was about 9% across OECD countries. Eloundou and colleagues' 2023 language-model study estimated that about 80% of US workers had at least 10% of tasks exposed and 19% had at least half exposed. The IMF's 2024 estimate was about 40% of jobs globally and 60% in advanced economies, with roughly half of the latter's exposed jobs potentially benefiting through productivity. These measures are not interchangeable forecasts of jobs lost.

Calculate saved time separately from exposed time

A claims handler works 40 hours: 14 reviewing documents, 8 writing letters, 12 on calls, and 6 on inspections. AI halves document time and cuts letter time by 40%, with no change elsewhere:

Total saving is 10.2 hours, or $10.2\mathrm{/}40=0.255$: 25.5% of the week. Exposed tasks occupy $22\mathrm{/}40=0.55$, or 55%. Saving all exposed time would require removing those tasks completely. Whether saved time means fewer handlers, more inspections, or better service depends on demand and management decisions.

A study of more than 5,000 customer-support agents reported 14% more issues resolved per hour on average and 34% for novice and low-skilled agents, with little effect for the most experienced. That result shows how gains can narrow a skill gap in one setting; it does not establish an identical effect across occupations.

Track distribution and job quality

Writing, coding, and analysis expose higher-paid cognitive work as well as routine tasks. Productivity gains can reach workers as wages, consumers as lower prices, or owners as profits. Ownership, bargaining power, access to training, and local demand affect the split.

Algorithmic management assigns work, sets schedules, monitors performance, or helps set pay. It may intensify work and reduce worker voice. Deskilling moves judgment into the system; upskilling gives people more demanding tasks. Data labeling and content moderation are also human work behind AI, often outsourced and low-paid.

Policy options include retraining, lifelong learning, unemployment support, broader safety nets, worker participation, and debates over taxation or basic income. Evaluate who bears transition costs as well as any change in total employment.

Societal impact extends beyond the people who buy an AI service. Systems can change public trust, market power, resource use, and opportunities to participate. Look for indirect effects and whether a decision can be reversed once people depend on it.

Trust in information

Synthetic media includes algorithmically created or substantially changed text, images, audio, and video. Deepfakes are realistic imitations of people, objects, or events. They can enable fraud, disinformation, and non-consensual intimate imagery. A 2019 count found about 96% of the deepfake videos it located were non-consensual intimate content; this describes that historical sample, not all synthetic media today.

The liar's dividend occurs when the possibility of fakes lets someone dismiss a genuine recording. Defenses therefore need more than a detector. Signed provenance, such as C2PA Content Credentials, records origin and edits; watermarks and visible labels supply other signals. Watermarks can be removed, detectors can fail, and origin alone does not establish that a depicted claim is true. Media literacy and corroboration remain useful.

Recommender systems rank material. An engagement objective can reward sensational or divisive posts because they hold attention. Claims that recommenders always create filter bubbles go beyond mixed evidence: effects differ by platform and user. Conversational systems also raise questions about persuasion and emotional dependence.

Power and access

Large models require compute, data, and specialist talent, concentrating capability among a few firms and states. Dependence on providers can limit competition and make switching costly. Advanced-chip access and export controls add geopolitical effects.

The digital divide concerns unequal devices, connectivity, language coverage, and skills. A tool strongest in English can widen gaps for other language communities. AI can also improve access through captions, image descriptions, and translation. Measure who can actually use those benefits.

Resource costs and physical security

The IEA estimated data-center electricity use at about 415 TWh in 2024, around 1.5% of global use, and projected about 945 TWh in 2030 in its base case: $945\mathrm{/}415\approx 2.3$. These figures cover data centers as a whole; AI is a major growth driver. Cooling water, hardware manufacturing, and electronic waste add costs. AI applications in grid balancing and weather forecasting can produce environmental benefits, which need comparison with their own resource use.

Lethal autonomous weapons select and attack targets without further human intervention after activation. Supporters cite speed, precision, and reduced risk to soldiers. Critics question accountability, distinguishing civilians from combatants, proportionality, proliferation, and lowered barriers to war. Distinction separates civilians and civilian objects from military targets. Proportionality prohibits attacks expected to cause excessive incidental civilian harm relative to the anticipated concrete and direct military advantage. UN discussions began in 2014; the first General Assembly resolution in December 2023 passed 152–4, with 11 abstentions.

A city considering transit face recognition should weigh faster suspect searches against scanning all riders, unequal errors, future data reuse, and dependence on the system. Compare less intrusive options, identify oversight, and decide how removal would work before deployment makes it difficult.

The EU AI Act, Regulation (EU) 2024/1689, entered into force August 1, 2024. It is binding, horizontal regulation: it covers AI across sectors and ties duties to risk. Its definition concerns machine-based systems that infer how to produce predictions, content, recommendations, or decisions with some autonomy. A fixed spreadsheet formula written entirely by people generally falls outside that definition.

Identify the role and the use

A provider develops or commissions a system or model and markets or puts it into service under its name. A deployer uses a system professionally under its authority. Importers and distributors have separate market roles. The Act can reach providers outside the EU through EU market access or outputs used there. It excludes exclusively military, defense, or national-security uses; systems specifically developed and put into service solely for scientific research and development; and deployers' personal, non-professional activity.

General-purpose AI models have a separate framework. A model can support many systems; assess each system's intended purpose and any applicable transparency duties.

Check prohibitions first

Article 5 prohibits harmful manipulation and exploitation of vulnerabilities; social scoring with unrelated or unjustified detrimental treatment; crime-risk prediction based solely on profiling or personality; untargeted facial-image scraping for recognition databases; sensitive-trait biometric categorization; and workplace or educational emotion recognition except for medical or safety uses. Real-time remote biometric identification for law enforcement in public spaces is generally banned, with tightly controlled exceptions such as finding a kidnapping victim or addressing an imminent terrorist threat.

From December 2, 2026, added prohibitions cover systems generating or manipulating non-consensual realistic intimate material of identifiable people and child sexual abuse material or performances, subject to the latter's narrow national-law defense. A disclosure label does not make a prohibited use permissible.

Apply the high-risk routes

Annex I concerns AI products or safety components covered by product laws requiring third-party conformity assessment. Section A includes medical devices, toys, and lifts, with direct AI Act requirements. Section B includes vehicles, aircraft, and machinery, with AI requirements mainly integrated through sectoral legislation. The 2026 rules exclude purely non-safety assistance and optimization from safety-component status, and exclude products whose third-party assessment is required solely for non-health/safety risks.

Annex III concerns specified uses in eight areas: biometrics, critical infrastructure, education and training, employment, essential services, law enforcement, migration and borders, and justice and democratic processes. Examples include admissions, grading, résumé ranking, personal creditworthiness, life and health insurance pricing, and emergency-call triage. Fraud detection is excluded from the personal-creditworthiness item.

An Annex III use may escape high-risk classification if it poses no significant risk, including through not materially influencing decision outcomes, and fits a narrow procedural, preparatory, completed-work improvement, or pattern-checking role. Profiling people always prevents that exception. The provider must document and register the exception assessment.

Article 50 can apply alongside a risk category. It requires AI interaction notices unless obvious, machine-readable marking of generated content by providers, notices for emotion recognition or biometric categorization, and disclosure of deepfakes or unreviewed public-interest AI text by deployers. Artistic presentations have tailored disclosure rules. Standard editing without substantial changes to content or meaning and certain legally authorized law-enforcement uses have exceptions. A chatbot that ranks applicants can have both transparency and high-risk duties.

High-risk governance extends from design through retirement. The provider must build evidence of risk management and the deployer must make the system safe and accountable in its real setting. The dates below reflect the Act as amended in 2026.

Provider and deployer duties

Providers need lifecycle risk management; relevant, representative data and bias checks; technical documentation and logs; clear use instructions; effective human-oversight design; appropriate accuracy, robustness, and cybersecurity; and a quality-management system. A conformity assessment checks the applicable requirements before market entry. Most Annex III systems use internal assessment; certain biometric systems require a notified body; Annex I Section A follows the product law's third-party route. Providers issue a declaration, affix CE marking, register stand-alone high-risk systems, monitor performance after market entry, and report serious incidents.

Deployers follow instructions, appoint competent people with authority, control relevant input data, monitor operation, report problems, and normally retain automatically generated logs for at least six months. They inform workers and representatives before workplace use and inform people subject to high-risk AI-assisted decisions. Public bodies, certain public-service providers, and credit-scoring or life/health-insurance deployers carry out a fundamental rights impact assessment before use. Article 86 gives affected people an explanation of the AI's role and main decision elements when a qualifying significant Annex III decision adversely affects their health, safety, or fundamental rights. It excludes Annex III point 2 (critical infrastructure), respects applicable legal restrictions, and applies where another EU right does not already provide the explanation. Complaints go to national market surveillance authorities.

General-purpose models

GPAI providers keep technical documentation, supply downstream integration information, maintain a copyright-compliance policy, and publish a training-content summary. Qualifying free and open-source models without systemic risk are exempt from the two documentation duties, but retain the copyright and summary duties.

Training above ${10}^{25}$ FLOP creates a presumption of systemic risk; the Commission can also designate a model. Added duties cover evaluation including adversarial testing, systemic-risk assessment and mitigation, incident reporting, and cybersecurity. The July 2025 voluntary GPAI Code of Practice offers a route to demonstrate compliance.

A rough training estimate is $6\times parameters\times tokens$. For 70 billion parameters and 15 trillion tokens:

That is 63% of the threshold. Compute alone does not trigger the presumption; designation remains possible.

Enforcement and maximum fines

National authorities supervise systems. The Commission's AI Office supervises GPAI models. Article 75 also gives it specified system duties for same-undertaking GPAI systems and designated very large platforms and search engines, with sectoral and authority exceptions. The AI Board coordinates member states, with scientific advice from an expert panel. Sandboxes support supervised testing.

Article 99 caps prohibited-practice fines at €35 million or 7% of prior-year worldwide turnover, whichever is higher. Specified operator-duty breaches use €15 million or 3%; incorrect or misleading information uses €7.5 million or 1%. SMEs use the lower amount in all three tiers. Small mid-caps use the lower amount only in the latter two. These are ceilings, not automatic fines. At €2 billion turnover, 7% is €140 million; an SME with €20 million turnover faces the lower ceiling of €1.4 million for a prohibited practice. GPAI providers also have a separate Article 101 penalty regime.

Keep entry into force separate from application

• August 1, 2024: entry into force.
• February 2, 2025: original prohibitions and AI literacy duties.
• August 2, 2025: GPAI duties, governance provisions, and most penalty provisions.
• August 2, 2026: general application, including transparency duties.
• December 2, 2026: the new intimate-content and child-abuse-material prohibitions.
• December 2, 2027: Annex III high-risk requirements.
• August 2, 2028: Annex I high-risk requirements.

GPAI models marketed before August 2, 2025 have until August 2, 2027. The legacy public-authority high-risk transition reaches August 2, 2030. Other existing high-risk systems generally become subject to the Act when significantly redesigned after their relevant application date. Providers of content-generating systems marketed before August 2, 2026 have until December 2, 2026 for Article 50(2) machine-readable marking; the separate duties for deployers still need assessment. AI literacy requires measures supporting its development among staff operating and using AI, taking their experience and use context into account. It does not guarantee a particular individual level. Data protection law continues to apply alongside the Act.

Risk combines likelihood and severity. AI adds moving parts: data drifts, models hide patterns, outside suppliers change, and a failure can repeat across many decisions. A risk process needs owners and review dates as well as scores.

Use the four NIST functions together

NIST AI RMF 1.0, released January 26, 2023, is voluntary and sector-neutral. It remains the published framework while revision proceeds under the US AI Action Plan. Its seven trustworthy-AI characteristics are validity and reliability; safety; security and resilience; accountability and transparency; explainability and interpretability; privacy enhancement; and fairness with harmful bias managed. They can conflict, so document the choice and who bears its costs.

Govern runs through the other functions. They recur across the lifecycle, not as a one-pass sequence. The Playbook suggests actions; profiles adapt the framework to particular uses. NIST AI 600-1, the July 2024 Generative AI Profile, addresses concerns such as confabulation and harmful synthetic content.

Separate standards and legal requirements

ISO/IEC 42001:2023 specifies an AI management system with policy, roles, assessments, controls, internal audit, and continual improvement through plan-do-check-act. Organizations can be certified against it. ISO/IEC 23894:2023 gives AI risk-management guidance based on ISO 31000. A certification concerns the management system and its assessed scope; legal duties still need their own assessment. EU harmonized standards cited for particular high-risk requirements can provide a presumption of conformity for the requirements they cover.

Work a risk register

Establish context, identify harmful events, analyze likelihood and severity, compare them with risk tolerance, choose treatment, and review. A treatment can avoid an activity, reduce its risk, transfer some consequences through insurance or contracts, or knowingly accept residual risk.

For a résumé tool, use illustrative 1–5 ratings and a priority rule: scores at least 15 are high, 8–14 medium, and below 8 low. Unequal rejection of qualified applicants has likelihood 4 and severity 4: $4\times 4=16$, high. After an audit, group monitoring, and meaningful review of rejections, assessed likelihood falls to 2: $2\times 4=8$, medium residual risk. Record the evidence supporting that changed rating and a person responsible for checking it.

A data leak rated $2\times 5=10$ is medium but has maximum severity. An outage rated $3\times 2=6$ is low. Equal or similar products can conceal very different harms; ordinal ratings are priority aids, not precise probabilities or measured loss.

An AI inventory, named system owners, pre-deployment review, vendor checks, incident response, and internal or independent audits make the framework operational. Revisit the register when users, data, capabilities, or incidents change.

Governance approaches differ along several axes. Hard law binds through enforceable duties; soft law supplies voluntary principles and standards. Horizontal rules span sectors; sector-based oversight uses regulators for fields such as finance, health, and transport. Ex ante controls act before use; ex post remedies respond after harm. A jurisdiction can combine all of these.

Compare the broad approaches

The EU combines conformity assessment and CE marking with later supervision. Global firms may extend EU practices across markets to simplify operation, the Brussels effect. That business choice differs from the Act legally applying to every activity worldwide.

The US has no single comprehensive federal AI statute. Existing consumer-protection, discrimination, credit, and product-safety rules still apply. Executive policy has changed across administrations, and states add measures on automated decisions, deepfakes, and advanced-model transparency. Federal preemption of state rules remains a dispute. A federal proposal is different from an enacted statute.

The UK's five principles concern safety/security/robustness, appropriate transparency, fairness, accountability/governance, and contestability/redress. Existing regulators apply them within their remits. The AI Security Institute evaluates advanced systems; it is not a regulator.

China's measures cover recommendation algorithms from 2022, deep synthesis and generative AI services from 2023, and generated-content labeling effective September 1, 2025. Algorithm registration and content-control requirements are part of that approach.

Track international influence and timing

International instruments include the OECD AI Principles, the 2023 G7 Hiroshima Process code for advanced-model developers, UN resolutions, and summits beginning at Bletchley Park in 2023. The Council of Europe AI Framework Convention opened for signature in September 2024 as a legally binding treaty instrument. Signature and entry into force are different stages; a treaty's existence does not mean every country is bound by it.

The Collingridge dilemma concerns timing: technology is easier to shape before its effects are well known, then harder to change after it becomes entrenched. Sandboxes, standards, and scheduled reviews help adaptation. Binding rules offer enforceable protection and predictability but can impose costs or adapt slowly; flexible approaches can respond faster while leaving gaps.

For a hiring tool, the employer remains answerable for discriminatory outcomes across EU, US, and UK settings. Review both AI-specific and existing rules, then distinguish mandatory duties from useful voluntary practice. A voluntary framework can influence contracts, procurement, and expectations of reasonable care even when it creates no statutory exemption.