|

The Bias Hidden in the Software: Automated Decisions, Disability, and the Rights That Still Apply

Editor’s note: this is reporting on a documented area of research and policy. It describes what analysts and researchers have found and what Canadian law provides. It takes no political position.

More of the decisions that shape a life now begin with software. Whether a job application clears the first screen. Whether a benefit is approved, flagged, or cut. Whether a request for care is scored as routine or suspect. Automated tools increasingly sit at these gates, and a growing body of research finds the tools can carry bias against people with disabilities.

This is worth understanding on its own terms, apart from any election or party. Here is what the research documents, why it reaches Canada, and what the law still requires.

How bias gets into an automated decision

The mechanism is not mysterious. Many AI systems learn from historical data, and that data reflects a world built around people without disabilities. A resume-screening tool trained on past hires can learn to penalize gaps in work history, which are common for someone managing a health condition. Video-interview software can misread atypical speech, eye contact, or facial expression. A fixed online assessment can be impossible to finish with a screen reader or without a standard set of hand movements. In each case no one told the system to discriminate. It reproduces the patterns in its training data, and a disability is often the thing that reads as a deviation from the pattern.

Researchers have documented this across hiring, benefits, housing, and health care. A 2025 socio-legal study in the Journal of Law and Society examined how employers’ AI hiring systems can facilitate discrimination. In the United States, the lawsuit Mobley v. Workday is testing whether an AI hiring platform can itself be held responsible when its screening tools disproportionately exclude applicants with disabilities and older applicants. It is a US case under US law, it remains in pre-trial stages, and it has decided nothing. The concern researchers keep returning to is that these tools embed old bias at scale, behind a surface of neutral objectivity, which makes the discrimination harder to see and harder to challenge. We have looked before at how much these systems already gather about you in the data you are giving away.

Why a US policy document put this in the news

Much of the recent attention traces to a US policy blueprint. Project 2025 is a document produced by the Heritage Foundation, a conservative US think tank, proposing a restructuring of the federal government. In February 2026, a peer-reviewed article in the journal First Monday connected that blueprint to concerns about artificial intelligence and disability, and several US disability organizations have published their own analyses of what its proposals could mean. Those are US documents about US policy, and they argue positions we are not going to weigh here.

The part that outlasts any single administration, and that does not stop at the border, is the technology. That is where this piece stays.

Why it reaches Canada

Two reasons. First, the software is borderless. The same hiring platforms, benefits systems, and risk-scoring models built and trained in the United States are sold to and used by Canadian employers, insurers, and governments. The bias travels with the code, the same way convenience and dependence travel together when a voice assistant becomes your memory. Second, Canadian institutions have started to respond directly.

In December 2025, Accessibility Standards Canada published CAN-ASC-6.2:2025, Accessible and Equitable Artificial Intelligence Systems, a National Standard of Canada and the first edition of that standard. The standard states that it is intended to align with the Accessible Canada Act, the Canadian Human Rights Act, and the United Nations Convention on the Rights of Persons with Disabilities, and that it “can be used for conformity assessment.” Its requirements cover the design, procurement, deployment and monitoring of AI systems, and include a clause requiring organizations that use AI to make or assist decisions to offer people an alternative decision-making process, either without AI or with direct human oversight, and to give people information in accessible formats about how to correct, contest, change or reverse an AI-assisted decision.

Two limits on that are worth stating plainly. Accessibility Standards Canada describes its standards as voluntary, applying to federally regulated entities, and says of itself that it “has no power, nor does it undertake, to enforce conformance” with them. The standard is published free on the department’s website in multiple formats.

What the law still requires

Human rights obligations do not switch off because a computer is in the loop, and the bodies that administer them have said so directly. In joint principles for the responsible use of AI, the Ontario Human Rights Commission and Ontario’s Information and Privacy Commissioner put the duty on the organization rather than the software: “Institutions using AI systems must prevent and remedy discrimination effectively and ensure that benefits from the use of AI are universal and free from discrimination.” They add that “human rights law requires that developers, providers and institutions ensure that they do not infringe substantive equality rights,” and warn that applying one system uniformly across diverse groups, “though seemingly neutral, may actually result in adverse impact discrimination.” Those principles are addressed to Ontario’s public sector, so they are guidance rather than a rule that binds every employer in the country, but the underlying duties they describe sit in the Canadian Human Rights Act and in every provincial and territorial human rights code.

It is worth being precise about what follows from that and what does not. The two commissions recommend that individuals “should be provided with at least a right of review for automated decision processes that do not involve high risk,” and given the choice of opting out of high-risk automated decisions “in preference of a human decision maker.” That is a recommendation to institutions, not an entitlement that already exists everywhere. What does exist is the underlying prohibition on discrimination: if you believe an automated decision produced a discriminatory outcome on the basis of disability, the human rights commission or tribunal in your jurisdiction is the body that considers that complaint, and the organization that chose to deploy the tool is the one answerable for it.

This article is general information, not legal advice. For a specific situation, a community legal clinic or a human rights office in your province or territory can help.

Sources

Similar Posts