Transcript:
Jen Carnig: Welcome to Keep Me Posted, a podcast from Spitfire Strategies about the intersection of race, rights, democracy and justice in the digital age.
Each episode of Keep Me Posted has a short conversation with leading experts and advocates in law, civil rights and technology. I'm your host, Jen Carnig, Chief Advocacy Officer at Spitfire. You can follow the show on Twitter at @KeepMePostedPod.
State governments are increasingly relying on AI tools and systems to determine whether people qualify for public benefits and to what extent they receive them. For people with disabilities, this can mean losing critical support without warning or explanation. Algorithms are designed to make decisions based on patterns, but disabilities are diverse, nuanced and sometimes not even physically apparent. Without proper oversight, these tools will continue to promote discriminatory outcomes.
This week I'm joined by lydia x.z. brown. Lydia x.z. brown is a policy counsel with the Center for Democracy and Technology’s Privacy and Data Project, focused on disability rights and algorithmic fairness and justice. Their work is investigated algorithmic harm and injustice in public benefits determinations, hiring algorithms and algorithmic surveillance that disproportionately impact disabled people — particularly multiply marginalized, disabled people.
I am thrilled to be joined now by lydia x.z. brown. Lydia, thank you so much for joining me today on Keep Me Posted and no pressure, but I have to tell you, I heard you recently at a Ford Foundation convening and you were one of the most compelling speakers I've heard. So I'm really excited for this conversation.
lydia x.z. brown: Thank you so much for having me, and I'm really excited to be part of the conversation, too.
Jen Carnig: I'd love to start with some background on this topic because the landscape of social benefits is broad, and for people with disabilities, the support is critical for day-to-day life. Can you talk about what these support systems could and really should look like and how crucial these benefits are, and what challenges are people facing and where is the need the greatest?
lydia x.z. brown: Different kinds of benefits serve different purposes, but unfortunately, disabled people in society face really high rates of impoverishment. The poverty rate for disabled people is consistently higher than that for non-disabled people and the disparities are even greater when you break that down by race and by gender. Disabled people of color, disabled women, disabled people in the LGBTQ community, all face higher rates of poverty than even the overall population of disabled people and as a result, a higher proportion of disabled people need access to public benefits and public assistance programs, which could include income from social security programs. It can include public housing assistance or subsidies like section eight programs. It can include Medicaid, which is not just access to healthcare, but for disabled people is also often the source of funding for long-term supports and services, which can be any kind of support or service that a disabled person might need to be able to stay alive, stay at home, to be able to keep a job.
And that can be services related to assistance with health care or feeding or services related to assistance with other daily activities, services related to health needs, taking medication, attending to chronic conditions that a person might have. And so disabled people are just more likely to need access to those types of benefits. Because number one, we're more likely to experience impoverishment, low income, low wages, or unemployment or precarious employment. And number two, for many disabled people, the only way to access necessary long-term support, services or care is through Medicaid funding, which itself is contingent upon a person having and keeping a lower income. Many benefits programs actually require disabled people to be able to prove eligibility, not just on the basis of a disability, if it is disability specific, but on the basis of poverty. For example, in social security, disabled people are subjected to an asset cap on how much money they are allowed to possess and an income cap, how much money they are allowed to earn.
And so to be continually eligible for, say, social security benefits and Medicaid benefits, which Medicaid is the only insurance provider that will pay for long-term supports and services in most cases, and most disabled people, again are not independently wealthy and so can't pay out of pocket for supports and services that they might need on a daily basis, then disabled people who are receiving those particular benefits are trapped in a cycle of poverty. You are not allowed to not be poor or you lose access to benefits.
Jen Carnig: So something you emphasize in your work is how difficult it is for people who have lost benefits to regain access to them and you're certainly indicating that right now. Can you tell us a little bit more about the process and the implications that that has on people with disabilities?
lydia x.z. brown: For one, the difficulty in trying to obtain benefits once you've lost your benefits is that it requires knowledge. It requires resources, and frankly it requires having the wherewithal to be able to pursue some kind of administrative or legal remedy, and that can require specific skills or capacities that many disabled people simply might not have, not for lack of innate capability and not for lack of intelligence, but simply because the vast majority of disabled people, like the vast majority of people in general, aren't lawyers, aren't policy experts, and many disabled people on top of that are dealing with the full-time job of being a disabled person in a society that is profoundly ableist, and may not have unlimited reserves of mental and emotional energy. May be experiencing anxiety, may have reading and cognitive disabilities that affect the ability to process the information necessary to be able to figure out what the steps are to attempt to appeal a negative determination, that you are not eligible for the same amount of benefits or that your benefits eligibility has been terminated altogether.
lydia x.z. brown: And lastly, may not have the executive functioning skills necessary to be able to participate in that process. And those are the skills of being able to sort information, to initiate tasks, to track multiple aspects of the same task, to follow through on a task and to see a task through to completion. And those skills, the skills of executive functioning are often profoundly impacted by learning disabilities and by cognitive disabilities. And one of the most common experiences that people whose executive functioning is impacted by their disabilities have and will cite, other than describing the ways that it might impact activities of daily living, is that having trouble with executive functioning can have a profound impact on a person's ability to participate in or to follow through on bureaucratic or administrative processes. So just those experiences alone, facing the compounded barriers of disability related reasons that a person's ability to challenge an administrative decision are impacted, in addition to the requirements that a disabled person might have to go through to prove that you do in fact, qualify for benefits can make it so hard.
And if we just take social security, for example, most people who apply for social security benefits are initially denied. And for many disabled people who are approved eventually to receive social security income, have to appeal in order to do so. And to have an attorney representing you drastically increases the likelihood that you might be approved for social security. Whereas if you are unrepresented, the likelihood drastically decreases conversely, but another hurdle with social security are redeterminations, and a redetermination is after you've received benefits for so many years, you need to be reassessed by a medical professional who needs to assert that you do, in fact, still have the same disability. And I've just heard so many stories from disabled folks who will say, basically I'm being asked to prove I still have Downs Syndrome. I didn't know that it was possible for Downs Syndrome to go away. Wait, I'm being asked to prove I still have cerebral palsy. Wasn't aware it was possible for cerebral palsy to stop happening. The redetermination process in theory, right, is meant to root out someone who perhaps needed benefits for a time related to their disability. That is, that they are disabled and because of their disability, they are deemed unable to work. And that person may have, perhaps because of their disability, been unable to work for a time, but then later regained the ability to work or gained the ability to work for the first time. And that's what the redetermination process is meant to capture. But for many people, the nature of their specific disability and the demands that are expected of someone to be able to work in our current society simply might not be compatible.
And that just might not change. Like, you know, if your cerebral palsy impacts specifically the way that you personally can work and you are not able to be employed according to ableist assumptions about what it means to have a job, then is that really going to change from one year to the next? Is it going to change over a five-year period or not? And to be redetermined to be sufficiently disabled and eligible for social security, you need to see a medical professional who needs to confirm you still have this disability. And I think, you know, one of my friends even mentioned at one point that the paperwork that she had in particular literally asked, you know, how many years have you had this disability? And, you know, the answer is like, well, uh, my entire life? Now that might not be true, right? Like for all disabilities, some disabilities you might acquire because of an accident or illness, like, you know, long COVID. But in the case of Downs Syndrome, you're definitely born with it and you definitely have it for the rest of your life. It just comes back to how processes for being able to just basically prove, in fact, you need access to a certain benefit, and benefits that frankly are not particularly robust in some cases, that most disabled people I know who are receiving social security income live on about $800 a month. That $800 a month is supposed to go toward literally everything. Who can live off of $800 a month? And yet, that's the amount that many disabled people are reliant upon if that is what they're receiving in social security. And to lose access to that $800 a month, can mean losing access to all income.
So the question of being able to keep one's benefits can be a matter of life or death, if that care includes the services of a support worker or a care attendant who has to provide daily assistance to you, up to and including medications, physically positioning you, assisting you with food, with eating, can mean that you are able to function, that your most basic physical needs are attended to, or that you're left to develop bedsores, to spend hours unable to eat, to go without medication, to miss windows when you're supposed to take specific medications and then to experience the side effects of that medication and/or the experience of mini withdrawal, of not being able to have the medication at the times that you're supposed to be having it depending on what kind of medication that might be.
Let me add that too. If you are facing the loss of your primary source of income, you're facing the loss of necessary support and care that you were depending on, how are you under those conditions supposed to be able to mount an effective appeal?
Jen Carnig: Yes, there's really no answer. Right? I don't know who would have the resources or capacity to take on such an onerous process or any process. So I always hear this story about curb cuts. Before there were curb cuts, it was really challenging for people in a wheelchair or people with other challenges to be able to get around. And then with curb cuts, lo and behold, it actually helped so many other people. It's easier to push the stroller. It's good for kids. I'm curious if there's some metaphor here for that. If we're able to create systems and policies and practices that are supportive for the individuals that you're speaking about, what are the greater benefits for the larger good, too? Cause I'm confident it's going to improve things for a lot of people.
lydia x.z. brown: You know, our current benefits system isn't really working. People who need section eight housing vouchers for affordable housing and assistance with paying their rent are put on waiting lists that, you know, I hear from friends in different states that are all saying it's eight years, 10 years, 12 years. That's to get on the waiting list. Where are you supposed to go while you're waiting for a voucher? Where are you supposed to actually go? People will tell me, you know, I need Medicaid because it's the funder that will cover my services. But in almost any state that I go to in order to stay on Medicaid, I basically am not allowed to have an income and with very, very, very few exceptions, and our current benefits system doesn't work because it assumes number one, that you need to be poor and that you need to stay poor, and number two, in some states, the perverse incentive is well, if we make it extraordinarily hard to get benefits, extraordinarily hard to keep benefits in theory, that's supposed to be, oh, we want to encourage you to get a job.
Well, that's not humane in any way whatsoever. It is forcing you to figure out how to work. Well, most people who are receiving benefits probably do want to work, you know, and if only because our society so highly stigmatizes and shames perceived or actual inability to work, according to what's considered a job, but most people do not think “I want to literally do nothing ever, and also have everyone that I ever encounter think that I'm lazy and irresponsible.” Most people don't think that again, if only because of our social conditioning that so highly stigmatizes not working, and yet our benefits system assumes that it needs to be made undesirable to have to have benefits. And you need to be able to prove that you're not committing fraud. You need to be able to prove that you truly are deserving and you're not making it up.
And our benefits system assumes that it's supposed to be a stop gap or a last resource. And if we redesigned the way that our social services and public benefits worked to account for the fact that so many different people need assistance in so many different ways, over the course of their life span, and that shouldn't be tied to enforced poverty or having to undergo extremely invasive medical examinations or having to essentially degrade yourself to be able to prove I do, in fact, need this assistance and to give meaningful assistance, you know, that might look very different, but we don't have that system set up right now. That's not what we have.
So just in thinking about the way our social services and public benefit systems are designed, they are not designed to provide in a humane way for people who need assistance of some form or another, we don't have access to universal healthcare in this country. We do not have access to guaranteed housing security in this country. We have more empty available housing units in this country than necessary to house people who are currently experiencing homelessness.
And yet people continue to be unhoused. People continue to be evicted. People continue to be foreclosed upon. People continue to be forced out because of discrimination, or prevented from accessing units to begin with because of discrimination. So, you know, our current societal and political setup does not contemplate a society in which we care about meeting one another's needs. It is a society that only considers whether someone is deemed sufficiently worthy or productive to be afforded minimal protections as a person.
Jen Carnig: So in my work at Spitfire, I think a lot about the power of story. And I know the center for democracy and technology, where you work, is deeply invested in making the issues you're talking about really personal and presenting the real human stakes of these policies. I know you have the report challenging the use of algorithm driven decision-making in benefits determinations affecting people with disabilities. And in that, CDT takes stories of people from across the country who have lost or had their benefits cut after state governments adopted algorithms for benefits determination. What stories did you come across that were really striking to you?
lydia x.z. brown: You know, what happened to Bradley Ledgerwood and Tammy Dobbs in Arkansas was particularly horrifying. Bradley and Tammy are both people who have cerebral palsy and need significant amounts of support in their daily lives like, you know, a really large number of disabled people do. And under the prior program that Arkansas had in place before they implemented their new algorithmic determination process was that they were able to receive a certain number of hours of approved medical care in order to be able to stay at home. The number was 56, which wasn't necessarily accounting for hours of care that weren't paid for. In Bradley's case, his care was provided by his parents and the state's program allowed his parents to receive paid reimbursement for, again, up to 56 hours of that care.
And in fact, in Bradley's case, his mother quit a job that paid quite well so that should be able to stay at home and provide support full-time for Bradley, who is himself a very active member of his community, he’s an alderman at the town. Tammy, on the other hand received the same type of funding, 56 hours a week. And she hired a professional support worker. The way that that funding worked was they could look into who they needed to hire to take care of attending to their respective needs related to dressing, related to physical positioning, related to eating and other tasks that they're not able to perform on their own. But then the state decided to use a new algorithm that would calculate how many hours of care each person in that Medicaid program would actually receive. Previously, a nurse would actually be able to exercise individual discretion in determining what amount of care a person could be approved for.
But now a computer program would actually make that determination automatically. And essentially overnight in 2016, both Bradley and Tammy were told that their hours would be cut nearly in half from 56 to 32.
Jen Carnig: So when you have uncovered these kinds of stories, have there been changes? Are governments responding to kind of being shown how these algorithms are impacting people? Are you seeing a change in how state governments use these tools since you released the report?
lydia x.z. brown: Bradley and Tammy both reached out to Legal Aid of Arkansas, where one of the leading advocates in this area, Kevin De Liban, began to litigate and ultimately this led to litigation in the state of Arkansas that challenged the nature of that algorithm and its implementation, and this went back and forth a couple of times in the court and ultimately, the state legislature moved in a different direction after repeated litigation around its implementation.
But unfortunately, not every state is thinking about, you know, how harmful algorithmic decision-making might be. That algorithms can't necessarily account for individual difference. That algorithms can't necessarily account for differences within and among people who have the same diagnosis, or algorithms that don't account for why, how, and to what extent different disabled people might need assistance with different areas within their life and how their needs might fluctuate. Even for the same person, that same person might require substantially more support one day than another, and an algorithm is not going to be able to capture that or to allow a person to understand how they're being assessed and how the algorithm is making its decision. Let alone to be able to challenge that. Over the last year now, for example, the state of Colorado, the state's department of healthcare policy and financing has been working to create a person centered budget algorithm.
That's the name of it. That's the name of it, the person centered algorithm, because you know, algorithms are so person-centered, with a private vendor, and they've hosted a number of public opportunities for comment, requested for public assistance in the development of that algorithm. And yet, they ironically, despite a lot of public opposition and expressed concern, they're moving forward with implementation of the new algorithmic decision-making process. And they are attempting to claim that this algorithm will not be determinative of what kinds of services or amount of services a person will need, but the algorithm is meant to identify the amount of care that a person will be approved for. It's supposed to be, in theory, part of an individualized, person-centered process, but I don't understand, as an advocate or as a human being, how a computer program is supposed to be individualized and person-centered.
Jen Carnig: This feels really big and heavy and challenging. Just to give folks maybe a little bit of a higher note in the bad news, are there things that are giving you a feeling of hope or confidence right now? Or, you know, are there things that we can do to get involved and kind of support the advocacy that folks like you are doing?
lydia x.z. brown: I think an increasing number of people are aware of the role that algorithmic decision-making plays in our lives now, and more folk,s directly impacted or not, who are concerned with issues of civil rights and social justice are cognizant of at least some of the ways in which algorithms can directly implicate people's rights, people's lives and people's freedom. And as more people from marginalized communities, including the disabled community, become aware of just how insidious algorithmic decision-making can be and how ever present it is in everything, from public benefit systems to rental applications, to employment and job applications, and even to policing, right, that more folks will and already have become empowered to speak out, to work for change and to advocate for regulators in legislative and regulatory capacities, to be able to limit the capacity of algorithmic decision-making tools to cause harm.
And, you know, there've been a handful of bills proposed just over the past year in the current legislative session at the state level, that look at ways in which algorithms might be monitored or audited, ways in which algorithms and what they can be used for might be limited. A couple of cities in California have actually banned the use of predictive policing tools and the state is considering a bill that would look at the use of algorithms by public entities. The New York City Council looked at legislation that would require bias auditing and hiring algorithms and, you know, bias auditing isn't an answer. It's not a solution, but certainly having that conversation that algorithmic decision making can be unjust, can deepen, exacerbate and perpetuate existing forms of discrimination and structural oppression and therefore we need to look at them very carefully. We need to consider how we are deploying them and we need to consider what safeguards we place on the possible use and application of them.
The fact that more legislators and regulators are having that conversation, the fact that more ordinary folks are empowered to understand just how pervasive algorithms are, that it's not just fun jokes about what Facebook shows you, but actually sometimes life or death matters. It gives me hope to know that we're not just screaming into a void in the wilderness somewhere. I have screamed into the wilderness, and it can be very cathartic but not very effective in trying to move change.
Jen Carnig: I love ending on a laugh. Lydia x.z. brown, I'm very grateful for your time today. Thank you so much for joining us on Keep Me Posted.
lydia x.z. brown: Thank you so much. And it was a pleasure to talk to you and I hope to see you in the fight.
Jen Carnig: I want to again thank lydia x.z. brown, policy counsel at the Center for Democracy and Technology’s Privacy and Data Project. Next time on Keep Me Posted, I'll speak with Azza Altiraifi. I hope you'll join us.
Until then, please follow the show on Twitter at @KeepMePostedPod and rate, review and subscribe to us on Apple podcasts.
Keep me posted is produced by Spitfire Strategies. Trendel Lightburn is our senior editor. Our production team is Gabrielle Connor, Ben Christiason, Maggie Cass and Nima Shirazi. To learn more, visit us at spitfirestrategies.com.
The music is by UI. I'm your host, Jen Carnig. Thanks so much for listening. Stay well.