Lessons from the Robodebt debacle

How to avoid algorithmic decision-making mistakes: lessons from the Robodebt debacle

The University of Queensland has a research alliance looking at Trust, Ethics and Governance and one of the teams has recently published an interesting summary of How to avoid algorithmic decision-making mistakes: lessons from the Robodebt debacleThis is based on an open paper Algorithmic decision-making and system destructiveness: A case of automatic debt recovery. The web summary article is a good discussion of the Australian 2016 robodebt scandal where an unsupervised algorithm issued nasty debt collection letters to a large number of welfare recipients without adequate testing, accountability, or oversight. It is a classic case of a simplistic and poorly tested algorithm being rushed into service and having dramatic consequences (470,000 incorrectly issued debt notices). There is, as the article points out, also a political angle.

UQ’s experts argue that the government decision-makers responsible for rolling out the program exhibited tunnel vision. They framed welfare non-compliance as a major societal problem and saw welfare recipients as suspects of intentional fraud. Balancing the budget by cracking down on the alleged fraud had been one of the ruling party’s central campaign promises.

As such, there was a strong focus on meeting financial targets with little concern over the main mission of the welfare agency and potentially detrimental effects on individual citizens. This tunnel vision resulted in politicians’ and Centrelink management’s inability or unwillingness to critically evaluate and foresee the program’s impact, despite warnings. And there were warnings.

What I find even more disturbing is a point they make about how the system shifted the responsibility for establishing the existence of the debt from the government agency to the individual. The system essentially made speculative determinations and then issued bills. It was up to the individual to figure out whether or not they had really been overpaid or there was a miscalculation. Imagine if the police used predictive algorithms to fine people for possible speeding infractions who then had to prove they were innocent or pay the fine.

One can see the attractiveness of such a “fine first then ask” approach. It reduces government costs by shifting the onerous task of establishing the facts to the citizen. There is a good chance that many who were incorrectly billed will pay anyway as they are intimidated and don’t have the resources to contest the fine.

It should be noted that this was not the case of an AI gone bad. It was, from what I have read, a fairly simple system.

Predatory community

Projects that seek to create new communities of marginalized people to draw them in to risky speculative markets rife with scams and fraud are demonstrating

Through a Washington Post article I discovered Molly White who has been documenting the alt-right and now the crypto community. She has a blog at Molly White and a site that documents the problems of crypto at Web3 is going just great. There is, of course, a connection between the alt-right and crypto broculture, something that she talks about in posts like Predatory community which is about crypto promotions try to build community and are now playing the inclusive card – aiming at marginalized communities and trying to convince them that now they can get in on the action and build community. She calls this “predatory community.”

Groups that operate under the guise of inclusion, regardless of their intentions, are serving the greater goal of crypto that keeps the whole thing afloat: finding ever more fools to buy in so that the early investors can take their profits. And it is those latecomers who are left holding the bag in the end.

With projects that seek to provide services and opportunities to members of marginalized groups who have previously not had access, but on bad terms that ultimately disadvantaged them, we see predatory inclusion.22 With projects that seek to create new communities of marginalized people to draw them in to risky speculative markets rife with scams and fraud, we are now seeing predatory community.

Michael GRODEN Obituary

I just found out that Michael GRODEN (1947 – 2021) passed away a year ago. Groden was a member of CSDH/SCHN when it was called COCH/COSH and gave papers at our conferences. He developed an hypertext version of Ulysses that was never published because of rights issues. He did, however, talk about it. He did, however, publish about his ideas about hypertext editions of complex works like Ulysses. See his online CV for more.

Wordle – A daily word game

Wordle Logo

Guess the hidden word in 6 tries. A new puzzle is available each day.

Well … I finally played Wordle – A daily word game after reading about it. It was a nice clean puzzle that got me thinking about vowels. I like the idea that there is one a day as I was immediately tempted to try another and another … Instead the one-a-day gives it a detachment. I can see why the New York Times would buy it, it is the sort of game that would bring in potential subscribers.

We Might Be in a Simulation. How Much Should That Worry Us?

We may not be able to prove that we are in a simulation, but at the very least, it will be a possibility that we can’t rule out. But it could be more than that. Chalmers argues that if we’re in a simulation, there’d be no reason to think it’s the only simulation; in the same way that lots of different computers today are running Microsoft Excel, lots of different machines might be running an instance of the simulation. If that was the case, simulated worlds would vastly outnumber non-sim worlds — meaning that, just as a matter of statistics, it would be not just possible that our world is one of the many simulations but likely.

The New York Times has a fun opinion piece to the effect that We Might Be in a Simulation. How Much Should That Worry Us? This follows on Nick Bostrom’s essay Are you living in a computer simulation? that argues that either advanced posthuman civilizations don’t run lots of simulations of the past or we are in one.

The opinion is partly a review of a recent book by David Chalmers, Reality+: Virtual Worlds and the Problems of Philosophy (which I haven’t read.) Chalmers thinks there is a good chance we are in a simulation, and if so, there are probably others.

I am also reminded of Hervé Le Tellier’s novel The Anomaly where a plane full of people pops out of the clouds for the second time creating an anomaly where there are two instances of each person on the plane. This is taken as a glitch that may indicate that we are in a simulation raising all sorts of questions about whether there are actually anomalies that might be indications that this really is a simulation or a complicated idea in God’s mind (think Bishop Berkeley’s idealism.)

For me the challenge is the complexity of the world I experience. I can’t help thinking that a posthuman society modelling things really doesn’t need such a rich world as I experience. For that matter, would there really be enough computing to do it? Is this simulation fantasy just a virtual reality version of the singularity hypothesis prompted by the new VR technologies coming on stream?

The Future of Digital Assistants Is Queer

AI assistants continue to reinforce sexist stereotypes, but queering these devices could help reimagine their relationship to gender altogether.

Wired has a nice article on how the The Future of Digital Assistants Is Queer. The article looks at the gendering of virtual assistants like Siri and how it is not enough to just offer male voices, but we need to queer the voices. It mentions the ethical issue of how voice conveys information like whether the VA is a bot or not.

Why people believe Covid conspiracy theories: could folklore hold the answer?

Using Danish witchcraft folklore as a model, the researchers from UCLA and Berkeley analysed thousands of social media posts with an artificial intelligence tool and extracted the key people, things and relationships.

The Guardian has a nice story on Why people believe Covid conspiracy theories: could folklore hold the answer? This reports on research using folklore theory and artificial intelligence to understand conspiracies.

The story maps how Bill Gates connects the coronavirus with 5G for conspiracy fans. They use folklore theory to understand the way conspiracies work.

Folklore isn’t just a model for the AI. Tangherlini, whose specialism is Danish folklore, is interested in how conspiratorial witchcraft folklore took hold in the 16th and 17th centuries and what lessons it has for today.

Whereas in the past, witches were accused of using herbs to create potions that caused miscarriages, today we see stories that Gates is using coronavirus vaccinations to sterilise people. …

The research also hints at a way of breaking through conspiracy theory logic, offering a glimmer of hope as increasing numbers of people get drawn in.

The story then addresses the question of what difference the research might make. What good would a folklore map of a conspiracy theory do? The challenge of research is the more information clearly doesn’t work in a world of information overload.

The paper the story is based on is Conspiracy in the time of corona: automatic detection of emerging Covid-19 conspiracy theories in social media and the news, by Shadi Shahsavari, Pavan Holur, Tianyi Wang , Timothy R Tangherlini and Vwani Roychowdhury.

Diggin’ in the Carts: Japanese video game music history

Meet the men and women responsible for creating the most iconic tunes in video game history.

We finished up the Replaying Japan 2021 conference today. The conference was online using Zoom and Gather Town where there was a hidden easter egg with a link to Diggin’ in the Carts: Japanese video game music history, a 5 part documentary from Red Bull that is quite good. The 5 15 minute episodes are part of the first season. Not sure if there will be other seasons, but there is a related radio show with multiple seasons. The documentary episodes nicely feature the composers and experts talking about the Japanese history along with other musicians commenting on the influence of the early music which would have been heard over and over in houses with Japanese consoles.

The creator of the show is Nick Dwyer who is interviewed here about the documentary and associated radio show..

Right Research: Modelling Sustainable Research Practices in the Anthropocene – Open Book Publishers

This timely volume responds to an increased demand for environmentally sustainable research, and is outstanding not only in its interdisciplinarity, but its embrace of non-traditional formats, spanning academic articles, creative acts, personal reflections and dialogues.

Open Book Publishers has just published the book I helped edit, Right Research: Modelling Sustainable Research Practices in the Anthropocene. The book gathers essays that came out of the last Around the World Conference that the Kule Institute for Advanced Research ran on Sustainable Research.

The Around the  World econferences we ran were experiments in trying to find a more sustainable way to meet and exchange ideas that involved less flying. It is good to see this book out in print.