Tag Archives: artificial intelligence

Must we always use LLMs this way?

As with my last post on AI, let me open with a statement:

Just because most people hold AI a certain way, doesn’t mean that we always must, either now or in the future.

As I noted in that piece, and my shades of gray post, I am well aware — and agree with — the notion that a lot of frontier AI models have disturbing ethical concerns that we shouldn’t dismiss, including concentration of power and wealth, environmental concerns, unethical scraping and training, the financial bubble they seem to be creating, and more. The most poisonous of AI cheerleaders are accelerating enshittification while cutting quality all over the place. AI slop is also a real problem, which I’ll discuss in a bit.

Before I dive in, I need to mention that no AI assistance was used for writing this post, and I have been using em dashes — correctly — for years before LLMs did. Secondly, while popular discourse seems to use “AI” and “LLM” interchangeably, the AI field is much more broad and much more old than LLMs. I will, nonetheless, adopt the common “AI” jargon here, but be more specific when necessary to differentiate.

In The Reverse Centaur’s Guide to Life After AI by Cory Doctorow, he presented two ideas I particularly want to focus on here:

  1. The present and future of LLMs outside of the cloud-hosted frontier models
  2. What differentiates productive use of AI from slop

The two are related, in fact. Doctorow discussed how he never uses AI to write anything, but he sometimes uses it as an assistant to help him write. Is that contradictory? Not really. He gave this example: he remembered a quote from a podcast, which he wanted to cite and attribute correctly. But he couldn’t remember which podcast or who said it. So he fired up Whisper, an open-weight model you can download and run locally, even on meagre hardware. Whisper is a speech-to-text model. Doctorow ran likely podcast episodes (from the target range and set of podcasts he listens to) through the model, and at that point it was a regular search. He found the quote, then went back and listened to the episode at that time spot to verify its accuracy.

It would have taken so much time to do this without Whisper that he simply would not have done it otherwise. His writing was better because of AI. Not because AI wrote any words for him, but because it helped him find the right ideas and attributions. He didn’t even have to trust its transcription; it pointed him at the right place, and then he could transcribe with precision.

I too have used Whisper in much the same way. A little while back, I digitized many hours of old family audio recordings going back to the 1970s. Thanks to Whisper, which I ran locally primarily during hours of being fully solar-powered, I have a searchable index to these gems. A person I know recently asked me if I knew any way to transcribe some old family recordings he had. I told him I could run it through Whisper. It wouldn’t have great punctuation, but most of the words would be there. This meant a lot to him and his family; what I did in 2 minutes would have taken him many days (I presume he is not a fast typist as well.)

So we can start by naming: AI isn’t just chatbots, and AI can help even if it isn’t writing anything.

Doctorow goes on to ask the question: what happens after the AI bubble bursts? I do believe it will, as Ed Zitron has carefully documented for us for a long time now. The environmental damage done by frontier training and inference in the last few years will, of course, be done. But as he points out, we’ll be left with a plethora of surprisingly-capable open weight models, a community that has made significant strides running them on low-end devices (even phones), and a massive buildup of clean energy that hopefully will be finally able to offset some polluting energy once these heavy users drop off the grid.

So then what? Will the world just throw away the likes of Whisper and Gemma? Obviously not. Doctorow imagines, credibly, a situation in which individuals and universities tinker with locally-run models, iterating on them and improving them with the more traditional scale of resources available to individuals and universities. In other words, this future sounds a lot like the earlier days of Open Source.

As Ed Zitron has pointed out, the real costs of frontier AI have been hidden as these companies rack up fantastical losses. When the bubble bursts, the real costs of frontier AI are likely to be high enough that CEO’s won’t be able to ignore the poor ROI. But we’ll be left with these artifacts, these open-weight starting points, that people can use.

That brings me to my central thesis: a lot of anti-AI sentiment is focused on the real downsides of the hyped uses of AI, and fails to account for the more ethical advances being made now, and likely to come in the future.

Varying quality of AI use

We are all familiar with AI slop. I’ve seen it, and have an increasingly skeptical eye when I see “new” projects announced in even places as niche as a BBS community (where I see things like new terminal emulators announced, written in shockingly-poor nodejs, and such). I’ve seen documents that take many pages to say what amounts to nothing intelligble to a human, with perfect grammar. I’ve seen pull requests that have a description that amounts to several pages of slop. I’ve personally seen an AI agent do things like correctly detect passing an untrusted argument to a shell, then propose a broken quoting algorithm to fix it (rather than proper sanitization and exec() or, better yet, avoiding the call altogether.)

I’ve also seen AI tools very quickly and accurately identify bugs that had passed human review, even professional security reviews, for years. I’ve seen them bring more joy to some people’s jobs by handling the more drudgery parts of the jobs. One of my favorite podcasts, Oxide and Friends, had an episode called Engineering Rigor in the LLM age and credibly described ways in which careful use of LLMs can increase quality and safety. Yes, I said increase. (Oxide and Friends are not hype cheerleaders by any means; these are very serious people with solid technical chops, and I know some of them.)

What accounts for this disconnect?

We can point to many factors, but Doctorow asks a fundamental question: who is in charge? Is it the person or the computer? Does the AI service people, or do people service the AI?

In some of the most ridiculously AI-first companies, it is very clearly the latter. Mandates from CEOs to increase AI usage lead to perverse outcomes that sacrifice quality. The AI is in charge. I’ve never seen an AI be any good at design questions, and it’s no wonder than all these vibe-coded apps have the same generic feeling of crappiness.

But when the AI is used pragmatically as a junior research assistant, that’s a different approach. Its output is greeted skeptically, its actions distrusted, and it’s never asked to design something. These are the cases where I’ve seen it make a meaningful difference.

It’s not black-and-white

All this adds up to why I was opposed to the Debian proposals that would have banned LLM use. Those votes are hard to change. They would have left Debian in a vulnerable position of being unable to react to accurate security findings if they originated with an LLM, or left Debian unable to adopt ethical use of models.

It would be akin to saying “you should never buy a PC because Windows 11 is terrible.” I agree Windows 11 is terrible, but saying something like that is so dismissive of all those that have invested decades of their time in building something better.

So, I find AI hype to be uncredible and dangerous. I also find knee-jerk reactions to AI to be the same. They are actually a form of capitulation: they let the AI megacorps dictate the terms of the conversation to revolve around them.

Case study: Debian’s DFSG scanner

For a long time, my two biggest annoyances with Debian have been the difficulty of packaging Rust and Go packages with their long dependency trees, and how long it takes a new package to get reviewed. About a year ago, the latter process was set on a path to reform, and it has gotten dramatically better. It also cuts off a potential connection with someone that hasn’t yet heard of Linux. Some people may not jump from Windows 11 (or the ChatGPTs of the world) immediately, but pretending that it’s ChatGPT or nothing does a disservice to all those that are trying to do research etchically.

We should state at the outset that Debian takes compliance with the Debian Free Software Guidelines (DFSG) extremely seriously. Every new package gets a review, and these reviews often catch things that upstream repos and other distros don’t. One common example would be a package that has a stated copyright and license, but then includes files that bear a different copyright and license.

You can imagine reviewing packages at this level of detail isn’t a fun task for people, and it’s hard to find volunteers to do it. Perhaps the answer to this should be “Debian should stop caring so much more than everyone else”, and perhaps you’re right (and perhaps you aren’t).

In any case, the DFSG team that reviews this added what amounts to an LLM-backed license linter. Its job is to produce an initial assessment that the license statements documented in debian/copyright are correct, thorough, and in compliance with DFSG. This report was not taken to be authoritative, but was designed to help with some initial assessments.

There were some predictable reactions. People were angry that the DFSG team had added this tool without any communication about it (justifiably so, I’d say). Many were angry just because of LLM. People made assumptions about what kind of model it used, what kind of quality it would produce, how definitive it would be in the project.

It is worth noting here that Debian is uniquely decentralized. There is a central policy, but maintainers are free to maintain their packages how they like. They don’t have to use a standard location for a Git repo, or even use Git.

One person said essentially, “it’s fine for others to use whatever tool they want, but I shouldn’t have to be exposed to LLM slop.” The other perspective boiled down to, “why should they keep the reports secret? If you want to proactively fix things, or flag false positives, why not?”

As Andreas Tille wrote, “I have read the LLM reviews carefully, and in many cases their findings inspired additional commits to improve my packaging. In some cases, they even helped me catch mistakes that led me to reject an upload myself before human inspection.”

Joey Hess raises an interesting risk: that LLMs will remove the incentive for Debian to actually fix its cumbersome processes, by letting an LLM paper them over. I have seen this tendency in other contexts, both at work and at home, as well. It’s a real risk, especially if Debian doesn’t tackle the packaging problem for Rust and Go.

The Mastodon post that prompted me to write this began a chain of events that started with “Uploaders will be forced to address LLMs issues without human inspection.” I don’t believe that will be the case; it goes too far against the Debian ethos.

I encourage everyone to take a deep breath. We are all getting used to a shifting landscape. We will all make missteps, myself included. The issues are complicated, the picture isn’t black and white, and there is incredible hype around the ecosystem that can make finding nuance difficult.

Let’s be kind to each other, fellow humans.

But, I do not care if you are kind to Claude.

Artificial Intelligence: Shades of Gray

AI sure is a hot topic right now, and I see a lot of people arguing about it. To a lot of people around here, I’m the “computer person” they know and I get asked a lot about AI.

I’m going to suggest a lot of things can be true at once. For instance:

  • LLMs are changing how we work and will continue to do so.
  • LLMs are vastly over-hyped by vested interests, and may be in a bubble.

Or how about:

  • Huge investment in GenAI is having many negative consequences, ranging from environmental to causing affordability problems in many industries that use hardware (ie, everywhere)
  • Useful results can be had from models that run on local hardware, even battery-powered hardware, which may have negligible harm or even some benefit

And:

  • GenAI is further concentrating wealth and power in megacorps, with the effect of squeezing out the smaller players even more.
  • GenAI is lowering the cost of entry for people without a lot of resources already.

I have sympathy for the naysayers; those that say it’s nothing but a stochastic parrot. But I don’t have a lot of sympathy for the naysayers that deny ever using it; you can’t form a credible argument against something without having an understanding of it informed by experience.

I also have sympathy for the cheerleaders. I have seen some impressive things from AI; for instance, a story from an engineer who has a child with a rare disease without a credible cure. The engineer did a lot of research on it, started feeding research papers into AI to analyze, and the AI started finding correlations between different areas of research that humans hadn’t yet found — leading to a positive result for the child.

To be fair, I have rarely seen an AI deliver a 100% correct answer on anything with any real level of complexity. I have seen it both waste more time than it saves, and save a ton of time.

My point here is: It is neither always fantastic nor always terrible.

Let me talk you through an example.

I am a fan of inbox zero for email. That is, the inbox should be empty. Unfortunately, mine has 8000 messages in it. According to the oldest messages in my inbox, I last had inbox zero 8 years ago. But really, only a handful are older than 2020. I guess something must have happened that year…

I’ve been chipping away at this for quite some time now. The problem is, there are certain emails in there that really do still need some action – maybe it’s photos to save off into our photo collection, for instance. But when looking at things sorted by date or thread, there are old shipping confirmations next to phishing attempts and family photos. One can’t just scan down the list.

I’ve tried all the usual tricks, most of which involve selecting groups of message that are easy to bulk erase, or at least easy to scan visually for the occasional thing worth saving. Sort by sender or subject line, for instance. Then I can, for instance, delete all the old messages from the shopping sites I commonly use all at once. But then they start using different senders and different subject lines and that doesn’t get all of them. I’ve tried keyword searches for this sort of thing too. Still, that got me down to about 8000 messages.

So I thought: why not see if an LLM could help me classify these? Maybe it could categorize them, and then I could look at emails grouped by category.

I have one machine with a discrete GPU, an Nvidia RTX 4070. It’s a desktop machine I don’t use all that often. But I set up Ollama on it, running in a Docker container. Ollama runs models locally.

I should also mention at this point that we are solar-powered, and this time of year is a time of peak production of excess solar, because it is sunny and not much heat or AC is required. So that machine is solar-powered and isn’t causing environmental harm. In any case, charging the EV uses much more power than that GPU.

I figured I would do this in two passes. First, ask the LLM to classify each message (or a sampling of them would probably work too), letting it pick its own categories for each. Then, look at the patterns that emerge and give it a single, much smaller, set of broad categories to use and rerun it over that.

Then I can easily select messages from my Maildirs by category and process them in bulk.

I used open-interpreter pointing to that GPU on my network to help me write the scripts for this. It didn’t get things right on its own; for instance, it didn’t call the Ollama API correctly, and insisted on appending “/cur” to the path to the Maildir (which was not going to fly with Python’s maildir module). It took roughly an hour to classify those 8000 messages (or, as I had it do, the first 2000 characters of them), and then the same to do it a second time. I had it output lines in the form of “filename\tcategory” and hand-wrote the shell script that processed those.

In the end, was it useful? Yes, quite. Its classifications weren’t perfect (and it didn’t even follow my prompt perfectly; sometimes it would give me a long discussion on why it picked a certain category rather than just that category, and occasionally it picked categories not on the list). But then, neither were my manual keyword searches. So far I’ve gotten rid of nearly 1000 more messages. Several categories were a “visual scan for sanity and then delete all” sort of thing.

My emails never left my network. I didn’t rely on a cloud AI to process them. I didn’t contribute to global warming (this may have even been a case of saving energy, since it no doubt will offset quite a bit of manual time that would keep screens and room lights energized and so forth). I used about as much energy as watching a movie on a TV.

Did it complete the task for me entirely autonomously? Also no. AI isn’t a mind reader and it can’t possibly evaluate exactly what my thought process would be for a given task. But it can do a decent enough job to save me some time.

Still, this didn’t require hyperscaler datacenters. AI even runs on-phone (Google Translate being one of the most useful AI-driven apps I’ve ever seen, and it can run on-device).