I used Claude Code and GSD to build the accessibility tool I’ve always wanted
Because of a severe mobility impairment—spinal muscular atrophy—I use a Mac for almost everything I do, and I have a particularly unique way of interacting with it. One of my biggest challenges—aside from typing—is the rather mundane act of scrolling.
I’m unable to manipulate a scroll wheel or perform a swiping gesture. That means the usual methods for scrolling on a mouse or trackpad are out. The method I do use is probably something you haven’t thought about in a long time—I click on and drag the scrollbar.[1] In most instances, it’s not too bad. But it can be very annoying in cases where the scrollable area is very long or in newer apps that tend to hide the scrollbar when not in use.
For years now, I’ve been able to ease my scrolling burden with a fantastic browser extension called ScrollAnywhere. It works great on most web pages and I do use the web a lot. That said, there are a ton of places where it doesn’t work. Even in the web browser, it doesn’t work in reader mode and it doesn’t work in browser settings views and other built-in browser pages. And obviously it doesn’t work in other apps or in the operating system itself.
Several months after posting about my scrollbar woes, Ryan Ohs sent me an email with a delightful surprise. He had been learning about Apple’s Accessibility API and after reading my post he put together this quick and dirty drag and scroll program that worked at the operating system level. It was a cool experiment. And while I was grateful for the gesture, it didn’t turn out to be practically usable in my day-to-day workflow.
Time passed. I helplessly watched as app after app shipped with inaccessible scrollbars. I made do with a cumbersome workaround—macOS has a dwell action for scrolling. The way it works is when you enable dwelling, you hover your mouse over where you want to scroll and wait for a short delay. Some UI appears with arrows which you can then hover your mouse over to scroll the area in that direction. It’s slow and annoying but effective when needed.
In other cases where apps are built with customizable web technology—like Obsidian—I was able to write CSS styles to make scrollbars large and in charge.
Fast-forward to now. AI is everywhere. Many developers I follow are blogging or making videos about the moment they realized AI is a force to be reckoned with. Other developers I follow loathe the technology, citing ethical concerns over environmental impact and intellectual property theft. I have a ton of cognitive dissonance. It’s amazing and horrible all at the same time.
I would say that 90% of my current AI use is Copilot autocomplete in VS Code. This saves me a ton of time because I’m using an onscreen keyboard, pecking out code character by character. Yes, there is standard editor completion, especially when you’re using typed languages. But AI autocomplete goes far beyond, inferring what I want to do based on just a handful of characters. I can often complete an entire line or perhaps an entire code block after a half-dozen keystrokes in a fraction of the time it would have taken me to enter it manually.
The remaining 10% (I’m totally guesstimating here obviously) of my usage is queries I make to ChatGPT. This is usually stuff where I expect my query has one correct answer based on a consensus. Otherwise, I will turn to searching the web with Kagi. I’ve dabbled with things like Claude Code on small side projects, mostly as an AI dev assistant. I haven’t done yet what people call vibe coding.
There’s a new kind of coding I call “vibe coding”, where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. It’s possible because the LLMs (e.g. Cursor Composer w Sonnet) are getting too good. Also I just talk to Composer with SuperWhisper so I barely even touch the keyboard. I ask for the dumbest things like “decrease the padding on the sidebar by half” because I’m too lazy to find it. I “Accept All” always, I don’t read the diffs anymore. When I get error messages I just copy paste them in with no comment, usually that fixes it. The code grows beyond my usual comprehension, I’d have to really read through it for a while. Sometimes the LLMs can’t fix a bug so I just work around it or ask for random changes until it goes away. It’s not too bad for throwaway weekend projects, but still quite amusing. I’m building a project or webapp, but it’s not really coding - I just see stuff, say stuff, run stuff, and copy paste stuff, and it mostly works.
Until now.
The idea of vibe coding was previously off-putting to me. There are several reasons for that. One of which is that I enjoy coding. Another reason is responsibility and accountability. If I’m going to put something out there for people to use, particularly if I’m charging for it or promoting it, I want to know how the code works. I also don’t really enjoy working by chatting and doing multiple follow-up prompts. I don’t want to have to include all the right magic words to make the AI do what I’m talking about. I don’t want to be a prompt engineer.
But then I came across this post by Jeffrey Zeldman. He linked to an article about building an app from a Figma mockup. In that article, the author mentions using Claude Code and Get Shit Done, a “meta-prompting system” that makes Claude Code behave a certain way and essentially encapsulates context management and prompt engineering so that you don’t have to think about it.
I was intrigued and I had an idea. Maybe my scrolling problem was solvable after all. This was the perfect candidate for a vibe coding project. I didn’t have the mental bandwidth to learn enough Swift and native macOS programming to do it on my own. I’d considered hiring other developers to make assistive tech software for me before, but it never went anywhere, usually because of lack of interest. This seemed like a small enough project that AI could reasonably handle it.
And handle it, it did.
Introducing Scroll My Mac
Not only was I able to put together this native app for scrolling at the OS level, I was able to add a ton of quality of life features that make it really nice to use. Most people are not going to find this app useful. I get that. But it’s incredibly useful for me and a handful of people I know who have tried it.
The main purpose of Scroll My Mac is to allow me to click and drag my mouse over any area to scroll it like you would on a phone or tablet—no reliance on a scroll wheel, a finger swipe, or manipulating a tiny scrollbar. It achieves this goal with a handful of other features that make it more practical.
- You can toggle scroll mode with a keyboard shortcut or by the menu bar icon.
- If you enable click-through, you can still click on things normally.
- By default, doing a dragging action will invoke the scrolling engine. That means you cannot drag to select text, move windows around, or do drag-and-drop actions on a web page. For this reason there is an optional setting called click-and-hold passthrough. It could use a better name, but what it does is allow you to click and hold the mouse button down, keeping it stationary, for a short delay. After that delay, the scroll engine will deactivate and allow you to perform a drag action as you normally would. The delay is adjustable. This quality of life feature is necessary if you want to keep scroll mode on all the time.
- You can reconfigure the hotkey to be whatever key combo you want.
- You can have Scroll My Mac launch at login.
- If you’d like certain applications to be ignored by Scroll My Mac, you can add those to an excluded apps list in settings.
I’ve only had it for a few days and I’m already blown away by how much easier it is for me to use certain applications.
How I built it
Well, I didn’t really build it at all of course. This project was 100% vibe-coded.
I already had Claude Code installed. I followed the instructions for installing GSD. I should put a disclaimer here that I don’t think using this directly on your machine is a super great idea. I did it. But I think if I were doing a non-macOS project with it, I would install it in a VM. You’re probably relatively safe using Claude Code only, but once you start including prompts from external sources that you aren’t really monitoring, that’s when you set yourself up for prompt injection attacks.
Anyway, once I had everything set up, I ran the new project command. Right off the bat, what was fascinating about this process is that Claude began asking me a ton of questions about my project. It made me think of aspects of it I hadn’t considered.
Throughout the process, I would do a good bit of back and forth describing what I wanted and answering follow-up questions about it. Everything I discussed with the AI went into markdown files in the project. GSD runs its own software development lifecycle. There’s a discuss phase, a planning phase where it may do some research and map out a detailed plan, and the execute phase where the actual code is written. Lastly, it will do a verification phase. This is where it will ask you to verify things and describe issues. At every point in this process, the current state of things is written to markdown files. This means you don’t lose context over multiple chats.
One by one, we completed phases of development. First creating a settings window and keyboard shortcut. Then doing the basic scroll functionality. Then adding inertial scrolling. And so it went, discussing milestones and letting the AI do its thing.
I have the Claude Code Pro plan, which is $20 per month. I blew through my included usage pretty rapidly and was routinely dipping into extra usage. The entire project took roughly 8 hours (done over a few days) and cost me about $80 worth of extra usage. I believe this could have been mi less if I had not been using GSD and leaning into its whole planning phase (as opposed to using its quick task mode, which gives similar benefits with a bit less work on the planning and discussion phases).
Still though, if the app I ended up with was on the Mac App Store for $99, I would buy it in a heartbeat. I don’t know if this app could make any money or not, but I’m not charging for it. I’m making it available on GitHub for anyone who might need it. You don’t need to build it from source. I’m uploading releases as I get them out. I do have an Apple Developer account (I have an old Electron app called d20 on the App Store), so I was able to code-sign and notarize it. That means you can download it and run it without a bunch of security warnings. Still though, just know that the code was 100% written by AI and use it at your own risk.
What does it all mean?
There is a feeling of existential dread permeating the industry. It’s countered by what is possibly the biggest hype machine I’ve ever seen. I’ve felt both extremes of this over the past three or four days.
On one hand, I think it’s absolutely incredible to think that users with all kinds of different disabilities could create their own assistive technology by describing to AI what they need. It is hard to describe how slow that bottleneck is sometimes. There is an industry built up around assistive tech, but it often is either too expensive or doesn’t do what the person needs. And lots of things just don’t exist because there isn’t sufficient motivation to work on them (for example, the onscreen keyboard situation on Linux completely blows).
But now these apps can exist. All it takes is the motivation of one person to spend a day or so working with an AI to develop an assistive technology that exactly meets their needs. Right now, you have to be a developer to use this stuff. But it’s not hard to imagine this technology becoming more mainstream and easier for non-developers to use.
And that’s the other side of it—the existential dread. Is AI going to replace me, the developer? What will I do instead? My only skills are knowledge and information work (coding and design). I can barely move a few fingers on my hands, so I’m not going to be getting a manual labor job anytime soon. If we are to believe the biggest of the AI hype men, then it won’t matter because we will be living in a land of plenty where no one will have to work. Somehow I doubt that’s the case.
Until then, I continue to dabble in order to hopefully keep myself relevant. Only in personal projects, though. Vibe-coding is expressly forbidden in my current role and I’m grateful for that. We’ve been told that we are responsible for the code we ship and that means understanding why and how it works. It doesn’t mean I don’t get the occasional AI assist. But the way I use it is more akin to the way I used to use Stack Overflow.
I don’t know what’s going to happen, but I guess I’m joining the refrain of many a developer that has said it before me—it seems like this shit is here to stay.
You will see frequent use of the em dash in my writing. That’s not because AI wrote it for me, it’s because I love the em dash. I say don’t let AI claim our em dash and our sparkle ✨ emoji. ↩︎