Skip to content

The first version was one Excel script

A few days ago I announced my talk at Devopsdays Prague. This is the story behind one of its slides.

Before there was an audit, a sandbox or a talk, there was one small script. In May 2025 I needed to know which of more than a hundred feature flags were still referenced anywhere in an organisation's code. Not in one repository. In all of them.

The gap

LaunchDarkly has a code references feature, and it is genuinely useful. But it is organised by repository: each repository reports its own references, on whatever trigger it has been wired to. I needed one answer across the whole organisation, and that is not the shape the feature has.

GitHub could give me that answer. Its code search accepts an org: qualifier and looks across every repository at once. There was one catch, and I found it quickly: code search is only available through the REST API. GitHub's GraphQL API can search issues, repositories, users and discussions, but not code.

The script

So the first version was deliberately small: an Excel Office Script. A column of flag keys, one GitHub code search per key, and the number of matching files written next to each key. Zero matches turned yellow.

The script is public as a gist, with its token deactivated.

It was also one of the first times I let AI write a whole script end to end. The first prompt was simple. Then I ran it on the real list, and it stopped long before the end.

Code search is rate-limited far more tightly than the rest of GitHub's API. The fix took a few more rounds: read the rate-limit headers, wait until the window resets, carry on. It is not elegant, and it did not need to be. Once it could wait, it could run unattended, once a day or whenever I asked, and the only thing it depended on was GitHub.

Around the same time another team ran into GitHub's rate limits, and this was the code I showed them. A second script used the same technique against the LaunchDarkly API. That one stays private: I could not separate its internals from the idea cleanly enough to publish it.

The count was the easy part

Running the script was never the real work. The real work started with its output.

Every number had to be read and prioritised. For each flag worth removing, I opened the search, followed each reference to the code path behind it, and planned a manual session: a pull request that removed the check, then whatever became unused because of it, and sometimes the tangle that had grown around it. Removing one flag often took hours, and that was before review started.

The script made finding cheap. Nothing yet made removing cheap.

What I would share today

If I were publishing this today, I would not share the script. A script fixes the choices I happened to have: Excel, GitHub, one flag platform, one organisation. Yours will be different.

What travels better is a short series of prompts. Rewritten for your own tools, it looks roughly like this:

  1. "Write a script that reads feature flag keys from [my spreadsheet or CSV], runs an organisation-wide code search for each key in [my code host], and writes the number of matching files next to each key. Highlight the zeros."
  2. "Make it survive the code search rate limit: read the rate-limit headers, wait until the window resets, retry, and show progress while it waits."
  3. "Make it safe to re-run: clear the previous results first, mark failed lookups as failed instead of writing a number, and report how many rows were processed, skipped and failed, and how long it took."

There is a fourth prompt, and it is the one my talk on 5 October is about: what a zero in that yellow column really means. The slides will be on the talk page shortly.

Short link to this post: moudrick.net/blog/ff-script/