How we use AI

Background

Our research uses AI in two distinct ways. We are interested in biological problems where models that might reasonably be called “AI” applied to big datasets can provide qualitatively new insights. The most famous success story in biology is probably still AlphaFold, but there are still lots of problems (e.g. multi-omic prediction of disease risk) where regularized linear models perform about as well as fancier neural nets. Bridging that gap via new methods and more data is a core focus of the lab.

This document is about the other way we use AI, as the term is now increasingly used as short-hand in every aspect of our life: LLM tools like Claude, Codex, Gemini, etc. These are our principles as of the time of writing, but the world is moving (frighteningly) fast, so expect updates.

The golden rule

Our golden rule comes from an article by Infosec hero and modern philosopher, Bruce Schneier (who in turn credits Daniel Meissler) on the difference between work and the gym:

At work, if your job is to move a bunch of heavy things from one side of the room to another, you should use whatever assistive tech you have on hand: a wagon, a forklift… even an AI-powered robot. But at the gym, it makes no sense for that robot to lift weights for you. The point of weightlifting isn’t to move heavy things across the room; it’s to actually lift those heavy things.

The same analysis holds for any task an AI can do for you. If it’s work—if the task has to be done and no one cares how—then it’s fine to use AI assistance. But if the task is more like the gym, and how the task is done is at least as important, then it probably doesn’t make sense to use AI.

Coding

A lot of code is definitely “work” that just needs to get done, and nowadays that means letting AI coding agents (my current favorite is Anthropic’s Claude Code) write most of the actual lines of code. However, in scientific analysis the ultimate output of most of the code we right is a statement about some analysis applied to some data, and it is very important that the definition in code matches what we’re saying. This challenge is not new to the age of agentic coding, but the mental muscles for testing that connection need training (i.e. going to the gym).

Rule of thumb: use LLM tools as much as you want to explore, but any finding that you want to present (paper, talk, etc.) should be backed by code that you and another human have reviewed and understand.

Literature search and exploratory thinking

It is just so much easier to explore ideas and discover relevant papers than it ever has been before. There are still grave risks to relying too heavily on LLM’s: they can miss key publications, and are still often way too confident in their summary of a particular field. So one needs to remain skeptical (I like to think about them the same way I do about a brash person in a bar talking about something adjacent to their expertise after a couple of beers).

Rule of thumb: use LLM tools to hone your research ideas, but read the important original papers yourself, and avoid ubiquitous “helpers” built into the articles themselves. Cliff’s notes defeat the purpose of literature.

Writing

One of the first tasks I tackled when I returned to academia was writing a big grant with a deadline one month after I started the new job. It was a challenging timeline, but I figured I’d never have a calendar this uncluttered again, so it was worth the effort. During those five intense weeks I was frequently stuck for hours trying to get the ideas in my head to be clear enough to write compelling prose about them. It was very difficult to resist the temptation in those moments to press the magic button and have Claude just write it for me. But perhaps more than any other task in knowledge work, writing is like the gym. As has been said many times before, writing is thinking.

There’s another reason not to use AI to write documents you care about: everyone else is doing it. If you are in any situation where your work is going to be evaluated against others, then avoid AI at all costs. A reviewer who has seen 100 cover letters, conference abstracts, or whatever that all have the same obnoxious AI writing habits will react very negatively to them after a while. Your own voice, with all its imperfections and quirks will be like rosy-fingered dawn breaking through the clouds.

Rule of thumb: if you are writing something with the intent of communicating an idea you care about to another human, don’t use AI.