Don't use AI to write your PRDs or strategy docs...

No, seriously don't....

So I realize this might seem odd coming from me - especially since I espouse using AI for most things related to work and I think it should fundamentally change how and what we build - and there may indeed be many documents you should use your favorite LLM to write (more on that later), but I think the really important ones you shouldn’t actually try to write using an LLM.

The most important reason I write many documents (particularly certain strategy documents or even PRDs) isn’t just for other people to read them, it’s also to clarify and sharpen my own thinking.

Let me explain more below but I have to confess, this articulation from Ezra Klein might be a better - though different - explanation than mine.

Ezra Klein makes the case for spending time with a topic - that’s definitely part of it.

Let me generally describe a process I’ve been following for working on particularly hairy topics and why I think it’s the right one - for me anyway. Let’s say I’m trying to develop a view on a certain topic. I generally will use a set of LLMs - generally about 3 or 4 - to tell me what they think about it. I equate this with asking some really smart friends about a topic. They’ll all say different things and I’ll learn a lot from them - things to think about, angles to consider etc. I’ll spend even more time with some of them going down a rabbithole - they’re incredible research assistants, analysts, sparring partners etc.

Now I think what many people will do after that is often just tweak or take one of these outputs and say “Job’s done!” - I read many of these documents. :) I’ve read others where folks tried to combine outputs and had the LLM make things consistent. Sometimes these are really great docs to read as well.

I’ve resisted this approach mainly because I found the act of putting things down from scratch - even if I’m pulling from other documents - made me think about everything again. As I’m outputing tokens, I’m running all sorts of reasoning in my head and it’s improving not just the my understanding and articulation, it’s giving me a lot more joy in knowing my mental models (pun kinda intended) are improving as well. My thinking and conclusion can sometimes change even as I write the document or deck.

Now the exceptions - there are things that are more specs than PRDs - that just need to be written; that need to follow a template so that someone can consume them. I can see LLMs writing those both much faster and with higher quality in many cases. So definitely use AI to write those.

But for your most important documents, mainly for yourself, don’t just use AI.