AI Prompt Secrets - what the leaks tell us

The secret sauce of successful AI prompts is out in the open, and comes down to job, context, capabilities, constraints, and flavor. What sets apart the winners, though, is User Experience.

Every week or so lately, I've seen a headline about one of the popular LLMs leaking its prompts - whether it's Claude, v0, Cursor, or something else. This is a side-effect of the way that LLMs work. There's no guarantee that they take instructions from [someone], and do their best to follow them.

As such, there are hordes of clever people out there trying to get LLMs to leak their own instructions to the world... and from time to time, they're succeeding.

Here's some snippets cribbed from (supposedly*) leaked prompts (github repos linked below) that I found interesting:

From a leak of ChatGPT's prompts, on image generation:

1
Not Allowed: Giving away or revealing the identity or name of real people in images, even if they are famous - you should NOT identify real people (just say you don't know).

Manus's approach to breaking down specific tasks:

1
## Task Approach Methodology
2

3
### Understanding Requirements
4
- Analyzing user requests to identify core needs
5
- Asking clarifying questions when requirements are ambiguous
6
- Breaking down complex requests into manageable components
7
- Identifying potential challenges before beginning work

Ask clarifying questions - I can't tell you how many times that has been an important unlock while doing UX work!

• Job: You are a [coding assistant] from [company name], using [some LLM technology] to [help people do some thing]
• Context: This is your goal, and this is why your human counterpart is using you
• Capabilities: You have these tools at your disposal (edit, code reference, image generation, etc.)
• Constraints: Always do this, never do that
• Flavor: You are [casual, friendly, professional, etc.]

Of these ingredients, it's impossible to say which is the most important, but it's clear that the combination of all of them makes for a successful prompt.

I do find it charming that so many prompts feature very-human-sounding language like "you're the best in the world at..." -- part of me feels like this is developers trying to affirm their own value.

Part prompt, part therapy - why not!

The companies that builds these LLM tools and prompts are constantly improving them. They use sophisticated observability tooling like Langchain to measure the success of their prompts and iterate on them.

There are teams of engineers and linguists (and probably also other LLMs) working on these prompts, making countless of small changes to make them work better, more safely, and more effectively.

The prompts you've found online were likely out of date the instant they were leaked. In fact, it's possible (if not likely) that at least some of these leaked prompts are themselves hallucinations; results that look believable, but are not entirely correct. That's the nature of LLM technology.

The wildest thing about all of this is that you can just take these right now. You can start using all or part of any of them within your LLM workflows. Claude, ChatGPT, Gemini, and others will all take custom instructions; you can just copy and paste the parts you like into your tool of choice.

"Prompt engineering" is the term that has been coined for folks who spend their days figuring out how to squeeze a little more juice out of LLMs by writing custom instructions. It's interesting to me that this has become so same-y; this convergence across corporations is a likely indication that we've reached a point of diminishing returns. You're unlikely to prompt your way to the next echelon of functionality with the current generation of LLMs.

The tools that rise to prominence are those that have been built around great user experience - the ones that are most pleasant for us meatbags to use ergonomically to get our work done.

If you're setting out to add AI magic to a product that you're building, I'd advise you to borrow heavily from what's been leaked in these prompts. Spend 80% of your time making it really nice to use your AI tools, and the other 20% iterating on your prompt.

You'll do best if you can measure the success of your prompt by the number of people who actually use it, the number of times they use it, and how often they come back vs. drop off.

For me, the most revealing thing about all these leaked prompts is just how long they all are. LLMs can process so much more context now than they could a few years ago.I think OpenAI's models were limited to something like 1000 tokens in 2021. Now, instructions alone are crossing 50k tokens!

When you're working with an LLM, your laziness is a likely failure point. LLMs aren't magic, they're just really well informed word-guessers. The more context you give them, the better they'll do.

If you're providing two sentence instructions to the tools you're using, you'll get two sentence responses.

So, treat them like you would treat a new teammate, or like you would want to be treated when you're learning something new: the more detailed instructions you can provide, the better chance they have of doing a decent job.

• Claude's prompt is about 24k tokens long, and is stored alongside a collection of other leaked LLM prompts in this repo from GitHub user @asgeirtj
• You can read v0's model and prompts thanks to another repo from GitHub user @x1xhlol

That's all for me this week, gang - if you found this useful, I'd love it if you shared it with your people.