AI with ALLIE

The professional’s guide to quick AI bites for your personal life, work life, and beyond.

Hi there,

My Instagram Reel scripts and newsletters start with me, usually a long voice memo of what I actually think (and all my spicy hot takes). For a long time, the first draft from AI still came back a mess. Even the best model in the world gave me a jumbled pile of my own ideas, written in someone else’s voice for no reader in particular.

It's also one of the most common frustrations I hear from the enterprise marketing leaders I advise. They give AI seemingly everything, and it still doesn't sound like them or their brand.

The main fix is an eval. It's how I got AI to use a long voice memo I recorded to write an Instagram Reel script for a video that got 400K views, and it's one of the first things I teach in my AI Agent Mastermind.

An eval is basically a checklist

"Eval" is short for evaluation. Put simply, an eval lets AI grade its own work before handing it to you.

Most online guides on how to write evals are written for engineers, but you can steal this exact same idea for your everyday work—whether that’s business proposals, or team updates, or product launch video scripts. AI has no idea what “good enough” means for your work until you define it, and an eval, ladies and gents, is how you define it.

What goes in an eval

A good eval checklist has four parts:

1. Real examples. Show AI what great looks like. For Instagram Reel scripts, maybe you give it scripts that went viral or scripts that got a lot of positive feedback in your DMs. My eval uses these examples as the target to hit.

2. Pass or fail checks. Most eval guides, including those written by Anthropic employees, suggest keeping it under 10 pass/fail checks. My Instagram Reel eval has nine checks. So what counts as a check? For a team announcement email eval, one check might be "Did I thank at least one other person by name?"

3. Deal breakers. The things that can never ship. In an HR eval, that could be any employee data, any promised outcome, or a missing union sign-off.

4. The finish line. How AI knows it's done. For a job description eval: under 250 words, describes the company and team culture, lists three workflows the person will own, written in English and Spanish, both saved in Notion, and an email drafted to the recruiter. If you’re designing a product, a graphic, or a 3D world, you can also ask AI to deliver a final image or video.

Make every check pass or fail

Say your AI is replying to a customer support email. Before handing you its draft reply, it runs that reply through every pass/fail check in your eval. If even one check fails, your AI revises the draft and runs the entire checklist again from the beginning.

Why the whole list? Because one change can break something else. I call it the AI Butterfly Effect. You ask AI to make your email warmer, and suddenly, the refund details in paragraph two completely vanish. With an eval, by the time the draft reaches you, it has passed every check.

For this to work, aim for binary checks aka things that are either a yes or a no. "The design is strong" is way too subjective. "The design scores a 7 out of 10" doesn't work either, because AI is probabilistic, so the same design might score a 6 one time and a 7 the next. Everyone says to treat AI like an intern, and I can’t stand that analogy. I write every check for a tired teammate, or a star we just poached from a competitor: someone smart and capable who has zero patience for vague instructions.

Save the document edits you're making

Strap in, because this is an epic hack. For the last 15 years of writing, I'd start out with a crappy version, edit my way to the perfect email or blog post, and then hit publish. I used to think handing people my perfectly final sparkly versions was enough for them to “get it” and be able to replicate it later. But in fact, our deep-seated human judgment shows up in what we decided to edit. The draft was too long, so we cut it. Too formal, so we loosened it up. Not funny at all, so we made it a little funny. The delta matters.

Think of a home improvement show. You want to see the before, not just the after, and you want to hear why the designers made each call. So over the last several months, I've been giving AI my befores and afters, not just my finals. When AI can actually compare versions, it learns your judgment and applies it to the next task.

Start with your most consistent complaint

Not everything needs an eval (a one-off birthday party invitation for your uncle certainly doesn’t). Start with the work you do most often, the work where a mistake is most expensive, or the thing you're constantly complaining to AI about. You can even ask it, "Hey Claude, what do I keep yelling at you about?"

Anything AI can validate without your involvement belongs in an eval, and every eval frees up time for you to take on bigger projects, build relationships, and have more fun.

Build your first eval this week

Paste this into ChatGPT, Claude, Codex, or whichever AI tool you're allowed to use for work, along with five initial AI drafts and the final versions you actually sent:

“Treat this as an eval. Here are 5 drafts AI wrote for me and the versions I actually sent. Compare each pair line by line and list every change I made. Cluster my changes into patterns, and make a checklist based on the patterns you find, where every check is a yes-or-no question. Ask me for my deal breakers and how you'll know you're done. From now on, before you show me any draft, grade it against every check. If anything fails, fix it and run the whole checklist again. Only show me the draft once everything passes, with a one-line pass/fail for each check.”

Spot-check what it marks as passing for the first couple of weeks. AI grading its own homework is good, not perfect. Then save the eval where your AI will find it every time: in a skill, in a project, or wherever your AI says makes the most sense. I put a lot of my evals directly into my skill files. And extra bonus: use evals to measure the performance of new models, like yesterday’s Fable 5.2 release.

A few example checks that go way past "no typos":

• A quarterly business review: any miss over 10% names an owner and a recovery date.

• A weekly client update: no delivery date beats what engineering has confirmed.

• A family schedule: if any kid has Tuesday night plans, Wednesday karate is canceled.

For a couple hours of upfront work, I save hours every week.

The real goal is to never repeat yourself

As AI keeps improving, the goal is to stop prompting, and certainly to never repeat yourself.

We're heading toward a world where—even if you don’t talk to them directly—you'll have countless AI teammates working for you and your company around the clock. None of that matters if every output still requires your approval. If you're the bottleneck in your business, your growth is throttled and can only scale to 168 hours a week. And let’s be honest, most of that is going to sleeping, eating, and being stuck in traffic 😉

Your expertise has always lived in your head. Now it can live in a file that works while you sleep. Make that first eval, and let me know how it goes.

Stay curious, stay informed,

Allie

I’M HIRING. COME JOIN US.

Join a small remote team transforming the world of AI and business. We’re hiring people across growth, automation, writing, operations, business development - and you can even invent your own job and tell us why we need you. If you’re passionate about AI growing the capabilities and ambitions of non-engineers, take a look.

Feedback is a Gift

I would love to know what you thought of this newsletter and any feedback you have for me. Do you have a favorite part? Wish I would change something? Felt confused? Please reply and share your thoughts or just take the poll below so I can continue to improve and deliver value for you all.