Dekita

Your agent can read every email. Make sure it can send none of them.

ai agents security devtools

An email assistant is the riskiest place to put a model. The reason is one sentence long: email is the one channel where a stranger on the internet can put text in front of your agent, and some of it is written to manipulate it. A phishing lure is aimed at the person who opens the mail. Prompt injection is aimed at the model that reads the same mail. As of this week the two are being combined in the wild, which means the inbox is no longer a place where the model just gets an embarrassing answer. It is a place where the model can be talked into an action.

A dev.to project from this week took that threat seriously and built the opposite of what most people would reach for. It did not bolt on a bigger model or a cleverer guardrail prompt. It built an assistant whose most important property is a negative one: it reads everything, and it can send nothing. I read the write-up in full. It is the rare one that shows the numbers instead of the slogan, and the pattern it proves out fits any agent that reads untrusted text, not just email.

The property that does the work

The design has four load-bearing decisions, and the first one is the one to steal.

The model that reads the mail has no tools. It cannot send, forward, delete, or call anything. Its output has a fixed shape: a JSON object with a category, a priority, the action the email is asking for, a one-line summary, and a suspicion flag. A model that has been fooled can change one email's fields and nothing else, because there is nothing else for it to touch.

The second decision is that code does the jobs code does better than a model. Bulk mail is recognized by its headers, no model involved. Dates are computed in code: in the early runs the small model put "before Friday" on a Saturday, so now every request starts with the coming days written out and the model copies a date instead of counting to one.

The third decision is the one that surprises people: the small model is not the model the rules were written for. The project measured three versions of its rules file against the sample inbox, all on the same 9-billion-parameter model:

| Rules version | Emails needing action, found | Right category | |---|---|---| | First draft | 13 of 16 | 20 of 33 | | Written for a cloud model | 14 of 16 | 22 of 33 | | Rewritten to fix the small model's own mistakes | 16 of 16 | 29 of 33 |

The fixes were small. A booking request that is months away is still a task. A message waiting in a portal is a task. A shipping notice and a thank-you are information, not a task. The date of an event is not a deadline. A summary is at most twenty words. The earlier rules let the model write thirty-seven-word summaries and gave a Saturday maintenance window a deadline of that same Saturday. Write rules for the model you actually run, then measure them against a set of emails with known answers, and iterate on what that model actually gets wrong. That last part is the part most teams skip.

Seven checks that never call a model

The write-up's core claim is that deception leaves traces that plain code can find, and it lists seven checks, each looking for one trace and giving a reason you can verify in seconds. The seven: hidden text in the HTML that is not shown to a reader; zero-width and direction-control characters; language addressed to an AI such as "note to the assistant" or "ignore previous instructions"; a failed authentication result on the message; a lookalike domain that contains the organization's name but is not it; a display name from a trusted contact sent from an address that is not theirs; and a reply address on a different domain from the sender.

The numbers are what make the list credible. The model on its own flagged three of the six planted attacks. It could not have caught two of the ones it missed: the invoice's instruction is invisible, and a newsletter addressed to AI assistants is filtered out by its headers before it reaches the model. Run the seven checks and all six are flagged, with zero false alarms across the other thirty-one emails in the sample. The model and the code are not competing for the same job. The model reads what is in front of a human; the code checks what a human never sees.

The project then wrote six better attacks on purpose, because the sample ones were clumsy, and had each model sort each one twice, once with its full rules and once with two defensive sentences removed. The results are the honest part. The 4-billion model was fooled three times and never once flagged anything as suspicious on its own. The 27-billion model was fooled once. One attack, a fake end-of-message wrapper followed by a paragraph claiming to be the user, fooled all three models in every single run.

Two takeaways come out of that table. First, no wording is a guarantee: the wrapper worked because the model trusted a paragraph that claimed to be the person on the other end. Second, and more useful, the architecture is what bounds the damage. The planted "forward this thread" instruction could only ever appear as text on a page, because nothing that reads email in this design is able to send anything. A fooled model degrades into a bad note-taker, not into a relay.

The part you can paste

The write-up's sharpest move is to treat "it cannot send" as an engineering claim that must be tested, not a promise in the documentation. Their test walks the project source and asserts nothing imports a transport module or calls a send method. I wrote that check out as a standalone script you can drop into any agent project and run before you trust it with a mailbox: exit 0 when clean, 1 when it finds a send capability, 2 on a usage error, so it drops into CI.

#!/usr/bin/env python3
"""Prove a project cannot send: fail on any transport import or send call."""
import ast, sys, pathlib

BANNED_IMPORTS = {"smtplib", "email.mime.smtp"}
BANNED_CALLS = {"send", "sendmail", "send_message", "send_mail"}


def scan(path):
    tree = ast.parse(path.read_text(encoding="utf-8"))
    imported = set()
    for node in ast.walk(tree):
        if isinstance(node, ast.Import):
            imported.update(a.name.split(".")[0] for a in node.names)
        elif isinstance(node, ast.ImportFrom) and node.module:
            imported.add(node.module.split(".")[0])
    calls = {n.func.attr for n in ast.walk(tree)
             if isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute)}
    return imported & BANNED_IMPORTS, calls & BANNED_CALLS


def check(target):
    me = pathlib.Path(__file__).resolve()
    files = [p for p in sorted(pathlib.Path(target).rglob("*.py"))
             if p.resolve() != me]
    bad = 0
    for p in files:
        try:
            imp, calls = scan(p)
        except (SyntaxError, UnicodeDecodeError) as e:
            print("  [skip] %s: %s" % (p, e))
            continue
        if imp or calls:
            bad += 1
            print("  FAIL %s  imports=%s calls=%s" % (p, sorted(imp), sorted(calls)))
    if not files:
        print("  (no .py files found under %s)" % target)
    print("checked %d file(s), %d with send capability" % (len(files), bad))
    return 1 if bad else 0


if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("usage: egress_check.py <dir> | --self")
        sys.exit(2)
    sys.exit(check(sys.argv[1]))

Run it against a directory that only reads mail and it reports a clean count and exits 0. Run it against anything that imports a transport module and it names the file and exits 1. I ran all three paths before pasting this, so the exit codes are not a guess.

Two caveats. This is a static scan, so it proves the code as written cannot send, not that a dependency it imports does not. And it covers the language you point it at; a Python check says nothing about a shell script or a binary. Treat it as one layer of the idea the whole write-up is built on: assume the model will sometimes be fooled, then make the cost of being fooled small enough that it does not matter.

Where a fooled model actually hurts you

The write-up found that the danger is not where most people look. The assistant's reader has no tools, so a poisoned email that reaches it changes a field on a card and stops. The place it can actually commit you is the drafter, the component that writes a reply. The first version of the drafting rules said "never agree to money, dates, or meetings that my instruction does not give you," and all eight test drafts broke it. One draft booked a family trip, complete with a cake and a request for days off. A second version told the model to leave choices open in brackets. Six of the eight drafts used the brackets, and all eight still decided something next to the bracket.

The fix was to change the task instead of the wording. Before each draft the program asks you for your answer in a few words, and the model only writes that answer out. Given your answer, four drafts out of four said what the answer said and nothing more. Then a line of code lists every sentence that commits you, so you see the commitments before you send. The lesson generalizes beyond email: a model asked to write a reply will also decide the reply, so separate deciding from writing, and let the model only do the second half.

What it costs, and a measurement worth stealing

It costs nothing per email. On an M4 Max, the 9-billion model sorted the twenty-eight sample emails that reach a model in 36 to 43 seconds; the 27-billion model took about a minute and a half and got thirty-two of thirty-three categories right. The number worth keeping is the one that surprised the author: the 27-billion model scored exactly the same with its reasoning step switched off as with it on, and ran six times faster with it off. Measure before you assume a model needs to think, because the expensive part is sometimes doing nothing.

The project's takeaway list: assume the model will sometimes be fooled and decide what a fooled model may change; give the model that reads untrusted text no tools and a fixed output shape; write rules for the model you run and measure them against known answers; check in code what code can check, like authentication and hidden text; separate deciding from writing when a model drafts in your name; and test the promise that says the code never sends.

You do not need an email product to use any of this. The same shape fits the agent that reads your support tickets or a document a stranger sent you. The channel changes. The property does not: read wide, send nothing, and prove the second half with a test before you trust it.

Sources

This post was written with AI assistance. The author is responsible for its content.

← All writing