Skip to content

How we're tracking models

New models land monthly. Some are better at one thing and worse at four. Some change their prompt grammar without changing their name. A tool that pretends this is settled will quietly give you last quarter’s advice, so we keep a record instead, and this page is how that record reaches you.

The single most important rule in how we track models, and the one most often skipped elsewhere:

Published and measured are different kinds of fact. A provider’s price page, documented character cap or stated aspect support is published: useful, authoritative about intent, and sometimes wrong about behavior. What we got back when we actually ran the model is measured. We never let one stand in for the other, and every entry carries the date it was established.

This matters because the gap is real. A model documented as honoring a requested size may return the source’s dimensions instead. A platform documented as ignoring negative phrasing may in fact produce the opposite of what you asked. You only find that out by running it.

How tracking becomes the agent’s choices

Section titled “How tracking becomes the agent’s choices”

The record is not a reference shelf. Three things flow straight into what the Prompt Agent does:

Model fit per format. Each format (product photography, social graphic, display ad, multi-scene video) resolves to models that are actually good at it, not to whatever is newest. Where there is no good answer we say so rather than routing you somewhere plausible. Today that means avatar and talking-head video, and video-to-video editing, are open gaps with no live model behind them.

Text-rendering strength. If your image has copy rendered in it (a headline on a social graphic, a label on a pack shot), that narrows the field sharply, because models differ enormously here. The agent weights it when copy is part of the brief.

Negative-prompt policy, which is the finding we would most like you to take away. Platforms do not agree on how to say not that, and the disagreement is not cosmetic:

What the platform wants Who What we do
A flag Midjourney emit --no a, b
Its own separate field Ideogram, Wan strip it out and hand you a second block to paste there
Positive rewriting Nano Banana Pro, GPT Image, FLUX 3 Image, Veo, Kling, Hailuo, Seedance, Grok “no cars” becomes “empty street”, never bare negation
Nothing at all Runway, Luma Ray, FLUX.2 rewritten or dropped entirely

That last row is the one that costs people work. Runway documents that negative phrasing “is not supported and may produce unpredictable or even opposite results”, and FLUX.2 states plainly that it does not support negative prompts. Pasting a prompt with “no clutter” into either is not a wasted instruction. It is an instruction that can actively work against you.

Every copy target carries what that destination actually accepts, so the Key Prompt is rewritten for where it is going rather than handed over unchanged:

  • Character caps. Kling composes to a 2,500-character ceiling with a margin, so a long brief is condensed rather than truncated mid-sentence.
  • Aspect syntax, or the absence of it. Midjourney takes --ar 16:9 in the prompt string. Runway has a picker instead, so nothing is emitted for it and the tip tells you to set it yourself.
  • Native audio. Veo takes dialogue as attributed lines in the prompt; most others do not, and the same sentence is wasted text there.
  • Edits that ignore your size. On an edit, Recraft returns the source’s dimensions and Kling Image 3 returns its own size in the source’s orientation. Neither reads a requested aspect or resolution. Rather than leave those controls looking functional, the Studio disables them on that path and says why.

One distinction worth holding on to: a platform control is only ours to handle if the platform parses it out of the prompt string. A dropdown in their interface is yours to set after pasting: we will tell you it exists, but we will not pretend a line of text can reach it.

Not every model can run every operation. Some take a reference image and some take none. Some can start from a still, some can start and end on one, and a few cannot start from text at all. That set changes when providers ship, and it rarely changes with any announcement.

We read what each model’s endpoints actually accept, not a list of flags kept by hand. The distinction matters, and we learned it the expensive way: our own records claimed one model took character references when its endpoint had no way to receive them, and claimed three others could start from text when they could not. A record maintained beside the thing it describes drifts from it. A record derived from the thing cannot.

So the Studio asks one reader, every surface asks the same reader, and a check in our build fails if any surface starts deciding for itself.

  • An option a model cannot run is grayed, with the reason on the card, not hidden. You can see the whole roster and why a choice is unavailable, rather than wondering where a model went.
  • Nothing becomes a dead end. Click a grayed model and the operation gives way, with a note saying so. Changing the operation never moves your model out from under you: the thing you just touched is the thing that wins.
  • Generate refuses an impossible pair by name, and says which models can run what you asked for.
  • A prompt you saved months ago reopens as you saved it. If that pair is no longer possible (a model retired, an endpoint withdrawn), you are told, in those words, and nothing on the record is changed until you decide.

Before this, a request could be sent to a model that quietly ignored part of it. You would choose a reference-driven shot, the references would be dropped on the way out, the job would run as though you had asked for plain text-to-video, and you would be charged the full price for it.

That is the worst shape a failure can take: not an error you can act on, but a result that looks finished and is not what you asked for. A refusal costs you ten seconds. A silent substitution costs you the generation, the credits, and the time it takes to work out why the output is wrong.

The matrix in our model register is generated from the same reader the Studio uses, and a guard compares the two on every commit. If the documentation and the behavior ever disagree, the build fails, so the table cannot quietly go stale the way a hand-written one does.

Models move. We would rather the product noticed than left you to.

We read the public blind-vote and benchmark aggregators, cross-check them against our own runs, and never take a ranking as the whole answer:

  • Arena.ai — blind-vote rankings broken out by category, including image editing and image-to-video. The category-level view is the useful part: a model can lead overall and still lose on product and commercial work, which is what most of our users are actually making.
  • Artificial Analysis — arena scores alongside measured speed and price, with separate boards for image editing and image-to-video. The price-and-latency axis is what a leaderboard alone will not give you.

Where they don’t reach, we say so. Midjourney is absent from the blind-vote datasets (its access model does not integrate with those pipelines), so its standing in our roster rests on the qualitative case for parameter control and art direction, not on arena data. That is not a mark against it. It is a gap in the measurement, and it should be labeled as one rather than quietly filled in.

Recommend a model we have not run. Researched additions stay clearly separated from the live roster until the endpoint and the price are verified. A model we have read about is not a model we can route you to.

Let a stale entry pass as current. Every record carries the date it was established, and we re-audit when a platform ships a major version, when a documented syntax stops appearing in its own docs, or when a surface is retired. We date things because we have been caught out: a platform sunset once sat in our list for about four months before anyone noticed. Dating it is what makes the next one visible.

Pretend a price is a fact when it is a reading. Where a cost came from a pricing page rather than a provider’s own data feed, it is marked unverified, and it stays marked until it is checked.