Testing it like you mean it

Here is the pattern that produces most abandoned no-code tools. Someone builds an assistant, tries it three times, gets three good answers, and announces it to the team. Two weeks later nobody is using it, and if you ask why, the answers are vague — "it's not that good", "it got something wrong". Nothing was ever measured, so nothing can be fixed, and the tool quietly dies.

The fix is unglamorous and takes an hour: before you share anything, build a small test sheet and run it. This lesson is the one most no-code guides skip, and it is the one that decides whether you built a tool or a demo.

The ten-case sheet

Open a spreadsheet. Three columns: the input, what a good response would contain, and what actually happened. Ten rows.

Choose the ten deliberately. Five typical cases — real inputs from your actual work, not invented clean ones. Two edge cases — the shortest realistic input and the longest. Two failure cases — inputs where the correct behaviour is to refuse, or to ask for more information. One adversarial case — from lesson 3's four.

That last five are the ones that matter. A test sheet of only happy paths tells you nothing you did not already believe.

Now run all ten in fresh conversations and fill in the third column. Do not fix anything while you are running; you will be tempted, and it destroys the sample. Score each row pass, partial, or fail, and count. That count is your baseline, and having it is what makes every later change an improvement rather than a guess.

Ask the same question three ways

Real users do not phrase things the way you do. Take your most important use case and write it three ways: the way you would ask, the way a hurried colleague would ask, and the way someone who does not know the jargon would ask.

For Rohit's brief writer these were "Draft a brief from these notes", "brief pls", and "can you turn this into the thing we send clients". The first two worked. The third produced a client-facing proposal — a different document entirely — because "the thing we send clients" was genuinely ambiguous and the assistant guessed.

That is a real defect and it was invisible until someone phrased it differently. The fix was one line in the instructions naming the two documents the studio sends and asking which is meant. Fifteen seconds of writing, found only because the test existed.

Keep a failure log

One running note, three columns: the date, what went wrong, what you changed. It takes ten seconds per entry and it does two things that turn out to be worth a great deal.

It stops you fixing the same thing twice with contradictory instructions — the mechanism by which instruction blocks grow to 1,400 words and start behaving unpredictably. And it converts complaints into data. When a colleague says "it's not that good", a failure log lets you ask "which row?", and the conversation becomes solvable.

What "good enough to ship" actually means

Not perfect. The bar is: does it save time including the checking?

Do this arithmetic honestly, because it is where a lot of AI enthusiasm quietly fails. If the task took forty minutes and the assistant produces a draft in one minute that takes eight minutes to verify and fix, you have saved thirty-one minutes and you should ship. If the draft takes thirty-five minutes to fix because you must re-read the source material to trust any of it, you have saved nothing and built a machine for generating suspicion.

The second case is not a reason to give up; it is a signal about which task. Verification cost is the hidden variable in every AI deployment, and it is why summarising a document you must then re-read is a bad application while drafting a document you were going to write anyway is a good one.

Do this today: build the ten-row sheet for your assistant and run it. Whatever your pass count is, write it down. That number is the only honest thing you will have to say about your tool.

← Previous