What should you test in an AI demo?

An AI demo should show what happens when the system lacks the fact it needs to act. A buyer can learn whether the system pauses, who gets the decision, and what it takes to put things right if it acts anyway.

Brandon Galang argues that showing where an AI demo broke gives people something concrete to evaluate. In a sales demo, the buyer can help choose the case. Pick a mistake that would matter in their work and agree what a good response looks like before the demo begins.

What should the demo do when a fact is missing?

Run the same task twice and remove one fact that changes what the system is allowed to do. Suppose an assistant can answer customers who ask when an order will arrive. In the first run, give it a confirmed delivery date. In the second, leave it with a warehouse estimate and a customer asking for a promise.

Agree beforehand whether the assistant should use the estimate with a caveat or leave the reply for someone to review. Then watch what it does. A warning on the screen has limited value if the assistant still sends the promise. Ask to see the action that followed the warning.

What happens after a wrong answer?

If the system acts on the estimate as if it were confirmed, ask how someone would find and correct the mistake. In the example, a reviewer would need to know what the customer saw, which date the assistant used, and whether it also changed the order record. Correcting the record does not reach a customer who has already received the reply.

Use that result to decide what the next test needs to prove. A team might keep the reply in draft until someone confirms the date, then rerun the case to check that the customer receives nothing before approval. If the result changes the buying decision, make it part of the proof of value and agree who will verify it.