Proving an AI product holds at 1,000 concurrent users
A real-time AI product taken to 1,000 concurrent sessions and 7,607 conversation turns with every failure counter at zero and latency flat across a tenfold concurrency increase.
The problem
Anyone can demo an AI feature that works for one user. The question that decides whether a product is real is what happens at a thousand — and answering it honestly means being willing to find out that it does not hold.
What we did
- 01
Ran a staged programme from 100 to 500 to 1,000 concurrent sessions, measuring at each step instead of jumping to the headline number.
- 02
Treated every green result as suspect until a stated discriminator proved the test could actually detect failure. A check that cannot report a failure is not a check.
- 03
Found and fixed a rate limiter that was permanently locking users out after thirty messages — a defect that every naive retry probe reported as healthy.
- 04
Exercised the real write path, not just the read path, which immediately exposed a roughly thirty-fold capacity gap that read-only testing had hidden completely.
- 05
Caught a configuration defect where a cloned test environment was pointed at production, before it could inject tens of thousands of synthetic records.
The transferable lesson
Not one defect this programme found was in the product — every single one was in the apparatus or the record. That is the argument for testing the test before you trust it. Most teams never learn what their software does at scale because their first real load test is an outage.