Twik went from a soft launch to its biggest signup day in one evening. These are the fixes we shipped that week, in the order we needed them.
Twik had been in a quiet soft launch since the end of May: about 170 accounts, a few a day, mostly testers. On the evening of 5 August, Indian time, it went public. 9,007 people signed up that evening. The next day (UTC), 124,134 more did. Every one of them tried the same thing first: hold the mic, say something, and see if it typed back.
The backend was a single Fastify service that took an audio chunk, sent it to a speech-to-text provider, and returned text. That shape did not change during the week. Almost everything we changed was about what happens when the people and the providers both show up at once.
1. Put a ceiling on every request
The first fix went in on the morning of the 6th: a hard body limit on audio uploads and explicit timeouts on the transcription routes. Before that, a slow upload or a provider that hung could hold a connection open for as long as the client was willing to wait, and under load that is how a fine server turns into a stuck one.
2. Paying users go first
Fifteen minutes later we added subscriber prioritisation and surge handling to the audio routes. When the queue is deep, a request from someone who pays is not behind a free trial. It sounds obvious, but it has to exist before the surge, because you cannot ask a paying customer to wait while you write it.
3. One upstream is a single point of failure
Speech-to-text providers throttle hard at launch volume. We were hitting 429s, and every 429 was a user staring at a spinner. By lunchtime the service had a small router in front of its upstream speech endpoints: requests rotate across them, and a 429 fails over to the next one instead of reaching the user. That afternoon each upstream got its own cooldown timer, so one that was just throttled sits out for a while instead of being retried straight away.
// The idea, not the production code.
const ready = upstreams.filter(u => u.coolingUntil < now);
const next = ready[i++ % ready.length];
try {
return await transcribe(audio, next);
} catch (err) {
if (isRateLimited(err)) next.coolingUntil = now + COOLDOWN_MS;
throw err; // the caller retries on the next upstream
}4. The detour that lasted an afternoon
Mid-afternoon on the 6th we containerised the service and moved it to Google Cloud Run, on the theory that autoscaling would take care of the rest. The Docker image needed three quick fixes (Node 22 for pnpm, skipping install scripts, and file ownership for the node user). By the evening we had removed the Dockerfile and the deploy script and gone back to running the service directly. The bottleneck was the providers, not our CPU, and more instances only meant more 429s.
Scaling the part that is not the bottleneck just moves the queue somewhere you can't see it.
5. Measure the thing that hurt
That night we added two admin endpoints: requests per second, and the historical peak. Over the next two days came user growth over time in UTC, live subscriptions, and per-model usage. On the 9th, with real traffic patterns to look at, the slow admin queries got compound indexes and parallel fetches, and the free-tier limits moved into environment variables so we could change them without a deploy.
What I would keep
- Limits and timeouts on every route that touches a third party, from day one.
- A priority lane for paying users, built before you need it.
- Failover with per-upstream cooldowns for anything that can throttle you.
- Peak and per-second metrics before launch, not after.
Twik passed 500,000 accounts ten weeks after that evening. The service is still Fastify on EC2.