Engineering
Running a handful of automations is one challenge. Running millions, for thousands of teams, is another. As Shard's volume has grown, our engineering team has learned a few hard lessons about keeping the run engine fast and dependable.
Queue everything
A burst of deploys or messages shouldn't slow everything else down. Every trigger goes into a queue and is processed in order, so spikes are absorbed instead of passed on, and no event is lost if a worker restarts.
A reliable automation is one you can forget about, until it matters.
Make retries safe
Retries only help if running a step twice is harmless. We give every event a unique key so a repeated delivery runs the automation once, and every external call is designed to be repeated without side effects.
Observe everything
You can't fix what you can't see. Every step records its input, output, and duration, so we can trace a slow run or a failure in seconds, and customers can see the same detail in their own run history.
Ahmed Samir
Engineering Lead
Builds the infrastructure for what's next.
Similar Posts
