While speeding up the APIs on Smart Metrify, I put Redis in front of a lot of data. Latency improved, and then users started seeing old values right after changing them.
Try the version I shipped first.
These match for now. Top up the wallet to see what happens.
What went wrong
I had cached data that was written frequently, not only the data that was read constantly and rarely changed. Each read was faster, but a user could change something and immediately be shown the old value.
On a platform where people pay for electricity, a wrong number is worse than a slow one.
The fix
I traced the stale values to the cache and pulled caching back to the read-heavy paths only. On what remained, invalidation is driven by writes rather than waiting for a timer to expire. That’s more code to maintain, but it closes the window where a timer alone would keep serving old data.
Stale reads stopped, and the read-heavy endpoints kept their latency gains. Overall, the core APIs still went from about 480ms to under 200ms.
What I do differently now
A cache over frequently written data buys speed at the cost of correctness. Before caching anything, I now look at how often it’s written, not just how slow the read feels.
I took the same approach on HCMatrix, where Redis sits in front of the directory of employees and companies, because that data is read constantly and changes rarely.