{
  "version": "https://jsonfeed.org/version/1.1",
  "title": "Alex Rivera: blog",
  "home_page_url": "https://astro-agent-ready-portfolio.sohab.dev/blog",
  "feed_url": "https://astro-agent-ready-portfolio.sohab.dev/feed.json",
  "description": "Software engineer building developer tools and reliable backends.",
  "language": "en",
  "authors": [
    {
      "name": "Alex Rivera",
      "url": "https://astro-agent-ready-portfolio.sohab.dev"
    }
  ],
  "items": [
    {
      "id": "https://astro-agent-ready-portfolio.sohab.dev/blog/one-pipeline-forty-services",
      "url": "https://astro-agent-ready-portfolio.sohab.dev/blog/one-pipeline-forty-services",
      "title": "One pipeline, forty services",
      "summary": "What it took to move every team at Northwind onto one deploy path, and why the first deploy mattered more than any feature.",
      "content_html": "<p>At the start of 2025, Northwind had three ways to ship code. A bash script that only two people understood, a Jenkins job with forty parameters, and \"ask Priya\". By the end of the year, 42 services deployed through one pipeline: build, test, canary, promote. Nobody was told to move. This is what I think made the difference.</p>\n<div class=\"callout callout-tldr\" role=\"note\"><p class=\"callout-title\"><svg class=\"callout-icon\" viewBox=\"0 0 24 24\" width=\"16\" height=\"16\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\"><path d=\"M4 14a1 1 0 0 1-.78-1.63l9.9-10.2a.5.5 0 0 1 .86.46l-1.92 6.02A1 1 0 0 0 13 10h7a1 1 0 0 1 .78 1.63l-9.9 10.2a.5.5 0 0 1-.86-.46l1.92-6.02A1 1 0 0 0 11 14z\"></path></svg>TL;DR</p>\n<p>Make the first deploy take ten minutes. Copy the old scripts' good ideas. Publish the numbers every week, and let teams move when they are ready.</p>\n</div>\n<h2 id=\"start-with-the-boring-path\">Start with the boring path<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#start-with-the-boring-path\">#</a></h2>\n<p>The first version did less than any of the scripts it replaced. It built a container, ran the test suite, rolled it out to 5% of traffic, waited, and promoted. No custom steps, no plugins, no YAML anchors. One team, the search team, used it for a month while we fixed what they hit.</p>\n<p>That month mattered more than any feature. Every rough edge they found was one the next forty teams did not. By the time a second team asked, the answer to \"how do I start?\" was a single command:</p>\n<pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#FFA657\">npx</span><span style=\"color:#A5D6FF\"> shipline</span><span style=\"color:#A5D6FF\"> init</span><span style=\"color:#79C0FF\"> --service</span><span style=\"color:#A5D6FF\"> billing-events</span></span></code></pre>\n<p>It writes a <code>shipline.yaml</code> with the defaults, opens a pull request, and the first deploy runs when it merges.</p>\n<h2 id=\"steal-from-the-scripts-you-replace\">Steal from the scripts you replace<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#steal-from-the-scripts-you-replace\">#</a></h2>\n<p>The old scripts were ugly, but each one encoded a lesson. The bash script refused to deploy on Friday after 4pm. The Jenkins job posted to the team channel with the diff. \"Ask Priya\" checked the error budget before rolling out.</p>\n<p>All three made it into the pipeline as defaults. Teams that moved over did not lose anything they relied on, so the argument for staying shrank to \"we are used to it\". Habit is a real reason, so we did not fight it. We waited.</p>\n<figure class=\"media-figure\"><img src=\"https://astro-agent-ready-portfolio.sohab.dev/images/demo/projects/shipline.webp\" alt=\"A shipline run: four stages, the canary comparing v42 against v41\" loading=\"lazy\" decoding=\"async\"><figcaption>A run in progress. The canary stage compares the new version's error rate with the old one before promoting.</figcaption></figure>\n<h2 id=\"make-the-numbers-public\">Make the numbers public<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#make-the-numbers-public\">#</a></h2>\n<p>Every Monday a bot posted three numbers for every team: deploys last week, median time from merge to production, and rollbacks. Teams on the pipeline sat at about 12 minutes. Teams on the old scripts sat at hours or days.</p>\n<p>We never wrote a mandate. The chart did the persuading.</p>\n<figure class=\"media-figure\"><img src=\"https://astro-agent-ready-portfolio.sohab.dev/images/demo/blog/pipeline-adoption.webp\" alt=\"Bar chart of services on the shared pipeline by month in 2025, rising from 1 in January to 42 in December\" loading=\"lazy\" decoding=\"async\"><figcaption>Adoption by month in 2025. The service template in June made moving a single pull request.</figcaption></figure>\n<p>The jump in the middle of the year is the service template. Before it, moving meant editing your Dockerfile, your health checks and your alerts. After it, a new service started on the pipeline, and an old one moved in one pull request.</p>\n<h2 id=\"what-i-would-do-differently\">What I would do differently<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#what-i-would-do-differently\">#</a></h2>\n<ul>\n<li><strong>Ship rollbacks before canaries.</strong> We had the canary first, and the first bad deploy it caught still took eight minutes to roll back by hand.</li>\n<li><strong>Write the migration guide for the ugliest service first.</strong> The easy services moved on their own. The guide only mattered for the hard ones.</li>\n<li><strong>Count the people, not the services.</strong> Forty services is a nice number. The number I watch now is how many engineers deployed something this month without asking anyone.</li>\n</ul>\n<h2 id=\"where-it-is-now\">Where it is now<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#where-it-is-now\">#</a></h2>\n<p>The pipeline runs a few hundred deploys a week. An earlier post covers the part people ask about most: <a href=\"https://astro-agent-ready-portfolio.sohab.dev/blog/canary-deploys-without-a-mesh\">how the canary works without a service mesh</a>.</p>",
      "date_published": "2026-10-21T00:00:00.000Z",
      "tags": [
        "platform",
        "devtools"
      ],
      "image": "https://astro-agent-ready-portfolio.sohab.dev/blog/one-pipeline-forty-services/og.png"
    },
    {
      "id": "https://astro-agent-ready-portfolio.sohab.dev/blog/canary-deploys-without-a-mesh",
      "url": "https://astro-agent-ready-portfolio.sohab.dev/blog/canary-deploys-without-a-mesh",
      "title": "Canary deploys without a service mesh",
      "summary": "Weighted routing at the load balancer, two metrics and a ten-minute timer. A canary setup small enough to explain on a whiteboard.",
      "content_html": "<p>Every canary guide I read started with \"first, install a service mesh\". We have forty services and a small platform team, and a mesh would have been the most complicated thing we run. So we built the canary from parts we already had: the load balancer, Prometheus, and a timer.</p>\n<h2 id=\"two-deployments-one-weight\">Two deployments, one weight<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#two-deployments-one-weight\">#</a></h2>\n<p>A canary is two versions of a service running at the same time, with a small share of traffic on the new one. On Kubernetes that is two Deployments behind one Service. The share comes from the ingress controller, which supports weighted backends:</p>\n<pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#7EE787\">metadata</span><span style=\"color:#E6EDF3\">:</span></span>\n<span class=\"line\"><span style=\"color:#7EE787\">  annotations</span><span style=\"color:#E6EDF3\">:</span></span>\n<span class=\"line\"><span style=\"color:#7EE787\">    nginx.ingress.kubernetes.io/canary</span><span style=\"color:#E6EDF3\">: </span><span style=\"color:#A5D6FF\">\"true\"</span></span>\n<span class=\"line\"><span style=\"color:#7EE787\">    nginx.ingress.kubernetes.io/canary-weight</span><span style=\"color:#E6EDF3\">: </span><span style=\"color:#A5D6FF\">\"5\"</span></span></code></pre>\n<p>Shipline creates the canary Deployment with the new image, points this second ingress at it, and changes the weight in steps. It needs no sidecars, no new control plane, and no new on-call rotation.</p>\n<h2 id=\"the-gate\">The gate<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#the-gate\">#</a></h2>\n<p>Between steps, the pipeline asks two questions about the canary, compared with the stable version over the same ten minutes:</p>\n<ol>\n<li>Is the error rate more than 0.5 points higher?</li>\n<li>Is p95 latency more than 20% higher?</li>\n</ol>\n<p>If either answer is yes, the weight goes to zero and the run fails. If both are no, the weight moves to the next step: 5%, 25%, 50%, 100%.</p>\n<figure class=\"media-figure\"><img src=\"https://astro-agent-ready-portfolio.sohab.dev/images/demo/blog/canary-steps.webp\" alt=\"Step chart of traffic on the new version: 5% for ten minutes, then 25%, 50% and 100%, with a passed gate before each step\" loading=\"lazy\" decoding=\"async\"><figcaption>Each step holds for ten minutes. A gate between steps compares the canary with the stable version.</figcaption></figure>\n<p>We tried more metrics, but every extra one added false alarms without catching anything the first two missed. A CPU spike on a new version is usually a warm cache, not a bug.</p>\n<div class=\"callout callout-warning\" role=\"note\"><p class=\"callout-title\"><svg class=\"callout-icon\" viewBox=\"0 0 24 24\" width=\"16\" height=\"16\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\"><path d=\"m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3\"></path><path d=\"M12 9v4\"></path><path d=\"M12 17h.01\"></path></svg>Low traffic breaks the math</p>\n<p>At 5% of a service that gets ten requests a minute, the canary sees five requests in ten minutes. One error is a 20% error rate. For services under about 50 requests a minute, Shipline skips the 5% step and gates on synthetic checks instead.</p>\n</div>\n<h2 id=\"comparing-against-now-not-last-week\">Comparing against now, not last week<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#comparing-against-now-not-last-week\">#</a></h2>\n<p>The first version compared the canary with a fixed threshold. It failed every deploy during a partner outage, because the stable version was failing too. Comparing the canary with the stable version over the same window fixed that: if both are bad, the deploy is not the problem.</p>\n<h2 id=\"what-it-has-caught\">What it has caught<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#what-it-has-caught\">#</a></h2>\n<p>In five months the gate has stopped 23 deploys. Most were dull: a missing environment variable, a migration that ran before the code that needed it, a dependency bump that changed a timeout. One was a memory leak that only showed under real traffic, which no test would have found.</p>\n<p>The total cost is about 300 lines of TypeScript in the pipeline and one extra ingress per service. When someone asks how our canaries work, the answer fits on a whiteboard, and that turned out to be the main feature.</p>",
      "date_published": "2026-06-09T00:00:00.000Z",
      "tags": [
        "deploys",
        "kubernetes",
        "reliability"
      ],
      "image": "https://astro-agent-ready-portfolio.sohab.dev/blog/canary-deploys-without-a-mesh/og.png"
    },
    {
      "id": "https://astro-agent-ready-portfolio.sohab.dev/blog/the-on-call-handbook",
      "url": "https://astro-agent-ready-portfolio.sohab.dev/blog/the-on-call-handbook",
      "title": "The on-call handbook we actually read",
      "summary": "A twelve-page handbook, an alert review every Friday, and how platform pages went from 31 a week to 9.",
      "content_html": "<p>Last spring the platform on-call rotation was paged 31 times in a typical week, several of them at night. People traded shifts to avoid it. Twelve weeks later it was 9 a week, and nights were quiet most weeks. No new tools were involved. We wrote a short handbook and started deleting alerts.</p>\n<figure class=\"media-figure\"><img src=\"https://astro-agent-ready-portfolio.sohab.dev/images/demo/blog/pages-per-week.webp\" alt=\"Bar chart of pages per week over twelve weeks, falling from 31 to 9 after the handbook and alert review started in week 5\" loading=\"lazy\" decoding=\"async\"><figcaption>Pages per week. The handbook and the Friday alert review started in week 5.</figcaption></figure>\n<h2 id=\"the-old-runbooks\">The old runbooks<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#the-old-runbooks\">#</a></h2>\n<p>We had runbooks. They were in a wiki, written by whoever set up each alert, and most of them said \"check the dashboard and escalate if needed\". At 3am that tells you nothing.</p>\n<p>The problem was not missing documentation. It was documentation written for the person who already knew the answer.</p>\n<h2 id=\"one-page-per-alert\">One page per alert<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#one-page-per-alert\">#</a></h2>\n<p>The handbook has a page for every alert that can page someone, and every page has the same four parts:</p>\n<ol>\n<li><strong>What it means.</strong> One sentence, in words a new hire understands.</li>\n<li><strong>Is it urgent?</strong> What users are seeing right now, if anything.</li>\n<li><strong>First three steps.</strong> Commands you can paste, in order.</li>\n<li><strong>When to wake someone.</strong> A name and a condition, not \"if needed\".</li>\n</ol>\n<pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#79C0FF;font-weight:bold\">## Disk pressure on build nodes</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#E6EDF3;font-weight:bold\">**Means:**</span><span style=\"color:#E6EDF3\"> a build node is above 90% disk. New builds may fail.</span></span>\n<span class=\"line\"><span style=\"color:#E6EDF3;font-weight:bold\">**Urgent?**</span><span style=\"color:#E6EDF3\"> Only if the build queue is growing. Check the queue panel first.</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#FFA657\">1.</span><span style=\"color:#79C0FF\"> `kubectl get nodes -l role=build`</span><span style=\"color:#E6EDF3\"> and find the node.</span></span>\n<span class=\"line\"><span style=\"color:#FFA657\">2.</span><span style=\"color:#79C0FF\"> `shipline nodes drain &#x3C;node> --prune-images`</span></span>\n<span class=\"line\"><span style=\"color:#FFA657\">3.</span><span style=\"color:#E6EDF3\"> Watch the queue for five minutes.</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#E6EDF3;font-weight:bold\">**Wake someone:**</span><span style=\"color:#E6EDF3\"> the build-infra lead, if two or more nodes are above 90% at once.</span></span></code></pre>\n<p>If we could not fill in all four parts, the alert should not page anyone. That rule did most of the work.</p>\n<h2 id=\"the-friday-review\">The Friday review<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#the-friday-review\">#</a></h2>\n<p>Every Friday, the person coming off call spends thirty minutes going through every page from their week with one other engineer. Each alert gets one of three outcomes:</p>\n<ul>\n<li><strong>Keep.</strong> It was real and the page helped.</li>\n<li><strong>Fix.</strong> It was real, but the handbook page was wrong or missing. Fix it now.</li>\n<li><strong>Delete or downgrade.</strong> Nobody needed to wake up. Make it a ticket, or remove it.</li>\n</ul>\n<p>In the first four weeks we deleted 19 alerts and moved 11 to daytime tickets. Some people worried that deleting alerts would hide real problems. Nothing has slipped through so far, and the alerts that are left now get taken seriously.</p>\n<h2 id=\"why-people-read-it\">Why people read it<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#why-people-read-it\">#</a></h2>\n<p>The handbook is twelve pages long and lives in the repo next to the code, so changing an alert and its page happens in the same pull request. The old wiki had over a hundred pages, and nobody had read them all, including the people who wrote them.</p>\n<p>The best feedback came from a new engineer after their first shift: \"I got paged twice and didn't have to ask anyone anything.\" That is the whole goal of a handbook.</p>",
      "date_published": "2026-01-27T00:00:00.000Z",
      "tags": [
        "on-call",
        "teams",
        "writing"
      ],
      "image": "https://astro-agent-ready-portfolio.sohab.dev/blog/the-on-call-handbook/og.png"
    },
    {
      "id": "https://astro-agent-ready-portfolio.sohab.dev/blog/searching-ten-thousand-notes",
      "url": "https://astro-agent-ready-portfolio.sohab.dev/blog/searching-ten-thousand-notes",
      "title": "Searching ten thousand notes in 27 ms",
      "summary": "How a weekend tool for my own notes became notegrep, and what an inverted index buys you over grep.",
      "content_html": "<p>I have kept notes in plain Markdown files since 2016. There are 10,412 of them now: meeting notes, snippets, half-finished posts. For years I searched them with grep. Then the folder got big enough that grep took two seconds, and two seconds is long enough to lose your train of thought.</p>\n<p>This post is about the tool I wrote to fix that, and the one data structure that makes it fast.</p>\n<h2 id=\"why-grep-is-slow-here\">Why grep is slow here<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#why-grep-is-slow-here\">#</a></h2>\n<p>Grep reads every byte of every file on every search. Ripgrep is much smarter about it: it skips ignored files, uses SIMD and runs in parallel. On my notes it takes 410 ms, which is impressive. But it still reads everything, every time.</p>\n<p>The trick search engines use is to read everything once and remember where each word is.</p>\n<h2 id=\"the-inverted-index\">The inverted index<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#the-inverted-index\">#</a></h2>\n<p>An inverted index maps each word to the notes that contain it:</p>\n<pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span>postgres  → 112, 431, 2048, 7730, …</span></span>\n<span class=\"line\"><span>vacuum    → 431, 7730, 9120</span></span>\n<span class=\"line\"><span>autovac   → 7730</span></span></code></pre>\n<p>To find notes with \"postgres vacuum\", look up both lists and take the notes that appear in both. You never touch a file that does not contain the words. With sorted lists, the intersection is a single walk through each.</p>\n<h2 id=\"ranking\">Ranking<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#ranking\">#</a></h2>\n<p>Matching is only half of it. When 30 notes match, the best one should be first. notegrep uses BM25, the usual ranking formula. It scores a note higher when a query word appears often in that note and rarely in other notes, and it scales the score down for long notes so they do not win just by being long.</p>\n<p>Two small additions made the ranking feel right for notes:</p>\n<ul>\n<li><strong>Titles count three times.</strong> A note called \"Postgres vacuum\" is almost always the one you want.</li>\n<li><strong>Recent notes get a small boost.</strong> A note from last month beats an identical one from 2017.</li>\n</ul>\n<figure class=\"media-figure\"><img src=\"https://astro-agent-ready-portfolio.sohab.dev/images/demo/projects/notegrep.webp\" alt=\"Terminal running notegrep &#x22;postgres vacuum&#x22;: four ranked hits with highlighted matches, then index stats\" loading=\"lazy\" decoding=\"async\"><figcaption>Four ranked results in 27 ms, then the index stats.</figcaption></figure>\n<h2 id=\"keeping-the-index-fresh\">Keeping the index fresh<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#keeping-the-index-fresh\">#</a></h2>\n<p>An index is only useful if it is current. notegrep stores the size and modified time of each file, and on every search it checks whether anything changed. Changed files are re-read and their entries replaced. With a few edits a day, this takes a couple of milliseconds, so there is no daemon and no watch process.</p>\n<h2 id=\"the-numbers\">The numbers<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#the-numbers\">#</a></h2>\n<figure class=\"media-figure\"><img src=\"https://astro-agent-ready-portfolio.sohab.dev/images/demo/blog/search-benchmark.webp\" alt=\"Horizontal bar chart: grep 1,940 ms, ripgrep 410 ms, notegrep with a cold index 160 ms, notegrep 27 ms\" loading=\"lazy\" decoding=\"async\"><figcaption>One query over 10,412 notes, median of 50 runs.</figcaption></figure>\n<p>The whole index is 9 MB on disk and builds from scratch in 4 seconds. A cold search, where the index is read from disk for the first time, takes 160 ms. After that it takes 27 ms.</p>\n<pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#FFA657\">npx</span><span style=\"color:#A5D6FF\"> notegrep</span><span style=\"color:#A5D6FF\"> \"postgres vacuum\"</span><span style=\"color:#79C0FF\"> --top</span><span style=\"color:#79C0FF\"> 4</span></span></code></pre>\n<p>27 ms is fast enough that search feels like it is not there. That was the whole goal.</p>",
      "date_published": "2025-08-19T00:00:00.000Z",
      "tags": [
        "cli",
        "search",
        "performance"
      ],
      "image": "https://astro-agent-ready-portfolio.sohab.dev/blog/searching-ten-thousand-notes/og.png"
    },
    {
      "id": "https://astro-agent-ready-portfolio.sohab.dev/blog/idempotency-keys",
      "url": "https://astro-agent-ready-portfolio.sohab.dev/blog/idempotency-keys",
      "title": "Idempotency keys, explained",
      "summary": "Why a payments API needs them, the three cases a key has to handle, and the table that makes retries safe.",
      "content_html": "<p>At Brightpay I spent a good part of four years on one question: what happens when a client sends the same payment twice? This post is the answer I wish I had on day one.</p>\n<h2 id=\"the-problem-is-the-timeout\">The problem is the timeout<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#the-problem-is-the-timeout\">#</a></h2>\n<p>A client sends <code>POST /charges</code>. The API charges the card, writes the result, and sends the response. The response never arrives: a phone loses signal, a proxy times out. From the client's side, nothing happened. So it retries.</p>\n<p>Without protection, the retry charges the card again. Telling clients not to retry does not work, because retrying is the right thing to do when you do not know if a request succeeded.</p>\n<figure class=\"media-figure\"><img src=\"https://astro-agent-ready-portfolio.sohab.dev/images/demo/blog/retry-sequence.webp\" alt=\"Sequence diagram: the client&#x27;s first request charges the card but the response is lost; the retry with the same key returns the stored response\" loading=\"lazy\" decoding=\"async\"><figcaption>The retry carries the same key, so it gets the first attempt's answer instead of a second charge.</figcaption></figure>\n<h2 id=\"the-key\">The key<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#the-key\">#</a></h2>\n<p>The client makes up a random key per operation, not per request, and sends it with every attempt:</p>\n<pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#FF7B72\">POST</span><span style=\"color:#E6EDF3\"> /charges</span></span>\n<span class=\"line\"><span style=\"color:#7EE787\">Idempotency-Key</span><span style=\"color:#FF7B72\">:</span><span style=\"color:#A5D6FF\"> 7f3a9c2e-1b4d-4e8a-9f0c-2d6b8a1e5f37</span></span></code></pre>\n<p>The server stores the key with the response. When the same key arrives again, it returns the stored response and does nothing else.</p>\n<h2 id=\"three-cases\">Three cases<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#three-cases\">#</a></h2>\n<p>That sounds like a cache, but a key has to handle three situations, and the third is where bugs live:</p>\n<ol>\n<li><strong>New key.</strong> Insert it, do the work, store the response.</li>\n<li><strong>Finished key.</strong> Return the stored response, byte for byte.</li>\n<li><strong>Key in progress.</strong> The first attempt is still running. Return <code>409 Conflict</code> and let the client retry later. Do not wait, and do not start the work again.</li>\n</ol>\n<p>The table that handles all three:</p>\n<pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#FF7B72\">create</span><span style=\"color:#FF7B72\"> table</span><span style=\"color:#D2A8FF\"> idempotency_keys</span><span style=\"color:#E6EDF3\"> (</span></span>\n<span class=\"line\"><span style=\"color:#FF7B72\">  key</span><span style=\"color:#FF7B72\">          text</span><span style=\"color:#FF7B72\"> primary key</span><span style=\"color:#E6EDF3\">,</span></span>\n<span class=\"line\"><span style=\"color:#E6EDF3\">  account_id   </span><span style=\"color:#FF7B72\">bigint</span><span style=\"color:#FF7B72\"> not null</span><span style=\"color:#E6EDF3\">,</span></span>\n<span class=\"line\"><span style=\"color:#E6EDF3\">  request_hash </span><span style=\"color:#FF7B72\">bytea</span><span style=\"color:#FF7B72\"> not null</span><span style=\"color:#E6EDF3\">,</span></span>\n<span class=\"line\"><span style=\"color:#FF7B72\">  status</span><span style=\"color:#FF7B72\">       text</span><span style=\"color:#FF7B72\"> not null</span><span style=\"color:#FF7B72\"> check</span><span style=\"color:#E6EDF3\"> (</span><span style=\"color:#FF7B72\">status</span><span style=\"color:#FF7B72\"> in</span><span style=\"color:#E6EDF3\"> (</span><span style=\"color:#A5D6FF\">'started'</span><span style=\"color:#E6EDF3\">, </span><span style=\"color:#A5D6FF\">'finished'</span><span style=\"color:#E6EDF3\">)),</span></span>\n<span class=\"line\"><span style=\"color:#E6EDF3\">  response     jsonb,</span></span>\n<span class=\"line\"><span style=\"color:#E6EDF3\">  created_at   </span><span style=\"color:#FF7B72\">timestamptz</span><span style=\"color:#FF7B72\"> not null</span><span style=\"color:#FF7B72\"> default</span><span style=\"color:#FF7B72\"> now</span><span style=\"color:#E6EDF3\">()</span></span>\n<span class=\"line\"><span style=\"color:#E6EDF3\">);</span></span></code></pre>\n<p>The insert in case 1 is <code>insert ... on conflict do nothing</code>. If it inserted a row, this request owns the key. If it did not, read the row and you are in case 2 or 3. The primary key does the locking for you.</p>\n<div class=\"callout callout-important\" role=\"note\"><p class=\"callout-title\"><svg class=\"callout-icon\" viewBox=\"0 0 24 24\" width=\"16\" height=\"16\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\"><path d=\"M11.017 2.814a1 1 0 0 1 1.966 0l1.051 5.558a2 2 0 0 0 1.594 1.594l5.558 1.051a1 1 0 0 1 0 1.966l-5.558 1.051a2 2 0 0 0-1.594 1.594l-1.051 5.558a1 1 0 0 1-1.966 0l-1.051-5.558a2 2 0 0 0-1.594-1.594l-5.558-1.051a1 1 0 0 1 0-1.966l5.558-1.051a2 2 0 0 0 1.594-1.594z\"></path><path d=\"M20 2v4\"></path><path d=\"M22 4h-4\"></path><circle cx=\"4\" cy=\"20\" r=\"2\"></circle></svg>Check the request too</p>\n<p>Store a hash of the request body. If a key comes back with a different body, return <code>422</code>. It is a client bug, and returning the old response would hide it.</p>\n</div>\n<h2 id=\"the-part-outside-the-database\">The part outside the database<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#the-part-outside-the-database\">#</a></h2>\n<p>The charge itself goes to a card network, which is not in your transaction. If the process dies after the charge but before storing the response, the key is stuck in <code>started</code>.</p>\n<p>We fixed that with an outbox: the API writes the intent to charge in the same transaction as the key, and a separate relay talks to the card network and records the result. That pattern became <a href=\"https://astro-agent-ready-portfolio.sohab.dev/projects/outboxer\">Outboxer</a>, a small Go library I still maintain.</p>\n<h2 id=\"results\">Results<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#results\">#</a></h2>\n<p>Over the three years after we shipped keys, the API handled a few billion requests and produced zero double charges that we could trace to retries. Support went from weekly double-charge tickets to none. The keys cost one table and about two hundred lines of code.</p>",
      "date_published": "2025-05-12T00:00:00.000Z",
      "tags": [
        "backend",
        "reliability",
        "payments"
      ],
      "image": "https://astro-agent-ready-portfolio.sohab.dev/blog/idempotency-keys/og.png"
    },
    {
      "id": "https://astro-agent-ready-portfolio.sohab.dev/blog/hello-world",
      "url": "https://astro-agent-ready-portfolio.sohab.dev/blog/hello-world",
      "title": "Hello world",
      "summary": "A first post that doubles as the authoring guide for this site. Replace it with your own writing.",
      "content_html": "<p>This is the starter post. It shows what a post can do, and the source of this file is the reference. When you write your own first post, delete this one (or keep it as a draft with <code>draft: true</code>).</p>\n<h2 id=\"create-a-post\">Create a post<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#create-a-post\">#</a></h2>\n<pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#FFA657\">pnpm</span><span style=\"color:#A5D6FF\"> new:post</span><span style=\"color:#A5D6FF\"> \"My first post\"</span><span style=\"color:#79C0FF\"> --description</span><span style=\"color:#A5D6FF\"> \"One sentence about it.\"</span><span style=\"color:#79C0FF\"> --tags</span><span style=\"color:#A5D6FF\"> notes</span><span style=\"color:#79C0FF\"> --publish</span></span></code></pre>\n<p>That creates <code>content/posts/my-first-post.md</code>. The file name is the URL: <code>/blog/my-first-post</code>. Without <code>--publish</code> the post is a hidden draft, so you can write in peace and publish later.</p>\n<h2 id=\"frontmatter\">Frontmatter<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#frontmatter\">#</a></h2>\n<p>Every post starts with a block between <code>---</code> lines:</p>\n<ul>\n<li><code>title</code>, <code>description</code> and <code>published</code> are required. The description is used on cards, in search results and in feeds.</li>\n<li><code>published</code> can be a date (<code>2026-11-10</code>) or a date with a time (<code>2026-11-10T14:30</code>). Posts are listed newest first, and the time orders posts from the same day.</li>\n<li><code>tags</code> become topic pages and filter chips.</li>\n<li><code>projects</code> lists project names from <code>content/github-projects.ts</code>; the post then shows them as \"where I use this\" cards.</li>\n<li><code>cover</code> is optional: a finished 1200x630 JPEG under <code>content/public/</code>, with the title already on it.</li>\n<li><code>aliases</code> lists former slugs. If you rename a post, put the old name here and the old link redirects.</li>\n<li><code>draft: true</code> hides the post everywhere.</li>\n</ul>\n<h2 id=\"callouts\">Callouts<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#callouts\">#</a></h2>\n<div class=\"callout callout-tip\" role=\"note\"><p class=\"callout-title\"><svg class=\"callout-icon\" viewBox=\"0 0 24 24\" width=\"16\" height=\"16\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" aria-hidden=\"true\"><path d=\"M15 14c.2-1 .7-1.7 1.5-2.5 1-.9 1.5-2.2 1.5-3.5A6 6 0 0 0 6 8c0 1 .2 2.2 1.5 3.5.7.7 1.3 1.5 1.5 2.5\"></path><path d=\"M9 18h6\"></path><path d=\"M10 22h4\"></path></svg>Callouts</p>\n<p>GitHub alert syntax works: NOTE, TIP, IMPORTANT, WARNING, CAUTION and TLDR. Add a title after the type, as in this one.</p>\n</div>\n<h2 id=\"code\">Code<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#code\">#</a></h2>\n<p>Fenced code is syntax highlighted:</p>\n<pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#FF7B72\">export</span><span style=\"color:#FF7B72\"> const</span><span style=\"color:#D2A8FF\"> greet</span><span style=\"color:#FF7B72\"> =</span><span style=\"color:#E6EDF3\"> (</span><span style=\"color:#FFA657\">name</span><span style=\"color:#FF7B72\">:</span><span style=\"color:#79C0FF\"> string</span><span style=\"color:#E6EDF3\">) </span><span style=\"color:#FF7B72\">=></span><span style=\"color:#A5D6FF\"> `Hello, ${</span><span style=\"color:#E6EDF3\">name</span><span style=\"color:#A5D6FF\">}!`</span><span style=\"color:#E6EDF3\">;</span></span></code></pre>\n<p>A <code>prompt</code> block is a prompt readers can copy or open in Claude or ChatGPT:</p>\n<div class=\"prompt-box\"><div class=\"prompt-head\"><p class=\"prompt-title\">Try it with your agent</p></div><pre class=\"prompt-text\"><code>Explain what a callout is and when to use one in a blog post.</code></pre></div>\n<p>When readers need the same thing in different flavours, group the fences into tabs. Each label goes in brackets after the language:</p>\n<p><strong>npm</strong></p><pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#FFA657\">npm</span><span style=\"color:#A5D6FF\"> install</span><span style=\"color:#A5D6FF\"> zod</span></span></code></pre><p><strong>pnpm</strong></p><pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#FFA657\">pnpm</span><span style=\"color:#A5D6FF\"> add</span><span style=\"color:#A5D6FF\"> zod</span></span></code></pre><p><strong>bun</strong></p><pre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#FFA657\">bun</span><span style=\"color:#A5D6FF\"> add</span><span style=\"color:#A5D6FF\"> zod</span></span></code></pre>\n<p>Feeds show every block under its label, and the Markdown copy keeps this source. See the <a href=\"https://www.npmjs.com/package/rehype-code-group\">rehype-code-group docs</a> for rich tabs, vertical tabs and syncing.</p>\n<h2 id=\"figures\">Figures<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#figures\">#</a></h2>\n<p>A figure is a small Astro component whose script plays it while it is on screen. This one is registered in <code>content/figures/index.ts</code>:</p>\n<p><em>Figure: An example figure: write, check, ship, on a loop.</em> <a href=\"https://astro-agent-ready-portfolio.sohab.dev/blog/hello-world#figure-example\">Open the interactive figure</a></p>\n<p>To add your own, create a component next to <code>ExampleFigure.astro</code>, register it in that index with a caption and a text description, and put <code>::figure your-id</code> on a line of its own. Feeds and the Markdown copy use the caption and description instead of the component.</p>\n<h2 id=\"images\">Images<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#images\">#</a></h2>\n<figure class=\"media-figure\"><img src=\"https://astro-agent-ready-portfolio.sohab.dev/images/demo/blog/pages-per-week.webp\" alt=\"Bar chart of pages per week falling from 31 to 9\" loading=\"lazy\" decoding=\"async\"><figcaption>A titled image becomes a captioned figure</figcaption></figure>\n<p>Put image files in <code>content/public/images/</code> and reference them with a path starting with <code>/images/</code>.</p>\n<h2 id=\"before-you-publish\">Before you publish<a class=\"heading-anchor\" aria-label=\"Link to this section\" href=\"#before-you-publish\">#</a></h2>\n<p>Run <code>pnpm check-content</code>. It tells you what is missing or still a starter value.</p>",
      "date_published": "2025-03-04T00:00:00.000Z",
      "tags": [
        "meta",
        "writing"
      ],
      "image": "https://astro-agent-ready-portfolio.sohab.dev/blog/hello-world/og.png"
    }
  ]
}