<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Database on Buro notes</title><link>https://notes.burelli.xyz/tags/database/</link><description>Recent content in Database on Buro notes</description><generator>Hugo</generator><language>en-us</language><lastBuildDate>Thu, 22 Oct 2026 01:11:58 +0200</lastBuildDate><atom:link href="https://notes.burelli.xyz/tags/database/index.xml" rel="self" type="application/rss+xml"/><item><title>SQL optimization</title><link>https://notes.burelli.xyz/posts/engineering/sql-optimization/</link><pubDate>Wed, 21 Oct 2026 00:03:01 +0200</pubDate><guid>https://notes.burelli.xyz/posts/engineering/sql-optimization/</guid><description>&lt;h1 id="sql-optimization"&gt;SQL optimization&lt;/h1&gt;&#10;&lt;p&gt;Two goals:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;reduce roundtrips&lt;/strong&gt; between the caller (e.g. the BE) and the DB&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;optimize the queries&lt;/strong&gt;&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;h2 id="measure-first"&gt;Measure first&lt;/h2&gt;&#10;&lt;p&gt;Don&amp;rsquo;t guess, find what is actually slow: slow query log, &lt;code&gt;pg_stat_statements&lt;/code&gt;, &lt;code&gt;EXPLAIN ANALYZE&lt;/code&gt; (runs the query and shows real timings, unlike plain &lt;code&gt;EXPLAIN&lt;/code&gt;).&lt;/p&gt;&#10;&lt;h2 id="reading"&gt;Reading&lt;/h2&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;Ensure indexes&lt;/strong&gt; on the columns used in &lt;code&gt;WHERE&lt;/code&gt;, &lt;code&gt;JOIN&lt;/code&gt; and &lt;code&gt;ORDER BY&lt;/code&gt;.&#10;&lt;ul&gt;&#10;&lt;li&gt;Indexes are not free: they slow down writes and take space. Index what your queries use, not everything.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Composite indexes&lt;/strong&gt;: column order matters, put equality filters before range filters.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Covering indexes&lt;/strong&gt;: include the selected columns so the DB doesn&amp;rsquo;t need to read the table.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Check how you wrote the query&lt;/strong&gt;: select only the columns you need, avoid functions on indexed columns.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Avoid N+1&lt;/strong&gt; (classic with ORMs): fetch related rows for the whole batch at once with eager loading or &lt;code&gt;WHERE id IN (...)&lt;/code&gt;.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"&gt;&lt;code class="language-sql" data-lang="sql"&gt;&lt;span style="display:flex;"&gt;&lt;span&gt;&lt;span style="color:#66d9ef"&gt;CREATE&lt;/span&gt; &lt;span style="color:#66d9ef"&gt;INDEX&lt;/span&gt; idx_orders_customer_created &lt;span style="color:#66d9ef"&gt;ON&lt;/span&gt; orders (customer_id, created_at) INCLUDE (total);&#10;&lt;/span&gt;&lt;/span&gt;&lt;span style="display:flex;"&gt;&lt;span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span style="display:flex;"&gt;&lt;span&gt;&lt;span style="color:#66d9ef"&gt;EXPLAIN&lt;/span&gt; &lt;span style="color:#66d9ef"&gt;ANALYZE&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span style="display:flex;"&gt;&lt;span&gt;&lt;span style="color:#66d9ef"&gt;SELECT&lt;/span&gt; id, total &lt;span style="color:#66d9ef"&gt;FROM&lt;/span&gt; orders&#10;&lt;/span&gt;&lt;/span&gt;&lt;span style="display:flex;"&gt;&lt;span&gt;&lt;span style="color:#66d9ef"&gt;WHERE&lt;/span&gt; customer_id &lt;span style="color:#f92672"&gt;=&lt;/span&gt; &lt;span style="color:#ae81ff"&gt;42&lt;/span&gt; &lt;span style="color:#66d9ef"&gt;AND&lt;/span&gt; created_at &lt;span style="color:#f92672"&gt;&amp;gt;&lt;/span&gt; &lt;span style="color:#e6db74"&gt;&amp;#39;2026-01-01&amp;#39;&lt;/span&gt;;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id="reading-many-rows"&gt;Reading many rows&lt;/h3&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;&lt;strong&gt;Use pagination&lt;/strong&gt;: never load the whole table at once.&lt;/li&gt;&#10;&lt;li&gt;&lt;strong&gt;Load in batches&lt;/strong&gt; (e.g. 500 rows per call), process them, then write the results in batch (see below).&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;Prefer keyset pagination over &lt;code&gt;OFFSET&lt;/code&gt;, it stays fast on big tables:&lt;/p&gt;</description></item></channel></rss>