This post revisits Crunchy posts in the “load, storage, indexes, and partitioning” bucket: what we wrote, which version moved the needle, and what we would tell you to do now. Along the way: async I/O, more resilient COPY, LZ4 by default, richer BRIN shapes, skip scan, and smoother partition operations.

Async I/O: faster scans and vacuum on modern storage

In 2019 we benchmarked a BRIN index against a B-tree and a parallel sequential scan on the same time-series table. Sometimes BRIN won. Sometimes the parallel seq scan won: four workers chewing through the heap beat a clever index. That was the right lesson for Postgres 11: indexes are a tradeoff against what the executor can already do in parallel.

Postgres 18 made those heap reads substantially faster.

Async I/O lets backends queue multiple disk reads instead of waiting on each one. Sequential scans, bitmap heap scans (the path BRIN and many bitmap index plans finish with), and vacuum all benefit. Community benchmarks have shown up to ~3× on cold, latency-bound storage, a big deal for cloud disks. Defaults matter here: io_method = worker is on out of the box; on Linux 5.1+ you can try io_method = io_uring. See Get Excited About Postgres 18 for the operator view.

Postgres 19 builds on that: I/O workers can autoscale (io_min_workers / io_max_workers), read-ahead scheduling improved, and EXPLAIN (ANALYZE, IO) can show what the async subsystem is doing. Parallel query is still there; each worker can queue several reads and keep making progress while some of them are still in flight, so you get more useful work between waits. Parallel autovacuum workers also landed in 19 (autovacuum_max_parallel_workers and per-table autovacuum_parallel_workers), so maintenance can fan out, but the defaults are conservative: tune them when vacuum is falling behind on large tables.

One related default flip: JIT is off by default in Postgres 19 (it had been on since 12). The old cost model was unreliable, so large analytical or parallel scans that used to compile at runtime no longer will unless you turn jit back on. If those workloads matter to you, re-enable explicitly and re-check plans after upgrade.

Postgres 19 advice: Keep choosing indexes for selectivity, and re-test BRIN-vs-parallel plans on your storage with EXPLAIN (ANALYZE, BUFFERS, IO) after you upgrade. Tune effective_io_concurrency / maintenance_io_concurrency with the new defaults in mind (they rose to 16 in 18). Treat async I/O as more headroom for the heap paths, alongside good COPY, TOAST modeling, BRIN, and covering indexes.

Loading data: COPY is still king, and now more resilient

What we wrote: In Fast CSV and JSON Ingestion in PostgreSQL with COPY (2018, Postgres 10), Jonathan Katz showed the classic pattern: generate CSV or newline-delimited JSON, pipe it into psql, and let COPY ... FROM STDIN do the heavy lifting. Prefer COPY over row-by-row INSERT. Store JSON as jsonb. Add a GIN index when you need containment queries.

That advice was right then and is still right now. The load path itself did not need reinventing; it got more capable.

What changed:

Version	Change that matters for loads
16	COPY FROM can map a sentinel string to a column DEFAULT
17	ON_ERROR ignore skips bad type conversions and keeps loading; LOG_VERBOSITY reports what was skipped
18	REJECT_LIMIT caps how many bad rows you will tolerate; LOG_VERBOSITY silent quiets the noise; CSV handling of \. is clearer
19	Faster text/CSV parsing via SIMD; ON_ERROR SET_NULL turns invalid values into NULL; skip multiple header lines; COPY TO can emit JSON (and a single JSON array with FORCE_ARRAY) and can target partitioned tables directly

FREEZE on an initial load into a freshly created or truncated table is still the right performance trick when you want to skip a later freeze vacuum. That option predated these releases and remains useful.

Postgres 19 advice: Keep using COPY for bulk ingest, including imperfect feeds. A practical 19-era load looks like this:

COPY events (event_id, occurred_at, payload)
FROM STDIN
WITH (
  FORMAT csv,
  HEADER 2,
  DEFAULT '__DEFAULT__',
  ON_ERROR set_null,
  LOG_VERBOSITY verbose
);

HEADER 2 skips two lead-in lines (title row plus column names, or whatever your export prepends). ON_ERROR set_null keeps the row and nulls only the bad field instead of discarding the whole line. Prefer ON_ERROR ignore when a bad cell should drop the row entirely; pair that with REJECT_LIMIT when you want a hard cap on how many skips you will tolerate:

COPY events (event_id, occurred_at, payload)
FROM STDIN
WITH (
  FORMAT csv,
  HEADER 2,
  DEFAULT '__DEFAULT__',
  ON_ERROR ignore,
  REJECT_LIMIT 1000,
  LOG_VERBOSITY verbose
);

Here DEFAULT '__DEFAULT__' means: when the CSV cell is the literal string __DEFAULT__, use the column’s default expression. Pick any sentinel that will not appear as real data. (Avoid '\N' in CSV: readers often confuse it with the NULL marker.) Confirm final GA docs for edge cases around DEFAULT, ON_ERROR, and REJECT_LIMIT once 19 ships.

Prefer FREEZE for empty staging tables that you load once and then promote. And when you need to export partitioned data or hand JSON to another system, COPY TO handles the parent directly:

COPY events TO STDOUT WITH (FORMAT json);
COPY events TO STDOUT WITH (FORMAT json, FORCE_ARRAY);

Plain FORMAT json streams newline-delimited JSON (NDJSON/JSONL: one object per line), which is handy for batch pipelines. FORCE_ARRAY wraps the whole result in a single JSON array ([...]), which some web APIs expect.

The 2018 post’s JSON tip still stands: ingest into jsonb, not text json, unless you have a specific reason to preserve exact formatting.

TOAST: same mechanism, faster default compression

What we wrote: In Postgres TOAST: The Greatest Thing Since Sliced Bread? (2024), Elizabeth Christensen walked through the 8 kB page, the ~2 kB toast_tuple_target, storage strategies (PLAIN / EXTENDED / EXTERNAL / MAIN), and the practical guidance: updating a toasted row rewrites toast chunks, frequent access of compressed values has a cost, and large JSON or text that you query often belongs in a better data model, not in a giant toasted blob.

That mental model is still the right one. TOAST did not get replaced; it got a better default compressor.

What changed:

Version	Change
14	LZ4 becomes available via default_toast_compression = 'lz4' or per-column COMPRESSION lz4
19	LZ4 becomes the default (default_toast_compression flips from pglz to lz4); native REPACK (with CONCURRENTLY) rewrites tables without a long exclusive lock

We covered the compression decision tree in more depth in Postgres 19 Compression: from pglz to LZ4. The short version: LZ4 compresses and decompresses much faster than pglz, with a similar compression ratio, and still fails fast on incompressible data.

Postgres 19 advice: Keep Elizabeth’s modeling advice. Prefer structured columns for hot lookup data when you can. Prefer EXTERNAL when you have already compressed the payload yourself. On a fresh Postgres 19 cluster, LZ4 is what new toasted values get unless you override it, with no extra config step required.

After an upgrade from an older major version, existing toast values stay in whatever algorithm compressed them. New writes follow the new default. If you want to measure the win:

SELECT
  pg_column_size(payload) AS stored_bytes,
  octet_length(payload) AS raw_bytes
FROM events
WHERE length(payload) > 100
LIMIT 5;

Rebuild or rewrite only when storage or CPU on hot toasted columns is worth the effort. The default change already helps new data. When you do need a rewrite to reclaim bloat, Postgres 19’s native REPACK is the clean path; CONCURRENTLY keeps the table readable and writable while the new heap is built:

REPACK (CONCURRENTLY, ANALYZE) events;

CONCURRENTLY needs a primary key or index-based replica identity so Postgres can track and replay concurrent WAL updates, and it does not run on a partitioned parent (repack individual children instead). It also builds a full temporary copy of the relation, so plan on roughly 2× the table’s disk footprint while the rewrite runs. It is not MVCC-safe in the same narrow sense as TRUNCATE: a concurrent transaction that took a snapshot before the final swap and had not yet touched the table can briefly see it as empty. For blocking rewrites, plain REPACK still replaces the old VACUUM FULL / CLUSTER pair.

Neither plain nor concurrent REPACK recompresses existing toast values. After an upgrade, old pglz chunks stay pglz until those rows are updated (or you rewrite them some other way); new toasted writes follow the LZ4 default. Check with pg_column_compression() if you need to confirm.

BRIN: still tiny, richer opclasses, faster to build

What we wrote: In PostgreSQL BRIN Indexes: Big Data Performance With Minimal Storage (2019, Postgres 11), Jonathan showed BRIN crushing a B-tree on size for append-mostly sensor data: a 32 kB BRIN versus a 214 MB B-tree on the same timestamp column, with strong range-query performance when physical order and logical order lined up. Later posts (Avoiding the Pitfalls of BRIN Indexes and Paul Ramsey’s When Does BRIN Win?) sharpened when BRIN shines: high correlation with physical order. Random or shuffled layouts need a different shape of index.

What changed:

Version	Change
14	minmax_multi opclasses store multiple min/max values per range (great with outliers); Bloom BRIN opclasses work for equality on less-correlated data
17	Parallel CREATE INDEX for BRIN

The gains here are mostly from 14 and 17 (better opclasses and parallel builds). On 19, BRIN mostly inherits the surrounding I/O improvements: async reads speed the bitmap heap scan that follows a BRIN hit. The default single min/max summarizer remains a strong choice for clean append-only time series; when outliers show up, minmax_multi keeps ranges useful by tracking several min/max pairs per block range instead of one stretched span.

Postgres 19 advice: Start with Paul’s question: is this column correlated with physical order? If yes, use BRIN. On 14+, prefer multi-minmax when the table is large and not perfectly ordered:

CREATE INDEX scans_created_at_brin_idx
  ON scans
  USING brin (created_at timestamptz_minmax_multi_ops)
  WITH (pages_per_range = 32);

Tune pages_per_range to your typical filter width. The default is 128 (about 1 MB of heap per summary). 32 raises index resolution and tightens heap probes at the cost of a slightly larger (still tiny) index, which helps when time-series inserts have mild out-of-order jitter. For equality lookups on values that are not well ordered on disk (UUIDs, MAC addresses, some categorical keys), try a Bloom BRIN opclass, often enough without a full-size B-tree:

CREATE INDEX events_id_brin_bloom
  ON events
  USING brin (event_id uuid_bloom_ops);

On Postgres 17+, BRIN builds on large tables can use parallel workers automatically when maintenance_work_mem and parallel settings allow it. You do not need a special syntax: give the build enough memory and let it fan out.

And remember the 2019 twist ending: sometimes a parallel sequential scan beat BRIN. On 18+, async I/O speeds both the parallel seq scan and the bitmap heap scan that follows a BRIN hit. Re-run the bake-off on your storage, especially cloud disks, and pick the winner with fresh numbers.

BRIN is still the specialist for correlated, append-mostly data, not a general-purpose B-tree replacement. The upgrade is that you have more shapes to choose from: classic minmax, multi-minmax, and Bloom, plus parallel builds.

Covering indexes: INCLUDE still helps, and skip scan covers more queries

What we wrote: In Why Covering Indexes in Postgres Are Incredibly Helpful (2018, Postgres 11), Jonathan introduced B-tree INCLUDE columns so queries could satisfy SELECT lists from the index alone, including types that cannot be B-tree keys, like point. Vacuum the table so the visibility map allows true index-only scans. Be conservative: every included column costs write amplification and disk.

That tradeoff is unchanged. INCLUDE is still how you build a covering index without polluting the sort key.

What changed:

Version	Change
17	More efficient B-tree scans for multi-value lookups such as IN (...)
18	B-tree skip scan: a multicolumn index can be used when equality predicates hit later columns and omitted leading columns have low cardinality

Skip scan is the big unlock. An index on (visitor, visited_at) can now help more queries that filter on later columns when the leading column has low cardinality. Teams can often keep one well-designed composite index instead of adding a companion for every access pattern. We covered the mechanics in Get Excited About Postgres 18.

Postgres 19 advice: Still use INCLUDE when you know a hot query’s filter columns and its select list:

CREATE UNIQUE INDEX visits_visitor_visited_at_geocode_idx
  ON visits (visitor, visited_at)
  INCLUDE (geocode);

Still VACUUM (or let autovacuum catch up) before you expect Heap Fetches: 0.

For your other indexes, check whether skip scan already covers a query before you add another index, especially when the leading column is something like status, region, or tenant with a small number of distinct values. If an equality-only single-column index exists only to cover those “missing leading column” cases, try dropping it after you confirm the composite index serves the workload. Keep dedicated leading-key indexes for high-cardinality first columns and for inequality-driven access patterns skip scan cannot help.

Also lean on Postgres 17’s bulk-scan improvements: WHERE id IN (...) style lookups on B-trees got cheaper without any schema change. Re-benchmark before you invent a custom “optimize IN lists” pattern an upgrade already improved.

Partitioning: still for lifecycle, now smoother to operate

What we wrote: In Partitioning with Native Postgres and pg_partman (2022), Elizabeth Christensen framed partitioning as a scale and lifecycle tool: archive old ranges by detaching, create tomorrow’s child, let pg_partman handle the calendar. Prefer native declarative partitioning over trigger/inheritance, advice that goes back to Keith Fiske’s migrate-from-trigger-based guide (2020). Later posts sharpened the edges: default partitions as a safety net you monitor and drain, and retention with pg_partman so dropping a child returns space immediately instead of waiting on DELETE + vacuum.

The 2022 tutorial’s detach looked like this:

ALTER TABLE iot_thermostat DETACH PARTITION iot_thermostat07142022;

That still works. On Postgres 14+, you can do the same rotation online with CONCURRENTLY. The caveats section correctly noted that unique indexes generally must include the partition key. That part is still true.

What changed:

Version	Change
14	DETACH PARTITION ... CONCURRENTLY: online detach with lighter locking
15	Cleaner FK behavior when an UPDATE moves a row between partitions
17	Identity columns on partitioned tables; exclusion constraints when they equality-compare the partition key
18	VACUUM / ANALYZE ... ONLY on the parent; NOT VALID FKs on partitioned tables
19	Native MERGE PARTITIONS / SPLIT PARTITION; COPY TO works on partitioned parents; vacuumdb --analyze-only analyzes partitioned parents by default

Planner and pruning improvements landed steadily from 12 through 18, so larger partition counts plan more efficiently than they did in the early native era. You still want a practical partition size (day, month, quarter), but you have more leeway than early native partitioning allowed.

Postgres 19 advice: Keep partitioning for retention and operational boundaries first. Query speed is a bonus when predicates hit the partition key, not the reason to partition a 200 GB table.

For day-to-day rotation, prefer concurrent detach:

ALTER TABLE iot_thermostat
  DETACH PARTITION iot_thermostat07142022 CONCURRENTLY;

Keep using a default partition to catch bad timestamps and late-arriving keys, then move that data into real children. Keith’s default-partition guidance still applies. On 17+, you can put identity columns on the partitioned parent directly.

When you need to coarsen history (monthly → quarterly) or split a hot range, Postgres 19 can do it in SQL:

ALTER TABLE events
  MERGE PARTITIONS (events_2024_01, events_2024_02, events_2024_03)
  INTO events_2024_q1;

ALTER TABLE events
  SPLIT PARTITION events_2024_q1 INTO (
    PARTITION events_2024_01 FOR VALUES FROM ('2024-01-01') TO ('2024-02-01'),
    PARTITION events_2024_02 FOR VALUES FROM ('2024-02-01') TO ('2024-03-01'),
    PARTITION events_2024_03 FOR VALUES FROM ('2024-03-01') TO ('2024-04-01')
  );

Plan merge/split for a maintenance window today: they take ACCESS EXCLUSIVE on the parent and do not yet offer CONCURRENTLY. Under the hood they physically copy tuples into new child relations, so that exclusive lock lasts for the entire copy, not just a metadata swap. On high-throughput tables, prefer concurrent detach / attach patterns for zero-downtime rotation and reserve merge/split for quieter windows. Also budget for catalog work: objects defined directly on the source partitions (partition-local indexes and constraints) are dropped during the split or merge. Indexes and constraints defined on the parent are recreated on the new children automatically. For online work, concurrent detach (or attach a pre-built child) remains the smooth path.

Unique indexes still need the partition key if you want uniqueness across the set. pg_partman remains the right automation layer for create/retain schedules. Native DDL got richer, and the calendar automation still complements it.

Exporting a partitioned table is simpler too. Before Postgres 19, COPY parent TO ... failed with cannot copy from partitioned table, so you had to wrap it as COPY (SELECT * FROM parent) TO ... or target children one by one. In 19, COPY TO runs directly on the root relation and pulls rows from all descendants, which skips the extra planning and executor overhead of the SELECT wrapper and keeps bulk exports cheaper on CPU and memory.

That same path helps logical replication’s initial table sync. When you publish a partitioned parent with publish_via_partition_root = true, snapshot copy workers on the publisher use COPY TO. In 19 they can take the native direct parent path instead of the query-wrapper workaround, which streamlines the initial streaming sync.

What hasn’t changed

After all those version notes, here is the reassuring part. These lasting Crunchy recommendations (ideas we have been writing about for years) remain correct on Postgres 19:

1. Prefer COPY over row-by-row INSERT for bulk loads. Streaming CSV or newline-delimited JSON into COPY ... FROM STDIN is still the fast path.
2. Store JSON as jsonb, not text json, unless you specifically need to preserve exact formatting. Index with GIN when you need containment queries.
3. Keep hot data out of TOAST when you can. Large blobs you filter and join on belong in structured columns; TOAST is for oversized attributes, not your primary access path.
4. Indexes are a tradeoff, not free. They speed lookups and can help joins and sorts, and they cost disk, write amplification, and maintenance.
5. The planner may skip your index. Creating an index does not guarantee it will be used; verify with EXPLAIN (ANALYZE, BUFFERS).
6. B-tree is the default workhorse. Reach for other index types when the access pattern calls for them, not because B-tree is outdated.
7. BRIN wins on correlated, append-mostly columns. Tiny indexes for time-series and sensor-style data when physical order matches logical order.
8. Covering indexes (INCLUDE) enable index-only scans when the select list fits in the index after vacuum has updated the visibility map. Don’t over-include wide columns.
9. Partition for lifecycle and cheap drops first. Retention by detaching or dropping children beats giant DELETEs; query speed is a bonus when predicates hit the partition key.
10. Unique indexes on partitioned tables need the partition key if you want uniqueness across the whole set. Native declarative partitioning beats trigger/inheritance routing.
11. Use a default partition as a safety net, and drain it. Catch bad or late keys there, then move valid rows into real children.
12. pg_partman still handles the schedule. Native DDL got richer; automatic create/retain schedules are still an extension’s job.

The advice did not flip. Postgres got better at delivering it: more resilient loads, faster compression by default, more BRIN shapes, wider B-tree usefulness, quicker disk reads, and smoother partition operations. If you still follow those older posts on Postgres 19, keep the mental model, update the edges of the checklist, and re-test after upgrade with EXPLAIN (ANALYZE, BUFFERS, IO). ]]> ed55032136d4ae89d9e7b2b86b4032785d935eebcecd9334595f6461b06a38db Tue, 18 Aug 2026 15:00:00 EDT 2026-08-18T19:00:00.000Z 2026-08-18T19:00:00.000Z