--- title: "pgvector gets even better" publication: "Postgres Weekly" issue: 553 published_at: "2024-05-01" canonical_url: "https://postgresweekly.com/issues/553" --- Plus reducing bloat, linting migrations, and various ways of moving data to other places. | #​553 — May 1, 2024 [Web Version](https://postgresweekly.com/issues/553) Together with [![Ottertune logo\](https://res.cloudinary.com/cpress/image/upload/c_fill,g_auto,e_trim,w_450,h_100/e_make_transparent/co_white,e_outline:7/d92wjjhrmysrssm2xhjm.png)](https://ottertune.com/free-trial/postgresql?utm_source=postgres_weekly&utm_medium=email&utm_campaign=may_01_primary) Postgres Weekly [![\](https://res.cloudinary.com/cpress/image/upload/w_1280,e_sharpen:60,q_auto/s3diq7eox8hkpfsn0jgj.jpg)](https://www.crunchydata.com/blog/crunchy-bridge-for-analytics-your-data-lake-in-postgresql) [Crunchy Bridge for Analytics: Your Data Lake in Postgres?](https://www.crunchydata.com/blog/crunchy-bridge-for-analytics-your-data-lake-in-postgresql) — Marco ponders: _“What if you could use PostgreSQL to easily query your data lake with state-of-the-art analytics performance?”_ [Crunchy Bridge for Analytics](https://www.crunchydata.com/products/crunchy-bridge-for-analytics) is a new part of Crunchy Data’s Postgres platform offering S3 integration, CSV/Parquet querying support, and bidirectional data lake ↔ Postgres import/export. Marco Slot (Crunchy Data) [The 150x pgvector Speedup: A Year in Review](https://jkatz05.com/post/postgres/pgvector-performance-150x-speedup/) — [pgvector](https://github.com/pgvector/pgvector/) provides Postgres with extra data types, indexing, and querying facilities to work with high dimensional vectors (such as are common with LLMs and embeddings). The past year has seen a _huge_ leap in its capabilities and performance. Jonathan Katz 💡 [pgvector 0.7.0](https://github.com/pgvector/pgvector/blob/master/CHANGELOG.md#070-2024-04-29) has also just been released with new `halfvec` and `sparsevec` types, useful extra functions, support for [scalar and binary quantization](https://jkatz05.com/post/postgres/pgvector-scalar-binary-quantization/), and more. [![\](https://copm.s3.amazonaws.com/65fde9b6.png)](https://ottertune.com/free-trial/postgresql?utm_source=postgres_weekly&utm_medium=email&utm_campaign=may_01_primary) [Improve Postgres Performance and Reduce Costs with OtterTune](https://ottertune.com/free-trial/postgresql?utm_source=postgres_weekly&utm_medium=email&utm_campaign=may_01_primary) — Spending more on AWS does not equal better performance. OtterTune gets 146% more throughput than standard RDS settings by optimizing knobs and provides index and query recommendations, plus table and autovacuum insights. Get a free, 30-day trial. OtterTune sponsor [Postgres Bloat Minimization](https://supabase.com/blog/postgres-bloat) — Tables occupying more space than expected is a common cause of complaint, but there are a variety of strategies you can employ so that space is used more efficiently. Pavel Borisov **IN BRIEF:** - Peter Eisentraut looks back at [Postgres' supported platforms over time.](https://peter.eisentraut.org/blog/2024/04/29/postgresql-supported-platforms-over-time) - Timescale's CEO [reflects on Postgres as being 'the bedrock for the future of data'](https://www.timescale.com/blog/postgres-for-everything/), using community sentimenent and survey data to make his case. - 📅 If you want to keep up to date with the state of development around David Wheeler's proposed improvements to the Postgres extension packaging and distribution ecosystem, [there's a virtual 'mini-summit'](https://www.eventbrite.com/e/postgres-extension-ecosystem-mini-summit-tickets-851125899477) taking place in just a few hours (and another on May 15). - It looks like [PG Day Chicago 2024 went well!](https://hdombrovskaya.wordpress.com/2024/04/28/hey-chicago-we-did-it/) [_Papa's Got a Brand New RAG_](https://bonesmoses.org/2024/pg-phriday-brand-new-rag/) — RAG (Retrieval-Augmented Generation) is a technique used alongside large language models where you fetch data to include in prompts to improve the responses LLMs give. Shaun demonstrates bringing all the parts together to build a simple RAG-based query system in Python with [pg_vectorize](https://github.com/tembo-io/pg_vectorize) doing the heavy lifting on the Postgres side. Shaun M. Thomas 📄 [Nightly Postgres Backups via GitHub Actions](https://joshstrange.com/2024/04/26/nightly-postgres-backups-via-github-actions/) Josh Strange 📄 [When and Why Postgres Indexes are Ignored](https://notso.boringsql.com/posts/why-postgresql-indexes-are-ignored/) Clusterity s.r.o. 📄 [Benchmarking Connection Poolers: PgBouncer, PgCat and Supavisor](https://tembo.io/blog/postgres-connection-poolers) Binidxaba (Tembo) 📄 [Test and Release pgrx Extensions with pgxn-tools](https://justatheory.com/2024/04/pgxn-tools-pgrx/) David Wheeler 📄 [LDAP Authentication in PgBouncer Through PAM](https://www.percona.com/blog/ldap-authentication-in-pgbouncer-through-pam/) Jobin Augustine 🛠 Code and Tools [PeerDB: A Simple, Fast Way to Replicate Postgres Data Elsewhere](https://www.peerdb.io/) — A data replication system built by former Microsoft and Citus Data employees that targets use cases where you need to stream data from Postgres to data warehouses, queues, and other storage engines. This week’s [v0.13.0 release](https://github.com/PeerDB-io/peerdb/releases/tag/v0.13.0) introduces an Elasticsearch connector. [GitHub repo.](https://github.com/PeerDB-io/peerdb) PeerDB 🎉 The company behind PeerDB also [raised $3.6m in seed funding](https://blog.peerdb.io/peerdb-raises-3-6-million-seed-funding-to-revolutionize-data-movement-for-postgresql) this month. [pg_tier: Tier Data to External Amazon S3 Storage](https://github.com/tembo-io/pg_tier) — Once data becomes older or much less frequently accessed, it might be more cost effective to store it elsewhere. pg_tier makes it easier to push a table over to S3 while still maintaining the ability to query it (at the price of some added latency, naturally). Tembo [Squawk: A Linter for Postgres Migrations](https://squawkhq.com/) — Prevent unexpected downtime caused by database migrations and encourage best practices around schemas and SQL. Steve Dignam - [HypoPG 1.4.1](https://github.com/HypoPG/hypopg) – Hypothetical indexes for Postgres. Now with Postgres 17 support. - [pgmoneta 0.11](https://github.com/pgmoneta/pgmoneta/releases/tag/0.11.0) – Backup / restore solution. Now supports macOS and adds disk and network rate limiting features. - [Orafce 4.10.0](https://github.com/orafce/orafce) – Emulate a subset of functions and packages from Oracle Database to aid migrations. - [Lantern 0.2.5](https://github.com/lanterndata/lantern) – HNSW/[usearch](https://github.com/unum-cloud/usearch) vector extension for Postgres. - [pgwire 0.22](https://github.com/sunng87/pgwire) – The Postgres wire protocol in a Rust library. **Got a new email?** You can [change your address here](https://postgresweekly.com/edit_subscription/*|UID|*). --- **Bored of us?** [Click here to stop getting Postgres Weekly](https://postgresweekly.com/leave/*|UID|*). --- **Got a link for us?** Reply and tell us. We can't include everything but we'll look at anything you send. _Thanks!_ --- Published by Cooper Press Ltd. Fairfield Enterprise Centre, Louth, LN11 0LS, United Kingdom Postgres, PostgreSQL and the Slonik Logo are trademarks or registered trademarks of the PostgreSQL Community Association of Canada. [Stop getting this newsletter](https://postgresweekly.com/leave/*|UID|*).