Introducing the Postgres Extension Manager, which we fondly call 'pgxman' or 'xman' for short. In the following sections, we’ll cover an introduction to pgxman, extensions, the problems pgxman attempts to solve, and how it works. Check out pgxman.com to learn more.
pgxman is npm for Postgres extensions. pgxman simplifies the discovery and use of extensions so modern hackers can easily enhance the capabilities of their applications.
Instead of managing extension versions, build & run dependencies, operating system and platform architecture, pgxman automatically detects and streamlines extension operations (xOps) based on the local development environment.
pgxman integrates with the system package manager, ensuring the correct versions are installed without extra packages from any shared dependencies between extensions. pgxman’s automated build system creates [APT] packages for each Postgres version, platform, and OS supported by the extension. Extensions are built from a “buildkit” formula, written in YAML, and are contributed through GitHub.
INTRODUCTION TO EXTENSIONS
In their 1986 paper "The Design of Postgres," authors Michael Stonebraker and Lawrence Rowe prioritized user extendibility, writing “[Postgres should] provide user extendibility for data types, operators, and access methods.”
If you’re unfamiliar with Postgres extensions, they are 3rd party open-source plugins that add new capabilities to Postgres. Extension authors create remarkably novel database capabilities by modifying Postgres’ low level hooks, such as parser hooks, executor hooks, planner hooks.
Here are examples of popular Postgres extensions:
Amazingly, extensions remain compatible with Postgres’ future version releases and new core features. Modifications to Postgres’ low level hooks can be quite powerful - it’s like gene splicing a database. In the eternal words of Professor Charles Xaviar, “mutation is the key to evolution.”
DISCUSSION OF DESIGN GOALS
While there are over 1000 Postgres extensions today, modern hackers rarely discover them. During our monthly “SF Analytics Night” club, I ask how many people have used Postgres extensions — only one or two hands raise in the large group.
Extensions are underutilized for several reasons: poor discoverability, restricted support from managed Postgres providers, the need for existing knowledge or study before use, and complex management of extension versions and dependencies. By lowering the friction to use Postgres extensions with pgxman, modern hackers can enhance applications more easily.
1. Discoverability & freedom of choice — pgxman has LLM-enabled search: "pgxman.ai” Search with personalized use case questions to discover relevant extensions. pgxman.ai explains why extensions are recommended to add helpful context. Modern hackers have the freedom to install and run any trusted extension in their environment with pgxman.
2. Clean install, upgrade, uninstall anywhere in “the way a machine expects” — Integrating with the system package manager solves several problems. pgxman handles build and runtime dependencies that extensions need - including Postgres and any shared dependencies between extensions. pgxman ensures correct versions are installed without extra packages. Packages are uninstalled directly with your system package manager - no unmanaged files. There’s no longer a need to expand a tarball into your filesystem.
3. Automated build system — Each extension is built from a “buildkit” formula, written in YAML. Buildkits are contributed through GitHub (homebrew model) and supports extensions written in all languages. The automated build system creates [APT] packages for each Postgres version, platform, and OS supported by the extension.
MADE FOR MODERN HACKERS
Steps to build parquet_s3_fdw manually:
If you want to build advanced applications, start with pgxman and make it so.
FUNCTIONAL SOURCE LICENSE
pgxman is released under the Functional Source License (FSL). Open to all users, source available, for Permitted Purpose: “any purpose other than a Competing Use. A Competing Use means use of the Software in or for a commercial product or service that competes with the Software or any other product or service we offer using the Software as of the date we make the Software available.” You can do anything with FSL software except strip-mine the software and undermine the continued development. You can read it, learn from it, run it internally, modify it, and propose improvements. After 2 years, the license for pgxman converts to Open Source software under Apache 2.0. Converting to Apache 2.0 offers community protection if pgxman development slows. We encourage pgxman adoption broadly and welcome any service provider to contact us to obtain a license. ensure the service can scale to meet your needs.
Thank you Hydra team for excellent work on pgxman, and especially, thank you to Owen Ou, Simon Wijckmans, James Lovern, and my cofounder Jonathan Dance “JD”.
A big thank you to my wife, Rebecca, for her years of loving support and thanks to our tiny, fluffy dog, Titus (”Postgres pet of the week”) too.