System design hack: Postgres is a great pub/sub and job server (layerci.com)

Article summary

The article discusses using Postgres as a pub/sub and job server, highlighting its potential as a distributed message queue. The comments reveal that this approach has been used by enterprises for decades and can be viable with the right configuration. Postgres's 'channel' feature and the ability to use triggers make it a suitable option for this use case. However, some commenters argue that other tools like Redis or RabbitMQ might be more suitable for this purpose.

Main themes

• Postgres as pub/sub and job server
• Message queues and distributed systems
• Database configuration and optimization
• Tool choice and trade-offs
• Scalability and performance
• Deployment and development simplicity

What commenters say

• Using Postgres as a pub/sub and job server can be a viable option with the right configuration and features like channels and triggers.
• Other tools like Redis or RabbitMQ might be more suitable for this purpose due to their native support for message queues and pub/sub models.
• The approach of using Postgres for this purpose can be beneficial when already using Postgres for persistence, as it simplifies deployment and development.
• Some argue that polling is a bad practice and should be avoided, while others see it as a necessary evil in certain scenarios.
• The use of Postgres for pub/sub and job server can be a good default choice, but it may not be the best tool for every situation.
• Implementing a job server in Postgres can be done using the 'FOR UPDATE SKIP LOCKED LIMIT 1' trick, which provides an atomic way to fetch jobs.
• Batching jobs can add unnecessary complexity and premature optimization, and stream-processing is often a better approach.
• The choice of tool for pub/sub and job server ultimately depends on the specific needs and requirements of the project.