Postgres is a great pub/sub and job server (2019) (webapp.io)

Article summary

The article discusses using Postgres as a publish/subscribe and job server, highlighting its benefits such as data integrity and performance guarantees. It provides an example use case of processing CI runs with sequential workers, demonstrating how Postgres can handle job queuing and notification. The article concludes that using Postgres as a pub/sub and job server can be a good starting point, eliminating the need to learn new technology. This approach can simplify system design and reduce the complexity of introducing additional messaging systems.

Main themes

• Postgres as pub/sub and job server
• Database vs dedicated message queue
• Scalability and reliability
• System design and complexity
• Cloud-based services
• Transactional guarantees
• Performance and throughput
• Message queue libraries and frameworks

What commenters say

• Using a database as a message queue can add unnecessary complexity to an application.
• Postgres can be a suitable choice for a pub/sub and job server, especially for smaller projects or those with low message volumes.
• Dedicated message queues like Kafka or RabbitMQ are more suitable for high-volume or high-performance applications.
• The use of a database as a message queue can lead to issues with scalability and reliability, particularly in cases of high message volumes or worker failures.
• Some argue that the benefits of using a database as a message queue, such as transactional guarantees, outweigh the potential drawbacks.
• Others believe that using a cloud-based message queue service can simplify the development process and reduce the risk of errors.
• The choice between using a database or a dedicated message queue depends on the specific needs and constraints of the project, including performance requirements and team expertise.
• Some libraries and frameworks can simplify the process of using a database as a message queue, but may introduce additional complexity or security concerns.