Polars 2.0 (pola.rs)

Article summary

Polars 2.0 has been released, featuring out-of-core support, performance improvements, first-class SQL support, a new Map dtype, and stricter data type handling. The release includes significant performance enhancements, making Polars competitive with DataFusion and DuckDB in TPC-H and TPC-DS1 benchmarks. The new version also introduces a streaming engine and improved support for SQL queries. The developers aim to continue improving Polars, with plans to enhance out-of-core support, scaling, and GeoPolars.

Main themes

• Polars 2.0 release
• performance improvements
• SQL support
• competition with DuckDB and DataFusion
• replacement for Pandas
• GeoPolars development
• data processing innovation
• out-of-core support

What commenters say

• Polars 2.0 is a significant improvement over the previous version, offering better performance and new features.
• The competition between Polars, DuckDB, and DataFusion is driving innovation and improvement in the data processing space.
• Polars is becoming a viable replacement for Pandas in many use cases, thanks to its improved API and performance.
• Some users still prefer Pandas due to its familiarity and existing ecosystem, but others are switching to Polars for its cleanliness and speed.
• The development of GeoPolars is underway, which will provide geospatial capabilities to Polars.
• Polars' support for SQL and out-of-core processing makes it a strong contender for data-intensive applications.
• The choice between Polars, DuckDB, and Pandas ultimately depends on the specific use case and personal preference.
• Polars' performance improvements are notable, but some users have encountered limitations, such as the lack of support for nanosecond resolution timestamps.