Just before the start of the Data Cloud Summit 2024, Snowflake is announcing a new project designed to improve companies’ data foundations: Polaris Catalog. This technology aims to help companies access data from different systems more easily, with an eye toward flexibility and control.
With Polaris, Snowflake offers companies an option to index and organize data following the Apache Iceberg format. Apache Iceberg has become an open data table format that many companies rely on. The partnership between Polaris and Apache Iceberg works like this: users have a central place where any engine can find and access any Iceberg table.
Christian Kleinerman, EVP of Product, emphasizes that interoperability with other query engines strengthens Polaris. “This is not a Snowflake feature to work better with the Snowflake query engine. Of course it will integrate and interoperate very well, but we are bringing together a number of industry partners to make sure that we can give our mutual customers at the end of the day,” Kleinerman said. For now, AWS, Microsoft Azure, Google Cloud, Salesforce, Confluent, and Dremio have been unveiled as tech platforms to which to interoperate.
Open-source
Polaris Catalog relies on Iceberg’s open-source REST protocol, which provides an open standard for retrieving data from any engine that supports the Iceberg REST API. This includes Apache Flink, Apache Spark, Dremio, Python, and Trino, among others. Snowflake says it will soon release a public preview of the new project. Polaris Catalog can then run as an option in the Data Cloud or via containers in a company’s infrastructure. Within 90 days, Polaris Catalog should also become open-source.
Techzine is attending Data Cloud Summit 2024 this week. Keep an eye on the website for the latest developments on Snowflake.
Tip: Snowflake acquires TruEra for observability in Data Cloud