Databricks has announced the acquisition of Fennel AI for an undisclosed amount. Databricks aims to strengthen its data intelligence platform with real-time feature engineering capabilities.
The acquisition was previously reported by SiliconANGLE. According to Databricks, Fennel developed a modern, incremental compute engine that supports the development of more advanced data pipelines for batch, streaming, and real-time data. It improves the efficiency and timeliness of data, helping companies build more advanced AI models.
Feature engineering is becoming increasingly important in AI development as the industry evolves and large language models become more complex. It refers to the process of selecting, extracting, and transforming the most relevant parts of a dataset to build more effective AI models. This is crucial because models’ performance depends heavily on the quality of the features used during training.
Feature engineering often challenging
Databricks believes that Fennel’s platform will be valuable to customers because feature engineering is traditionally very complex and requires a lot of maintenance from complex extract, transform, and load pipelines. This is especially difficult with features that depend on both batch and real-time data, as consistency between the training and model environments is essential.
Fennel was founded in 2023 by CEO Nikhil Garg and CTO Abhay Bothra. They previously worked on AI infrastructure at Meta Platforms and Google Brain. The company provides a fully managed platform for creating and managing both features and their associated data pipelines. It increases the efficiency and timeliness of data by only recalculating changed data and ignoring the rest.
Cost optimization
Fennel’s platform offers a Python-native user experience. This makes writing complex features more accessible. There is no need to learn new programming languages or to first engage data engineering teams. In addition, the incremental computing technology helps optimize costs by avoiding redundant work.
By integrating this technology into its Data Intelligence Platform, Databricks claims that customers can iterate on their features faster. They can also improve the performance of their models with more reliable signals. It also supports the development of models with better personalization and context understanding.
Although Fennel operated largely under the radar and had not completed any major investment rounds, it did manage to attract large customers. They use the platform for applications such as fraud detection, credit risk assessment, security and trust, recommendation systems, and personalized rankings.
Although the amount Databricks paid for Fennel has not been disclosed, it is unlikely to have been a major drain on the company’s financial resources. In January, Databricks raised more than $15 billion through venture capital and debt financing.
After that funding round, Databricks indicated that the money would be used for strategic acquisitions and to strengthen its AI capabilities. Shortly thereafter, in February, it acquired data migration company BladeBridge, followed by this latest acquisition.