Databricks’ Mosaic AI will focus on stronger model quality, new AI governance tools, and compound AI systems.
The company announced this during the keynote of its annual Data + AI Summit. The innovations coming out of Mosaic AI should give companies the confidence to develop and then measure production-quality applications. With this, Databricks wants to assist companies in meeting expectations around generative AI. That’s exactly where change is needed, Databricks sees, as companies struggle to get generative AI out of the pilot phase and finally get it into production properly. Often, privacy, quality, and cost concerns are the obstacles.
Optimizing foundation models and regulating governance
Introducing the public preview of Mosaic AI Model Training will be a step forward. This component enriches open source foundation models with a company’s private data. In this way, models are fed with the knowledge specific to the organization, department or task. The output of a model thus becomes much more reliable and accurate because it can better understand the context of a query and find the necessary data. In addition, this tuned model is often cheaper than a larger model since it relies on fewer parameters and computing resources. A lot of information that an LLM has about everything you can think of is no longer needed; through Mosaic AI Model Training, it can be narrowed down exactly to what is truly relevant to the business situation.
Also read our backstory on how Mosaic technology is transforming Databricks’ entire platform.
With Mosaic AI Gateway, Databricks offers another additional feature. This feature, also in public preview, provides a unified interface for querying, managing and deploying open-source or proprietary models. It allows companies to switch between LLMs powering their applications more easily without making complicated changes to the application code. On top of that, Mosaic AI Gateway offers options for tracking who calls a model and setting rate limits. The latter can help keep generative AI costs manageable. There is also the option to build filtering mechanisms for security and personally identifiable information (PII), regardless of the model used.
Compound AI systems
Finally, Mosaic AI is getting updates related to compound AI systems. This term still deserves some explanation. Compound AI systems refer to a way of working that addresses AI tasks by combining multiple interactive components. These might include components such as model calls, retrievers, or external tools. A now well-known example of a compound AI system is RAG, which combines a vector database with tools for monitoring, evaluation, security, and governance. Combining these things should increase the accuracy of an LLM.
Tip: What is Retrieval-Augmented Generation?
Last month, Databricks introduced Mosaic AI Vector Search, a serverless vector database integrated into its Data Intelligence Platform, for RAG capabilities. During the Data + AI Summit, the public preview of Mosaic AI Agent Framework will be added. This component promises to simplify the development of quality RAG applications using foundation models and enterprise data. Developers can evaluate the quality of RAG applications with Mosaic AI Agent Evaluation and quickly redeploy the app. The evaluation tools automatically determine whether the outputs are high quality and feature a user interface where developers can receive feedback from human stakeholders. Collectively, these developments should help roll out production-quality generative AI.
Databricks also has plans to go further for compound AI systems. Namely, it sees this using tools that feed the systems with the capacity to deal with real-world situations. These include, for example, searching the Internet and making API calls. To support this, Databricks is coming out with the private preview of Mosaic AI Tools Catalog, which allows companies to arrange everything around governance and sharing and registering the tools. This, in turn, uses the Unity Catalog, another tool from Databricks.
Techzine is attending Databricks’ Data + AI Summit this week. Keep an eye on the website for the latest developments around the company.