Nvidia is busy acquiring the young cloud startup Lepton AI. This would be the second acquisition in a short time, following the recent acquisition of the synthetic data specialist Gretel Labs.
The Information, citing sources, reports that Nvidia is reportedly already at an advanced stage in its possible acquisition of Lepton AI. The amount of the acquisition is unknown, but it is estimated to be several hundred million dollars.
Lepton AI raised a seed amount of 11 million dollars at the start of its activities in 2023.
Training and inference
Lepton AI, founded by former researchers from Meta’s AI Lab and the creators of PyTorch, offers a cloud platform for optimizing AI workloads. This platform helps users train LLMs and perform inference.
The startup makes this possible via a visual interface for setting up clusters in the cloud. Users can also train multiple LLMs in parallel using a selection of different Nvidia GPUs. A tool is available to centrally manage the hardware required for each individual project.
The Lepton AI cloud platform also allows for the detection of errors that occur during training and affect the output. According to the AI specialist, the cloud platform’s processing speed is more than 600 tokens per second, with a latency of less than 10 milliseconds. An autoscale functionality automatically adds more hardware to customers’ inference environments when demand increases.
The cloud platform’s inference functionality uses open-source tooling such as vLLM. This tool merges similar LLM prompts into a single request, reducing the computing power required for LLM. Developers can further utilize the vLLM tooling to reduce memory usage.
No clear plans from Nvidia
It is unclear exactly what Nvidia intends to use the Lepton AI platform and technology for. Perhaps it is for its machine learning expertise, but it is also possible to continue to offer the startup’s products.
Lepton AI’s cloud platform already has a number of competitors that provide similar technology, such as AI cloud provider CoreWeave, which is going public today.
The possible acquisition of Lepton AI follows Nvidia’s recent acquisition of Gretel Labs. This acquisition, which probably cost more than 320 million dollars, gives the AI and chip giant tools for synthetic data for training LLM.
Also read: Synthetic data and the danger of ‘model collapse’.