BentoML

BentoML is an open-source inference platform that packages and serves AI models as scalable APIs via Python decorators, with managed BentoCloud offering per-second, scale-to-zero GPU billing.

BentoML gives software engineers a Pythonic way to turn models into production inference services: you define a service class, and BentoML handles packaging (a 'Bento'), API generation, batching, and serving. It supports a wide range of workloads — LLM applications, model inference APIs, asynchronous job queues, and multi-model pipelines — and is model- and framework-agnostic. The managed BentoCloud runs these services in the cloud with autoscaling and per-second, scale-to-zero billing, so idle deployments cost nothing. It offers tiers from a pay-as-you-go Starter to SLA-backed Scale and a bring-your-own-cloud (BYOC) Enterprise option, with GPU instances metered per second and priority access to high-end accelerators on higher plans. The open-source framework remains free and widely used, while BentoCloud provides the managed serving layer. In February 2026, BentoML was acquired by Modular AI; the open-source project continues, though post-acquisition BentoCloud pricing may be re-disclosed over time, so teams should confirm current terms.