Building the next generation of multimodal AI experiences with fal in Microsoft Foundry

Video projects don’t all kick off with the same starting point or aim for identical outcomes. A creative team may interpret a written brief in multiple ways, a retailer may transform an approved product image into a snappy video concept, or an app developer might weave together image and video generation to evolve an idea from a simple description into a full-blown scene. Each of these situations comes with different needs regarding visual quality, motion, creative control, turnaround speed, and budget. This highlights the necessity for a versatile multi-modal approach to achieving a comprehensive vision.

Today, Microsoft and fal are joining forces to assist developers in selecting the right tools for their specific video workloads. We are thrilled to share that Kling, LTX, and MiniMax video models are now available in public preview on the Microsoft Foundry as a result of our new partnership with fal. These first-party Azure services, offering text-to-video and image-to-video capabilities, empower creators and developers with more options for video generation that align with their unique projects.

This collaboration broadens the palette available in Microsoft Foundry, integrating video into the wider platform that developers use to craft AI applications. The aim is to enable developers to choose the appropriate model for each phase of their project, seamlessly linking those phases into a cohesive application.

“This year, fal has witnessed remarkable uptake in generative media across various sectors, from advertising and marketing to gaming. Partnering with Microsoft truly enhances our capability to empower the expansive creative ecosystem and ensures that AI solutions leveraging generative media embrace a multi-modal approach.” – Burkay Gur, Cofounder

fal stands as a comprehensive solution for generative media, designed for creators and developers alike. It boasts APIs for over 1500 models spanning image, video, audio, 3D, and virtual environments. This robust platform offers inference, training, and the necessary infrastructure to deploy generative media workloads effectively while boasting innovative post-trained models that set new benchmarks for speed and quality.

The need for such specialization is essential because generative media combines both creative and technical aspects. A captivating demo is merely a starting point; production applications must also handle real user traffic, scale according to demand, and strike a balance between output quality, latency, and cost.

Thanks to this partnership, developers can now access a curated selection of video models within Microsoft Foundry, sourced from the companies that created them. The initial model set provides teams with various options tailored to diverse creative needs, rather than relying on a single model for all tasks, with more models anticipated in the future.

https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FO-qztPyJHtU%3Ffeature%3Doembed&display_name=YouTube&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DO-qztPyJHtU&image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FO-qztPyJHtU%2Fhqdefault.jpg&type=text%2Fhtml&schema=youtube" allowfullscreen="" style="max-width: 100%

The initial model offerings include Kling v3 Pro, LTX-2.5 Fast, LTX-2.5 Pro, MiniMax H3 Max, and MiniMax H3 Max Turbo. These models provide developers with various options for generating video from text and animating existing images. A team rapidly experimenting with creative ideas may prioritise speed and cost-efficiency, while another team focused on producing fewer, high-quality assets will likely favour output quality more. The origin of the application, whether it starts from text or an existing image, combined with output quality and budget considerations, heavily influence the choice of model.

Kling: Transforming a still image into a dynamic scene

Kling v3 Pro pairs cinematic visuals and fluid motion with native audio generation and bespoke element support. Its image-to-video functionality enables developers to dictate how a static image evolves into an animated scene, making it ideal for applications demanding strong visual direction and movement.

LTX: A model for quick drafts and final outputs

For speed-focused iterations and previews, LTX-2.5 Fast serves as the optimized option, while LTX-2.5 Pro, designed for high-fidelity final outputs, guarantees quality. Both models facilitate text-to-video and image-to-video creation while synchronizing audio.

MiniMax: An alternative video generation strategy

MiniMax H3 Max and MiniMax H3 Max Turbo are post-trained versions of MiniMax H3, focusing on prompt adherence, aesthetics, and performance. These models give developers more choices to assess against their application’s creative brief, quality requirements, and performance standards.

“When working with video models, the intricacies matter: how closely the result aligns with your prompt, how well it maintains reference images/videoclips/audio, and how quickly you can explore new ideas. Kling offers developers detailed control over reference to video creation. With LTX, you get fast iterations and top-notch output through its Fast and Pro variants. H3 Max integrates fal’s post-training and optimisation enhancements to elevate quality and speed. Having these models at their disposal in Microsoft Foundry gives developers the flexibility to find what’s best for their application.” – Gökay Aydoğan, Creative Engineer

Integrating video generation into an enterprise application goes beyond merely selecting the right model. Teams need a robust platform to review different options, build with the models best fitted for their needs, and maintain those experiences as part of the overall application flow.

The initial range of video models will be hosted in the US, with Microsoft Foundry being the first to offer a US-hosted Kling 3 endpoint. Here are some key factors for enterprise teams to consider:

• Data residency: For businesses that require data to remain within specific boundaries, the hosting location is critical. Be sure to read each model’s data processing and storage policies to ensure compliance with your requirements.
• Model evaluation: Teams should assess each model based on quality, performance, and cost, and see how video generation ties in with the other models and talents necessary for their applications.
• Production readiness: As teams shift toward production, it’s vital to utilise the evaluation, optimisation, governance, security, observability, and infrastructure capabilities that accompany their chosen solution.

Pricing for video generation varies by model and output resolution, giving teams the flexibility to balance creative needs against budget considerations. The table below summarises the pricing in USD per minute of generated video for LTX and MiniMax models, simplifying comparison across different options.

Expect pricing to be refreshed and published on our Azure pricing page shortly.

Eager to get started on your next project? Join us at Ignite for BRK256: Video AI at Scale with fal on Microsoft Foundry. On Thursday, November 19, from 11:00 AM to 11:45 AM CT, Naomi Moneypenny and Alexander Hughes from Microsoft will share the stage with Burkay Gur and Lovis Odin from fal to delve into our partnership. Secure your spot today!

Explore the features of Kling v3 Pro, LTX-2.5 Fast, LTX-2.5 Pro, MiniMax H3 Max, and MiniMax H3 Max Turbo on Microsoft Foundry and discover the model that best suits your video creation needs.