Amazon Web Services detailed an architectural pattern for hosting multimodal generative pipelines on Amazon SageMaker AI using a single AWS vLLM-Omni Deep Learning Container. The implementation pairs a real-time endpoint for FLUX.2-klein-4B image generation with an asynchronous endpoint running Wan2.1-VACE-1.3B to produce short videos from text and image inputs.
Dual-Endpoint Pipeline Architecture
The architecture deploys the same pinned container image to two distinct SageMaker AI endpoints, configuring the container's target model through an environment variable. The setup uses a real-time endpoint for FLUX.2-klein-4B to return generated base64 PNG images directly inline. For video synthesis with Wan2.1-VACE-1.3B, an asynchronous endpoint processes queued requests and outputs final MP4 files to Amazon S3, preventing client connections from timing out during longer processing windows.
In the sample implementation, generated base64 images are resized, converted to JPEG data URLs, and embedded into a multipart request for the video endpoint. Custom headers route requests to vLLM-Omni's standard APIs, using /v1/images/generations for static image tasks and /v1/videos/sync for video rendering.
Resource Allocation and Operational Requirements
The reference deployment runs FLUX.2-klein-4B on ml.g6.xlarge instances and Wan2.1-VACE-1.3B on ml.g6e.xlarge instances. AWS noted that the video container uses variational autoencoder tiling to curb peak memory usage during video decoding. Running the setup requires Python 3.11 or later, appropriate AWS CLI or SDK credentials, specific SageMaker execution roles, and active GPU endpoint quotas in the host AWS Region.
AI-generated editorial illustration; not a documentary photograph.
Source: Generate images and video with vLLM-Omni on SageMaker AI – Part 2
SIGNAL TITANS
Leave a Reply