Batch Rendering

Batch rendering is designed for template-based video production, bulk transcoding, multi-version content generation, and other workloads that process many tasks at the same time. Meishe Cloud Editing uses an independent Task Scheduler to receive, manage, and distribute synthesis and transcoding tasks through a unified queue. A deployment can start on a single server and grow into a multi-Worker cluster as demand increases.

Both CPU and GPU editions of the compile and transcode services are available. Customers can choose a deployment model based on task volume, media specifications, target codecs, and existing infrastructure, without building a large cluster from the outset.

How it works

Synthesis and transcoding requests submitted in batches enter a unified task queue. The Task Scheduler manages the queue and dispatches tasks to available service instances. Each instance performs the assigned task and reads source media from, and writes rendered output to, shared storage.

Separating task submission and scheduling from task execution provides several benefits:

• Unified task management: synthesis and transcoding requests are managed through one queue, making it easier to handle a continuous stream of batch jobs.
• Parallel execution: multiple service instances can run on the same server and process tasks concurrently when resources permit.
• Scalable rendering capacity: additional Worker servers or service instances can be introduced as demand grows without changing how upstream applications submit tasks.
• Flexible compute options: CPU and GPU editions are available for both compile and transcode services, allowing deployments to balance cost, codec requirements, and delivery time.

Actual batch-processing capacity depends on timeline complexity, input and output resolutions, codecs, bitrates, server resources, and concurrency settings. Use representative production workloads for capacity testing before finalizing a production configuration.

Deployment options

Option 1: Single-server deployment

The Task Scheduler and one or more compile and transcode service instances can run on the same server. Depending on available resources, the deployment can use CPU services or GPU services.

Benefits

• A simple topology with fewer servers makes the solution quick to deploy and operate.
• Multiple service instances can run on one server to process several tasks in parallel.
• Suitable for validation, initial production rollout, lower task volumes, or relatively stable concurrency requirements.

As demand grows, the same task-submission workflow can be retained while the deployment expands to multiple GPUs or a multi-Worker cluster.

Option 2: Multiple GPUs on one server

A server can contain multiple NVIDIA GPUs and run several GPU editions of the compile or transcode service. Each service instance can use a different GPU, allowing multiple tasks to run concurrently on separate GPU resources.

Benefits

• Expands rendering capacity within one server and improves the use of installed GPU resources without adding Worker nodes.
• Service instances can be aligned with individual GPUs so that tasks run on different devices.
• Suitable when rack space or server count is constrained but concentrated concurrent processing is required.

GPU selection should not be based only on memory capacity or product family. Before deployment, consult NVIDIA's official Video Encode and Decode GPU Support Matrix to verify codec support, the number of NVENC/NVDEC engines, and concurrent-session limits for the target GPU. Validate the final configuration with representative media workloads.

Option 3: Controller with multiple Workers

In a distributed deployment, the controller node hosts the Task Scheduler and other controller services, while multiple Worker servers run compile and transcode service instances and execute tasks. The Task Scheduler manages the task queue and dispatches tasks. Additional Workers can be added when more processing capacity is required, enabling horizontal expansion across servers.

Benefits

• Multiple servers process tasks together, making this option suitable for growing workloads or distinct batch-processing peaks.
• Workers can run CPU or GPU service editions according to their hardware, allowing existing compute resources to be combined.
• Expansion occurs primarily at the execution layer, while upstream applications continue to use the same task submission and query flow.
• Each Worker can also contain multiple GPUs, creating a multi-Worker, multi-GPU topology for further capacity growth.

Every Worker must be able to access the same source media and rendered output. The platform supports NFS shared storage or object storage so tasks can run on different Workers without depending on files held locally by a specific server.

Option 4: Kubernetes cluster deployment

The platform supports deployment in a customer-managed Kubernetes (K8s) cluster. The existing cluster can manage containers for the Task Scheduler, compile services, transcode services, and related components.

Benefits

• Integrates the rendering services with the customer's existing container platform, release process, and operations practices.
• Containerized service instances make it easier to plan and manage CPU, GPU, and storage resources consistently.
• Suitable for customers with established K8s infrastructure who want rendering services managed as part of the same cluster.

K8s schedules containers and cluster resources, while the Task Scheduler manages and distributes Meishe synthesis and transcoding tasks. During implementation, the customer configures GPU resources, persistent storage, and network access according to the design of its own cluster.

Choosing a deployment option

Business situation	Suggested option	Primary value
Validation, lower task volume, or stable concurrency	Single-server deployment	Simple architecture and a quick path to batch processing
More GPU concurrency is required within one server	Multiple GPUs on one server	Concentrated use of multiple GPUs with no additional Worker nodes
Workload is growing or rendering must scale across servers	Controller with multiple Workers	Horizontal capacity expansion by adding Workers
An established container platform and cluster operations model already exist	Kubernetes cluster deployment	Integration with the existing container resource management model

These options support a gradual deployment path. A customer can begin with one server, add service instances or GPUs within that server, and introduce more Workers when a single server no longer meets demand. Customers with an existing K8s platform can also adopt cluster deployment directly. This approach allows rendering resources to grow with the workload and avoids committing excessive infrastructure at the initial stage.

Planning a batch-rendering environment

Evaluate the following factors before finalizing a production design:

1. Workload characteristics: estimate peak task volume, video duration, timeline complexity, and acceptable completion time.
2. Codec requirements: identify input and output codecs and resolutions, then verify that the selected GPU supports the required hardware encoding and decoding capabilities.
3. Service-to-GPU assignment: in a multi-GPU server, explicitly assign each GPU service instance to its intended device to prevent resource overlap.
4. Shared storage: for a multi-Worker deployment, plan NFS or object-storage paths, permissions, and network bandwidth in advance.
5. Capacity validation: test representative timelines and media with different CPU, GPU, instance-count, and concurrency configurations to find a combination that meets business requirements.

This planning process helps balance task throughput, per-task completion time, infrastructure cost, and operational complexity for the current workload.