API Reference

Batch Processing

Batch processing in MediaX SDK is handled through the Queue class, providing a mechanism to run multiple Job instances in parallel with configurable concurrency and robust error isolation.

Queue Class

Concurrent Job Processing

Manages multiple media processing jobs with configurable parallelism

Configurable concurrency limits
Event-driven job monitoring
Robust error isolation between jobs
FIFO job execution order

Constructor

queue-constructor.ts
1
typescript

Parameters:

concurrency?numberMaximum number of concurrent jobs (default: 2)

Methods

add()

add(job: Job): void

Add a single job to the queue for processing

queue.add(job);

on()

on(event: string, handler: Function): void

Register event listeners for job lifecycle and queue state

queue.on("jobStart", (job) => console.log("Started:", job));

Events

The queue emits the following events for comprehensive job lifecycle monitoring:

jobStart(job: Job)Emitted when a job starts processing
jobProgress(job: Job, progress: any)Emitted periodically with job progress updates
jobDone(job: Job)Emitted when a job completes successfully
jobError(job: Job, error: Error)Emitted if a job fails
empty()Emitted when all jobs in the queue are finished

Usage Examples

Basic Queueing

basic-queue-example.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
typescript

Batching Multiple Jobs

batch-multiple-jobs.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
typescript

Note: All jobs will be executed with the specified concurrency limit until the queue is empty. Perfect for processing large batches of media files.

Batching Pipeline Jobs

Important: Queue does not directly accept Pipeline objects. However, since Pipeline.run() returns Jobs, those can be queued manually.

pipeline-queue-example.ts
1
2
3
4
5
6
7
8
9
10
11
12
typescript

Queue Behavior

FIFO Processing

Jobs execute in First-In-First-Out order respecting concurrency limits

Slot Management

Completed jobs free up slots for queued jobs automatically

Error Isolation

Errors in one job do not stop the queue; subsequent jobs continue

Completion Detection

Queue is considered finished when all jobs complete and empty event fires

Key Characteristics:

  • Jobs execute in FIFO order respecting concurrency limits
  • Completed jobs automatically free up slots for queued jobs
  • Errors in one job do not stop the queue; subsequent jobs continue
  • Queue is finished when all jobs complete and empty event fires

Advanced Queue Management

advanced-queue-management.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
typescript

Performance Considerations

Optimal Concurrency

Set concurrency based on your system capabilities:

  • CPU-intensive tasks: Number of CPU cores
  • I/O-intensive tasks: 2-4x CPU cores
  • Mixed workloads: Start with 2-3, monitor and adjust

Memory Management

Large video files and high concurrency can consume significant memory. Monitor system resources and adjust concurrency accordingly.

Timeout Settings

Set appropriate timeouts for different job types. Video processing can take significantly longer than audio extraction.

Scale Your Media Processing

With batch processing mastered, explore other API references to build comprehensive media processing workflows with MediaX SDK.