Task Scheduling for AI Workloads

ABSTRACT

The rapid growth of AI workloads has exposed limitations in traditional CPU and GPU scheduling. CPUs provide fairness and responsiveness but struggle with parallelism and memory-intensive operations, while GPUs deliver high-throughput execution yet rely on CPU coordination for tasks such as data preprocessing and kernel management. Hybrid CPU–GPU scheduling addresses these challenges by dynamically distributing tasks to leverage both CPU flexibility and GPU parallelism. This study evaluates CPU-only, GPU-only, and hybrid approaches across benchmarks for workload scaling, composition, resource constraints, and real-world AI scenario test suites. Results from experts and experimentation show that hybrid scheduling dramatically improves throughput for compute-bound, parallel workloads while maintaining high GPU utilization, though latency-sensitive inference tasks may incur slight overhead. These findings underscore the importance of adaptive, workload-aware scheduling strategies for hybrid AI architectures.