Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Performance Concepts and Key Metrics
- Latency, throughput, power consumption, and resource utilization.
- Distinguishing between system-level and model-level bottlenecks.
- Profiling strategies for inference versus training phases.
Profiling on Huawei Ascend
- Leveraging CANN Profiler and MindInsight.
- Diagnostic analysis of kernels and operators.
- Managing offload patterns and memory mapping.
Profiling on Biren GPU
- Exploring Biren SDK performance monitoring features.
- Optimizing kernel fusion, memory alignment, and execution queues.
- Conducting power and temperature-aware profiling.
Profiling on Cambricon MLU
- Utilizing BANGPy and Neuware performance tools.
- Gaining kernel-level visibility and interpreting logs effectively.
- Integrating the MLU profiler with various deployment frameworks.
Graph and Model-Level Optimization
- Strategies for graph pruning and quantization.
- Operator fusion and restructuring computational graphs.
- Standardizing input sizes and optimizing batch processing.
Memory and Kernel Optimization
- Refining memory layout and data reuse patterns.
- Managing buffers efficiently across different chipsets.
- Applying platform-specific kernel-level tuning techniques.
Cross-Platform Best Practices
- Achieving performance portability through abstraction strategies.
- Developing shared tuning pipelines for multi-chip environments.
- Case study: optimizing an object detection model across Ascend, Biren, and MLU.
Summary and Future Directions
Requirements
- Hands-on experience with AI model training or deployment pipelines.
- Solid understanding of GPU/MLU compute principles and model optimization techniques.
- Foundational knowledge of performance profiling tools and associated metrics.
Target Audience
- Performance engineers.
- Machine learning infrastructure teams.
- AI system architects.
