CATLASS Project Documentation

1 Practices

Step-by-step practices for learning how to use and develop each layer of CATLASS, covering complete operator development, testing, tuning, and model integration.

Development Workflow

• 01Quick Start: Prepare the template library environment, then build and run operator examples
• 02Host-side Matmul Assembly: Learn how to assemble Matmul on the host side
• 03Kernel Development: Explore Kernel code, including template assembly, Arguments, Params, and key functions
• 04Block Mmad Development: Explore Block Mmad code, template assembly, and major interfaces
• 05Block Scheduler Development: Explore Block Scheduler code, template assembly, and major interfaces
• 06Tile Development: Explore Tile Copy and Tile Mmad code, template assembly, and major interfaces
• 07Epilogue Adaptation: Adapt GEMM epilogues at the Host and Kernel layers, and develop Epilogue Block and Tile components
• 08Debugging and Analysis: Use debugging tools, locate precision issues, and analyze performance bottlenecks
• 09Example Contribution Guide: Follow the complete process for designing, developing, testing, and merging an example
• 10Innovative Example Development Guide: Follow the complete workflow for developing an innovative example
• 11Matmul Optimization Guide: Improve performance through Tiling parameter tuning and different Dispatch policies
• 12 Example Integration: Adapt examples and integrate them into full networks (contributions welcome)

Debugging and Analysis

• Ascend C Dump: Use Ascend C operator debugging APIs
• msDebug: Use msDebug in a CATLASS example project
• Performance Analysis Tools: Use performance analysis tools such as msProf and Profiling
• Print Debugging: Print diagnostic information during operator debugging
• Precision Analysis Basics: Learn the fundamentals of precision analysis
• Precision Issue Debugging: Locate precision issues in examples
• Performance Bottleneck Analysis and Optimization: Analyze and optimize performance bottlenecks

Specialized Practices

This section contains specialized practice documents contributed by both internal and external developers.

• TLA example refactoring (contributions welcome)
• Migrating from Atlas A2 to Ascend 950: Recommended approach for migrating existing Atlas A2 operators to Ascend 950
• Conv Kernel Development: Develop Conv operators
• Conv Kernel Optimization: Optimize the performance of Conv operators
• FA Kernel Optimization: Optimize the performance of FA operators
• Fused Operator Optimization: Explore performance tuning cases for CV fused operators
• Direct Kernel Invocation: Invoke newly developed operators directly with <<<>>>

2 Design

Project Overview

Introduces the project positioning, layered modular design, and repository structure.

Kernel Design

Basics

• Atlas A2 Hardware: Overview of the Atlas A2 hardware architecture
• Atlas A2 GEMM Instruction Set: Hardware instructions used by Atlas A2 GEMM examples

Core Design

• 01Example Design: Summary and index of example design documents in the repository
• 02Swizzle Policies: Swizzle policies that affect the execution order of basic compute blocks on AI Cores
• 03Dispatch Policies: DispatchPolicy, an important template parameter of Block Mmad
• 04Matrix Multiplication Template Summary: Summary of Matmul example templates, theoretical templates, engineering optimizations, and usage
• 05Adaptive Sliding Window Tiling: Adaptive sliding window Tiling policy
• 06 Low-precision Topics (contributions welcome)

TLA Design

• 01Layout: TLA Layout structures and related interfaces
• 02LayoutTag: Legacy layout tags and interfaces, including RowMajor, ColumnMajor, zN, and nZ
• 03Tensor: Tensor structures

EVG Design

• 01EVG Design Overview: EVG positioning, layering, execution model, and graph organization
• 02EVG Extension Guide: When to add a ComputeFn or node and the constraints for implementing them
• 03EVG Quick Start: Basic EVG integration using Matmul + Add as an example

3 API Documentation

CATLASS APIs

• API Index: Entry point for CATLASS API documentation
• GEMM API: General matrix multiplication interfaces
• EVG API: EVG integration, parameter ordering, and commonly used nodes

Related APIs

• Ascend C API: Ascend C API reference

4 Appendix

Technical Articles

Fundamentals

• C++ Templates Explained
• Ascend C Operator Development Documentation

Advanced Topics

Content covering concepts, troubleshooting, performance optimization, and best practices is to be added.

Training Videos

Ascend Community Courses

• Ascend C Operator Development (Beginner)
• Ascend C Operator Development (Intermediate)
• Ascend C Operator Development (Advanced)

CATLASS Courses

• Code Power Special: Mastering the Fundamentals of the CATLASS Template Library: The first CATLASS course, introducing the project, operator quick start, development roadmap, and community collaboration
• Code Power Special: Hands-on CATLASS Operator Development: The second CATLASS course, using a basic Matmul operator to explain NPU-based matrix multiplication theory and implementation
• Code Power Special: Deep Optimization with the CATLASS Template Library: The third CATLASS course, introducing optimization techniques for operator development with CATLASS