All Machine Learning Pages

Machine Learning

Understanding State Space Models

A look at State Space Models like S4, H3, and Mamba

Machine Learning

Writing CUDA Kernels for PyTorch

This page explores the basics of programming with CUDA, and shows how to build custom PyTorch operations that run on Nvidia GPUs

Machine Learning

Multi-Query & Grouped-Query Attention

Description of Multi-Query Attention (MQA), and Grouped-Query Attention (GQA) in transformer models.

Machine Learning

Mixture of Experts Pattern for Transformer Models

This page explores the Mixture of Experts pattern and its application to transformer models.

Machine Learning

Distributed Training and DeepSpeed

An examination of Distributed Training Techniques with PyTorch and DeepSpeed

Machine Learning