Tinkerd

Machine Learning

Feb 23, 2025

Attention in Transformers

Part II of An Interpretability Guide to Language Models

Machine Learning

Dec 31, 2024

Zero-Layer Transformers

Part I of An Interpretability Guide to Language Models

Machine Learning

Jul 8, 2024

Understanding State Space Models

A look at State Space Models like S4, H3, and Mamba

Machine Learning

Apr 20, 2024

Writing CUDA Kernels for PyTorch

This page explores the basics of programming with CUDA, and shows how to build custom PyTorch operations that run on Nvidia GPUs

Cloud

Mar 10, 2024

AWS Networking Crash Course

A crash course on AWS Networking, covering VPCs, Subnets, Internet Gateways, NAT Gateways, and more.

Machine Learning

Dec 24, 2023

Multi-Query & Grouped-Query Attention

Description of Multi-Query Attention (MQA), and Grouped-Query Attention (GQA) in transformer models.

Machine Learning

Aug 27, 2023

Mixture of Experts Pattern for Transformer Models

This page explores the Mixture of Experts pattern and its application to transformer models.

Machine Learning

Jun 18, 2023

Distributed Training and DeepSpeed

An examination of Distributed Training Techniques with PyTorch and DeepSpeed

Machine Learning

May 23, 2023

Language Model Fine-Tuning with LoRA

This page explore Low-Rank Adaptations, LoRA, as a method for fine-tuning language models.

Machine Learning

Mar 26, 2023

BERT Encoder Layer

This page explains the inner workings of the BERT Encoder Layer.

Machine Learning

Mar 26, 2023

BERT Embeddings

This page explains the concept of embeddings in neural networks and illustrates the function of the BERT Embedding Layer.

Machine Learning

Mar 14, 2023

BERT Tokenization

This page examines the tokenization logic used to prepare inputs for BERT.

Cloud

Nov 17, 2020

Machine Learning Pipelines with Kubeflow

This page shows how to automated machine learning workflows using Kubeflow Pipelines.