CUDA Templates for Linear Algebra Subroutines

Source to source compiler for PyTorch. It makes PyTorch programs faster on single accelerators and distributed.

A Fusion Code Generator for NVIDIA GPUs

A high-throughput and memory-efficient inference and serving engine for LLMs

Tensors and Dynamic neural networks in Python with strong GPU acceleration

AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.

Class notes for the course "Long Term Memory in AI - Vector Search and Databases" COS 495 @ Princeton Fall 2023

An experimental open-source attempt to make GPT-4 fully autonomous.

Source code for Twitter's Recommendation Algorithm

A Python-level JIT compiler designed to make unmodified PyTorch programs faster.

A TensorFlow implementation of this Nvidia paper: https://arxiv.org/pdf/1604.07316.pdf with some changes

Tutel MoE: An Optimized Mixture-of-Experts Implementation

A tiny scalar-valued autograd engine and a neural net library on top of it with PyTorch-like API

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

Optimizing SGEMM kernel functions on NVIDIA GPUs to a close-to-cuBLAS performance.

A compressed alternative to matrix multiplication using state-of-the art compression ROBE-Z

Training materials associated with NVIDIA's CUDA Training Series (www.olcf.ornl.gov/cuda-training-series/)

Enabling PyTorch on Google TPU

Stepwise optimizations of DGEMM on CPU, reaching performance faster than Intel MKL eventually, even under multithreading.

A Learnable LSH Framework for Efficient NN Training

Highly optimized DGEMV on CPU with both serial and parallel performance better than MKL and OpenBLAS.

Solutions to Stanford CS231n Spring 2018 Course Assignments.