Stars
Framework-Agnostic RL Environments for LLM Fine-Tuning
R package for performing 2-sample MR using IEU OpenGWAS database
DeepEP: an efficient expert-parallel communication library
A roadmap for "generative AI" learning resources
A resource of adverse drug effects extracted from FDA structured product labels
Code for implementing Mendelian randomization investigations
Stop messing around with finicky sampling parameters and just use DRµGS!
A bibliography and survey of the papers surrounding o1
Tile primitives for speedy kernels
A list of awesome tools for complex trait genetics.
Entropy Based Sampling and Parallel CoT Decoding
Reproduction of OpenGenome dataset curated by the Evo team
Improving Alignment and Robustness with Circuit Breakers
Sparsify transformers with SAEs and transcoders
Influence Functions with (Eigenvalue-corrected) Kronecker-Factored Approximate Curvature
Implementation of Influence Function approximations for differently sized ML models, using PyTorch
A library for making RepE control vectors
RAFT, or Retrieval-Augmented Fine-Tuning, is a method comprising of a fine-tuning and a RAG-based retrieval phase. It is particularly suited for the creation of agents that realistically emulate a …
Impact of typos and common misspellings on LLM task performance.
A library for mechanistic interpretability of GPT-style language models
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash attention v2.