Notes on Machine Learning
Table of Contents
Reinforcement Learning Resources
Collection of Reinforcement Learning Resources
Reinforcement Learning: An Introduction by Richard Sutton and Andrew Barto (2nd Edition, in progress) This is a great introductory book exploring Reinforcement learning.
RL Course by David Silver This course by David Silver covers introductory topics, Markov Decision Processes, Dynamic Programming, Model-Free Prediction, Model-Free Control, Value Function Approximation, Policy Gradient Methods, Integrating Learning and Planning, and Exploration and Exploitation. Additional Course Resources available here
aikorea/awesome-rl A curated list of Reinforcement learning resources
Neural Networks: Zero to Hero by Andrej Karpathy Lecture series that goes over the construction of neural networks in code.
Large Language Models (LLMs)
Key Research Papers and Resources
Language Models are Few-Shot Learners (GPT-3) The seminal paper introducing GPT-3, which demonstrated impressive few-shot learning capabilities in language models.
Constitutional AI: A Research Agenda Anthropic’s research on developing AI systems that are both capable and aligned with human values.
Llama 2: Open Foundation and Fine-Tuned Chat Models Meta’s research on their open-source LLM, including architecture improvements and safety considerations.
Scaling Laws for Neural Language Models Important paper from OpenAI showing how model performance scales with compute, dataset size, and parameter count.
Learning Resources
State of GPT - Andrej Karpathy Comprehensive overview of how GPT models work and their practical applications.
The Illustrated Transformer Visual guide to understanding transformer architecture, the foundation of modern LLMs.
Hugging Face Course Free course covering transformer models, fine-tuning, and practical NLP tasks.
Tools and Libraries
Transformers Library Hugging Face’s popular library for working with pre-trained models.
LangChain Framework for developing applications powered by language models.
OpenAI Cookbook Collection of examples and guides for using OpenAI’s models effectively.
Interesting Recent Developments
- Mixture of Experts (MoE): Research on scaling language models efficiently using specialized sub-networks
- Retrieval-Augmented Generation (RAG): Combining LLMs with external knowledge bases
- Constitutional AI: Approaches for making AI systems more aligned and controllable
- Multimodal Models: Integration of text, images, and other modalities (e.g., GPT-4V, Claude 3)
Benchmarks and Evaluation
HELM Holistic evaluation framework for language models.
Big-Bench Collaborative benchmark for testing large language model capabilities.
MMLU Massive Multitask Language Understanding benchmark for testing model knowledge.
Ethics and Safety Resources
AI Alignment Research Overview Anthropic’s overview of key challenges in AI alignment.
Machine Learning Safety Curated list of resources on AI safety and ethics.