GitHub Jupyter Notebook Weekly Trending Repositories

lyogavin/airllm

AirLLM 70B inference with single 4GB GPU

Quickstart | Configurations | MacOS | Example notebooks | FAQ

AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. You can even run 405B Llama 3.1 on 8GB, DeepSeek-V3 (671B) on ~12GB, and Kimi K3 (2.8T) — the largest open-source model released to date — on under 4GB, because sparse MoE models stream one expert at a time rather than a whole layer.

AI Agents Recommendation:

Updates

[2026/07] Kimi K3 (2.8T) support: the largest open-source model runs on a single card in 3.72GB of VRAM, measured end to end on one RTX 6000 Ada. Per-expert streaming loads only the experts a token actually routes to. K3 brings three requirements of its own: pip install compressed-tensors flash-attn (its model code mandates flash attention regardless of what you request), a CUDA 12 build of torch, since no prebuilt flash-attn wheel exists for CUDA 13 yet, and transformers 4.56.x, as its remote code does not load on 5.x.

[2026/06] v3.0: FP8 model support + the latest models. Run DeepSeek-V3 (671B) on ~12GB and Qwen3-235B on ~3GB, plus Qwen3, Llama 3.x/4, DeepSeek V2/V3, Phi-4, Gemma and more — all through a single AutoModel.

[2024/08/20] v2.11.0: Support Qwen2.5

[2024/08/18] v2.10.1 Support CPU inference. Support non sharded models. Thanks @NavodPeiris for the great work!

[2024/07/30] Support Llama3.1 405B (example notebook). Support 8bit/4bit quantization.

[2024/04/20] AirLLM supports Llama3 natively already. Run Llama3 70B on 4GB single GPU.

[2023/12/25] v2.8.2: Support MacOS running 70B large language models.

[2023/12/20] v2.7: Support AirLLMMixtral.

[2023/12/20] v2.6: Added AutoModel, automatically detect model type, no need to provide model class to initialize model.

[2023/12/18] v2.5: added prefetching to overlap the model loading and compute. 10% speed improvement.

[2023/12/03] added support of ChatGLM, QWen, Baichuan, Mistral, InternLM!

[2023/12/02] added support for safetensors. Now support all top 10 models in open llm leaderboard.

[2023/12/01] airllm 2.0. Support compressions: 3x run time speed up!

[2023/11/20] airllm Initial version!

Star History

Quickstart

1. Install package

First, install the airllm pip package.

pip install airllm

2. Inference

Then, initialize AirLLMLlama2, pass in the huggingface repo ID of the model being used, or the local path, and inference can be performed similar to a regular transformer model.

(You can also specify the path to save the splitted layered model through layer_shards_saving_path when init AirLLMLlama2.

from airllm import AutoModel

MAX_LENGTH = 128
# just pass a hugging face repo id — works with almost any popular model:
model = AutoModel.from_pretrained("Qwen/Qwen3-32B")

# go bigger with the exact same one line:
#model = AutoModel.from_pretrained("Qwen/Qwen3-235B-A22B")     # 235B, runs in ~3GB
#model = AutoModel.from_pretrained("deepseek-ai/DeepSeek-V3")  # 671B, runs in ~12GB

# or use a model's local path...
#model = AutoModel.from_pretrained("/home/ubuntu/.cache/huggingface/hub/models--Qwen--Qwen3-32B/snapshots/...")

input_text = [
        'What is the capital of United States?',
        #'I like',
    ]

input_tokens = model.tokenizer(input_text,
    return_tensors="pt", 
    return_attention_mask=False, 
    truncation=True, 
    max_length=MAX_LENGTH, 
    padding=False)
           
generation_output = model.generate(
    input_tokens['input_ids'].cuda(), 
    max_new_tokens=20,
    use_cache=True,
    return_dict_in_generate=True)

output = model.tokenizer.decode(generation_output.sequences[0])

print(output)

Note: During inference, the original model will first be decomposed and saved layer-wise. Please ensure there is sufficient disk space in the huggingface cache directory.

Model Compression - 3x Inference Speed Up!

We just added model compression based on block-wise quantization-based model compression. Which can further speed up the inference speed for up to 3x , with almost ignorable accuracy loss! (see more performance evaluation and why we use block-wise quantization in this paper)

How to enable model compression speed up:

Step 1. make sure you have bitsandbytes installed by pip install -U bitsandbytes
Step 2. make sure airllm verion later than 2.0.0: pip install -U airllm
Step 3. when initialize the model, passing the argument compression ('4bit' or '8bit'):

model = AutoModel.from_pretrained("garage-bAInd/Platypus2-70B-instruct",
                     compression='4bit' # specify '8bit' for 8-bit block-wise quantization 
                    )

What are the differences between model compression and quantization?

Quantization normally needs to quantize both weights and activations to really speed things up. Which makes it harder to maintain accuracy and avoid the impact of outliers in all kinds of inputs.

While in our case the bottleneck is mainly at the disk loading, we only need to make the model loading size smaller. So, we get to only quantize the weights' part, which is easier to ensure the accuracy.

Configurations

When initialize the model, we support the following configurations:

compression: supported options: 4bit, 8bit for 4-bit or 8-bit block-wise quantization, or by default None for no compression
profiling_mode: supported options: True to output time consumptions or by default False
layer_shards_saving_path: optionally another path to save the splitted model
hf_token: huggingface token can be provided here if downloading gated models like: meta-llama/Llama-2-7b-hf
prefetching: prefetching to overlap the model loading and compute. By default, turned on. For now, only AirLLMLlama2 supports this.
delete_original: if you don't have too much disk space, you can set delete_original to true to delete the original downloaded hugging face model, only keep the transformed one to save half of the disk space.

MacOS

Just install airllm and run the code the same as on linux. See more in Quick Start.

make sure you installed mlx and torch
you probably need to install python native see more here
only Apple silicon is supported

Example [python notebook] (https://github.com/lyogavin/airllm/blob/main/air_llm/examples/run_on_macos.ipynb)

Example Python Notebook

Example colabs here:

example of other models (ChatGLM, QWen, Baichuan, Mistral, etc):

ChatGLM:

from airllm import AutoModel
MAX_LENGTH = 128
model = AutoModel.from_pretrained("THUDM/chatglm3-6b-base")
input_text = ['What is the capital of China?',]
input_tokens = model.tokenizer(input_text,
    return_tensors="pt", 
    return_attention_mask=False, 
    truncation=True, 
    max_length=MAX_LENGTH, 
    padding=True)
generation_output = model.generate(
    input_tokens['input_ids'].cuda(), 
    max_new_tokens=5,
    use_cache= True,
    return_dict_in_generate=True)
model.tokenizer.decode(generation_output.sequences[0])

QWen:

from airllm import AutoModel
MAX_LENGTH = 128
model = AutoModel.from_pretrained("Qwen/Qwen-7B")
input_text = ['What is the capital of China?',]
input_tokens = model.tokenizer(input_text,
    return_tensors="pt", 
    return_attention_mask=False, 
    truncation=True, 
    max_length=MAX_LENGTH)
generation_output = model.generate(
    input_tokens['input_ids'].cuda(), 
    max_new_tokens=5,
    use_cache=True,
    return_dict_in_generate=True)
model.tokenizer.decode(generation_output.sequences[0])

Baichuan, InternLM, Mistral, etc:

from airllm import AutoModel
MAX_LENGTH = 128
model = AutoModel.from_pretrained("baichuan-inc/Baichuan2-7B-Base")
#model = AutoModel.from_pretrained("internlm/internlm-20b")
#model = AutoModel.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
input_text = ['What is the capital of China?',]
input_tokens = model.tokenizer(input_text,
    return_tensors="pt", 
    return_attention_mask=False, 
    truncation=True, 
    max_length=MAX_LENGTH)
generation_output = model.generate(
    input_tokens['input_ids'].cuda(), 
    max_new_tokens=5,
    use_cache=True,
    return_dict_in_generate=True)
model.tokenizer.decode(generation_output.sequences[0])

To request other model support: here

Supported Models

AirLLM works out of the box with virtually every popular open LLM — just pass its Hugging Face ID to AutoModel.from_pretrained(...). That covers all the major families:

Llama (2 / 3 / 3.1 / 3.3 / 4) · Qwen (1 / 2 / 2.5 / 3, including MoE and FP8) · DeepSeek (V2 / V3 / R1) · Mistral & Mixtral · Phi · Gemma · ChatGLM · Baichuan · InternLM · Yi — and most new models the day they're released.

Tiny GPU, huge models

The trick: AirLLM only ever keeps one layer on the GPU at a time, so the VRAM you need depends on the model's layer size — not its total size. That's how a 671B model fits on a hobbyist card:

Model	Size	GPU VRAM
Qwen3 / Mistral / Phi (≈8B)	8B	~1–2 GB
Qwen3-30B / Mixtral (MoE)	30–47B	~1–3 GB
Qwen3-235B (MoE)	235B	~3 GB
Llama 3.x 70B (full precision)	70B	~4 GB
Llama 3.1 405B	405B	~8 GB
DeepSeek-V3	671B	~12 GB

Same one line of code for all of them — no special setup.

Acknowledgement

A lot of the code are based on SimJeg's great work in the Kaggle exam competition. Big shoutout to SimJeg:

GitHub account @SimJeg, the code on Kaggle, the associated discussion.

FAQ

1. MetadataIncompleteBuffer

safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer

If you run into this error, most possible cause is you run out of disk space. The process of splitting model is very disk-consuming. See this. You may need to extend your disk space, clear huggingface .cache and rerun.

2. ValueError: max() arg is an empty sequence

Most likely you are loading QWen or ChatGLM model with Llama2 class. Try the following:

For QWen model:

from airllm import AutoModel #<----- instead of AirLLMLlama2
AutoModel.from_pretrained(...)

For ChatGLM model:

from airllm import AutoModel #<----- instead of AirLLMLlama2
AutoModel.from_pretrained(...)

3. 401 Client Error....Repo model ... is gated.

Some models are gated models, needs huggingface api token. You can provide hf_token:

model = AutoModel.from_pretrained("meta-llama/Llama-2-7b-hf", #hf_token='HF_API_TOKEN')

4. ValueError: Asking to pad but the tokenizer does not have a padding token.

Some model's tokenizer doesn't have padding token, so you can set a padding token or simply turn the padding config off:

input_tokens = model.tokenizer(input_text,
   return_tensors="pt", 
   return_attention_mask=False, 
   truncation=True, 
   max_length=MAX_LENGTH, 
   padding=False  #<-----------   turn off padding 
)

Citing AirLLM

If you find AirLLM useful in your research and wish to cite it, please use the following BibTex entry:

@software{airllm2023,
  author = {Gavin Li},
  title = {AirLLM: scaling large language models on low-end commodity computers},
  url = {https://github.com/lyogavin/airllm/},
  version = {0.0},
  year = {2023},
}

Contribution

Welcomed contributions, ideas and discussions!

If you find it useful, please ⭐ or buy me a coffee! 🙏

DataExpert-io/data-engineer-handbook

This is a repo with links to everything you'd ever want to learn about data engineering

The Data Engineering Handbook

This repo has all the resources you need to become an amazing data engineer!

Join the free Databricks AI boot camp on August 3rd here and signup for Databricks Free Edition here

Getting started

If you are new to data engineering, start by following this 2024 breaking into data engineering roadmap

If you are here for the 4-week free beginner boot camp you can check out:

If you are here for the 6-week free intermediate boot camp you can check out

For more applied learning:

Check out the projects section for more hands-on examples!
Check out the interviews section for more advice on how to pass data engineering interviews!
Check out the books section for a list of high quality data engineering books
Check out the communities section for a list of high quality data engineering communities to join
Check out the newsletter section to learn via email

Resources

Great list of over 25 books

Great list of over 10 communities to join:

Top must-join communities for DE:

Top must-join communities for ML:

Companies:

Orchestration
- Mage
- Astronomer
- Prefect
- Dagster
- Airflow
- Kestra
- Shipyard
- Hamilton
Data Lake / Cloud
- Tabular
- Microsoft
- Databricks
- Onehouse
- Delta Lake
- Ilum
- DuckLake
- Apache Iceberg
- Apache Polaris
- Lakekeeper
Data Warehouse
Data Quality
- dbt
- Metaplane
- Gable
- Great Expectations
- Streamdal
- Coalesce
- Soda
- DQOps
- HEDDA.IO
- Dingo
Education Companies
Analytics / Visualization
- Preset
- Starburst
- Metabase
- Looker Studio
- Tableau
- Power BI
- Hex
- Apache Superset
- Evidence
- Redash
- Lightdash
Data Integration
- Cube
- Fivetran
- Airbyte
- dlt
- Sling
- Meltano
- Estuary
- Arpe.io
Semantic Layers
- Cube
- dbt Semantic Layer
Modern OLAP
- Apache Druid
- ClickHouse
- Apache Pinot
- Apache Kylin
- DuckDB
- QuestDB
- StarRocks
LLM application library
Real-Time Data
Data Lineage
- OpenLineage

Data Engineering blogs of companies:

Data Engineering Whitepapers:

Social Media Accounts

Here's the mostly comprehensive list of data engineering creators: (You have to have at least 5k followers somewhere to be added!)

YouTube

Name	YouTube Channel	Follower Count
ByteByteGo	ByteByteGo	1,000,000+
Data with Baraa	Data with Baraa	195,000+
Zach Wilson	Data with Zach	150,000+
Shashank Mishra	E-learning Bridge	100,000+
Seattle Data Guy	Seattle Data Guy	100,000+
TrendyTech	TrendyTech	100,000+
Darshil Parmar	Darshil Parmar	100,000+
Andreas Kretz	Andreas Kretz	100,000+
The Ravit Show	The Ravit Show	100,000+
Guy in a Cube	Guy in a Cube	100,000+
Adam Marczak	Adam Marczak	100,000+
nullQueries	nullQueries	100,000+
TECHTFQ by Thoufiq	TECHTFQ by Thoufiq	100,000+
SQLBI	SQLBI	100,000+
Alex Freberg	Alex The Analyst	100,000+
Ankur Ranjan	Big Data Show	100,000+
Prashanth Kumar Pandey	ScholarNest	77,000+
ITVersity	ITVersity	67,000+
Soumil Shah	Soumil Shah	50,000
Ansh Lamba	Ansh Lamba	18,000+
Azure Lib	Azure Lib	10,000+
Advancing Analytics	Advancing Analytics	10,000+
Kahan Data Solutions	Kahan Data Solutions	10,000+
Ankit Bansal	Ankit Bansal	10,000+
Mr. K Talks Tech	Mr. K Talks Tech	10,000+
Samuel Focht	Python Basics	10,000+
Mehdi Ouazza	Mehdio DataTV	3,000+
Alex Merced	Alex Merced Data	N/A
John Kutay	John Kutay	N/A
Emil Kaminski	Databricks For Professionals	5,000+

Name	LinkedIn Profile	Follower Count
Zach Wilson	Zach Wilson	400,000+
Chip Huyen	Chip Huyen	250,000+
Shashank Mishra	Shashank Mishra	100,000+
Seattle Data Guy	Ben Rogojan	100,000+
TrendyTech	Sumit Mittal	100,000+
Darshil Parmar	Darshil Parmar	100,000+
Andreas Kretz	Andreas Kretz	100,000+
ByteByteGo (Alex Xu)	Alex Xu	100,000+
Azure Lib (Deepak Goyal)	Deepak Goyal	100,000+
Alex Freberg	Alex Freberg	100,000+
SQLBI (Marco Russo)	Marco Russo	50,000+
Ankit Bansal	Ankit Bansal	50,000+
Marc Lamberti	Marc Lamberti	50,000+
Ankur Ranjan	Ankur Ranjan	48,000+
ITVersity (Durga Gadiraju)	Durga Gadiraju	48,000+
Prashanth Kumar Pandey	Prashanth Kumar Pandey	37,000+
Alex Merced	Alex Merced	30,000+
Ijaz Ali	Ijaz Ali	24,000+
Mehdi Ouazza	Mehdi Ouazza	20,000+
Ananth Packkildurai	Ananth Packkildurai	18,000+
Ansh Lamba	Ansh Lamba	13,000+
Manojkumar Vadivel	Manojkumar Vadivel	12,000+
Advancing Analytics	Simon Whiteley	10,000+
Li Yin	Li Yin	10,000+
Jaco van Gelder	Jaco van Gelder	10,000+
Joseph Machado	Joseph Machado	10,000+
Eric Roby	Eric Roby	10,000+
Simon Späti	Simon Späti	10,000+
Constantin Lungu	Constantin Lungu	10,000+
Lakshmi Sontenam	Lakshmi Sontenam	9,500+
Dani Pálma	Daniel Pálma	9,000+
Soumil Shah	Soumil Shah	8,000+
Arnaud Milleker	Arnaud Milleker	7,000+
Dimitri Visnadi	Dimitri Visnadi	7,000+
Lenny	Lenny A	6,000+
Dipankar Mazumdar	Dipankar Mazumdar	5,000+
Daniel Ciocirlan	Daniel Ciocirlan	5,000+
Hugo Lu	Hugo Lu	5,000+
Tobias Macey	Tobias Macey	5,000+
Marcos Ortiz	Marcos Ortiz	5,000+
Julien Hurault	Julien Hurault	5,000+
John Kutay	John Kutay	5,000+
Hassaan Akbar	Hassaan Akbar	5,000+
Subhankar	Subhankar	5,000+
Nitin	Nitin	N/A
Hassaan	Hassaan	5000+
Javier de la Torre	Javier	5000+

X/Twitter

Name	X/Twitter Profile	Follower Count
ByteByteGo	alexxubyte	100,000+
Dan Kornas	@dankornas	66,000+
Zach Wilson	EcZachly	30,000+
Seattle Data Guy	SeattleDataGuy	10,000+
SQLBI	marcorus	10,000+
Joseph Machado	startdataeng	5,000+
Alex Merced	@amdatalakehouse	N/A
John Kutay	@JohnKutay	N/A
Mehdi Ouazza	mehd_io	N/A

Instagram

Name	Instagram Profile	Follower Count
Sundas Khalid	sundaskhalidd	300,000+
Zach Wilson	eczachly	150,000+
Andreas Kretz	learndataengineering	5,000+
Alex Merced	@alexmercedcoder	N/A

TikTok

Name	TikTok Profile	Follower Count
Zach Wilson	@eczachly	70,000+
Alex Freberg	@alex_the_analyst	10,000+
Mehdi Ouazza	@mehdio_datatv	N/A

Great Podcasts

Great list of 20+ newsletters

Top must follow newsletters for data engineering:

Glossaries:

Design Patterns

Courses / Academies

DataExpert.io course use code HANDBOOK10 for a discount!
LearnDataEngineering.com
Technical Freelancer Academy Use code zwtech for a discount!
IBM Data Engineering for Everyone
PrepNPlaced Open Learning - 153 hours of free SQL, Python, PySpark, and Power BI lessons with MCQs and a free skill test
Qwiklabs
DataCamp
Udemy Courses from Shruti Mantri
Rock the JVM teaches Spark (in Scala), Flink and others
Data Engineering Zoomcamp by DataTalksClub
Efficient Data Processing in Spark
Scaler
DataTeams - Data Engingeer hiring platform
Udemy Courses from Daniel Blanco
DeepLearning.AI Data Engineering Professional Certificate

Certifications Courses

microsoft/AI-For-Beginners

12 Weeks, 24 Lessons, AI for All!

Artificial Intelligence for Beginners - A Curriculum


AI For Beginners - Sketchnote by @girlie_mac

Explore the world of Artificial Intelligence (AI) with our 12-week, 24-lesson curriculum! It includes practical lessons, quizzes, and labs. The curriculum is beginner-friendly and covers tools like TensorFlow and PyTorch, as well as ethics in AI

🌐 Multi-Language Support

Supported via GitHub Action (Automated & Always Up-to-Date)

Prefer to Clone Locally?

This repository includes 50+ language translations which significantly increases the download size. To clone without translations, use sparse checkout:

Bash / macOS / Linux:
git clone --filter=blob:none --sparse https://github.com/microsoft/AI-For-Beginners.git
cd AI-For-Beginners
git sparse-checkout set --no-cone '/*' '!translations' '!translated_images'
CMD (Windows):
git clone --filter=blob:none --sparse https://github.com/microsoft/AI-For-Beginners.git
cd AI-For-Beginners
git sparse-checkout set --no-cone "/*" "!translations" "!translated_images"
This gives you everything you need to complete the course with a much faster download.

If you wish to have additional translations languages supported are listed here

Join the Community

What you will learn

Mindmap of the Course

In this curriculum, you will learn:

Different approaches to Artificial Intelligence, including the "good old" symbolic approach with Knowledge Representation and reasoning (GOFAI).
Neural Networks and Deep Learning, which are at the core of modern AI. We will illustrate the concepts behind these important topics using code in two of the most popular frameworks - TensorFlow and PyTorch.
Neural Architectures for working with images and text. We will cover recent models but may be a bit lacking in the state-of-the-art.
Less popular AI approaches, such as Genetic Algorithms and Multi-Agent Systems.

What we will not cover in this curriculum:

Find all additional resources for this course in our Microsoft Learn collection

Business cases of using AI in Business. Consider taking Introduction to AI for business users learning path on Microsoft Learn, or AI Business School, developed in cooperation with INSEAD.
Classic Machine Learning, which is well described in our Machine Learning for Beginners Curriculum.
Practical AI applications built using Cognitive Services. For this, we recommend that you start with modules Microsoft Learn for vision, natural language processing, Generative AI with Azure OpenAI Service and others.
Specific ML Cloud Frameworks, such as Azure Machine Learning, Microsoft Fabric, or Azure Databricks. Consider using Build and operate machine learning solutions with Azure Machine Learning and Build and Operate Machine Learning Solutions with Azure Databricks learning paths.
Conversational AI and Chat Bots. There is a separate Create conversational AI solutions learning path, and you can also refer to this blog post for more detail.
Deep Mathematics behind deep learning. For this, we would recommend Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville, which is also available online at https://www.deeplearningbook.org/.

For a gentle introduction to AI in the Cloud topics you may consider taking the Get started with artificial intelligence on Azure Learning Path.

Content

	Lesson Link	PyTorch/Keras/TensorFlow	Lab
0	Course Setup	Setup Your Development Environment
I	Introduction to AI
01	Introduction and History of AI	-	-
II	Symbolic AI
02	Knowledge Representation and Expert Systems	Expert Systems / Ontology /Concept Graph
III	Introduction to Neural Networks
03	Perceptron	Notebook	Lab
04	Multi-Layered Perceptron and Creating our own Framework	Notebook	Lab
05	Intro to Frameworks (PyTorch/TensorFlow) and Overfitting	PyTorch / Keras / TensorFlow	Lab
IV	Computer Vision	PyTorch / TensorFlow	Explore Computer Vision on Microsoft Azure
06	Intro to Computer Vision. OpenCV	Notebook	Lab
07	Convolutional Neural Networks & CNN Architectures	PyTorch /TensorFlow	Lab
08	Pre-trained Networks and Transfer Learning and Training Tricks	PyTorch / TensorFlow	Lab
09	Autoencoders and VAEs	PyTorch / TensorFlow
10	Generative Adversarial Networks & Artistic Style Transfer	PyTorch / TensorFlow
11	Object Detection	TensorFlow	Lab
12	Semantic Segmentation. U-Net	PyTorch / TensorFlow
V	Natural Language Processing	PyTorch /TensorFlow	Explore Natural Language Processing on Microsoft Azure
13	Text Representation. Bow/TF-IDF	PyTorch / TensorFlow
14	Semantic word embeddings. Word2Vec and GloVe	PyTorch / TensorFlow
15	Language Modeling. Training your own embeddings	PyTorch / TensorFlow	Lab
16	Recurrent Neural Networks	PyTorch / TensorFlow
17	Generative Recurrent Networks	PyTorch / TensorFlow	Lab
18	Transformers. BERT.	PyTorch /TensorFlow
19	Named Entity Recognition	TensorFlow	Lab
20	Large Language Models, Prompt Programming and Few-Shot Tasks	PyTorch
VI	Other AI Techniques
21	Genetic Algorithms	Notebook
22	Deep Reinforcement Learning	PyTorch /TensorFlow	Lab
23	Multi-Agent Systems
VII	AI Ethics
24	AI Ethics and Responsible AI	Microsoft Learn: Responsible AI Principles
IX	Extras
25	Multi-Modal Networks, CLIP and VQGAN	Notebook

Each lesson contains

Pre-reading material
Executable Jupyter Notebooks, which are often specific to the framework (PyTorch or TensorFlow). The executable notebook also contains a lot of theoretical material, so to understand the topic you need to go through at least one version of the notebook (either PyTorch or TensorFlow).
Labs available for some topics, which give you an opportunity to try applying the material you have learned to a specific problem.
Some sections contain links to MS Learn modules that cover related topics.

Getting Started

🎯 New to AI? Start Here!

If you're completely new to AI and want quick, hands-on examples, check out our Beginner-Friendly Examples! These include:

🌟 Hello AI World - Your first AI program (pattern recognition)
🧠 Simple Neural Network - Build a neural network from scratch
🖼️ Image Classifier - Classify images with detailed comments
💬 Text Sentiment - Analyze positive/negative text

These examples are designed to help you understand AI concepts before diving into the full curriculum.

📚 Full Curriculum Setup

We have created a setup lesson to help you with setting up your development environment. - For Educators, we have created a curricula setup lesson for you too!
How to Run the code in a VSCode or a Codespace

Follow these steps:

Fork the Repository: Click on the "Fork" button at the top-right corner of this page.

Clone the Repository: git clone https://github.com/microsoft/AI-For-Beginners.git

Don't forget to star (🌟) this repo to find it easier later.

Meet other Learners

Join our official AI Discord server to meet and network with other learners taking this course and get support.

If you have product feedback or questions whilst building visit our Azure AI Foundry Developer Forum

Quizzes

A note about quizzes: All quizzes are contained in the Quiz-app folder in etc\quiz-app, or Online Here They are linked from within the lessons the quiz app can be run locally or deployed to Azure; follow the instruction in the quiz-app folder. They are gradually being localized.

Help Wanted

Do you have suggestions or found spelling or code errors? Raise an issue or create a pull request.

Special Thanks

✍️ Primary Author: Dmitry Soshnikov, PhD
🔥 Editor: Jen Looper, PhD
🎨 Sketchnote illustrator: Tomomi Imura
✅ Quiz Creator: Lateefah Bello, MLSA
🙏 Core Contributors: Evgenii Pishchik

Other Curricula

Our team produces other curricula! Check out:

LangChain

Azure / Edge / MCP / Agents

Generative AI Series

Core Learning

Copilot Series

Getting Help

If you get stuck or have any questions about building AI apps. Join fellow learners and experienced developers in discussions about MCP. It's a supportive community where questions are welcome and knowledge is shared freely.

If you have product feedback or errors while building visit:

microsoft/generative-ai-for-beginners

21 Lessons, Get Started Building with Generative AI

21 Lessons teaching everything you need to know to start building Generative AI applications

🌐 Multi-Language Support

Supported via GitHub Action (Automated & Always Up-to-Date)

Prefer to Clone Locally?

This repository includes 50+ language translations which significantly increases the download size. To clone without translations, use sparse checkout:

Bash / macOS / Linux:
git clone --filter=blob:none --sparse https://github.com/microsoft/generative-ai-for-beginners.git
cd generative-ai-for-beginners
git sparse-checkout set --no-cone '/*' '!translations' '!translated_images'
CMD (Windows):
git clone --filter=blob:none --sparse https://github.com/microsoft/generative-ai-for-beginners.git
cd generative-ai-for-beginners
git sparse-checkout set --no-cone "/*" "!translations" "!translated_images"
This gives you everything you need to complete the course with a much faster download.

Generative AI for Beginners (Version 3) - A Course

Learn the fundamentals of building Generative AI applications with our 21-lesson comprehensive course by Microsoft Cloud Advocates.

🌱 Getting Started

This course has 21 lessons. Each lesson covers its own topic so start wherever you like!

Lessons are labeled either "Learn" lessons explaining a Generative AI concept or "Build" lessons that explain a concept and code examples in both Python and TypeScript when possible.

For .NET Developers checkout Generative AI for Beginners (.NET Edition)!

Each lesson also includes a "Keep Learning" section with additional learning tools.

What You Need

To run the code of this course, you can use either:

Azure OpenAI Service - Lessons: "aoai-assignment"
Microsoft Foundry Models - Lessons: "githubmodels" (GitHub Models is retiring at the end of July 2026 - use Microsoft Foundry Models instead)
OpenAI API - Lessons: "oai-assignment"
Foundry Local - Run models fully offline on your own device, no cloud subscription required
Basic knowledge of Python or TypeScript is helpful - *For absolute beginners check out these Python and TypeScript courses
A GitHub account to fork this entire repo to your own GitHub account

We have created a Course Setup lesson to help you with setting up your development environment.

Don't forget to star (🌟) this repo to find it easier later.

🧠 Ready to Deploy?

If you are looking for more advanced code samples, check out our collection of Generative AI Code Samples in both Python and TypeScript.

🗣️ Meet Other Learners, Get Support

Join our official Microsoft Foundry Discord server to meet and network with other learners taking this course and get support.

Ask questions or share product feedback in our Microsoft Foundry Developer Forum on Github.

🚀 Building a Startup?

Visit Microsoft for Startups to find out how to get started building with Azure credits today.

🙏 Want to help?

Do you have suggestions or found spelling or code errors? Raise an issue or Create a pull request

📂 Each lesson includes:

A short video introduction to the topic
A written lesson located in the README
Python and TypeScript code samples supporting Azure OpenAI and OpenAI API
Links to extra resources to continue your learning

🗃️ Lessons

#	Lesson Link	Description	Video	Extra Learning
00	Course Setup	Learn: How to Setup Your Development Environment	Video Coming Soon	Learn More
01	Introduction to Generative AI and LLMs	Learn: Understanding what Generative AI is and how Large Language Models (LLMs) work.	Video	Learn More
02	Exploring and comparing different LLMs	Learn: How to select the right model for your use case	Video	Learn More
03	Using Generative AI Responsibly	Learn: How to build Generative AI Applications responsibly	Video	Learn More
04	Understanding Prompt Engineering Fundamentals	Learn: Hands-on Prompt Engineering Best Practices	Video	Learn More
05	Creating Advanced Prompts	Learn: How to apply prompt engineering techniques that improve the outcome of your prompts.	Video	Learn More
06	Building Text Generation Applications	Build: A text generation app using Azure OpenAI / OpenAI API	Video	Learn More
07	Building Chat Applications	Build: Techniques for efficiently building and integrating chat applications.	Video	Learn More
08	Building Search Apps Vector Databases	Build: A search application that uses Embeddings to search for data.	Video	Learn More
09	Building Image Generation Applications	Build: An image generation application	Video	Learn More
10	Building Low Code AI Applications	Build: A Generative AI application using Low Code tools	Video	Learn More
11	Integrating External Applications with Function Calling	Build: What is function calling and its use cases for applications	Video	Learn More
12	Designing UX for AI Applications	Learn: How to apply UX design principles when developing Generative AI Applications	Video	Learn More
13	Securing Your Generative AI Applications	Learn: The threats and risks to AI systems and methods to secure these systems.	Video	Learn More
14	The Generative AI Application Lifecycle	Learn: The tools and metrics to manage the LLM Lifecycle and LLMOps	Video	Learn More
15	Retrieval Augmented Generation (RAG) and Vector Databases	Build: An application using a RAG Framework to retrieve embeddings from a Vector Databases	Video	Learn More
16	Open Source Models and Hugging Face	Build: An application using open source models available on Hugging Face	Video	Learn More
17	AI Agents	Build: An application using an AI Agent Framework	Video	Learn More
18	Fine-Tuning LLMs	Learn: The what, why and how of fine-tuning LLMs	Video	Learn More
19	Building with SLMs	Learn: The benefits of building with Small Language Models	Video Coming Soon	Learn More
20	Building with Mistral Models	Learn: The features and differences of the Mistral Family Models	Video Coming Soon	Learn More
21	Building with Meta Models	Learn: The features and differences of the Meta Family Models	Video Coming Soon	Learn More

🌟 Special thanks

Special thanks to John Aziz for creating all of the GitHub Actions and workflows

Bernhard Merkle for making key contributions to each lesson to improve the learner and code experience.

🎒 Other Courses

Our team produces other courses! Check out:

LangChain

Azure / Edge / MCP / Agents

Generative AI Series

Core Learning

Copilot Series

Getting Help

If you have product feedback or errors while building visit:

Lordog/dive-into-llms

《动手学大模型Dive into LLMs》系列编程实践教程

《动手学大模型》系列编程实践教程

项目动机/ 教程目录/ 贡献者列表

💡 Updates

2025/06/06 感谢各位朋友们的关注和积极反馈！我们从以下两个方面对本教程进行了更新：

上线国产化《大模型开发全流程》公益教程（含PPT、实验手册和视频），此处特别感谢华为昇腾社区的支持！
在原系列编程实践教程的基础上进行内容更新，并增加了新的主题（数学推理、GUI Agent、大模型对齐、隐写术等）！

🎯 项目动机

《动手学大模型》系列编程实践教程，由上海交通大学《自然语言处理前沿技术》（NIS8021）、《人工智能安全技术》课程（NIS3353）讲义拓展而来（教师：张倬胜），旨在提供大模型相关的入门编程参考。本教程属公益性质、完全免费。通过简单实践，帮助同学们快速入门大模型，更好地开展课程设计或学术研究。

📚 教程目录

教程内容	简介	地址
微调与部署	预训练模型微调与部署指南：想提升预训练模型在指定任务上的性能？让我们选择合适的预训练模型，在特定任务上进行微调，并将微调后的模型部署成方便使用的Demo！	[课件] [教程] [脚本]
提示学习与思维链	大模型的API调用与推理指南：“AI在线求鼓励？大模型对一些问题的回答令人大跌眼镜，但它可能只是想要一句「鼓励」”	[课件] [教程] [脚本]
知识编辑	语言模型的编辑方法和工具：想操控语言模型在对指定知识的记忆？让我们选择合适的编辑方法，对特定知识进行编辑，并将对编辑后的模型进行验证！	[课件] [教程] [脚本]
数学推理	如何让大模型学会数学推理？让我们快速蒸馏一个迷你R1！	[课件] [教程] [脚本]
模型水印	语言模型的文本水印：在语言模型生成的内容中嵌入人类不可见的水印	[课件] [教程] [脚本]
越狱攻击	想要得到更好的安全，要先从学会攻击开始。让我们了解越狱攻击如何撬开大模型的嘴！	[课件] [教程] [脚本]
大模型隐写	“看不见的墨水”！想让大模型在流畅回答的同时，悄悄携带只有“自己人”能识别的信息吗？大模型隐写告诉你！	[课件] [教程] [脚本]
多模态模型	作为能够更充分模拟真实世界的多模态大语言模型，其如何实现更强大的多模态理解和生成能力？多模态大语言模型是否能够帮助实现AGI？	[课件] [教程] [脚本]
GUI智能体	想要饭来张口、解放双手？那么让我们一起来让AI Agent替你点外卖、回消息、购物比价吧！	[课件] [教程] [脚本]
智能体安全	大模型智能体迈向了未来操作系统之旅。然而，大模型在开放智能体场景中能意识到风险威胁吗？	[课件] [教程] [脚本]
RLHF安全对齐	基于PPO的RLHF实验指南：本教程”十分危险“，阅读后请检查你的大模型是否在冷笑。	[课件] [教程] [脚本]

🔥 新上线：国产化《大模型开发全流程》

✨ 我们联合华为昇腾推出的《大模型开发全流程》公益教程正式上线！前沿技术+代码实践，手把手带你玩转AI大模型 ✨:

在《动手学大模型》原系列教程的基础上，我们联合华为开发了《大模型开发全流程》系列课程。本系列教程基于昇腾基础软硬件开发，覆盖PPT、实验手册、视频等教程形式。该教程分为初级、中级、高级系列，面向不同的大模型实践需求，旨在将前沿技术通过代码实践的方式，为相关研究者、开发者由浅入深地提供快速上手、应用昇腾已支持模型和全新模型迁移调优的全流程开发指南。
🚀 前往昇腾社区探索《大模型开发全流程》系列课程：

👉《大模型开发学习专区》@ 昇腾社区 👈
✨ 课程内容展示 ✨

🙏 免责声明

本教程所有内容仅仅来自于贡献者的个人经验、互联网数据、日常科研工作中的相关积累。所有技巧仅供参考，不保证百分百正确。若有任何问题，欢迎提交 Issue 或 PR。另本项目所用徽章来自互联网，如侵犯了您的图片版权请联系我们删除，谢谢。

🤝 欢迎贡献

本教程目前是一个正在进行中的项目，如有疏漏在所难免，欢迎任何的PR及issue讨论。

❤️ 贡献者列表

感谢以下老师和同学对本项目的支持与贡献：

《动手学大模型》系列教程开发团队：

上海交通大学：张倬胜、袁童鑫、马欣贝、何志威、杜巍、赵皓东、吴宗儒、吴铮、董凌众、张玉龙
新加坡国立大学：费豪

《大模型开发全流程》系列教程开发团队：

上海交通大学：张倬胜、刘功申、陈星宇、程彭洲、董凌众、何志威、鞠天杰、马欣贝、吴铮、吴宗儒、闫子赫、姚杳、袁童鑫、赵皓东;
华为昇腾社区：ZOMI、谢乾、程黎明、楼梨华、焦泽昱

🌟 Star History

jackfrued/Python-100-Days

Python - 100天从新手到大师

作者：骆昊

说明1：最近有一个非常火的岗位叫做 FDE（前沿部署工程师），国内首套 FDE 实战课已经为大家准备好了，请点击传送门。

说明2：如果访问 GitHub 比较慢的话，可以关注我的知乎账号（Python-Jack），上面的“从零开始学Python”专栏比较适合初学者，其他的专栏如“数据思维和统计思维”、“基于Python的数据分析”、“说走就走的AI之旅”、“AI智能体开发”等也在持续创作和更新中，欢迎大家关注、点赞和评论。如果有一起打卡学习或付费咨询的需求，可以加入付费交流群，新用户可以通过下方二维码付费之后添加我的私人微信（微信号：jackfrued），然后邀请大家进入付费学习打卡群，添加微信时请备注好自己的称呼和需求，我会为大家提供力所能及的帮助。

说明3：本项目对应的部分视频已经同步到 Bilibili，有兴趣的小伙伴可以点赞、投币、关注，一键三连支持一下！

Python应用领域和职业发展分析

简单的说，Python是一个“优雅”、“明确”、“简单”的编程语言。

学习曲线低，非专业人士也能上手
开源系统，拥有强大的生态圈
解释型语言，完美的平台可移植性
动态类型语言，支持面向对象和函数式编程
代码规范程度高，可读性强

Python在以下领域都有用武之地。

后端开发 - Python / Java / Go / PHP
DevOps - Python / Shell / Ruby
数据采集 - Python / C++ / Java
量化交易 - Python / C++ / R
数据科学 - Python / R / Julia / Matlab
机器学习 - Python / R / C++ / Julia
自动化测试 - Python / Shell

作为一名Python开发者，根据个人的喜好和职业规划，可以选择的就业领域也非常多。

Python后端开发工程师（服务器、云平台、数据接口）
Python运维工程师（自动化运维、SRE、DevOps）
Python数据分析师（数据分析、商业智能、数字化运营）
Python数据科学家（机器学习、深度学习、算法专家）
Python爬虫工程师（不推荐此赛道！！！）
Python测试工程师（自动化测试、测试开发）

说明：目前，数据科学赛道是非常热门的方向，因为不管是互联网行业还是传统行业都已经积累了大量的数据，各行各业都需要数据科学家从已有的数据中发现更多的商业价值，从而为企业的决策提供数据的支撑，这就是所谓的数据驱动决策。

给初学者的几个建议：

Make English as your working language. （让英语成为你的工作语言）
Practice makes perfect. （熟能生巧）
All experience comes from the mistakes you've made. （所有的经验都源于犯过的错误）
Don't be a freeloader. （学会分享，不要只当伸手党）
Embrace AI to boost your productivity.（拥抱AI，提升效率）

Day01~20 - Python语言基础

Day01 - 初识Python

Python简介
- Python编年史
- Python优缺点
- Python应用领域
安装Python环境
- Windows环境
- macOS环境

Day02 - 第一个Python程序

编写代码的工具
你好世界
注释你的代码

Day03 - Python语言中的变量

一些常识
变量和类型
变量命名
变量的使用

Day04 - Python语言中的运算符

算术运算符
赋值运算符
比较运算符和逻辑运算符
运算符和表达式应用
- 华氏和摄氏温度转换
- 计算圆的周长和面积
- 判断闰年

Day05 - 分支结构

使用if和else构造分支结构
使用match和case构造分支结构
分支结构的应用
- 分段函数求值
- 百分制成绩转换成等级
- 计算三角形的周长和面积

Day06 - 循环结构

for-in循环
while循环
break和continue
嵌套的循环结构
循环结构的应用
- 判断素数
- 最大公约数
- 猜数字游戏

Day07 - 分支和循环结构实战

例子1：100以内的素数
例子2：斐波那契数列
例子3：寻找水仙花数
例子4：百钱百鸡问题
例子5：CRAPS赌博游戏

Day08 - 常用数据结构之列表-1

创建列表
列表的运算
元素的遍历

Day09 - 常用数据结构之列表-2

列表的方法
- 添加和删除元素
- 元素位置和频次
- 元素排序和反转
列表生成式
嵌套列表
列表的应用

Day10 - 常用数据结构之元组

元组的定义和运算
打包和解包操作
交换变量的值
元组和列表的比较

Day11 - 常用数据结构之字符串

字符串的定义
- 转义字符
- 原始字符串
- 字符的特殊表示
字符串的运算
- 拼接和重复
- 比较运算
- 成员运算
- 获取字符串长度
- 索引和切片
字符的遍历
字符串的方法
- 大小写相关操作
- 查找操作
- 性质判断
- 格式化
- 修剪操作
- 替换操作
- 拆分与合并
- 编码与解码
- 其他方法

Day12 - 常用数据结构之集合

创建集合
元素的变量
集合的运算
- 成员运算
- 二元运算
- 比较运算
集合的方法
不可变集合

Day13 - 常用数据结构之字典

创建和使用字典
字典的运算
字典的方法
字典的应用

Day14 - 函数和模块

定义函数
函数的参数
- 位置参数和关键字参数
- 参数的默认值
- 可变参数
用模块管理函数
标准库中的模块和函数

Day15 - 函数应用实战

例子1：随机验证码
例子2：判断素数
例子3：最大公约数和最小公倍数
例子4：数据统计
例子5：双色球随机选号

Day16 - 函数使用进阶

高阶函数
Lambda函数
偏函数

Day17 - 函数高级应用

装饰器
递归调用

Day18 - 面向对象编程入门

类和对象
定义类
创建和使用对象
初始化方法
面向对象的支柱
面向对象案例
- 例子1：数字时钟
- 例子2：平面上的点

Day19 - 面向对象编程进阶

可见性和属性装饰器
动态属性
静态方法和类方法
继承和多态

Day20 - 面向对象编程应用

扑克游戏
工资结算系统

Day21~30 - Python语言应用

Day21 - 文件读写和异常处理

打开和关闭文件
读写文本文件
异常处理机制
上下文管理器语法
读写二进制文件

Day22 - 对象的序列化和反序列化

JSON概述
读写JSON格式的数据
包管理工具pip
使用网络API获取数据

Day23 - Python读写CSV文件

CSV文件介绍
将数据写入CSV文件
从CSV文件读取数据

Day24 - Python读写Excel文件-1

Excel简介
读Excel文件
写Excel文件
调整样式
公式计算

Day25 - Python读写Excel文件-2

Excel简介
读Excel文件
写Excel文件
调整样式
生成统计图表

Day26 - Python操作Word和PowerPoint文件

操作Word文档
生成PowerPoint

Day27 - Python操作PDF文件

从PDF中提取文本
旋转和叠加页面
加密PDF文件
批量添加水印
创建PDF文件

Day28 - Python处理图像

入门知识
用Pillow处理图像
使用Pillow绘图

Day29 - Python发送邮件和短信

发送电子邮件
发送短信

Day30 - 正则表达式的应用

正则表达式相关知识
Python对正则表达式的支持
- 例子1：输入验证
- 例子2：内容提取
- 例子3：内容替换
- 例子4：长句拆分

Day31~35 - 其他相关内容

Python语言进阶

重要知识点
数据结构和算法
函数的使用方式
面向对象相关知识
迭代器和生成器
并发编程

Web前端入门

用HTML标签承载页面内容
用CSS渲染页面
用JavaScript处理交互式行为
Vue.js入门
Element的使用
Bootstrap的使用

玩转Linux操作系统

操作系统发展史和Linux概述
Linux基础命令
Linux中的实用程序
Linux的文件系统
Vim编辑器的应用
环境变量和Shell编程
软件的安装和服务的配置
网络访问和管理
其他相关内容

Day36~45 - 数据库基础和进阶

Day36 - 关系型数据库和MySQL概述

关系型数据库概述
MySQL简介
安装MySQL
MySQL基本命令

Day37 - SQL详解之DDL

建库建表
删除表和修改表

Day38 - SQL详解之DML

insert操作
delete操作
update操作

Day39 - SQL详解之DQL

投影和别名
筛选数据
空值处理
去重
排序
聚合函数
嵌套查询
分组操作
表连接
- 笛卡尔积
- 内连接
- 自然连接
- 外连接
窗口函数
- 定义窗口
- 排名函数
- 取数函数

Day40 - SQL详解之DCL

创建用户
授予权限
召回权限

Day41 - MySQL新特性

JSON类型
窗口函数
公共表表达式

Day42 - 视图、函数和过程

视图
- 使用场景
- 创建视图
- 使用限制
函数
- 内置函数
- 用户自定义函数（UDF）
过程
- 创建过程
- 调用过程

Day43 - 索引

执行计划
索引的原理
创建索引
- 普通索引
- 唯一索引
- 前缀索引
- 复合索引
注意事项

Day44 - Python接入MySQL数据库

安装三方库
创建连接
获取游标
执行SQL语句
通过游标抓取数据
事务提交和回滚
释放连接
编写ETL脚本

Day45 - Hive实战

Hive概述
环境搭建
常用命令
基本语法
建表操作
写入数据
常用函数
分组聚合
抽样操作
排序操作
横向展开
性能优化

Day46~60 - 实战Django

Day46 - Django快速上手

Web应用工作机制
HTTP请求和响应
Django框架概述
5分钟快速上手

Day47 - 深入模型

关系型数据库配置
使用ORM完成对模型的CRUD操作
管理后台的使用
Django模型最佳实践
模型定义参考

Day48 - 静态资源和Ajax请求

加载静态资源
Ajax概述
用Ajax实现投票功能

Day49 - Cookie和Session

实现用户跟踪
cookie和session的关系
Django框架对session的支持
视图函数中的cookie读写操作

Day50 - 报表和日志

通过HttpResponse修改响应头
使用StreamingHttpResponse处理大文件
使用xlwt生成Excel报表
使用reportlab生成PDF报表
使用ECharts生成前端图表

Day51 - 日志和调试工具栏

配置日志
配置Django-Debug-Toolbar
优化ORM代码

Day52 - 中间件的应用

什么是中间件
Django框架内置的中间件
自定义中间件及其应用场景

Day53 - 前后端分离开发入门

返回JSON格式的数据
用Vue.js渲染页面

Day54 - RESTful架构和DRF入门

REST概述
DRF库使用入门
前后端分离开发
JWT的应用

Day55 - RESTful架构和DRF进阶

使用CBV
数据分页
数据筛选

Day56 - 使用缓存

网站优化第一定律
在Django项目中使用Redis提供缓存服务
在视图函数中读写缓存
使用装饰器实现页面缓存
为数据接口提供缓存服务

Day57 - 接入三方平台

文件上传表单控件和图片文件预览
服务器端如何处理上传的文件

Day58 - 异步任务和定时任务

网站优化第二定律
配置消息队列服务
在项目中使用Celery实现任务异步化
在项目中使用Celery实现定时任务

Day59 - 单元测试

Day60 - 项目上线

Python中的单元测试
Django框架对单元测试的支持
使用版本控制系统
配置和使用uWSGI
动静分离和Nginx配置
配置HTTPS
配置域名解析

Day61~65 - 网络数据采集

Day61 - 网络数据采集概述

网络爬虫的概念及其应用领域
网络爬虫的合法性探讨
开发网络爬虫的相关工具
一个爬虫程序的构成

Day62 - 数据抓取和解析

使用requests三方库实现数据抓取
页面解析的三种方式
- 正则表达式解析
- XPath解析
- CSS选择器解析

Day63 - Python中的并发编程

Day64 - 使用Selenium抓取网页动态内容

安装Selenium
加载页面
查找元素和模拟用户行为
隐式等待和显示等待
执行JavaScript代码
Selenium反爬破解
设置无头浏览器

Day65 - 爬虫框架Scrapy简介

Scrapy核心组件
Scrapy工作流程
安装Scrapy和创建项目
编写蜘蛛程序
编写中间件和管道程序
Scrapy配置文件

Day66~80 - Python数据分析

Day66 - 数据分析概述

数据分析师的职责
数据分析师的技能栈
数据分析相关库

Day67 - 环境准备

安装和使用anaconda
- conda相关命令
安装和使用jupyter-lab
- 安装和启动
- 使用小技巧

Day68 - NumPy的应用-1

创建数组对象
数组对象的属性
数组对象的索引运算
- 普通索引
- 花式索引
- 布尔索引
- 切片索引
案例：使用数组处理图像

Day69 - NumPy的应用-2

数组对象的相关方法
- 获取描述性统计信息
- 其他相关方法

Day70 - NumPy的应用-3

数组的运算
- 数组跟标量的运算
- 数组跟数组的运算
通用一元函数
通用二元函数
广播机制
Numpy常用函数

Day71 - NumPy的应用-4

向量
行列式
矩阵
多项式

Day72 - 深入浅出pandas-1

创建Series对象
Series对象的运算
Series对象的属性和方法

Day73 - 深入浅出pandas-2

创建DataFrame对象
DataFrame对象的属性和方法
读写DataFrame中的数据

Day74 - 深入浅出pandas-3

数据重塑
- 数据拼接
- 数据合并
数据清洗
- 缺失值
- 重复值
- 异常值
- 预处理

Day75 - 深入浅出pandas-4

数据透视
- 获取描述性统计信息
- 排序和头部值
- 分组聚合
- 透视表和交叉表
数据呈现

Day76 - 深入浅出pandas-5

计算同比环比
窗口计算
相关性判定

Day77 - 深入浅出pandas-6

索引的使用
- 范围索引
- 分类索引
- 多级索引
- 间隔索引
- 日期时间索引

Day78 - 数据可视化-1

安装和导入matplotlib
创建画布
创建坐标系
绘制图表
- 折线图
- 散点图
- 柱状图
- 饼状图
- 直方图
- 箱线图
显示和保存图表

Day79 - 数据可视化-2

高阶图表
- 气泡图
- 面积图
- 雷达图
- 玫瑰图
- 3D图表

Day80 - 数据可视化-3

Seaborn
Pyecharts

Day81~90 - 机器学习

Day81 - 浅谈机器学习

人工智能发展史
什么是机器学习
机器学习应用领域
机器学习的分类
机器学习的步骤
第一次机器学习

Day82 - k最近邻算法

距离的度量
数据集介绍
kNN分类的实现
模型评估
参数调优
kNN回归的实现

Day83 - 决策树和随机森林

决策树的构建
- 特征选择
- 数据分裂
- 树的剪枝
实现决策树模型
随机森林概述

Day84 - 朴素贝叶斯算法

贝叶斯定理
朴素贝叶斯
算法原理
- 训练阶段
- 预测阶段
- 代码实现
算法优缺点

Day85 - 回归模型

回归模型的分类
回归系数的计算
新数据集介绍
线性回归代码实现
回归模型的评估
引入正则化项
线性回归另一种实现
多项式回归
逻辑回归

Day86 - K-Means聚类算法

算法原理
数学描述
代码实现

Day87 - 集成学习算法

算法分类
AdaBoost
GBDT
XGBoost
LightGBM

Day88 - 神经网络模型

基本构成
工作原理
代码实现
模型优缺点

Day89 - 自然语言处理入门

词袋模型
词向量
NPLM和RNN
Seq2Seq
Transformer

Day90 - 机器学习实战

数据探索
特征工程
模型训练
模型评估
模型部署

Day91~99 - 团队项目开发

第91天：团队项目开发的问题和解决方案

软件过程模型
- 经典过程模型（瀑布模型）
  - 可行性分析（研究做还是不做），输出《可行性分析报告》。
  - 需求分析（研究做什么），输出《需求规格说明书》和产品界面原型图。
  - 概要设计和详细设计，输出概念模型图（ER图）、物理模型图、类图、时序图等。
  - 编码 / 测试。
  - 上线 / 维护。
  瀑布模型最大的缺点是无法拥抱需求变化，整套流程结束后才能看到产品，团队士气低落。
- 敏捷开发（Scrum）- 产品所有者、Scrum Master、研发人员 - Sprint
  - 产品的Backlog（用户故事、产品原型）。
  - 计划会议（评估和预算）。
  - 日常开发（站立会议、番茄工作法、结对编程、测试先行、代码重构……）。
  - 修复bug（问题描述、重现步骤、测试人员、被指派人）。
  - 发布版本。
  - 评审会议（Showcase，用户需要参与）。
  - 回顾会议（对当前迭代周期做一个总结）。
  补充：敏捷软件开发宣言
  - 个体和互动 高于流程和工具
  - 工作的软件 高于详尽的文档
  - 客户合作 高于合同谈判
  - 响应变化 高于遵循计划
  角色：产品所有者（决定做什么，能对需求拍板的人）、团队负责人（解决各种问题，专注如何更好的工作，屏蔽外部对开发团队的影响）、开发团队（项目执行人员，具体指开发人员和测试人员）。
  
  准备工作：商业案例和资金、合同、憧憬、初始产品需求、初始发布计划、入股、组建团队。
  
  敏捷团队通常人数为8-10人。
  
  工作量估算：将开发任务量化，包括原型、Logo设计、UI设计、前端开发等，尽量把每个工作分解到最小任务量，最小任务量标准为工作时间不能超过两天，然后估算总体项目时间。把每个任务都贴在看板上面，看板上分三部分：to do（待完成）、in progress（进行中）和done（已完成）。
项目团队组建
- 团队的构成和角色
- 编程规范和代码审查（flake8、pylint）
- Python中的一些“惯例”（请参考《Python惯例-如何编写Pythonic的代码》）
- 影响代码可读性的原因：
  - 代码注释太少或者没有注释
  - 代码破坏了语言的最佳实践
  - 反模式编程（意大利面代码、复制-黏贴编程、自负编程、……）
团队开发工具介绍
- 版本控制：Git、Mercury
- 缺陷管理：Gitlab、Redmine
- 敏捷闭环工具：禅道、JIRA
- 持续集成：Jenkins、Travis-CI
请参考《团队项目开发的问题和解决方案》。

项目选题和理解业务

选题范围设定
- CMS（用户端）：新闻聚合网站、问答/分享社区、影评/书评网站等。
- MIS（用户端+管理端）：KMS、KPI考核系统、HRS、CRM系统、供应链系统、仓储管理系统等。
- App后台（管理端+数据接口）：二手交易类、报刊杂志类、小众电商类、新闻资讯类、旅游类、社交类、阅读类等。
- 其他类型：自身行业背景和工作经验、业务容易理解和把控。
需求理解、模块划分和任务分配
- 需求理解：头脑风暴和竞品分析。
- 模块划分：画思维导图（XMind），每个模块是一个枝节点，每个具体的功能是一个叶节点（用动词表述），需要确保每个叶节点无法再生出新节点，确定每个叶子节点的重要性、优先级和工作量。
- 任务分配：由项目负责人根据上面的指标为每个团队成员分配任务。

制定项目进度表（每日更新）

模块	功能	人员	状态	完成	工时	计划开始	计划结束	备注
评论	添加评论	王大锤	正在进行	50%	4	2018/8/7	2018/8/7
	删除评论	王大锤	等待	0%	2	2018/8/7	2018/8/7
	查看评论	白元芳	正在进行	20%	4	2018/8/7	2018/8/7	需要进行代码审查
	评论投票	白元芳	等待	0%	4	2018/8/8	2018/8/8

OOAD和数据库设计

UML（统一建模语言）的类图
通过模型创建表（正向工程），例如在Django项目中可以通过下面的命令创建二维表。
```
python manage.py makemigrations app
python manage.py migrate
```
使用PowerDesigner绘制物理模型图。
通过数据表创建模型（反向工程），例如在Django项目中可以通过下面的命令生成模型。
```
python manage.py inspectdb > app/models.py
```

第92天：Docker容器技术详解

Docker简介
安装Docker
使用Docker创建容器（Nginx、MySQL、Redis、Gitlab、Jenkins）
构建Docker镜像（Dockerfile的编写和相关指令）
容器编排（Docker-compose）
集群管理（Kubernetes）

第93天：MySQL性能优化

基本原则
InnoDB引擎
索引的使用和注意事项
数据分区
SQL优化
配置优化
架构优化

第94天：网络API接口设计

设计原则
- 关键问题
- 其他问题
文档撰写

第95天：[使用Django开发商业项目](./Day91-100/95.使用Django开发商业项目.md)

项目开发中的公共问题

数据库的配置（多数据库、主从复制、数据库路由）
缓存的配置（分区缓存、键设置、超时设置、主从复制、故障恢复（哨兵））
日志的配置
分析和调试（Django-Debug-ToolBar）
好用的Python模块（日期计算、图像处理、数据加密、三方API）

REST API设计

项目中的重点难点剖析

使用缓存缓解数据库压力 - Redis
使用消息队列做解耦合和削峰 - Celery + RabbitMQ

第96天：软件测试和自动化测试

单元测试

测试的种类
编写单元测试（unittest、pytest、nose2、tox、ddt、……）
测试覆盖率（coverage）

Django项目部署

部署前的准备工作
- 关键设置（SECRET_KEY / DEBUG / ALLOWED_HOSTS / 缓存 / 数据库）
- HTTPS / CSRF_COOKIE_SECUR / SESSION_COOKIE_SECURE
- 日志相关配置
Linux常用命令回顾
Linux常用服务的安装和配置
uWSGI/Gunicorn和Nginx的使用
- Gunicorn和uWSGI的比较
  - 对于不需要大量定制化的简单应用程序，Gunicorn是一个不错的选择，uWSGI的学习曲线比Gunicorn要陡峭得多，Gunicorn的默认参数就已经能够适应大多数应用程序。
  - uWSGI支持异构部署。
  - 由于Nginx本身支持uWSGI，在线上一般都将Nginx和uWSGI捆绑在一起部署，而且uWSGI属于功能齐全且高度定制的WSGI中间件。
  - 在性能上，Gunicorn和uWSGI其实表现相当。
使用虚拟化技术（Docker）部署测试环境和生产环境

性能测试

AB的使用
SQLslap的使用
sysbench的使用

自动化测试

使用Shell和Python进行自动化测试
使用Selenium实现自动化测试
- Selenium IDE
- Selenium WebDriver
- Selenium Remote Control
测试工具Robot Framework介绍

第97天：电商网站技术要点剖析

商业模式和需求要点
物理模型设计
第三方登录
缓存预热和查询缓存
购物车的实现
支付功能集成
秒杀和超卖问题
静态资源管理
全文检索方案

第98天：项目部署上线和性能调优

MySQL数据库调优
Web服务器性能优化
- Nginx负载均衡配置
- Keepalived实现高可用
代码性能调优
- 多线程
- 异步化
静态资源访问优化
- 云存储
- CDN

第99天：面试中的公共问题

计算机基础
Python基础
Web框架相关
爬虫相关问题
数据分析
项目相关

第100天 - 补充内容

面试宝典
- Python 面试宝典
- SQL 面试宝典（数据分析师）
- 商业分析面试宝典
- 机器学习面试宝典
机器学习数学基础
深度学习
- 计算机视觉
- 大语言模型

karpathy/micrograd

A tiny scalar-valued autograd engine and a neural net library on top of it with PyTorch-like API

micrograd

A tiny Autograd engine (with a bite! 😃). Implements backpropagation (reverse-mode autodiff) over a dynamically built DAG and a small neural networks library on top of it with a PyTorch-like API. Both are tiny, with about 100 and 50 lines of code respectively. The DAG only operates over scalar values, so e.g. we chop up each neuron into all of its individual tiny adds and multiplies. However, this is enough to build up entire deep neural nets doing binary classification, as the demo notebook shows. Potentially useful for educational purposes.

Installation

pip install micrograd

Example usage

Below is a slightly contrived example showing a number of possible supported operations:

from micrograd.engine import Value

a = Value(-4.0)
b = Value(2.0)
c = a + b
d = a * b + b**3
c += c + 1
c += 1 + c + (-a)
d += d * 2 + (b + a).relu()
d += 3 * d + (b - a).relu()
e = c - d
f = e**2
g = f / 2.0
g += 10.0 / f
print(f'{g.data:.4f}') # prints 24.7041, the outcome of this forward pass
g.backward()
print(f'{a.grad:.4f}') # prints 138.8338, i.e. the numerical value of dg/da
print(f'{b.grad:.4f}') # prints 645.5773, i.e. the numerical value of dg/db

Training a neural net

The notebook demo.ipynb provides a full demo of training an 2-layer neural network (MLP) binary classifier. This is achieved by initializing a neural net from micrograd.nn module, implementing a simple svm "max-margin" binary classification loss and using SGD for optimization. As shown in the notebook, using a 2-layer neural net with two 16-node hidden layers we achieve the following decision boundary on the moon dataset:

Training a GPT

For a more advanced example, see microgpt, which trains and samples from a full GPT-2-like transformer in pure, dependency-free Python. It builds on a more efficient and better version of the autograd engine here (storing local gradients at forward time instead of per-op backward closures), and is the complete algorithm in a single file — everything else is just efficiency. See also the accompanying explainer post for a detailed walkthrough.

Tracing / visualization

For added convenience, the notebook trace_graph.ipynb produces graphviz visualizations. E.g. this one below is of a simple 2D neuron, arrived at by calling draw_dot on the code below, and it shows both the data (left number in each node) and the gradient (right number in each node).

from micrograd import nn
n = nn.Neuron(2)
x = [Value(1.0), Value(-2.0)]
y = n(x)
dot = draw_dot(y)

Running tests

To run the unit tests you will have to install PyTorch, which the tests use as a reference for verifying the correctness of the calculated gradients. Then simply:

python -m pytest

License

MIT

charliedream1/ai_quant_trade

股票AI操盘手：从学习、模拟到实盘，一站式平台。包含股票知识、策略实例、大模型、因子挖掘、传统策略、机器学习、深度学习、强化学习、图网络、高频交易、C++部署和聚宽实例代码等，可以方便学习、模拟及实盘交易

🤖 AI量化交易操盘手

一站式AI量化交易平台 · 从学习、模拟到实盘

ENGLISH VERSION

_{👇 关注公众号 · 加入知识星球，获取更多实战内容与视频教程}

🔥 新特性 • 📖 简介 • 🚀 快速开始 • 📊 量化策略 • 🤖 大模型 • ⛏️ 因子挖掘 • 💾 数据 • 🛠️ 工具 • 🎁 资源

✨ 核心亮点

🎯 定位	📌 说明
🏦 一站式平台	从学习、模拟到实盘，全流程覆盖
📈 多元策略	大模型、因子挖掘、传统策略、机器学习、深度学习、强化学习、图网络、高频交易
📚 资源汇总	全网资源汇总、实战案例、论文解读、代码实现
🛠️ 辅助工具	辅助盯盘、股票推荐等实用操盘工具
🌍 多市场覆盖	覆盖股票、基金、加密货币等多个市场
🚀 实盘部署	支持 Python/C++/CPU/GPU 等多种部署方式

🔥 新特性

时间	特性
2026.07.25	🆕 上班"摸鱼炒股"神器 V2：模块化盯盘系统 + 预警监控 + K线图 + 多源 fallback
2025.08.09	🆕 推理型股价预测大模型训练教程（预测准确率提升20%，且可解析）
2025.05.17	🆕 Unsloth推理型股价预测大模型（代码见本仓库、详细指南+模型见星球）
2025.01.03	大模型金融市场分析（视频教程见星球或公众号）

📂 2023 年更新

时间	特性
2023.04.09	StructBERT市场情绪分析
2023.03.28	强化学习多股票交易：年化收益53%
2023.02.28	机器学习自动挖掘5000个因子及股票趋势预测
2023.02.05	利用EXCEL看盘
2023.01.01	本地深度强化学习策略

📂 2022 年更新

时间	特性
2022.11.07	Wind本地实盘模拟
2022.08.03	基础回测框架 + 双均线策略

📖 简介

适合人群

🏢 机构投资者
👨‍💻 散户（有编程基础）
🌱 散户（无编程基础）

项目结构

ai_quant_trade
├── ai_notes ........... 金融量化交易知识（Markdown / Jupyter Notebook 知识体系）
│   ├── 资源 ........... 持续收录全网优秀资源
│   ├── 实战 ........... 各类工具、框架、库的使用及踩坑实录
│   └── 热点 ........... 金融市场热点、技术热点、论文解读
├── docs ............... 本仓库使用说明文档
├── egs_aide ........... 辅助操盘工具（看盘神器等）
├── egs_alpha .......... 因子库 & 因子挖掘
├── egs_data ........... 数据获取及处理（Wind / 开源工具）
├── egs_fin_nlp ........ 文本分析（情感分析等）
├── egs_llm ............ 大模型应用（股价预测 / 金融分析）
├── egs_online_platform  在线投研平台策略（优矿 / 聚宽）
├── egs_trade .......... 本地量化炒股策略
│   ├── paper_trade .... 实盘模拟（Wind万得）
│   ├── rl ............. 强化学习炒股
│   ├── ms_qlib ........ 微软Qlib框架
│   └── vanilla ........ 传统规则类策略
├── quant_brain ........ 核心算法库
├── runtime ............ 模型部署和实际使用
├── tools .............. 辅助工具
├── requirements.txt
└── README.md

🚀 快速开始

本仓库暂未封装为 Python 包，请克隆整个项目后，进入各 egs 目录查看详细的 使用说明 和 原理介绍。

# 1. 克隆仓库
git clone https://github.com/charliedream1/ai_quant_trade.git

# 2. 安装依赖
pip install -r requirements.txt

# 3. 进入对应示例目录，查看 README 开始使用
cd egs_trade/rl/a002_finRL_tutorial/a01_Stock_NeurIPS2018

📊 本地量化策略

📁 代码目录：egs_trade

🎯 每个实例均配备完善的教程，从原理、使用到代码解读。

可在本地构建一套独立的量化交易系统，涵盖以下策略类型：

类别	策略	状态
🤖 AI策略	强化学习、图网络、深度学习、机器学习、高频交易、因子挖掘、大模型	✅ / 🔨
📐 传统策略	规则类策略（双均线、投资组合管理等）	✅

🧠 强化学习策略

📁 代码目录：egs_trade/rl

自从2017年 AlphaGo 与柯洁围棋大战之后，深度强化学习大火。

相比于机器学习和深度学习，强化学习以最终目标为导向（以交互作为目标），而很多其他方法考虑的是孤立的子问题（如"股价预测"、"大盘预测"、"交易决策"等），并不能直接获得交互的动作。强化学习则直接面向"完成命令者的任务"，可以获得一连串的动作序列。

策略列表：

序号	策略	论文
1	原型	—
2	FinRL教程0-NeurIPS2018	Practical Deep Reinforcement Learning Approach for Stock Trading

回测结果：

序号	策略	市场	年化收益	最大回撤	夏普率
1	原型	中国A股	—	—	—
2	FinRL教程0-NeurIPS2018	美股道琼斯30	53.1%	-10.4%	2.17

📐 传统策略

传统策略虽然看似昨日黄花，但其可操作性更强，仍有一定使用价值。深度学习和机器学习往往需要配合规则使用。

双均线策略 + 简易手写回测框架
- 详细使用教程
- 包含策略代码 + 自建纯手写回测框架
- 包含良好的绘图，指示买点和卖点
- 🎯 目标：通过这个实例了解量化交易的完整框架构建方式
投资组合管理7节教学

💰 实盘交易

📁 代码目录：egs_trade

实盘模拟

Wind本地实盘模拟：双均线策略
- 利用 Wind 软件实现的实盘模拟
- Wind 常作为各大金融机构的首选数据源，由于价格较高，更适合机构使用
- 🏢 使用对象：机构

🛠️ 辅助操盘工具

📁 代码目录：egs_aide

利用EXCEL看盘 V1
- 👀 看盘时不容易被发现
- 📋 可自定义添加要盯盘的股票
- ⚡ 可利用 Excel 快速计算和处理数据
上班"摸鱼炒股"神器 V2：模块化盯盘系统 🆕
- 🏗️ 架构重构：从单文件升级为模块化 excel_monitor 包，Sheet Handler 模式，每个 Sheet 刷新互相隔离
- 🔔 预警监控：自定义涨跌幅/价格上下限，触发后整行变红 + 弹窗提醒
- 📈 K 线图：Excel 里点按钮即画 K 线（mplfinance 蜡烛图 + 均线），无需切软件
- 💰 资金情绪：新增 Sheet 聚合北向资金 + 微博舆情 + 新闻情绪 + 股吧热门
- 🔍 股票池选股：内置全 A 股，代码/名称/拼音首字母模糊搜索 + 下拉框直接选，不用再查代码
- 🔄 多源 fallback：qstock 主源 + akshare/东财/腾讯/网易/efinance 备选源，单个挂了自动切换
- ⚙️ 配置热重载：YAML + Excel "配置" Sheet，自选股/刷新间隔在 Excel 里改完热生效，无需重启
- 🧪 开箱即用：一条命令自动生成 Excel 模板（7 个 Sheet），不再需要额外准备股票列表
- ✅ 单元测试：pytest 覆盖核心逻辑（156 项），可长时间稳定运行
Streamlit实时行情监控
- 🌐 基于 Web 的实时行情看板

⛏️ 因子挖掘

📁 代码目录：egs_alpha

因子挖掘策略

序号	策略	论文
1	机器学习自动挖掘5000个因子及股票趋势预测	—

因子库

序号	因子库
1	alpha101
2	stockstats
3	ta_lib

💾 数据处理

📁 代码目录：egs_data

各类常见数据源使用详解
统一数据源接口

📝 文本分析

📁 代码目录：egs_fin_nlp

序号	工具
1	StructBERT市场情绪分析

🤖 大模型应用

📁 代码目录：egs_llm

序号	工具
1	大模型金融市场分析（视频教程见星球或公众号）
2	Unsloth推理型股价预测模型训练（代码开源、详细指南+模型见星球）

🌟 a_全网优秀资源（重点推荐）

📁 目录：a_全网优秀资源

⭐ 本仓库精华版块：从全网海量资料中筛选、整理、点评的优质量化资源，一站式获取！

🎯 这是什么？

这是本仓库最核心的"资源宝库"——我们花费大量精力从全网数万份资料中筛选、整理并附上点评，按量化交易全流程分类，方便你快速找到所需工具和资料，少走弯路。

与本仓库其他版块的区别：

版块	定位	特点
`a_全网优秀资源` ⭐	实战资源整合	收录全网优秀项目，附点评与对比
`egs_trade`	完整策略实战	从0到1的策略实现教程
`egs_llm`	大模型应用	LLM 在金融的落地实践
`ai_notes`	知识笔记	理论、概念、踩坑实录

✨ 四大特色

🔍 优中选优：从全网海量资源中精选，避免你重复踩坑
📂 分类清晰：按量化交易全流程（数据→策略→回测→交易）分类，便于按需查找
📝 含点评解读：不只是罗列链接，附有优缺点分析、上手指南
🔄 持续更新：紧跟技术发展，持续收录新资源

📚 资源分类一览

序号	类别	核心内容
📚 `00_基础知识`	入门学习	股票学习指南、入门教程
🎓 `00_学习资源`	资源汇总	GitHub量化资源、开源项目汇总
📊 `01_数据`	数据获取	数据获取工具、新闻数据、多模态数据
🏗️ `02_综合框架`	主流量化框架	Qlib、WonderTrader 等详解
🔄 `03_回测框架`	回测工具	Backtrader、PyAlgoTrade、Zipline、RQAlpha、QuantDigger 等
⛏️ `04_因子`	因子库	Alpha101、ta_lib、stockstats、alphalens 等
💹 `05_交易策略`	策略资源	传统/机器学习/深度学习/强化学习/图神经网络/研报复现/投资组合
🛠️ `06_辅助工具`	辅助工具	K线形态识别、金融建模
📊 `07_可视化`	可视化库	量化图表与可视化
🧠 `08_知识图谱`	知识图谱	传统方案与大模型方案
⚡ `09_高频交易`	高频交易	加密货币高频交易
🤖 `10_大模型`	LLM 金融应用	FinGPT、FinRobot、TradingAgents、Agent、RAG、Skill包等
🌐 `11_投研平台`	在线平台	免费量化平台汇总
💻 `12_交易平台`	交易接口	EasyTrader、VNPy 等

🔥 重点推荐内容

🤖 大模型在金融的应用：覆盖 FinGPT、FinMem、Self-Reflective、Stock-chain、TradingAgents、FinRobot 等最新研究与实战
🛠️ Skill 包合集（60+）：包含缠论、技术分析、量化统计、基本面分析、加密货币、宏观分析等专业 Skill
📊 回测框架多维对比：Backtrader、Zipline、RQAlpha、PyAlgoTrade、QuantDigger 等多框架实测对比
🔬 研报复现：精选高质量券商研报并附复现代码
💹 交易策略全套：从传统双均线到强化学习、图神经网络，覆盖各类型策略资源

💡 使用建议：进入 a_全网优秀资源目录按需浏览；如对某个项目感兴趣，可点击查看详细的介绍和点评。

📚 编程及AI基础知识

为了便于维护，已将原有的 ai_wiki 目录内容（系统操作、编程基础、AI基础、AI实践等）独立同步至仓库 AI大模型避坑指南。

里面记录了大量实际开发中遇到的问题和解决方案，并实时追踪前沿技术发展，欢迎大家关注和 Star ⭐

✨ AI大模型避坑指南

Github: https://github.com/charliedream1/ai_wiki

Gitee（国内镜像）: https://gitee.com/charlie1/ai_wiki.git

简介: 分享各种实用案例，追踪前沿技术发展，囊括 AI 全栈知识，涵盖大模型、编程技术、机器学习、深度学习、强化学习、图神经网络、语音识别、NLP 及图像识别等领域

🌐 在线投研平台

📁 代码目录：egs_online_platform

国内量化平台如聚宽、优矿、米筐、果仁和 BigQuant 等，感兴趣的读者可自行尝试。

投研平台是为量化爱好者（宽客）量身打造的云平台，提供免费股票数据获取、精准的回测功能、高速实盘交易接口、易用的 API 文档、由易入难的策略库，便于快速实现和验证策略。

⚠️ 注意：如下策略仅在所述回测段有效，没有进行详细的调优和全周期验证。没有策略能保证全周期有效，如实盘使用请慎重。

聚宽平台

🔗 聚宽平台 · 欢迎关注我：量客攻城狮

具体策略详细介绍和源码请点击对应策略链接查看

聚宽使用介绍：egs_online_platform/聚宽_JoinQuant

该部分代码仅能在 聚宽平台 运行

股票量化策略：

策略	收益	最大回撤
机器学习-动态因子选择策略	12.3%	38.93%
小市值+多均线量化炒股	58.4%	46.61%
龙虎榜-看长做短	41.82%	26.89%
强势股+趋势线判断+止损止盈	10.09%	21.449%

股票分析研究：

📖 量化资源集合

(我们在知乎上2.6万阅读的文章) 史上最全AI股票量化交易工具和开源项目汇总

我们将所有工具重新进行了分类并点评，收录在 ai_notes 文件夹下，方便大家查找。

🎯 开发中：

陆续对所有工具进行点评，方便选择
陆续记录各工具的优缺点，形成对比表，方便选型
陆续记录使用方法：我们不做大而全的教程，只列举最常用且实用的功能，让你快速上手

🎁 配套资源

本代码仓秉承 收费与免费并行 的原则。

💎 收费资源 — 知识星球

知识星球官网注册，用户权益有保障。星球内容介绍

👇 下方扫描二维码或点击链接，进入星球查看更详细的介绍 🎏

星球视频介绍：

星球使用指南：https://mp.weixin.qq.com/s/SGc49e0xf24q5aUbf3rO0g?token=2028063978&lang=zh_CN
学习路线及群内资源使用：https://mp.weixin.qq.com/s/3-U048mc0riVsdETrKr77g

星球加入链接：

AI智投星球：AI量化交易速成、前沿技术、实战案例、资源库
AI速成营：深入补充编程、大模型、AI基础、原理及金融方向实战及求职等的速成和案例分享，与 AI智投星球形成互补

星球介绍：

👇 扫码查看"星球"更详细的介绍（里面有搞笑漫画哦）！

🎯 本代码仓会持续更新，但部分代码转为私有化维护仅在星球中可见，对应功能会在仓库中标注。

🆓 免费资源

微信公众号

🔥 最新资讯实时关注 🎁 关注并点赞任一篇文章，私信管理员，领取精美量化资料包一份！

代码仓（永久免费）

✨ AI量化交易操盘手

Github: https://github.com/charliedream1/ai_quant_trade

Gitee（国内镜像）: https://gitee.com/charlie1/ai_quant_trade.git

本仓库配套项目

✨ AI驯龙笔记

Github: https://github.com/charliedream1/ai_wiki

Gitee（国内镜像）: https://gitee.com/charlie1/ai_wiki.git

简介: 分享各种实用案例，追踪前沿技术发展，囊括 AI 全栈知识，涵盖大模型、编程技术、机器学习、深度学习、强化学习、图神经网络、语音识别、NLP 及图像识别等领域

💖 打赏我

您的支持是我前进的动力，即便"1毛钱"我也很开心，感谢您的打赏和支持 (^o^)/

💬 讨论

欢迎在 Github Discussions 中发起讨论。

🐛 技术支持

欢迎在 Github Issues 中提交问题
加入知识星球，获取更多技术支持
- AI智投星球：专注AI量化交易知识分享
- 大模型避坑指南：专注编程、大模型、AI应用赋能

❓ 常见问题

请查看文档 → 常见问题

📄 引用

@misc{ai_quant_trade,
  author={Yi Li},
  title={ai_quant_trade},
  year={2022},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/charliedream1/ai_quant_trade}},
}

如果本项目对你有帮助，请给一个 Star ⭐ 支持一下！

neo4j-labs/llm-graph-builder

Neo4j graph construction from unstructured data using LLMs

Knowledge Graph Builder

Transform unstructured data (PDFs, DOCs, TXTs, YouTube videos, web pages, etc.) into a structured Knowledge Graph stored in Neo4j using the power of Large Language Models (LLMs) and the LangChain framework.

This application allows you to upload files from various sources (local machine, GCS, S3 bucket, or web sources), choose your preferred LLM model, and generate a Knowledge Graph.

Getting Started

Prerequisites

Python 3.12 or higher (for local/separate backend deployment)
Neo4j Database 5.23 or later with APOC installed.
- Neo4j 5.23 is required because the backend uses the Cypher variable-scope subquery syntax (CALL (variable) { ... }), which is not supported by earlier Neo4j 5.x releases such as 5.20.
- Neo4j Aura databases (including the free tier) are supported.
- If using Neo4j Desktop, you will need to deploy the backend and frontend separately (docker-compose is not supported).

Backend Setup

Create a .env file in the backend folder by copying backend/example.env.

Pre-configure user credentials in the .env file to bypass the login dialog:

NEO4J_URI=<your-neo4j-uri>
NEO4J_USERNAME=<your-username>
NEO4J_PASSWORD=<your-password>
NEO4J_DATABASE=<your-database-name>

Run:

cd backend
python3.12 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
uvicorn score:app --reload

Key Features

Knowledge Graph Creation

Seamlessly transform unstructured data into structured Knowledge Graphs using advanced LLMs.
Extract nodes, relationships, and their properties to create structured graphs.

Schema Support

Use a custom schema or existing schemas configured in the settings to generate graphs.

Graph Visualization

View graphs for specific or multiple data sources simultaneously in Neo4j Bloom.

Chat with Data

Interact with your data in the Neo4j database through conversational queries.
Retrieve metadata about the source of responses to your queries.
For a dedicated chat interface, use the standalone chat application with the /chat-only route.

LLMs Supported

OpenAI
Gemini
Diffbot
Azure OpenAI (dev deployed version)
Anthropic (dev deployed version)
Fireworks (dev deployed version)
Groq (dev deployed version)
Amazon Bedrock (dev deployed version)
Ollama (dev deployed version)
Deepseek (dev deployed version)
Other OpenAI-compatible base URL models (dev deployed version)

Token Usage Tracking

Easily monitor and track your LLM token usage for each user and database connection.
Enable this feature by setting the TRACK_USER_USAGE environment variable to true in your backend configuration.
View your daily and monthly token consumption and limits, helping you manage usage and avoid overages.
You can check your remaining token limits at any time using the provided API endpoint.

Embedding Model Selection

Choose from a variety of embedding models to generate vector embeddings for your data. This can be configured from the frontend in Graph Settings > Processing Configuration > Select Embedding Model.
Supported model providers include OpenAI, Gemini, Amazon Titan, and Sentence Transformers.
Your selected embedding model is saved to your user profile when TRACK_USER_USAGE is enabled.

Local Configuration

You have two ways to configure the embedding model locally:

With User Tracking (TRACK_USER_USAGE=true):
- Set TRACK_USER_USAGE to true in your backend .env file.
- Provide your token tracking database credentials (TOKEN_TRACKER_DB_URI, TOKEN_TRACKER_DB_USERNAME, etc.).
- Select your desired embedding model from the frontend. Your selection will be saved and automatically used in subsequent sessions.
Without User Tracking (TRACK_USER_USAGE=false):
- Set TRACK_USER_USAGE to false.
- Specify the embedding model and provider directly in your backend .env file using EMBEDDING_MODEL and EMBEDDING_PROVIDER.
- If these variables are not set, the application defaults to a Sentence Transformer model.
- In this mode, the embedding model cannot be changed from the frontend.

Getting Started

Prerequisites

Neo4j Database 5.23 or later with APOC installed.
- Neo4j 5.23 is required because the backend uses the Cypher variable-scope subquery syntax (CALL (variable) { ... }), which is not supported by earlier Neo4j 5.x releases such as 5.20.
- Neo4j Aura databases (including the free tier) are supported.
- If using Neo4j Desktop, you will need to deploy the backend and frontend separately (docker-compose is not supported).

Deployment Options

Local Deployment

Using Docker-Compose

Run the application using the default docker-compose configuration.

Supported LLM Models:
By default, only OpenAI and Diffbot are enabled. Gemini requires additional GCP configurations.
Use the VITE_LLM_MODELS_PROD variable to configure the models you need. Example:
```
VITE_LLM_MODELS_PROD="gemini_3.5_flash,openai_gpt_5.4_mini,diffbot,anthropic_claude_4.5_haiku"
```

Anthropic Models: Use the latest Claude model in your config:

LLM_MODEL_CONFIG_ANTHROPIC_CLAUDE_4_7_OPUS="claude-opus-4-7,anthropic_api_key"

Input Sources:
By default, the following sources are enabled: local, YouTube, Wikipedia, AWS S3, and web.
To add Google Cloud Storage (GCS) integration, include gcs and your Google client ID:
```
VITE_REACT_APP_SOURCES="local,youtube,wiki,s3,gcs,web"
VITE_GOOGLE_CLIENT_ID="your-google-client-id"
```

Chat Modes

Configure chat modes using the VITE_CHAT_MODES variable:

By default, all modes are enabled: vector, graph_vector, graph, fulltext, graph_vector_fulltext, entity_vector, and global_vector.
To specify specific modes, update the variable. For example:
```
VITE_CHAT_MODES="vector,graph"
```

Running Backend and Frontend Separately

For development, you can run the backend and frontend independently.

Frontend Setup

Create a .env file in the frontend folder by copying frontend/example.env.
Update environment variables as needed.
Run:
```
cd frontend
```

yarn yarn run dev


#### **Backend Setup**
1. Create a `.env` file in the `backend` folder by copying `backend/example.env`.
2. Pre-configure user credentials in the `.env` file to bypass the login dialog:
```bash
NEO4J_URI=<your-neo4j-uri>
NEO4J_USERNAME=<your-username>
NEO4J_PASSWORD=<your-password>
NEO4J_DATABASE=<your-database-name>

Run:
```
cd backend
```

python -m venv envName source envName/bin/activate pip install -r requirements.txt uvicorn score:app --reload


---

### **Cloud Deployment**

Deploy the application on **Google Cloud Platform** using the following commands:

#### **Frontend Deployment**
```bash
gcloud run deploy dev-frontend \
--source . \
--region us-central1 \
--allow-unauthenticated

Backend Deployment

gcloud run deploy dev-backend \
  --set-env-vars "OPENAI_API_KEY=<your-openai-api-key>" \
  --set-env-vars "DIFFBOT_API_KEY=<your-diffbot-api-key>" \
  --set-env-vars "NEO4J_URI=<your-neo4j-uri>" \
  --set-env-vars "NEO4J_USERNAME=<your-username>" \
  --set-env-vars "NEO4J_PASSWORD=<your-password>" \
  --source . \
  --region us-central1 \
  --allow-unauthenticated

For local llms (Ollama)

Pull the docker image of ollama
```
docker pull ollama/ollama
```

Run the ollama docker image

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Execute any llm model, e.g., llama3

docker exec -it ollama ollama run llama3

Configure env variable in docker compose.

LLM_MODEL_CONFIG_ollama_<model_name>
# example
LLM_MODEL_CONFIG_ollama_llama3=${LLM_MODEL_CONFIG_ollama_llama3-llama3,http://host.docker.internal:11434}

Configure the backend API url

VITE_BACKEND_API_URL=${VITE_BACKEND_API_URL-backendurl}

Open the application in browser and select the ollama model for the extraction.
Enjoy Graph Building.

Usage

Connect to a Neo4j Aura Instance, which can be either AURA DS or AURA DB, by passing the URI and password through the backend environment, filling in the login dialog, or dragging and dropping the Neo4j credentials file.
To differentiate, we have added different icons. For AURA DB, there is a database icon, and for AURA DS, there is a scientific molecule icon right under the Neo4j Connection details label.
Choose your source from a list of unstructured sources to create a graph.
Change the LLM (if required) from the dropdown, which will be used to generate the graph.
Optionally, define the schema (nodes and relationship labels) in the entity graph extraction settings.
Either select multiple files to 'Generate Graph', or all the files in 'New' status will be processed for graph creation.
View the graph for individual files using 'View' in the grid, or select one or more files and 'Preview Graph'.
Ask questions related to the processed/completed sources to the chatbot. Also, get detailed information about your answers generated by the LLM.

ENV

Env Variable Name	Mandatory/Optional	Default Value	Description

BACKEND ENV
OPENAI_API_KEY	Optional		An OpenAI Key is required to use OpenAI LLM model to authenticate and track requests
DIFFBOT_API_KEY	Mandatory		API key is required to use Diffbot's NLP service to extract entities and relationships from unstructured data
BUCKET_UPLOAD_FILE	Optional		Bucket name to store uploaded file on GCS
BUCKET_FAILED_FILE	Optional		Bucket name to store failed file on GCS while extraction
USER_AGENT	Optional	llm-graph-builder	Name of the user agent to track Neo4j database activity
ENABLE_USER_AGENT	Optional	true	Boolean value to enable/disable Neo4j user agent
DUPLICATE_TEXT_DISTANCE	Optional	5	This value is used to find distance for all node pairs in the graph and is calculated based on node properties
DUPLICATE_SCORE_VALUE	Optional	0.97	Node score value to match duplicate nodes
EFFECTIVE_SEARCH_RATIO	Optional	1	Ratio used for effective search calculations
GRAPH_CLEANUP_MODEL	Optional	openai_gpt_5_mini	Model name to clean up graph in post processing
MAX_TOKEN_CHUNK_SIZE	Optional	10000	Maximum token size to process file content
YOUTUBE_TRANSCRIPT_PROXY	Mandatory		Proxy key to process YouTube videos for getting transcripts
IS_EMBEDDING	Optional	true	Flag to enable text embedding
KNN_MIN_SCORE	Optional	0.8	Minimum score for KNN algorithm
GCP_LOG_METRICS_ENABLED	Optional	False	Flag to enable Google Cloud logs
NEO4J_URI	Optional	neo4j://database:7687	URI for Neo4j database
NEO4J_USERNAME	Optional	neo4j	Username for Neo4j database
NEO4J_PASSWORD	Optional	password	Password for Neo4j database
GCS_FILE_CACHE	Optional	False	If set to True, will save files to process into GCS. If False, will save files locally
ENTITY_EMBEDDING	Optional	False	If set to True, it will add embeddings for each entity in the database
LLM_MODEL_CONFIG_ollama_<model_name>	Optional		Set ollama config as model_name,model_local_url for local deployments

FRONTEND ENV
VITE_BLOOM_URL	Mandatory	Bloom URL	URL for Bloom visualization
VITE_REACT_APP_SOURCES	Mandatory	local,youtube,wiki,s3	List of input sources that will be available
VITE_CHAT_MODES	Mandatory	vector,graph+vector,graph,hybrid	Chat modes available for Q&A
VITE_ENV	Mandatory	DEV or PROD	Environment variable for the app
VITE_LLM_MODELS	Optional	openai_gpt_5_mini,gemini_flash_latest,anthropic_claude_4.5_haiku	Supported models for the application
VITE_BACKEND_API_URL	Optional	localhost	URL for backend API
VITE_TIME_PER_PAGE	Optional	50	Time per page for processing
VITE_CHUNK_SIZE	Optional	5242880	Size of each chunk of file for upload
VITE_GOOGLE_CLIENT_ID	Optional		Client ID for Google authentication
VITE_LLM_MODELS_PROD	Optional	openai_gpt_5_mini,gemini_flash_latest,anthropic_claude_4.5_haiku	To distinguish models based on environment (PROD or DEV)
VITE_AUTH0_CLIENT_ID	Mandatory if you are enabling Authentication otherwise it is optional		Okta OAuth Client ID for authentication
VITE_AUTH0_DOMAIN	Mandatory if you are enabling Authentication otherwise it is optional		Okta OAuth Client Domain
VITE_SKIP_AUTH	Optional	true	Flag to skip authentication
VITE_CHUNK_OVERLAP	Optional	20	Variable to configure chunk overlap
VITE_TOKENS_PER_CHUNK	Optional	100	Variable to configure tokens count per chunk. This gives flexibility for users who may require different chunk sizes for various tokenization tasks
VITE_CHUNK_TO_COMBINE	Optional	1	Variable to configure number of chunks to combine for parallel processing

Example Environment Files

Refer to the example environment files for additional variables and configuration:

Cloud Build Deployment

You can deploy the backend and the frontend to Google Cloud Run using Cloud Build, either manually or via automated triggers.

Automated Deployment (Recommended)

Connect your repository to Google Cloud Build:
- In the Google Cloud Console, go to Cloud Build > Triggers.
- Create a new trigger and select your repository.
- Set the trigger to run on push to your desired branch (main, staging, or dev).
- Cloud Build will automatically use the cloudbuild.yaml file in the root of your repository.
Configure Substitutions and Secrets:
- In the trigger settings, add required substitutions (e.g., _OPENAI_API_KEY, _DIFFBOT_API_KEY, etc.) as environment variables or use Secret Manager for sensitive data.
Push your code:
- When you push to the configured branch, Cloud Build will build and deploy your backend (and optionally frontend) to Cloud Run using the steps defined in cloudbuild.yaml.

Manual Deployment

Set up Google Cloud SDK and authenticate:

gcloud auth login
gcloud config set project <YOUR_PROJECT_ID>

Run Cloud Build manually:

gcloud builds submit --config cloudbuild.yaml \
  --substitutions=_REGION=us-central1,_REPO=cloud-run-repo,_OPENAI_API_KEY=<your-openai-key>,_DIFFBOT_API_KEY=<your-diffbot-key>,_BUCKET_UPLOAD_FILE=<your-bucket>,_BUCKET_FAILED_FILE=<your-bucket>,_PROJECT_ID=<your-project-id>,_GCS_FILE_CACHE=False,_TRACK_USER_USAGE=False,_TOKEN_TRACKER_DB_URI=...,_TOKEN_TRACKER_DB_USERNAME=...,_TOKEN_TRACKER_DB_PASSWORD=...,_TOKEN_TRACKER_DB_DATABASE=...,_DEFAULT_DIFFBOT_CHAT_MODEL=...,_YOUTUBE_TRANSCRIPT_PROXY=...,_EMBEDDING_MODEL=...,
    _EMBEDDING_PROVIDER=...,_BEDROCK_EMBEDDING_MODEL_KEY=...,_LLM_MODEL_CONFIG_OPENAI_GPT_5_2=...,_LLM_MODEL_CONFIG_OPENAI_GPT_5_MINI=...,_LLM_MODEL_CONFIG_GEMINI_2_5_FLASH=...,_LLM_MODEL_CONFIG_GEMINI_2_5_PRO=...,_LLM_MODEL_CONFIG_DIFFBOT=...,_LLM_MODEL_CONFIG_GROQ_LLAMA3_1_8B=...,_LLM_MODEL_CONFIG_ANTHROPIC_CLAUDE_4_5_SONNET=...,_LLM_MODEL_CONFIG_ANTHROPIC_CLAUDE_4_5_HAIKU=...,_LLM_MODEL_CONFIG_LLAMA4_MAVERICK=...,_LLM_MODEL_CONFIG_FIREWORKS_QWEN3_6=...,_LLM_MODEL_CONFIG_FIREWORKS_GPT_OSS=...,_LLM_MODEL_CONFIG_FIREWORKS_DEEPSEEK_V3=...,_LLM_MODEL_CONFIG_BEDROCK_NOVA_MICRO_V1=...,_LLM_MODEL_CONFIG_BEDROCK_NOVA_LITE_V1=...,_LLM_MODEL_CONFIG_BEDROCK_NOVA_PRO_V1=...,_LLM_MODEL_CONFIG_OLLAMA_LLAMA3=...

Replace the values in angle brackets with your actual configuration and secrets.
LLM_MODEL_CONFIG_FIREWORKS_QWEN3_6 is the app-facing config key for the fireworks_qwen3_6 model option and should map to the Fireworks serverless slug accounts/fireworks/models/qwen3p6-plus.
You can omit or add substitutions as needed for your deployment.

Monitor the build:
- The build and deployment process will be visible in the Cloud Build console.
Access your deployed service:
- After deployment, your backend will be available at the Cloud Run service URL shown in the Cloud Console.

Note:

The cloudbuild.yaml file supports multiple environments (main, staging, dev) based on the branch name.
The frontend build and deployment steps are commented out by default. Uncomment them in cloudbuild.yaml if you wish to deploy the frontend as well.

For more details, see the comments in cloudbuild.yaml.

Links

LLM Knowledge Graph Builder Application

Neo4j Workspace

Reference

Demo of application

Contact

For any inquiries or support, feel free to raise GitHub Issues

Happy Graph Building!

HandsOnLLM/Hands-On-Large-Language-Models

Official code repo for the O'Reilly Book - "Hands-On Large Language Models"

Hands-On Large Language Models

Welcome! In this repository you will find the code for all examples throughout the book Hands-On Large Language Models written by Jay Alammar and Maarten Grootendorst which we playfully dubbed:

"The Illustrated LLM Book"

Through the visually educational nature of this book and with almost 300 custom made figures, learn the practical tools and concepts you need to use Large Language Models today!

The book is available on:

We advise to run all examples through Google Colab for the easiest setup. Google Colab allows you to use a T4 GPU with 16GB of VRAM for free. All examples were mainly built and tested using Google Colab, so it should be the most stable platform. However, any other cloud provider should work.

Chapter	Notebook
Chapter 1: Introduction to Language Models
Chapter 2: Tokens and Embeddings
Chapter 3: Looking Inside Transformer LLMs
Chapter 4: Text Classification
Chapter 5: Text Clustering and Topic Modeling
Chapter 6: Prompt Engineering
Chapter 7: Advanced Text Generation Techniques and Tools
Chapter 8: Semantic Search and Retrieval-Augmented Generation
Chapter 9: Multimodal Large Language Models
Chapter 10: Creating Text Embedding Models
Chapter 11: Fine-tuning Representation Models for Classification
Chapter 12: Fine-tuning Generation Models

Tip

You can check the setup folder for a quick-start guide to install all packages locally and you can check the conda folder for a complete guide on how to setup your environment, including conda and PyTorch installation. Note that the depending on your OS, Python version, and dependencies your results might be slightly differ. However, they should this be similar to the examples in the book.

Reviews

"Jay and Maarten have continued their tradition of providing beautifully illustrated and insightful descriptions of complex topics in their new book. Bolstered with working code, timelines, and references to key papers, their book is a valuable resource for anyone looking to understand the main techniques behind how Large Language Models are built."

Andrew Ng - founder of DeepLearning.AI

"This is an exceptional guide to the world of language models and their practical applications in industry. Its highly-visual coverage of generative, representational, and retrieval applications of language models empowers readers to quickly understand, use, and refine LLMs. Highly recommended!"

Nils Reimers - Director of Machine Learning at Cohere | creator of sentence-transformers

"I can’t think of another book that is more important to read right now. On every single page, I learned something that is critical to success in this era of language models."

Josh Starmer - StatQuest

"If you’re looking to get up to speed in everything regarding LLMs, look no further! In this wonderful book, Jay and Maarten will take you from zero to expert in the history and latest advances in large language models. With very intuitive explanations, great real-life examples, clear illustrations, and comprehensive code labs, this book lifts the curtain on the complexities of transformer models, tokenizers, semantic search, RAG, and many other cutting-edge technologies. A must read for anyone interested in the latest AI technology!"

Luis Serrano, PhD - Founder and CEO of Serrano Academy

"Hands-On Large Language Models brings clarity and practical examples to cut through the hype of AI. It provides a wealth of great diagrams and visual aids to supplement the clear explanations. The worked examples and code make concrete what other books leave abstract. The book starts with simple introductory beginnings, and steadily builds in scope. By the final chapters, you will be fine-tuning and building your own large language models with confidence."

Leland McInnes - Researcher at the Tutte Institute for Mathematics and Computing | creator of UMAP and HDBSCAN

Bonus content!

We attempted to put as much information into the book without it being overwhelming. However, even with a 400-page book there is still much to discover!

We continue to create more guides that compliment the book and go more in-depth into new and exciting topics:

A Visual Guide to Mamba	A Visual Guide to Quantization	The Illustrated Stable Diffusion

A Visual Guide to Mixture of Experts	A Visual Guide to Reasoning LLMs	The Illustrated DeepSeek-R1

Citation

Please consider citing the book if you consider it useful for your research:

@book{hands-on-llms-book,
  author       = {Jay Alammar and Maarten Grootendorst},
  title        = {Hands-On Large Language Models},
  publisher    = {O'Reilly},
  year         = {2024},
  isbn         = {978-1098150969},
  url          = {https://www.oreilly.com/library/view/hands-on-large-language/9781098150952/},
  github       = {https://github.com/HandsOnLLM/Hands-On-Large-Language-Models}
}

borglab/gtsam

GTSAM is a library of C++ classes that implement smoothing and mapping (SAM) in robotics and vision, using factor graphs and Bayes networks as the underlying computing paradigm rather than sparse matrices.

GTSAM: Georgia Tech Smoothing and Mapping Library

Important Note

The develop branch is officially in "Pre 4.3" mode. We envision several API-breaking changes as we switch to C++17 and away from boost.

In addition, features deprecated in 4.2 will be removed. Please use the stable 4.2 release if you need those features. However, most are easily converted and can be tracked down (in 4.2) by disabling the cmake flag GTSAM_ALLOW_DEPRECATED_SINCE_V42.

What is GTSAM?

GTSAM is a C++ library that implements smoothing and mapping (SAM) in robotics and vision, using Factor Graphs and Bayes Networks as the underlying computing paradigm rather than sparse matrices.

CI Status	Platform	Compiler
	Ubuntu 22.04, MacOS 13-14, Windows	gcc/clang,MSVC
	Latest Windows/Ubuntu/Mac	-
	See pypi files; no Windows	-

On top of the C++ library, GTSAM includes wrappers for MATLAB & Python.

Documentation

C++ API Docs: https://gtsam.org/doxygen/
Python API Docs: https://borglab.github.io/gtsam/

Quickstart

In the root library folder execute:

#!bash
mkdir build
cd build
cmake ..
make check  # optional, runs all unit tests
make install

Prerequisites:

A modern compiler:
- Mac: at least xcode-14.2
- Linux: at least clang-11 or gcc-9
- Windows: at least msvc-14.2
CMake >= 3.16
- Ubuntu: sudo apt-get install cmake

Optional Boost prerequisite:

Boost is now optional. Two cmake flags govern its behavior:

GTSAM_USE_BOOST_FEATURES = ON|OFF: some of our timers and concept checking in the tests still depend on boost.
GTSAM_ENABLE_BOOST_SERIALIZATION = ON|OFF: serialization of factor graphs, factors, etc still is done using boost

If one or both of these flags are ON, you need to install Boost >= 1.70 - Mac: brew install boost - Ubuntu: sudo apt-get install libboost-all-dev - Windows: We highly recommend using the vcpkg package manager. For other installation methods or troubleshooting, please see the guidance in the cmake/HandleBoost.cmake script.

Optional prerequisites - used automatically if findable by CMake:

Intel Threaded Building Blocks (TBB) (Ubuntu: sudo apt-get install libtbb-dev)
Intel Math Kernel Library (MKL) (Ubuntu: installing using APT)
- See INSTALL.md for more installation information
- Note that MKL may not provide a speedup in all cases. Make sure to benchmark your problem with and without MKL.

GTSAM 4 Compatibility

GTSAM 4 introduces several new features, most notably Expressions and a Python toolbox. It also introduces traits, a C++ technique that allows optimizing with non-GTSAM types. That opens the door to retiring geometric types such as Point2 and Point3 to pure Eigen types, which we also do. A significant change which will not trigger a compile error is that zero-initializing of Point2 and Point3 is deprecated, so please be aware that this might render functions using their default constructor incorrect.

There is a flag GTSAM_ALLOW_DEPRECATED_SINCE_V43 for newly deprecated methods since the 4.3 release, which is on by default, allowing anyone to just pull version 4.3 and compile.

Wrappers

We provide support for MATLAB and Python wrappers for GTSAM. Please refer to the linked documents for more details.

Citation

If you are using GTSAM for academic work, please use the following citation:

@software{gtsam,
  author       = {Frank Dellaert and GTSAM Contributors},
  title        = {borglab/gtsam},
  month        = May,
  year         = 2022,
  publisher    = {Georgia Tech Borg Lab},
  version      = {4.2a8},
  doi          = {10.5281/zenodo.5794541},
  url          = {https://github.com/borglab/gtsam)}}
}

To cite the Factor Graphs for Robot Perception book, please use:

@book{factor_graphs_for_robot_perception,
    author={Frank Dellaert and Michael Kaess},
    year={2017},
    title={Factor Graphs for Robot Perception},
    publisher={Foundations and Trends in Robotics, Vol. 6},
    url={http://www.cs.cmu.edu/~kaess/pub/Dellaert17fnt.pdf}
}

If you are using the IMU preintegration scheme, please cite:

@book{imu_preintegration,
    author={Christian Forster and Luca Carlone and Frank Dellaert and Davide Scaramuzza},
    title={IMU preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation},
    year={2015}
}

The Preintegrated IMU Factor

GTSAM includes a state of the art IMU handling scheme based on

Todd Lupton and Salah Sukkarieh, "Visual-Inertial-Aided Navigation for High-Dynamic Motion in Built Environments Without Initial Conditions", TRO, 28(1):61-76, 2012. [link]

Our implementation improves on this using integration on the manifold, as detailed in

Luca Carlone, Zsolt Kira, Chris Beall, Vadim Indelman, and Frank Dellaert, "Eliminating conditionally independent sets in factor graphs: a unifying perspective based on smart factors", Int. Conf. on Robotics and Automation (ICRA), 2014. [link]
Christian Forster, Luca Carlone, Frank Dellaert, and Davide Scaramuzza, "IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation", Robotics: Science and Systems (RSS), 2015. [link]

If you are using the factor in academic work, please cite the publications above.

In GTSAM 4 a new and more efficient implementation, based on integrating on the NavState tangent space and detailed in this document, is enabled by default. To switch to the RSS 2015 version, set the flag GTSAM_TANGENT_PREINTEGRATION to OFF.

Additional Information

There is a GTSAM users Google group for general discussion.

Read about important GTSAM-Concepts here. A primer on GTSAM Expressions, which support (superfast) automatic differentiation, can be found on the GTSAM wiki on BitBucket.

See the INSTALL file for more detailed installation instructions. Our CI/CD process is detailed in workflows.md.

GTSAM is open source under the BSD license, see the LICENSE and LICENSE.BSD files.

Please see the examples/ directory and the USAGE file for examples on how to use GTSAM.

GTSAM was developed in the lab of Frank Dellaert at the Georgia Institute of Technology, with the help of many contributors over the years, see THANKS.

microsoft/ai-agents-for-beginners

18 Lessons to Get Started Building AI Agents

AI Agents for Beginners - A Course

A course teaching everything you need to know to start building AI Agents

🌐 Multi-Language Support

Supported via GitHub Action (Automated & Always Up-to-Date)

Prefer to Clone Locally?

This repository includes 50+ language translations which significantly increases the download size. To clone without translations, use sparse checkout:

Bash / macOS / Linux:
git clone --filter=blob:none --sparse https://github.com/microsoft/ai-agents-for-beginners.git
cd ai-agents-for-beginners
git sparse-checkout set --no-cone '/*' '!translations' '!translated_images'
CMD (Windows):
git clone --filter=blob:none --sparse https://github.com/microsoft/ai-agents-for-beginners.git
cd ai-agents-for-beginners
git sparse-checkout set --no-cone "/*" "!translations" "!translated_images"
This gives you everything you need to complete the course with a much faster download.

If you wish to have additional translation languages supported, they are listed here.

🌱 Getting Started

This course has lessons covering the fundamentals of building AI Agents. Each lesson covers its own topic so start wherever you like!

There is multi-language support for this course. Go to our available languages here.

If this is your first time building with Generative AI models, check out our Generative AI For Beginners course, which includes 21 lessons on building with GenAI.

Don't forget to star (🌟) this repo and fork this repo to run the code.

Meet Other Learners, Get Your Questions Answered

If you get stuck or have any questions about building AI Agents, join our dedicated Discord Channel in the Microsoft Foundry Discord.

What You Need

Each lesson in this course includes code examples, which can be found in the code_samples folder. You can fork this repo to create your own copy.

The code examples in these exercises utilize Microsoft Agent Framework with Microsoft Foundry Agent Service V2:

Microsoft Foundry - Azure Account Required

This course uses the following AI Agent frameworks and services from Microsoft:

Some code samples also support alternative OpenAI-compatible providers such as MiniMax, which offers large-context models (up to 204K tokens). See the Course Setup for configuration details.

For more information on running the code for this course, go to the Course Setup.

🙏 Want to help?

Do you have suggestions or found spelling or code errors? Raise an issue or Create a pull request

📂 Each lesson includes

A written lesson located in the README and a short video
Python code samples using Microsoft Agent Framework with Microsoft Foundry
Links to extra resources to continue your learning

🗃️ Lessons

Lesson	Text & Code	Video	Extra Learning
Intro to AI Agents and Agent Use Cases	Link	Video	Link
Exploring AI Agentic Frameworks	Link	Video	Link
Understanding AI Agentic Design Patterns	Link	Video	Link
Tool Use Design Pattern	Link	Video	Link
Agentic RAG	Link	Video	Link
Building Trustworthy AI Agents	Link	Video	Link
Planning Design Pattern	Link	Video	Link
Multi-Agent Design Pattern	Link	Video	Link
Metacognition Design Pattern	Link	Video	Link
AI Agents in Production	Link	Video	Link
Using Agentic Protocols (MCP, A2A and NLWeb)	Link	Video	Link
Context Engineering for AI Agents	Link	Video	Link
Managing Agentic Memory	Link	Video
Exploring Microsoft Agent Framework	Link
Building Computer Use Agents (CUA)	Link		Link
Deploying Scalable Agents	Link		Link
Creating Local AI Agents	Link		Link
Securing AI Agents	Link		Link

🎒 Other Courses

Our team produces other courses! Check out:

LangChain

Azure / Edge / MCP / Agents

Generative AI Series

Core Learning

Copilot Series

🌟 Community Thanks

Thanks to Shivam Goyal for contributing important code samples demonstrating Agentic RAG.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos is subject to those third-parties' policies.

Getting Help

If you get stuck or have any questions about building AI apps, join:

If you have product feedback or errors while building visit:

Infrasys-AI/AISystem

AISystem 主要是指AI系统，包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术

AI System & AI Infra

大模型内容太多啦！！因此最新大模型的内容归档在 AIFoundation https://github.com/chenzomi12/AIFoundation/ 上面，欢迎大家移步过去哦！！

文字课程内容正在一节节补充更新，尽可能抽空继续更新正在 AISys ，希望您多多鼓励和参与进来！！！

文字课程开源在 AISys，系列视频托管B 站和油管，PPT 开源在github，欢迎取用！！！

课程背景

这个开源课程英文名字叫做AI System(AISys)，中文名字叫做AI 系统。

本开源课程主要是跟大家一起探讨和学习人工智能、深度学习的系统设计，而整个系统是围绕着 ZOMI 在工作当中所积累、梳理、构建 AI 系统全栈的内容。希望跟所有关注 AI 开源课程的好朋友一起探讨研究，共同促进学习讨论。

课程内容大纲

课程主要包括以下五大模块：

教程内容	简介	地址
AI 系统全栈概述	AI 基础知识和 AI 系统的全栈概述的AI 系统概述，以及深度学习系统的系统性设计和方法论，主要是整体了解 AI 训练和推理全栈的体系结构内容。	[Slides]
AI 芯片与体系架构	作为 AI 的硬件体系架构主要是指 AI 芯片，这里就很硬核了，从CPU、GPU 的芯片基础到 AI 芯片的原理、设计和应用场景范围，AI 芯片的设计不仅仅考虑针对 AI 计算的加速，还需要充分考虑到AI 的应用算法、AI 框架等中间件，而不是停留在天天喊着吊打英伟达和 CUDA，实际上芯片难以用起来。	[Slides]
AI 编程与计算架构	进阶篇介绍 AI 编程与计算架构，将站在系统设计的角度，思考在设计现代机器学习系统中需要考虑的编译器问题，特别是中间表达乃至后端优化。	[Slides]
AI 推理系统与引擎	实际应用推理系统与引擎，讲了太多原理身体太虚容易消化不良，还是得回归到业务本质，让行业、企业能够真正应用起来，而推理系统涉及一些核心算法和注意的事情也分享下。	[Slides]
AI 框架核心技术	介绍 AI 框架核心技术，首先介绍任何一个 AI 框架都离不开的自动微分，通过自动微分功能后就会产生表示神经网络的图和算子，然后介绍 AI 框架前端的优化，还有最近很火的大模型分布式训练在 AI 框架中的关键技术。	[Slides]

课程设立目的

本课程主要为本科生高年级、硕博研究生、AI 系统从业者设计，帮助大家：

完整了解 AI 的计算机系统架构，并通过实际问题和案例，来了解 AI 完整生命周期下的系统设计。
介绍前沿系统架构和 AI 相结合的研究工作，了解主流框架、平台和工具来了解 AI 系统。

课程部分

一. AI 系统概述

编号	名称	具体内容
1	AI 系统	算法、框架、体系结构的结合，形成 AI 系统

二. AI 芯片体系结构

编号	名称	具体内容
1	AI 计算体系	神经网络等 AI 技术的计算模式和计算体系架构
2	AI 芯片基础	CPU、GPU、NPU 等芯片体系架构基础原理
3	图形处理器 GPU	GPU 的基本原理，英伟达 GPU 的架构发展
4	英伟达 GPU 详解	英伟达 GPU 的 Tensor Core、NVLink 深度剖析
5	国外 AI 处理器	谷歌、特斯拉等专用 AI 处理器核心原理
6	国内 AI 处理器	寒武纪、燧原科技等专用 AI 处理器核心原理
7	AI 芯片黄金 10 年	对 AI 芯片的编程模式和发展进行总结

三. AI 编译原理

编号	名称	具体内容
1	传统编译器	传统编译器 GCC 与 LLVM，LLVM 详细架构
2	AI 编译器	AI 编译器发展与架构定义，未来挑战与思考
3	前端优化	AI 编译器的前端优化(算子融合、内存优化等)
4	后端优化	AI 编译器的后端优化(Kernel 优化、AutoTuning)
5	多面体	待更 ing...
6	PyTorch2.0	PyTorch2.0 最重要的新特性：编译技术栈

四. AI 推理系统

编号	名称	具体内容
1	推理系统	推理系统整体介绍，推理引擎架构梳理
2	轻量网络	轻量化主干网络，MobileNet 等 SOTA 模型介绍
3	模型压缩	模型压缩 4 件套，量化、蒸馏、剪枝和二值化
4	转换&优化	AI 框架训练后模型进行转换，并对计算图优化
5	Kernel 优化	Kernel 层、算子层优化，对算子、内存、调度优化

五. AI 框架核心技术

编号	名称	具体内容
1	AI 框架基础	AI 框架的作用、发展、编程范式
2	自动微分	自动微分的实现方式和原理
3	计算图	计算图的概念，图优化、图执行、控制流表达

知识清单

备注

这个仓已经到达疯狂的 10G 啦（ZOMI 把所有制作过程、高清图片都原封不动提供），如果你要 git clone 会非常的慢，因此建议优先到 Releases · chenzomi12/AISystem 来下载你需要的内容

非常希望您也参与到这个开源课程中，B 站给 ZOMI 留言哦！

欢迎大家使用的过程中发现 bug 或者勘误直接提交代码 PR 到开源社区哦！

请大家尊重开源和 ZOMI 的努力，引用 PPT 的内容请规范转载标明出处哦！

ArcInstitute/evo2

Genome modeling and design across all domains of life

Evo 2: Genome modeling and design across all domains of life

Evo 2 is a state of the art DNA language model for long context modeling and design. Evo 2 models DNA sequences at single-nucleotide resolution at up to 1 million base pair context length using the StripedHyena 2 architecture. Evo 2 was pretrained using Savanna. Evo 2 was trained autoregressively on OpenGenome2, a dataset containing 8.8 trillion tokens from all domains of life.

We describe Evo 2 in our paper: "Genome modeling and design across all domains of life with Evo 2".

Note

Evo 2 published: read more in Nature.
Evo 2 20B released: 40B-level performance with double the speed, read more here.
Light install for 7B models: option compatible with more hardware, see Installation.

Setup

This repo is for running Evo 2 locally for inference or generation, using our Vortex inference code. For training and finetuning, see the section here. You can run Evo 2 without any installation using the Nvidia Hosted API. You can also self-host an instance using Nvidia NIM. See the Nvidia NIM section for more information.

Requirements

Evo 2 is built on the Vortex inference repo, see the Vortex github for more details and Docker option.

System requirements

[OS] Linux (official) or WSL2 (limited support)
[Software]
- CUDA: 12.1+ with compatible NVIDIA drivers
- cuDNN: 9.3+
- Compiler: GCC 9+ or Clang 10+ with C++17 support
- Python 3.11 or 3.12
Recommended Torch 2.6.x or 2.7.x

FP8 and Transformer Engine requirements

The 40B, 20B, and 1B models require FP8 via Transformer Engine for numerical accuracy and a Nvidia Hopper GPU. The 7B models can run in bfloat16 without Transformer Engine on any supported GPU.

Model	FP8 (Transformer Engine) Required
`evo2_7b` / `evo2_7b_262k` / `evo2_7b_base`	No
`evo2_20b`	Yes
`evo2_40b` / `evo2_40b_base`	Yes
`evo2_1b_base`	Yes

Always validate model outputs after configuration changes or on different hardware by using the tests.

Installation

Full install

Install Transformer Engine and Flash Attention first, then install Evo 2. We recommend using conda to install Transformer Engine:

conda install -c nvidia cuda-nvcc cuda-cudart-dev
conda install -c conda-forge transformer-engine-torch=2.3.0
pip install flash-attn==2.8.0.post2 --no-build-isolation
pip install evo2

Light install (7B models only, no Transformer Engine)

Evo 2 7B models can run without Transformer Engine or FP8-capable hardware. If you run into issues installing Flash Attention, see the Flash Attention GitHub for system requirements and troubleshooting.

# A compatible PyTorch must be installed before flash attention, for example: pip install torch==2.7.1 --index-url https://download.pytorch.org/whl/cu128
pip install flash-attn==2.8.0.post2 --no-build-isolation
pip install evo2

From source

git clone https://github.com/arcinstitute/evo2
cd evo2
pip install -e .

Verify installation

python -m evo2.test.test_evo2_generation --model_name evo2_7b  # or evo2_1b_base, evo2_20b, evo2_40b

Docker

Evo 2 can be run using Docker (shown below), Singularity, or Apptainer.

docker build -t evo2 .
docker run -it --rm --gpus '"device=0"' -v ./huggingface:/root/.cache/huggingface evo2 bash

Note: The volume mount (-v) preserves downloaded models between container runs and specifies where they are saved.

Once inside the container:

python -m evo2.test.test_evo2_generation --model_name evo2_7b

Usage

Checkpoints

We provide the following model checkpoints, hosted on HuggingFace:

Checkpoint Name	Description
`evo2_40b`	40B parameter model with 1M context
`evo2_20b`	20B parameter model with 1M context
`evo2_7b`	7B parameter model with 1M context
`evo2_40b_base`	40B parameter model with 8K context
`evo2_7b_base`	7B parameter model with 8K context
`evo2_1b_base`	Smaller 1B parameter model with 8K context
`evo2_7b_262k`	7B parameter model with 262K context
`evo2_7b_microviridae`	7B parameter base model fine-tuned on Microviridae genomes

Note: The 40B model requires multiple H100 GPUs. Vortex automatically handles device placement, splitting the model across available CUDA devices.

Optional: Triton inference kernels

Evo 2 can dispatch Triton kernels from Vortex PR #77. They require vtx>=1.1.0. Enable them when loading a model for faster inference:

from evo2 import Evo2
evo2_model = Evo2('evo2_7b', use_kernels=True) # enable inference kernels

to test, pass --use_kernels to the test scripts:

python -m evo2.test.test_evo2_generation --model_name evo2_7b --use_kernels

Forward

Evo 2 can be used to score the likelihoods across a DNA sequence.

import torch
from evo2 import Evo2

evo2_model = Evo2('evo2_7b')

sequence = 'ACGT'
input_ids = torch.tensor(
    evo2_model.tokenizer.tokenize(sequence),
    dtype=torch.int,
).unsqueeze(0).to('cuda:0')

outputs, _ = evo2_model(input_ids)
logits = outputs[0]

print('Logits: ', logits)
print('Shape (batch, length, vocab): ', logits.shape)

Embeddings

Evo 2 embeddings can be saved for use downstream. We find that intermediate embeddings work better than final embeddings, see our paper for details.

import torch
from evo2 import Evo2

evo2_model = Evo2('evo2_7b')

sequence = 'ACGT'
input_ids = torch.tensor(
    evo2_model.tokenizer.tokenize(sequence),
    dtype=torch.int,
).unsqueeze(0).to('cuda:0')

layer_name = 'blocks.28.mlp.l3'

outputs, embeddings = evo2_model(input_ids, return_embeddings=True, layer_names=[layer_name])

print('Embeddings shape: ', embeddings[layer_name].shape)

Generation

Evo 2 can generate DNA sequences based on prompts.

from evo2 import Evo2

evo2_model = Evo2('evo2_7b')

output = evo2_model.generate(prompt_seqs=["ACGT"], n_tokens=400, temperature=1.0, top_k=4)

print(output.sequences[0])

Notebooks

We provide example notebooks.

The BRCA1 scoring notebook shows zero-shot BRCA1 variant effect prediction. This example includes a walkthrough of:

Performing zero-shot BRCA1 variant effect predictions using Evo 2
Reference vs alternative allele normalization

The generation notebook shows DNA sequence completion with Evo 2. This example shows:

DNA prompt based generation and 'DNA autocompletion'
How to get and prompt using phylogenetic species tags for generation

The exon classifier notebook demonstrates exon classification using Evo 2 embeddings. This example shows:

Running the Evo 2 based exon classifier
Performance metrics and visualization

The sparse autoencoder (SAE) notebook explores interpretable features learned by Evo 2. This example includes:

Running and visualizing Evo 2 SAE features
Demonstrating SAE features on a part of the E. coli genome

Nvidia NIM

Evo 2 is available on Nvidia NIM and hosted API.

The quickstart guides users through running Evo 2 on the NVIDIA NIM using a python or shell client after starting NIM. An example python client script is shown below. This is the same way you would interact with the Nvidia hosted API.

#!/usr/bin/env python3
import requests
import os
import json
from pathlib import Path

key = os.getenv("NVCF_RUN_KEY") or input("Paste the Run Key: ")

r = requests.post(
    url=os.getenv("URL", "https://health.api.nvidia.com/v1/biology/arc/evo2-40b/generate"),
    headers={"Authorization": f"Bearer {key}"},
    json={
        "sequence": "ACTGACTGACTGACTG",
        "num_tokens": 8,
        "top_k": 1,
        "enable_sampled_probs": True,
    },
)

if "application/json" in r.headers.get("Content-Type", ""):
    print(r, "Saving to output.json:\n", r.text[:200], "...")
    Path("output.json").write_text(r.text)
elif "application/zip" in r.headers.get("Content-Type", ""):
    print(r, "Saving large response to data.zip")
    Path("data.zip").write_bytes(r.content)
else:
    print(r, r.headers, r.content)

Very long sequences

You can use Savanna or Nvidia BioNemo for embedding long sequences. Vortex can currently compute over very long sequences via teacher prompting, however please note that forward pass on long sequences may currently be slow.

Dataset

The OpenGenome2 dataset used for pretraining Evo2 is available on HuggingFace . Data is available either as raw fastas or as JSONL files which include preprocessing and data augmentation.

Training and Finetuning

Evo 2 was trained using Savanna, an open source framework for training alternative architectures.

To train or finetune Evo 2, you can use Savanna or Nvidia BioNemo which provides a Evo 2 finetuning tutorial here.

Citation

If you find these models useful for your research, please cite the relevant papers

@article{Brixi2026,
    author  = {Brixi, Garyk and Durrant, Matthew G. and Ku, Jerome and Naghipourfar, Mohsen and Poli, Michael and Sun, Gwanggyu and Brockman, Greg and Chang, Daniel and Fanton, Alison and Gonzalez, Gabriel A. and King, Samuel H. and Li, David B. and Merchant, Aditi T. and Nguyen, Eric and Ricci-Tam, Chiara and Romero, David W. and Schmok, Jonathan C. and Taghibakhshi, Ali and Vorontsov, Anton and Yang, Brandon and Deng, Myra and Gorton, Liv and Nguyen, Nam and Wang, Nicholas K. and Pearce, Michael T. and Simon, Elana and Adams, Etowah and Amador, Zachary J. and Ashley, Euan A. and Baccus, Stephen A. and Dai, Haoyu and Dillmann, Steven and Ermon, Stefano and Guo, Daniel and Herschl, Michael H. and Ilango, Rajesh and Janik, Ken and Lu, Amy X. and Mehta, Reshma and Mofrad, Mohammad R. K. and Ng, Madelena Y. and Pannu, Jaspreet and Ré, Christopher and St. John, John and Sullivan, Jeremy and Tey, Joseph and Viggiano, Ben and Zhu, Kevin and Zynda, Greg and Balsam, Daniel and Collison, Patrick and Costa, Anthony B. and Hernandez-Boussard, Tina and Ho, Eric and Liu, Ming-Yu and McGrath, Thomas and Powell, Kimberly and Pinglay, Sudarshan and Burke, Dave P. and Goodarzi, Hani and Hsu, Patrick D. and Hie, Brian L.},
    title   = {Genome modelling and design across all domains of life with Evo 2},
    journal = {Nature},
    year    = {2026},
    doi     = {10.1038/s41586-026-10176-5},
    url     = {https://doi.org/10.1038/s41586-026-10176-5},
}

dscripka/openWakeWord

An open-source audio wake word (or phrase) detection framework with a focus on performance and simplicity.

openWakeWord

openWakeWord is an open-source wakeword library that can be used to create voice-enabled applications and interfaces. It includes pre-trained models for common words & phrases that work well in real-world environments.

Quick Links

Updates

2024/02/11

v0.6.0 of openWakeWord released. See the releases for a full descriptions of new features and changes.

2023/11/09

Added example scripts under examples/web that demonstrate streaming audio from a web application into openWakeWord.

2023/10/11

Significant improvements to the process of training new models, including an example Google Colab notebook demonstrating how to train a basic wake word model in <1 hour.

2023/06/15

v0.5.0 of openWakeWord released. See the releases for a full descriptions of new features and changes.

Demo

You can try an online demo of the included pre-trained models via HuggingFace Spaces right here!

Note that real-time detection of a microphone stream can occasionally behave strangely in Spaces. For the most reliable testing, perform a local installation as described below.

Installation

Installing openWakeWord is simple and has minimal dependencies:

pip install openwakeword

On Linux systems, both the onnxruntime package and tflite-runtime packages will be installed as dependencies since both inference frameworks are supported. On Windows, only onnxruntime is installed due to a lack of support for modern versions of tflite.

To (optionally) use Speex noise suppression on Linux systems to improve performance in noisy environments, install the Speex dependencies and then the pre-built Python package (see the assets here for all .whl versions), adjusting for your python version and system architecture as needed.

sudo apt-get install libspeexdsp-dev
pip install https://github.com/dscripka/openWakeWord/releases/download/v0.1.1/speexdsp_ns-0.1.2-cp38-cp38-linux_x86_64.whl

Many thanks to TeaPoly for their Python wrapper of the Speex noise suppression libraries.

Usage

For quick local testing, clone this repository and use the included example script to try streaming detection from a local microphone. You can individually download pre-trained models from current and past releases, or you can download them using Python (see below).

Adding openWakeWord to your own Python code requires just a few lines:

import openwakeword
from openwakeword.model import Model

# One-time download of all pre-trained models (or only select models)
openwakeword.utils.download_models()

# Instantiate the model(s)
model = Model(
    wakeword_models=["path/to/model.tflite"],  # can also leave this argument empty to load all of the included pre-trained models
)

# Get audio data containing 16-bit 16khz PCM audio data from a file, microphone, network stream, etc.
# For the best efficiency and latency, audio frames should be multiples of 80 ms, with longer frames
# increasing overall efficiency at the cost of detection latency
frame = my_function_to_get_audio_frame()

# Get predictions for the frame
prediction = model.predict(frame)

Additionally, openWakeWord provides other useful utility functions. For example:

# Get predictions for individual WAV files (16-bit 16khz PCM)
from openwakeword.model import Model

model = Model()
model.predict_clip("path/to/wav/file")

# Get predictions for a large number of files using multiprocessing
from openwakeword.utils import bulk_predict

bulk_predict(
    file_paths = ["path/to/wav/file/1", "path/to/wav/file/2"],
    wakeword_models = ["hey jarvis"],
    ncpu=2
)

See openwakeword/utils.py and openwakeword/model.py for the full specification of class methods and utility functions.

Recommendations for Usage

Noise Suppression and Voice Activity Detection (VAD)

While the default settings for openWakeWord will work well in many cases, there are adjustable parameters in openWakeWord that can improve performance in some deployment scenarios.

On supported platforms (currently only X86 and Arm64 linux), Speex noise suppression can be enabled by setting the enable_speex_noise_suppression=True when instantiating an openWakeWord model. This can improve performance when relatively constant background noise is present.

Second, a voice activity detection (VAD) model from Silero is included with openWakeWord, and can be enabled by setting the vad_threshold argument to a value between 0 and 1 when instantiating an openWakeWord model. This will only allow a positive prediction from openWakeWord when the VAD model simultaneously has a score above the specified threshold, which can significantly reduce false-positive activations in the present of non-speech noise.

Threshold Scores for Activation

All of the included openWakeWord models were trained to work well with a default threshold of 0.5 for a positive prediction, but you are encouraged to determine the best threshold for your environment and use-case through testing. For certain deployments, using a lower or higher threshold in practice may result in significantly better performance.

User-specific models

If the baseline performance of openWakeWord models is not sufficient for a given application (specifically, if the false activation rate is unacceptably high), it is possible to train custom verifier models for specific voices that act as a second-stage filter on predictions (i.e., only allow activations through that were likely spoken by a known set of voices). This can greatly improve performance, at the cost of making the openWakeWord system less likely to respond to new voices.

Project Goals

openWakeWord has four high-level goals, which combine to (hopefully!) produce a framework that is simple to use and extend.

Be fast enough for real-world usage, while maintaining ease of use and development. For example, a single core of a Raspberry Pi 3 can run 15-20 openWakeWord models simultaneously in real-time. However, the models are likely still too large for less powerful systems or micro-controllers. Commercial options like Picovoice Porcupine or Fluent Wakeword are likely better suited for highly constrained hardware environments.
Be accurate enough for real-world usage. The included models are typically have false-accept and false-reject rates below the annoyance threshold for the average user. This is obviously subjective, by a false-accept rate of <0.5 per hour and a false-reject rate of <5% is often reasonable in practice. See the Performance & Evaluation section for details about how well the included models can be expected to perform in practice.
Have a simple model architecture and inference process. Models process a stream of audio data in 80 ms frames, and return a score between 0 and 1 for each frame indicating the confidence that a wake word/phrase has been detected. All models also have a shared feature extraction backbone, so that each additional model only has a small impact to overall system complexity and resource requirements.
Require little to no manual data collection to train new models. The included models (see the Pre-trained Models section for more details) were all trained with 100% synthetic speech generated from text-to-speech models. Training new models is a simple as generating new clips for the target wake word/phrase and training a small model on top of of the frozen shared feature extractor. See the Training New Models section for more details.

Future releases of openWakeWord will aim to stay aligned with these goals, even when adding new functionality.

Pre-Trained Models

openWakeWord comes with pre-trained models for common words & phrases. Currently, only English models are supported, but they should be reasonably robust across different types speaker accents and pronunciation.

The table below lists each model, examples of the word/phrases it is trained to recognize, and the associated documentation page for additional detail. Many of these models are trained on multiple variations of the same word/phrase; see the individual documentation pages for each model to see all supported word & phrase variations.

Model	Detected Speech	Documentation Page
alexa	"alexa"	docs
hey mycroft	"hey mycroft"	docs
hey jarvis	"hey jarvis"	docs
hey rhasspy	"hey rhasspy"	TBD
current weather	"what's the weather"	docs
timers	"set a 10 minute timer"	docs

Based on the methods discussed in performance testing, each included model aims to meet the target performance criteria of <5% false-reject rates and <0.5/hour false-accept rates with appropriate threshold tuning. These levels are subjective, but hopefully are below the annoyance threshold where the average user becomes frustrated with a system that often misses intended activations and/or causes disruption by activating too frequently at undesired times. For example, at these performance levels a user could expect to have the model process continuous mixed content audio of several hours with at most a few false activations, and have a failed intended activation in only 1/20 attempts (and a failed retry in only 1/400 attempts).

If you have a new wake word or phrase that you would like to see included in the next release, please open an issue, and we'll do a best to train a model! The focus of these requests and future release will be on words and phrases that have broad general usage versus highly specific application.

Model Architecture

openWakeword models are composed of three separate components:

A pre-processing function that computes melspectrogram of the input audio data. For openWakeword, an ONNX implementation of Torch's melspectrogram function with fixed parameters is used to enable efficient performance across devices.
A shared feature extraction backbone model that converts melspectrogram inputs into general-purpose speech audio embeddings. This model is provided by Google as a TFHub module under an Apache-2.0 license. For openWakeWord, this model was manually re-implemented to separate out different functionality and allow for more control of architecture modifications compared to a TFHub module. The model itself is series of relatively simple convolutional blocks, and gains its strong performance from extensive pre-training on large amounts of data. This model is the core component of openWakeWord, and enables the strong performance that is seen even when training on fully-synthetic data.
A classification model that follows the shared (and frozen) feature extraction model. The structure of this classification model is arbitrary, but in practice a simple fully-connected network or 2 layer RNN works well.

Performance and Evaluation

Evaluating wake word/phrase detection models is challenging, and it is often very difficult to assess how different models presented in papers or other projects will perform when deployed with respect to two critical metrics: false-reject rates and false-accept rates. For clarity in definitions:

A false-reject is when the model fails to detect an intended activation from a user.

A false-accept is when the model inadvertently activates when the user did not intend for it to do so.

For openWakeWord, evaluation follows two principles:

The false-reject rate should be determined from wakeword/phrases that represent realistic recording environments, including those with background noise and reverberation. This can be accomplished by directly collected data from these environments, or simulating them with data augmentation methods.
The false-accept rate should be determined from audio that represents the types of environments that would be expected for the deployed model, not just on the training/evaluation data. In practice, this means that the model should only rarely activate in error, even in the presence of hours of continuous speech and background noise.

While other wakeword evaluation standards do exist, for openWakeWord it was decided that a custom evaluation would better indicate what performance users can expect for real-world deployments. Specifically:

false-reject rates are calculated from either clean recordings of the wakeword that are mixed with background noise at realistic signal-to-noise ratios (e.g., 5-10 dB) and reverberated with room Impulse Response Functions (RIRs) to better simulate far-field audio, or manually collected data from realistic deployment environments (e.g., far-field capture with normal environment noise).
false-accept rates are determined by using the Dinner Party Corpus dataset, which represents ~5.5 hours of far-field speech, background music, and miscellaneous noise. This dataset sets a realistic (if challenging) goal for how many false activations might occur in a similar situation.

To illustrate how openWakeWord can produce capable models, the false-accept/false-reject curves for the included "alexa" model is shown below along with the performance of a strong commercial competitor, Picovoice Porcupine. Other existing open-source wakeword engines (e.g., Snowboy, PocketSphinx, etc.) are not included as they are either no longer maintained or demonstrate performance significantly below that of Porcupine. The positive test examples used were those included in Picovoice's repository, a fantastic resource that they have freely provided to the community. Note, however, that the test data was prepared differently compared to Picovoice's implementation (see the Alexa model documentation for more details).

For at least this test data and preparation, openWakeWord produces a model that is more accurate than Porcupine.

As a second illustration, the false-accept/false-reject rate of the included "hey mycroft" model is shown below along with the performance of a custom Picovoice Porcupine model and Mycroft Precise. In this case, the positive test examples were manually collected from a male speaker with a relatively neutral American english accent in realistic home recording scenarios (see the Hey Mycroft model documentation for more details).

Again, for at least this test data and preparation, openWakeWord produces a model at least as good as existing solutions.

However, in should noted that for both of these tests sample sizes are small and there are issues (1, 2) with the evaluation of the other libraries that suggest these results should be interpreted cautiously. As such, the only claim being made is that openWakeWord models are broadly competitive with comparable offerings. You are strongly encouraged to test openWakeWord to determine if it will meet the requirements of your use-case.

Finally, to give evidence that the core methods behind openWakeWord (i.e., pre-trained speech embeddings and high-quality synthetic speech) are effective across a wider range of wake word/phrase structure and length, the table below shows the performance on the Fluent Speech Commands test set using an openWakeWord model and the baseline method shown in a related paper by the dataset authors. While both models were trained on fully-synthetic data, due to fundamentally different data synthesis & preparation, training, and evaluation approaches, the numbers below are likely not directly comparable. Rather, the important conclusion is that openWakeWord is a viable approach for the task of spoken language understanding (SLU).

Model	Test Set Accuracy	Link
openWakeWord	~97.5%	NA
encoder-decoder	~94.9%	paper

If you are aware of other open-source wakeword/phrase libraries that should be added to these comparisons, or have suggestions on how to improve the evaluation more generally, please open an issue! We are eager to continue improving openWakeWord by learning how others are approaching this problem.

Other Performance Details

Model Robustness

Due to a combination of variability in the generated speech and the extensive pre-training from Google, openWakeWord models also demonstrate some additional performance benefits that are useful for real-world applications. In testing, three in particular have been observed.

The trained models seem to respond reasonably well to wakewords and phrases that are whispered. This is somewhat surprising behavior, as the text-to-speech models used for producing training data generally do not create synthetic speech that has acoustic qualities similar to whispering.
The models also respond relatively well to wakewords and phrases spoken at different speeds (within reason).
The models are able to handle some variability in the phrasing of a given command. This behavior was not entirely a surprise, given that others have reported similar benefits when training end-to-end spoken language understanding systems. For example, the included pre-trained weather model will typically still respond correctly to a phrase like "how is the weather today" despite not training directly on that phrase (though false rejections rates will likely be higher, on average, compared to phrases closer to the training data).

Background Noise

While the models are trained with background noise to increase robustness, in some cases additional noise suppression can improve performance. Setting the enable_speex_noise_suppression=True argument during openWakeWord model initialization will use the efficient Speex noise suppression algorithm to pre-process the audio data prior to prediction. This can reduce both false-reject rates and false-accept rates, though testing in a realistic deployment environment is strongly recommended.

Training New Models

openWakeWord includes an automated utility that greatly simplifies the process of training custom models. This can be used in two ways:

A simple Google Colab notebook with an easy to use interface and simple end-to-end process. This allows anyone to produce a custom model very quickly (<1 hour) and doesn't require any development experience, but the performance of the model may be low in some deployment scenarios.
A more detailed notebook that describes the training process in more details, and enables more customization. This can produce high quality models, but requires more development experience.

For a collection of models trained using the notebooks above by the Home Assistant Community (and with much gratitude to @fwartner), see the excellent repository here.

For users interested in understanding the fundamental concepts behind model training there is a more detailed, educational tutorial notebook also available. However, this specific notebook is not intended for training production models, and the automated process above is recommended for that purpose.

Fundamentally, a new model requires two data generation and collection steps:

Generate new training data for the desired wakeword/phrase using open-source speech-to-text systems (see Synthetic Data Generation for more details). These models and the generation code are hosted in a separate repository. The number of generated examples required can vary, a minimum of several thousand is recommended and performance seems to increase smoothly with increasing dataset size.
Collect negative data (e.g., audio where the wakeword/phrase is not present) to help the model have a low false-accept rate. This also benefits from scale, and the included models were all trained with ~30,000 hours of negative data representing speech, noise, and music. See the individual model documentation pages for more details on training data curation and preparation.

Language Support

Currently, openWakeWord only supports English, primarily because the pre-trained text-to-speech models used to generate training data are all based on english datasets. It's likely that speech-to-text models trained on other languages would also work well, but non-english models & datasets are less commonly available.

Future release road maps may have non-english support. In particular, Mycroft.AIs Mimic 3 TTS engine may work well to help extend some support to other languages.

FAQ

Is there a Docker implementation for openWakeWord?

While there isn't an official Docker implementation, @dalehumby has created one that works very well!

Can openWakeWord be run in a browser with javascript?

While the ONNX runtime does support javascript, much of the other functionality required for openWakeWord models would need to be ported. This is not currently on the roadmap, but please open an issue/start a discussion if this feature is of particular interest.
As a potential work-around for some applications, the example scripts in examples/web demonstrate how audio can be captured in a browser and streaming via websockets into openWakeWord running in a Python backend server.
Other potential options could include projects like pyodide (see here) for a related issue.

Is there a C++ version of openWakeWord?

While the ONNX runtime also has a C++ API, there isn't an official C++ implementation of the full openWakeWord library. However, @synesthesiam has created a C++ version of openWakeWord with basic functionality implemented.

Is openWakeWord suitable for edge devices and microcontrollers?

openWakeWord is generally small and efficient, but likely not enough to be suitable for deployment on very low power edge devices. For example, some experimentation by other openWakeWord users & contributors indicates that it may still take several seconds to process a single 80 ms frame on an ESP32-S3 with quantized openWakeWord models. Instead, I would recommend the excellent microWakeWord library from @kahrendt. It uses a similar synthetic-only training data approach and can produce high quality models that are efficient enough to run on very low power edge devices.

Why are there three separate models instead of just one?

Separating the models was an intentional choice to provide flexibility and optimize the efficiency of the end-to-end prediction process. For example, with separate melspectrogram, embedding, and prediction models, each one can operate on different size inputs of audio to optimize overall latency and share computations between models. It certainly is possible to make a combined model with all of the steps integrated, though, if that was a requirement of a particular use case.

I still get a large number of false activations when I use the pre-trained models, how can I reduce these?

First, review the recommendations for usage and ensure that these options do not improve overall system accuracy. Second, experiment with custom verifier models, if possible. If neither of these approaches are helping, please open an issue with details of the deployment environment and the types of false activations that you are experiencing. We certainly appreciate feedback & requests on how to improve the base pre-trained models!

Acknowledgements

I am very grateful for the encouraging and positive response from the open-source community since the release of openWakeWord in January 2023. In particular, I want to acknowledge and thank the following individuals and groups for their feedback, collaboration, and development support:

License

All of the code in this repository is licensed under the Apache 2.0 license. All of the included pre-trained models are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license due to the inclusion of datasets with unknown or restrictive licensing as part of the training data. If you are interested in pre-trained models with more permissive licensing, please raise an issue and we will try to add them to a future release.

datawhalechina/happy-llm

📚 从零开始构建大模型

Happy-LLM

中文 | English

📚 在线阅读地址

📚 从零开始构建大模型

深入理解 LLM 核心原理，动手实现你的第一个大模型

🎯 项目介绍

很多小伙伴在看完 Datawhale开源项目： self-llm 开源大模型食用指南后，感觉意犹未尽，想要深入了解大语言模型的原理和训练过程。于是我们（Datawhale）决定推出《Happy-LLM》项目，旨在帮助大家深入理解大语言模型的原理和训练过程。

本项目是一个系统性的 LLM 学习教程，将从 NLP 的基本研究方法出发，根据 LLM 的思路及原理逐层深入，依次为读者剖析 LLM 的架构基础和训练过程。同时，我们会结合目前 LLM 领域最主流的代码框架，演练如何亲手搭建、训练一个 LLM，期以实现授之以鱼，更授之以渔。希望大家能从这本书开始走入 LLM 的浩瀚世界，探索 LLM 的无尽可能。

✨ 你将收获什么？

📚 Datawhale 开源免费 完全免费的学习本项目所有内容
🔍 深入理解 Transformer 架构和注意力机制
📚 掌握预训练语言模型的基本原理
🧠 了解现有大模型的基本结构
🏗️ 动手实现 一个完整的 LLaMA2 模型
⚙️ 掌握训练 从预训练到微调的全流程
🚀 实战应用 RAG、Agent 等前沿技术

📖 内容导航

章节	关键内容	状态
学习与环境准备	分章依赖、硬件建议与实践入口	✅
前言	本项目的缘起、背景及读者建议	✅
第一章 NLP 基础概念	什么是 NLP、发展历程、任务分类、文本表示演进	✅
第二章 Transformer 架构	注意力机制、Encoder-Decoder、手把手搭建 Transformer	✅
第三章预训练语言模型	Encoder-only、Encoder-Decoder、Decoder-Only 模型对比	✅
第四章大语言模型	LLM 定义、训练策略、涌现能力分析	✅
第五章动手搭建大模型	实现 LLaMA2、训练 Tokenizer、预训练小型 LLM	✅
第六章大模型训练实践	预训练、有监督微调、LoRA/QLoRA 高效微调	✅
第七章大模型应用	模型评测、RAG 检索增强、Agent 智能体	✅
第八章 Agentic-RL	GRPO、OPD、Search-R1、ReTool（Coding Agent-RL）	✅
Extra Chapter LLM Blog	优秀的大模型学习笔记/Blog ，欢迎大家来 PR ！	🚧

第六章正文已覆盖 Pretrain、SFT 与 PEFT 等核心训练流程，建议结合第六章实践说明和学习与环境准备一起阅读。

第八章聚焦 GRPO、OPD、Search-R1 与 ReTool。更多 Agentic RL 算法、训练代码与实验实践，可以前往作者持续维护的另一个仓库 agentic-rl-lab；该仓库更新频率更高，会持续跟进新的算法与环境。

Extra Chapter LLM Blog

大模型都这么厉害了，微调0.6B的小模型有什么意义？ @不要葱姜蒜 2025-7-11
Transformer 整体模块设计解读 @ditingdapeng 2025-7-14
文本数据处理详解 @蔡鋆捷 2025-7-14
Qwen3-"VL"——超小中文多模态模型的“拼接微调”之路 @ShaohonChen 2025-7-30
S1: Thinking Budget with vLLM @不要葱姜蒜 2025-8-03
CDDRS: 使用细粒度语义信息指导增强的RAG检索方法 @Hongru0306 2025-8-21
大模型生成 Token 的方式有哪些？ @不要葱姜蒜 2025-10-17

如果大家在学习 Happy-LLM 项目或 LLM 相关知识中有自己独到的见解、认知、实践，欢迎大家 PR 在 Extra Chapter LLM Blog 中。请遵守 Extra Chapter LLM Blog 的 PR 规范，我们会视 PR 内容的质量和价值来决定是否合并或补充到 Happy-LLM 正文中来。

模型下载

模型名称	下载地址
Happy-LLM-Chapter5-Base-215M	🤖 ModelScope
Happy-LLM-Chapter5-SFT-215M	🤖 ModelScope

ModelScope 创空间体验地址：🤖 创空间

PDF 版本下载

本 Happy-LLM PDF 教程完全开源免费。为防止各类营销号加水印后贩卖给大模型初学者，我们特地在 PDF 文件中预先添加了不影响阅读的 Datawhale 开源标志水印，敬请谅解～

Happy-LLM PDF : https://github.com/datawhalechina/happy-llm/releases/tag/v1.0.2

PPT 资源下载

本项目配套教学讲义PPT课件资源获取链接：https://github.com/HZAI-ZJNU/happy-llm-ppt 或可在本项目的 Releases 页面下载。

💡 如何学习

本项目适合大学生、研究人员、LLM 爱好者。在学习本项目之前，建议具备一定的编程经验，尤其是要对 Python 编程语言有一定的了解。最好具备深度学习的相关知识，并了解 NLP 领域的相关概念和术语，以便更轻松地学习本项目。

如果你计划复现章节代码，建议先阅读学习与环境准备。仓库当前按章节拆分依赖，不同章节建议使用独立的 Python 环境，以减少版本冲突。

本项目分为两部分——基础知识与实战应用。第1章～第4章是基础知识部分，从浅入深介绍 LLM 的基本原理。其中，第1章简单介绍 NLP 的基本任务和发展，为非 NLP 领域研究者提供参考；第2章介绍 LLM 的基本架构——Transformer，包括原理介绍及代码实现，作为 LLM 最重要的理论基础；第3章整体介绍经典的 PLM，包括 Encoder-Only、Encoder-Decoder 和 Decoder-Only 三种架构，也同时介绍了当前一些主流 LLM 的架构和思想；第4章则正式进入 LLM 部分，详细介绍 LLM 的特点、能力和整体训练过程。第5章～第8章是实战应用部分，将逐步带领大家深入 LLM 的底层细节。其中，第5章将带领大家基于 PyTorch 亲手搭建一个 LLM，并实现预训练、有监督微调的全流程；第6章将引入目前业界主流的 LLM 训练框架 Transformers，带领学习者基于该框架快速、高效地实现 LLM 训练过程；第7章介绍 LLM 的评测、检索增强生成（Retrieval-Augmented Generation，RAG）和智能体（Agent）；第8章进一步介绍 GRPO、OPD，以及 Search-R1 与 ReTool 两类 Agentic RL 实践。你可以根据个人兴趣和需求，选择性地阅读相关章节。

在阅读本书的过程中，建议你将理论和实际相结合。LLM 是一个快速发展、注重实践的领域，我们建议你多投入实战，复现本书提供的各种代码，同时积极参加 LLM 相关的项目与比赛，真正投入到 LLM 开发的浪潮中。我们鼓励你关注 Datawhale 及其他 LLM 相关开源社区，当遇到问题时，你可以随时在本项目的 issue 区提问。

最后，欢迎每一位读者在学习完本项目后加入到 LLM 开发者的行列。作为国内 AI 开源社区，我们希望充分聚集共创者，一起丰富这个开源 LLM 的世界，打造更多、更全面特色 LLM 的教程。星火点点，汇聚成海。我们希望成为 LLM 与普罗大众的阶梯，以自由、平等的开源精神，拥抱更恢弘而辽阔的 LLM 世界。

中国计算机学会(CCF) × Datawhale × GitLink开源平台联合推出AI普惠课程，免费算力报名参加【报名地址】【GitLink 地址】

🤝 如何贡献

我们欢迎任何形式的贡献！

🐛 报告 Bug - 发现问题请提交 Issue
💡 功能建议 - 有好想法就告诉我们
📝 内容完善 - 帮助改进教程内容
🔧 代码优化 - 提交 Pull Request

🙏 致谢

核心贡献者

宋志学-项目负责人 (Datawhale成员)
邹雨衡-项目负责人 (Datawhale成员-对外经济贸易大学)
朱信忠-指导专家（Datawhale首席科学家-浙江师范大学杭州人工智能研究院教授）

Extra-Chapter 贡献者

ditingdapeng（内容贡献者-云原生基础架构工程师）
蔡鋆捷（内容贡献者-福州大学）
ShaohonChen （情感机器实验室研究员-西安电子科技大学在读硕士）
肖鸿儒, 庄健琨 (内容贡献者-同济大学)

特别感谢

感谢 @Sm1les 对本项目的帮助与支持
感谢所有为本项目做出贡献的开发者们 ❤️

Star History

⭐ 如果这个项目对你有帮助，请给我们一个 Star！

关于 Datawhale

扫描二维码关注 Datawhale 公众号，获取更多优质开源内容

📜 开源协议

本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。

mrdbourke/zero-to-mastery-ml

All course materials for the Zero to Mastery Machine Learning and Data Science course.

Zero to Mastery Machine Learning

Welcome! This repository contains all of the code, notebooks, images and other materials related to the Zero to Mastery Machine Learning Course on Udemy and zerotomastery.io.

Quick links

🎥 Watch the first 10 hours of the course on YouTube.
📚 Read the materials of the course in a beautiful online book.
🤔 Found something wrong with the code? Leave an issue.
❓ Got a question? Post a discussion (see the question template).

Updates

30 October 2024 - Add course book version of Milestone Project 2: Bulldozer Price Regression (updated for 2025 onwards)
12 September 2024 - Working on updating the materials for 2025, see progress in #105
12 October 2023 - Created an online book version of the course materials, see: https://dev.mrdbourke.com/zero-to-mastery-ml/

The following contents are listed in suggested chronological order.

But feel free to mix in match in anyway you feel fit.

Note: All of the datasets we use in the course are available in the data/ folder.

Section	Resource	Description
00	A 6 step framework for approaching machine learning projects	A guideline for different kinds of machine learning projects and how to break them down into smaller steps.
01	Introduction to NumPy	NumPy stands for Numerical Python. It's one of the most used Python libraries for numerical processing (which is what much of data science and machine learning is).
02	Introduction to pandas	pandas is a Python library for manipulating and analysing data. You can imagine pandas as a programmatic form of an Excel spreadsheet.
03	Introduction to Matplotlib	Matplotlib helps to visualize data. You can create plots and graphs programmatically based on various data sources.
04	Introduction to Scikit-Learn	Scikit-Learn or sklearn is full of data processing techniques as well as pre-built machine learning algorithms for many different tasks.
05	Milestone Project 1: End-to-end Heart Disease Classification	Here we'll put together everything we've gone through in the previous sections to create a machine learning model that is capable of classifying if someone has heart disease or not based on their health characteristics. We'll start with a raw dataset and work through performing an exploratory data analysis (EDA) on it before trying out several different machine learning models to see which performs best.
06	Milestone Project 2: End-to-end Bulldozer Price Prediction	In this project we'll work with an open-source dataset of bulldozer sales information. We'll use this data to build a machine learning model capable of predicting the sales price of a bulldozer based on several input parameters such as size and brand. Since this dataset isn't perfect, we'll work through several data preprocessing steps before building a model. And since we'll be working towards predicting a number (price of bulldozers), this project is known as regression project.
07	Milestone Project 3: Introduction to TensorFlow/Keras and Deep Learning	TensorFlow/Keras are deep learning frameworks written in Python. Originally created by Google and are now open-source. These frameworks allow you to build and train neural networks, one of the most powerful kinds of machine learning models. In this section we'll learn about deep learning and TensorFlow/Keras by building Dog Vision 🐶👁️, a neural network to identify dog breeds in images.
08	Communicating your work	One of the most important parts of machine learning and any software project is communicating what you've found/done. This module takes the learnings from the previous sections and gives tips and tricks on how you can communicate your work to others.

What this course focuses on

Create a framework for working through problems (6 step machine learning modelling framework)
Find tools to fit the framework
Targeted practice = use tools and framework steps to work on end-to-end machine learning modelling projects

How this course is structured

Section 1 - Getting your mind and computer ready for machine learning (concepts, computer setup)
Section 2 - Tools for machine learning and data science (pandas, NumPy, Matplotlib, Scikit-Learn)
Section 3 - End-to-end structured data projects (classification and regression)
Section 4 - Neural networks, deep learning and transfer learning with TensorFlow 2.0
Section 5 - Communicating and sharing your work

Student notes

Some students have taken and shared extensive notes on this course, see them below.

If you'd like to submit yours, leave a pull request.

Chester's notes - https://github.com/chesterheng/machinelearning-datascience
Sophia's notes - https://www.rockyourcode.com/tags/udemy-complete-machine-learning-and-data-science-zero-to-mastery/

ageron/handson-mlp

A series of Jupyter notebooks that walk you through the fundamentals of Machine Learning and Deep Learning in Python using Scikit-Learn, PyTorch, and Hugging Face libraries.

Hands-On Machine Learning with Scikit-Learn and PyTorch

The goal of this project is to teach you the fundamentals of Machine Learning in Python. It contains the example code and solutions to the exercises in the first edition of my new O'Reilly book Hands-on Machine Learning with Scikit-Learn and PyTorch (1st edition):

Note: If you are looking for the notebooks for the TensorFlow/Keras version of this book, check out ageron/handson-ml3.

Quick Start

Want to play with these notebooks online without having to install anything?

(recommended)

⚠ Colab provides a temporary environment: anything you do will be deleted after a while, so make sure you download any data you care about.

Just want to quickly look at some notebooks, without executing any code?

github.com's notebook viewer also works but it's not ideal: it's slower, the math equations are not always displayed correctly, and large notebooks often fail to open.

Want to install this project on your own machine?

Check out the installation instructions.

FAQ

Which Python version should I use?

I recommend Python 3.12. If you follow the installation instructions above, that's the version you will get. Versions 3.10 and 3.11 should work as well, but some libraries are not yet available for 3.13.

I'm getting an error when I call load_housing_data()

If you're getting an HTTP error, make sure you're running the exact same code as in the notebook (copy/paste it if needed). If the problem persists, please check your network configuration. If it's an SSL error, see the next question.

I'm getting an SSL error on MacOSX

You probably need to install the SSL certificates (see this StackOverflow question). If you downloaded Python from the official website, then run /Applications/Python\ 3.12/Install\ Certificates.command in a terminal (change 3.12 to whatever version you installed). If you installed Python using MacPorts, run sudo port install curl-ca-bundle in a terminal.

I've installed this project locally. How do I update it to the latest version?

See INSTALL.md

How do I update my Python libraries to the latest versions, when using Anaconda?

See INSTALL.md

Contributors

I would like to thank everyone who contributed to this project, either by providing useful feedback, filing issues or submitting Pull Requests.

GitHub Jupyter Notebook Weekly Trending Repositories

lyogavin/airllm

AI Agents Recommendation:

Updates

Star History

Table of Contents

Quickstart

1. Install package

2. Inference

Model Compression - 3x Inference Speed Up!

How to enable model compression speed up:

What are the differences between model compression and quantization?

Configurations

MacOS

Example Python Notebook

example of other models (ChatGLM, QWen, Baichuan, Mistral, etc):

To request other model support: here

Supported Models

Tiny GPU, huge models

Acknowledgement

FAQ

1. MetadataIncompleteBuffer

2. ValueError: max() arg is an empty sequence

3. 401 Client Error....Repo model ... is gated.

4. ValueError: Asking to pad but the tokenizer does not have a padding token.

Citing AirLLM

Sponsors

Run AI Agent Teams in the Cloud — Bloome

Contribution

DataExpert-io/data-engineer-handbook

The Data Engineering Handbook

Join the free Databricks AI boot camp on August 3rd here and signup for Databricks Free Edition here

Getting started

Resources

Great list of over 25 books

Great list of over 10 communities to join:

Companies:

Data Engineering blogs of companies:

Data Engineering Whitepapers:

Social Media Accounts

YouTube

LinkedIn

X/Twitter

Instagram

TikTok

Great Podcasts

Great list of 20+ newsletters

Glossaries:

Design Patterns

Courses / Academies

Certifications Courses

microsoft/AI-For-Beginners

Artificial Intelligence for Beginners - A Curriculum

🌐 Multi-Language Support

Supported via GitHub Action (Automated & Always Up-to-Date)

Join the Community

What you will learn

Content

Each lesson contains

Getting Started

🎯 New to AI? Start Here!

📚 Full Curriculum Setup

Meet other Learners

Quizzes

Help Wanted

Special Thanks

Other Curricula

LangChain

Azure / Edge / MCP / Agents

Generative AI Series

Core Learning

Copilot Series

Getting Help

microsoft/generative-ai-for-beginners

21 Lessons teaching everything you need to know to start building Generative AI applications

🌐 Multi-Language Support

Supported via GitHub Action (Automated & Always Up-to-Date)

Generative AI for Beginners (Version 3) - A Course

🌱 Getting Started

What You Need