Project Introduction

This is a digital avatar project, the core idea is to use QQ's C2C chat records as a dataset to fine-tune large models, making the model restore your unique expression style and chat patterns as much as possible.

This project includes a complete tutorial, covering:

• Decrypting and processing QQ databases
• Cleaning and converting chat data
• QLoRA fine-tuning workflow
• Testing and using fine-tuned models
• Accelerating training with Unsloth!

I know there are already quite a few similar projects out there, but maybe my tutorial, workflow, and code implementation can offer you something different or spark new ideas. If you find it useful, feel free to give it a star — it’ll make me happy!

The project still has its shortcomings:

• I’m not sure what they are for now
• (If you run into issues, feel free to open an Issue)
• But it’s already capable of fine-tuning Qwen3-8B with FP8 precision on a 4090 24G GPU (tested and working)

If you also want to create your own digital persona, give it a try!

—— X: @qqqqqf5

---

Project Version

V 0.1.1

Warning Good News

• This version of Qlora_qwen3.py has been tested on a 4090 (using generate_training_data_llm.py + run_finetune.py)
• Data cleaning has also been tested on the current version

TODO

• [Completed but untested] Add support for OSS models (and MXFP4? This is a computation for the 50 series, and I still can’t test it myself)

Challenges:

1. MOE models
2. Non-Qwen series models
3. My 3080 seems unable to run local tests (for either fine-tuning or MXFP4) Well, it’s actually not that hard — I’ve just been busy with other projects these past few days.

Changelog

Written in the commits — don’t feel like writing it here.