➡ ADVANCED Vision Finetuning Repo: https://trelis.com/advancedvision/
➡ ADVANCEDinference Repo: https://trelis.com/enterpriseserver...
➡ ADVANCEDfinetuning Repo: https://trelis.com/advancedfinetuni...
➡ Trelis Functioncalling Models and Scripts: https://trelis.com/functioncalling/
➡ ADVANCED Transcription Repo: https://trelis.com/advancedtranscrip...
➡ Oneclick Finetuning & Inference Templates: https://github.com/TrelisResearch/one...
➡ Trelis Newsletter: https://Trelis.Substack.com
➡ Trelis Resources and Support: https://Trelis.com/About

Affiliate Links (support the channel):
Vast AI https://cloud.vast.ai/?ref_id=98762
RunPod https://tinyurl.com/4b6ecbbn

*Video Resources*
Slides: https://docs.google.com/presentation/...
Oneclick RunPod / VastAI Templates: https://github.com/TrelisResearch/ins...
IDEFICS: https://huggingface.co/HuggingFaceM4/...
LLaVA: https://llava.hliu.cc/
Trelis Newsletter: Trelis.Substack.com

Chapters:
0:00 Finetuning Multimodal Models
0:16 Overview
1:30 LLaVA vs ChatGPT
4:53 Applications
5:37 Multimodal model architecture
9:05 Vision Encoder architecture
14:00 LLaVA 1.5 architecture
16:30 LLaVA 1.6 architecture
18:30 IDEFICS architecture
22:00 Data creation
24:11 Dataset creation
25:29 Finetuning
34:25 Inference and Evaluation
37:34 Data loading
40:00 LoRA setup
42:52 Recap so far
43.25 Evaluation pretraining
44:26 Training
45:40 Evaluation posttraining
46:45 Technical clarifications
50:29 Summary