둔비의 공부공간

https://arxiv.org/pdf/2306.00978https://github.com/mit-han-lab/llm-awq GitHub - mit-han-lab/llm-awq: [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Ac[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration - mit-han-lab/llm-awqgithub.com MLSys 2024 Best Paper Award AbstractActivation-aware Weig..

ICLR 2025 Accpeted https://openreview.net/forum?id=DmEHmZ89iB Single Teacher, Multiple Perspectives: Teacher Knowledge...Do diverse perspectives help students learn better? Multi-teacher knowledge distillation, which is a more effective technique than traditional single-teacher methods, supervises the student from...openreview.net   기존 KD와 다르게, Multi-teacher를 사용해서 더욱 다양한 관점을 student에게 전달하는 논문들이 ..

https://arxiv.org/abs/2308.10524 Dataset QuantizationState-of-the-art deep neural networks are trained with large amounts (millions or even billions) of data. The expensive computation and memory costs make it difficult to train them on limited hardware resources, especially for recent popular large languagearxiv.org ICCV 2023 Accepted.  일단, 우리가 생각하는 model quantization을 기대하고 논문을 접하면 안된다.그냥 data를..

ViT Quantization을 빠르게 보려고 하는데, 공개되어있는 github가 있길래 읽어봤다. https://github.com/HuangOwen/VVTQ GitHub - HuangOwen/VVTQ: Official PyTorch implementation of paper "Variation-aware Vision Transformer Quantization"Official PyTorch implementation of paper "Variation-aware Vision Transformer Quantization" - HuangOwen/VVTQgithub.com Accpeted는 안된 것 같다.     논문의 저자들이 하고자하는 이야기는 다음과 같다. "ViT Quantization은 불안정하며..