목록Papers (46)
둔비의 공부공간
AWQ: Activation-aware Weight Quantization for on-device LLM compression and acceleration
https://arxiv.org/pdf/2306.00978https://github.com/mit-han-lab/llm-awq GitHub - mit-han-lab/llm-awq: [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Ac[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration - mit-han-lab/llm-awqgithub.com MLSys 2024 Best Paper Award AbstractActivation-aware Weig..
SINGLE TEACHER, MULTIPLE PERSPECTIVES: TEACHERKNOWLEDGE AUGMENTATION FOR ENHANCED KNOWLEDGE DISTILLATION
ICLR 2025 Accpeted https://openreview.net/forum?id=DmEHmZ89iB Single Teacher, Multiple Perspectives: Teacher Knowledge...Do diverse perspectives help students learn better? Multi-teacher knowledge distillation, which is a more effective technique than traditional single-teacher methods, supervises the student from...openreview.net 기존 KD와 다르게, Multi-teacher를 사용해서 더욱 다양한 관점을 student에게 전달하는 논문들이 ..
Dataset Quantization
https://arxiv.org/abs/2308.10524 Dataset QuantizationState-of-the-art deep neural networks are trained with large amounts (millions or even billions) of data. The expensive computation and memory costs make it difficult to train them on limited hardware resources, especially for recent popular large languagearxiv.org ICCV 2023 Accepted. 일단, 우리가 생각하는 model quantization을 기대하고 논문을 접하면 안된다.그냥 data를..
Variation-aware Vision Transformer Quantization
ViT Quantization을 빠르게 보려고 하는데, 공개되어있는 github가 있길래 읽어봤다. https://github.com/HuangOwen/VVTQ GitHub - HuangOwen/VVTQ: Official PyTorch implementation of paper "Variation-aware Vision Transformer Quantization"Official PyTorch implementation of paper "Variation-aware Vision Transformer Quantization" - HuangOwen/VVTQgithub.com Accpeted는 안된 것 같다. 논문의 저자들이 하고자하는 이야기는 다음과 같다. "ViT Quantization은 불안정하며..
Learned step size quantization
블로그에 자세히 정리한다고 생각하고 글을 쓰다 보니, 논문 리뷰가 아니라 번역에 가깝게 옮겨놓게 되면서[작성중] 카테고리에 비공개로 쓰다가 만 논문들이 너무 많아졌다.그냥 한번 읽고, 기억나는 내용을 중점으로 정리하는 방법으로 바꿔야겠다. https://arxiv.org/abs/1902.08153 Learned Step Size QuantizationDeep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaining high accuracy as..
DaFKD: Domain-aware Federated Knowledge Distillation
https://openaccess.thecvf.com/content/CVPR2023/html/Wang_DaFKD_Domain-Aware_Federated_Knowledge_Distillation_CVPR_2023_paper.html CVPR 2023 Open Access RepositoryDaFKD: Domain-Aware Federated Knowledge Distillation Haozhao Wang, Yichen Li, Wenchao Xu, Ruixuan Li, Yufeng Zhan, Zhigang Zeng; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 20412-2..