목록분류 전체보기 (45)
둔비의 공부공간
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
(2022 CVPR, META AI) (github) Abstract Pre-trained모델을 갖고, 다른 하이퍼 파라미터로 fine tuned된 모델의 weights를 averaging했을때, 정확도와 안정성이 향상되는 것을 확인했다. (https://arxiv.org/pdf/2110.12899.pdf, https://proceedings.neurips.cc/paper/2020/file/0607f4c705595b911a4f3e7a127b44e0-Paper.pdf) 둘중 논문에서 하나의 pretrained model로 hyper-params를 바꿔가면서 파라미터를 했을때 비슷한 loss convex를 갖는다고 했다. 그래서 그 증명을 토대로 위와 같은 model-soups가 가능했다고 함 - 기존 en..
Prune and Tune Ensembles: Low-Cost Ensemble Learning With Sparse Independent Subnetworks
(AAAI 2022) https://github.com/tjwhitaker/prune-and-tune-ensembles GitHub - tjwhitaker/prune-and-tune-ensembles Contribute to tjwhitaker/prune-and-tune-ensembles development by creating an account on GitHub. github.com Abstract Deeplearning의 ensemble은 효과적이고 좋지만, computational cost가 비싸다는 단점이 있다. 그래서, 이 논문에서는 빠르고 작은 cost로 scratch부터 다양한 모델을 학습할 필요가 없는 ensemble방법을 소개한다. 일단 single parent network를 하나 ..
Learning both Weights and Connections for Efficient Neural Networks
https://arxiv.org/abs/1506.02626 (Standford, Nvidia NIPS 2015) Learning both Weights and Connections for Efficient Neural Networks Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems. Also, conventional networks fix the architecture before training starts; as a result, training cannot improve the architecture. To arxiv.org ..