목록Papers/Ensemble (2)
둔비의 공부공간
Training Independent Subnetworks For Robust Prediction
https://arxiv.org/abs/2010.06610 Training independent subnetworks for robust predictionRecent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network. However, these methods still require multiple forward paarxiv.org (ICLR 2021) Cambridge, Google, Standford Abs..
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
(2022 CVPR, META AI) (github) Abstract Pre-trained모델을 갖고, 다른 하이퍼 파라미터로 fine tuned된 모델의 weights를 averaging했을때, 정확도와 안정성이 향상되는 것을 확인했다. (https://arxiv.org/pdf/2110.12899.pdf, https://proceedings.neurips.cc/paper/2020/file/0607f4c705595b911a4f3e7a127b44e0-Paper.pdf) 둘중 논문에서 하나의 pretrained model로 hyper-params를 바꿔가면서 파라미터를 했을때 비슷한 loss convex를 갖는다고 했다. 그래서 그 증명을 토대로 위와 같은 model-soups가 가능했다고 함 - 기존 en..