AdaptOr: Objective-Centric Adaptation Framework for Language Models

Varování

Publikace nespadá pod Ekonomicko-správní fakultu, ale pod Fakultu informatiky. Oficiální stránka publikace je na webu muni.cz.
Autoři

ŠTEFÁNIK Michal NOVOTNÝ Vít GROVEROVÁ Nikola SOJKA Petr

Rok publikování 2022
Druh Článek ve sborníku
Konference Proceedings of the 60th Conference of Association of Computational Linguistics, ACL 2022
Fakulta / Pracoviště MU

Fakulta informatiky

Citace
www
Doi http://dx.doi.org/10.18653/v1/2022.acl-demo.26
Klíčová slova Adaptor library; domain adaptation; similarity search; vector space; embeddings
Popis Progress in natural language processing research is catalyzed by the possibilities given by the widespread software frameworks. This paper introduces the Adaptor library that transposes the traditional model-centric approach composed of pre-training + fine-tuning steps to the objective-centric approach, composing the training process by applications of selected objectives. We survey research directions that can benefit from enhanced objective-centric experimentation in multitask training, custom objectives development, dynamic training curricula, or domain adaptation. Adaptor aims to ease the reproducibility of these research directions in practice. Finally, we demonstrate the practical applicability of Adaptor in selected unsupervised domain adaptation scenarios.
Související projekty:

Používáte starou verzi internetového prohlížeče. Doporučujeme aktualizovat Váš prohlížeč na nejnovější verzi.