ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

From statwiki
Revision as of 18:08, 2 November 2020 by Mdadbin (talk | contribs)
Jump to: navigation, search

Presented by

Maziar Dadbin

Introduction

Motivation

Model details

Factorized embedding parameterization

Cross-layer parameter sharing

Inter-sentence coherence loss

SOPvsNSP.png

Removing dropout