ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

From statwiki

Revision as of 19:05, 2 November 2020 by Mdadbin (talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Jump to navigation Jump to search

Presented by

Maziar Dadbin

Introduction

Motivation

Model details

Factorized embedding parameterization

Cross-layer parameter sharing

Inter-sentence coherence loss

Relationship between convexity and smoothness.

Removing dropout

Retrieved from "http://wiki.math.uwaterloo.ca/statwiki/index.php?title=ALBERT:_A_Lite_BERT_for_Self-supervised_Learning_of_Language_Representations&oldid=43209"