Difference between revisions of "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"
From statwiki
| Line 14: | Line 14: | ||
===Inter-sentence coherence loss=== | ===Inter-sentence coherence loss=== | ||
| − | [[File: | + | [[File:ConvexSmooth.PNG|frame|Relationship between convexity and smoothness.]] |
===Removing dropout=== | ===Removing dropout=== | ||
Revision as of 19:05, 2 November 2020
Contents
Presented by
Maziar Dadbin
Introduction
Motivation
Model details
Factorized embedding parameterization
Cross-layer parameter sharing
Inter-sentence coherence loss
Relationship between convexity and smoothness.
