# Difference between revisions of "Functional regularisation for continual learning with gaussian processes"

(Created page with "== Presented by == Meixi Chen == Introduction == == Previous Work == == Motivation == == Model == == Conclusion == == Critiques == == References ==") |
(→Introduction) |
||

Line 2: | Line 2: | ||

Meixi Chen | Meixi Chen | ||

− | == Introduction == | + | == Introduction == |

+ | |||

+ | Continual Learning (CL) refers to the problem where different tasks are fed to model sequentially, such as training a natural language processing model on different languages over time. A major challenge in CL is model forgets how to solve earlier tasks. This paper proposed a new framework to regularize Continual Learning (CL) so that it doesn't forget previously learned tasks. This method, referred to as functional regularization for Continual Learning, leverages the Gaussian process to construct an approximate posterior belief over the underlying task-specific function. Then the posterior belief is utilized in optimization as a regularizer to prevent the model from completely deviating from the earlier tasks. The estimation of posterior functions is carried out under the framework of approximate Bayesian inference. | ||

== Previous Work == | == Previous Work == |

## Revision as of 17:29, 22 November 2020

## Contents

## Presented by

Meixi Chen

## Introduction

Continual Learning (CL) refers to the problem where different tasks are fed to model sequentially, such as training a natural language processing model on different languages over time. A major challenge in CL is model forgets how to solve earlier tasks. This paper proposed a new framework to regularize Continual Learning (CL) so that it doesn't forget previously learned tasks. This method, referred to as functional regularization for Continual Learning, leverages the Gaussian process to construct an approximate posterior belief over the underlying task-specific function. Then the posterior belief is utilized in optimization as a regularizer to prevent the model from completely deviating from the earlier tasks. The estimation of posterior functions is carried out under the framework of approximate Bayesian inference.