Conditional Layer Normalization (CLN) is used in paper: Unified Named Entity Recognition as Word-Word Relation Classification. In this tutorial, we will introduce it.
Conditional Layer Normalization
It is used to a pair \(x_i, x_j\) and is defined as:
The representation \(V_{ij} of the word pair \((x_i, x_j)\) can be considered as a combination of the representation \(h_i\) of \(x_i\) and \(h_j\) of \(x_j\), where the combination should imply that \(x_j\) is conditioned on \(x_i\).