Conditional Layer Normalization can allow us to normalize a representation based on different targets or features.
An Introduction to Conditional Layer Normalization
In this tutorial, we will use an example to show you how to use it.
In paper: Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization. Conditional Layer Normalization is used to transfer one content representation to different style.
Here we should notice parameters \(\gamma^s\) and \(\beta^s\) is learned, which are not transfered by a MLP.
Conditional Layer Normalization is also can be viewed as a feature fusion method. As to article:
An Introduction to Topic Attention in Deep Learning
We also can implement a conditional layer normalization based on different topic representations before computing topic attention.