In deep learning, we may use multiple features to inprove the final performance in one model. For example, in sentiment analysis, you may use sentiment lexicon feature and sentence context feature.
How to incorporate multiple features in one model?
If you plan to use two kinds of features, how to use them?
Paper: Lexicon Integrated CNN Models with Attention for Sentiment Analysis give us 3 methods to incorporate them.
Method 1: Naive concatenation
It looks like:
This method is a common way, many models use this method to incorporate multiple features.
For example, one feature is 1*200, the other is 1*200, then we will get a new feature with 1*400. In tensorflow, it is:
n = tf.concat([f1, f2], axis = -1)
Method 2: Multichannel concatenation
This method concatenates multiple features on channel dimension.
It looks like:
However, this method we should notice features have the same dimension, otherwise, we need pad them with 0 to the same.
In tensorflow, we can view it as:
n = tf.concat([f1, f2], axis = 1)
Method 3: Separate modeling
We do not incorporate them when inputing, we incorporate them in the final layer.
Here is an example:
Which method has the best performance?
This paper used 3 methods above to make an experiment, we can find method 3 is the best.
Here is the comparative result:
