Share AI Models and Methods

Contrastive Learning with Nearest Neighbors for Text Clustering

Contrastive learning is a supervised method, which is a good way to improve the text clustering. For example, in sentence clustering, it canĀ  pull together similar sentences and pushes away distant ones in the embedding space. We can find an example in paper: New Intent Discovery with Pre-training and Contrastive Learning

What is the contrastive learning?

There are some kinds of contrastive learning methods. In this article, we will introduce one.

We will build a contrastive learning based on an adjacency matrix.

What is the contrastive learning

From this paper, we can find temperature parameter can be 0.07

As to contrastive learning, we should notice: each sample is same to important. However, contrastive learning will make the model poor.

For example, if you cluster words using contrastive learning, different word contributes different weight for sentence representation, you have to modify loss object.