一.什么是文本相似度?
需要解决文本相似度必须确定两段文本的词汇相似(lexical similarity)
以及 语义相似(semantic similarity)
问题
具体文献参考: Text Similarities : Estimate the degree of similarity between two texts
二.代码实现
# -*- coding: utf-8 -*-
# @Time : 2021/04/15 09:01
# @Author :
from semantic_text_similarity.models import WebBertSimilarity
# from semantic_text_similarity.models import ClinicalBertSimilarity
web_model = WebBertSimilarity(device='cpu', batch_size=10) # defaults to GPU prediction
# clinical_model = ClinicalBertSimilarity(device='cuda', batch_size=10) #defaults to GPU prediction
# 单词,词组
web_model.predict([("furniture", "decoration design")])
# 句子
# print(web_model.predict([("She won an olympic gold medal","The women is an olympic champion")]))
参考:https://pypi.org/project/semantic-text-similarity/