gensim.word2vec 训练词向量

一、训练词向量

# 导入必要的包
import jieba
import numpy as np
import pandas as pd
from gensim.models.word2vec import Word2Vec
import joblib
from sklearn.svm import SVC

# 读取两个类别的语料
pos = pd.read_csv(weather_pos.txt, encoding=UTF-8, header=None)
neg = pd.read_csv(weather_neg.txt, encoding=UTF-8, header=None)

# 进行分词处理
pos[words] = pos[0].apply(lambda x: jieba.lcut(x))
neg[words] = neg[0].apply(lambda x: jieba.lcut(x))

# 将正负语料进行合并成训练语料然后并打上标签，正语料打上标签1，负语料打上标签0
x = np.concatenate((pos[words], neg[words]))
y = np.concatenate((np.ones(len(pos)), np.zeros(len(neg))))

# 训练词向量128维
word2vec = Word2Vec(x, size=128, window=3, min_count=5, sg=1, hs=1, iter=10, workers=25)
word2vec.save(word2vec.model)

二、使用词向量

1. 加载词向量

word2vec_model = Word2Vec.load(word2vec.model)

2. 将所有词的词向量保存

word2vec_model .wv.save_word2vec_format(word2vec.txt, binary=False)

3. 获取某个词的词向量

word2vec_model .wv[温度]

4. 获取某个词的相似词

word2vec_model .most_similar(天气)