[关闭]
@HaomingJiang 2016-08-12T14:29:41.000000Z 字数 5038 阅读 5820

Doc2Vec Sentiment Analysis Tutorial on the Airline Tweets

Tweets Textmining Doc2Vec



This is a tutorial about the procedue of using doc2vec to do sentiment analysis on airline tweets. Although the result is not very beautiful, by this tutorial you still can learn the procedue of sentiment analysis via Gensim Doc2Vec.

The main procedue is:
1. Text Preprocessing
2. Building Doc2Vec Model
3. Building Sentiment Classifier

Data Preparation

The airline tweets data can be collected from Kaggle. It is a small corpus. In order to get a great document vector, a larger data set is needed. A larger data set, which contains about 55,000 airline tweets can be downloaded here. In addition, I collect about 55,000 unlabeled data from twitter API. It can be downloaded here

PS: Here I renamed the labeled data as "Tweets_NAg.csv" for simplisity.

Text Preprocessing

A good introduction about how to preprocess tweet data can be found here. The main purpose of this step is to tokenize the meaningful word unit from original tweet. Based on the provided code of sentiment tokenizer, I did a slight modification including nomalizing the URLs and Usernames. The file you need is ReadACleanT.py

Put the files in the same fold. Let's load and clean the data.

  1. from ReadACleanT import clean_tweet
  2. import pandas as pd
  3. import random
  4. df = pd.read_csv('Tweets_NAg.csv')
  5. df = df[[u'airline_sentiment',u'text']]
  6. df.loc[:,'text'] = df.loc[:,'text'].map(clean_tweet)
  7. udf = pd.read_csv('../data/Tweets_Unlabeled.csv')
  8. udf = udf[[u'text']]
  9. udf.loc[:,'text'] = udf.loc[:,'text'].map(clean_tweet)

Constructing Doc2Vec Model

First we need to turn our data into TaggedDocument form, which is the input form of Doc2Vec.

  1. from gensim.models.doc2vec import TaggedDocument,Doc2Vec
  2. TotalNum = df.size/2
  3. TotalNum_Unlabed = udf.size
  4. TestNum =3000
  5. TrainNum=TotalNum-TestNum
  6. documents = [TaggedDocument(list(df.loc[i,'text']),[i]) for i in range(0,TotalNum)]
  7. documents_unlabeled = [TaggedDocument(list(udf.loc[i,'text']),[i+TotalNum]) for i in range(0,TotalNum_Unlabed)]
  8. documents_all = documents+documents_unlabeled
  9. Doc2VecTrainID = range(0,TotalNum+TotalNum_Unlabed)
  10. random.shuffle(Doc2VecTrainID)
  11. trainDoc = [documents_all[id] for id in Doc2VecTrainID]
  12. Labels = df.loc[:,'airline_sentiment']

After that we can construct the Doc2Vec model. According to the paper, I train two seprate models (i.e. DM and DBOW). Later I will concatenate two vectors into one. For the model parameter setting, you can refer to the document.

  1. import multiprocessing
  2. cores = multiprocessing.cpu_count()
  3. model_DM = Doc2Vec(size=400, window=8, min_count=1, sample=1e-4, negative=5, workers=cores, dm=1, dm_concat=1 )
  4. model_DBOW = Doc2Vec(size=400, window=8, min_count=1, sample=1e-4, negative=5, workers=cores, dm=0)

Next, we are going to build vocabulary for models.

  1. model_DM.build_vocab(trainDoc)
  2. model_DBOW.build_vocab(trainDoc)

Now, we are ready to train Doc2Vec model. Here I use both the testing data and training data to get the vector. It seems violate the normal process. However, this step is unsupervised, we don't use the true label information of testing set.
It will take a lot of time, it's time to have a break.

  1. for it in range(0,10):
  2. random.shuffle(Doc2VecTrainID)
  3. trainDoc = [documents_all[id] for id in Doc2VecTrainID]
  4. model_DM.train(trainDoc)
  5. model_DBOW.train(trainDoc)

Building Classifier

Here I simply use softmax to build the classifier. Neural network and SVM may achieve higher performance. Try them if you want.

  1. from sklearn.linear_model import LogisticRegression
  2. from sklearn.metrics import confusion_matrix
  3. import statsmodels.api as sm
  4. random.seed(1212)
  5. newindex = random.sample(range(0,TotalNum),TotalNum)
  6. testID = newindex[-TestNum:]
  7. trainID = newindex[:-TestNum]
  8. train_targets, train_regressors = zip(*[(Labels[id], list(model_DM.docvecs[id])+list(model_DBOW.docvecs[id])) for id in trainID])
  9. train_regressors = sm.add_constant(train_regressors)
  10. predictor = LogisticRegression(multi_class='multinomial',solver='lbfgs')
  11. predictor.fit(train_regressors,train_targets)

Let's use G-mean (Geometric mean of recalls) and accuracy to evaluate the model.

  1. accus=[]
  2. Gmeans=[]
  3. test_regressors = [list(model_DM.docvecs[id])+list(model_DBOW.docvecs[id]) for id in testID]
  4. test_regressors = sm.add_constant(test_regressors)
  5. test_predictions = predictor.predict(test_regressors)
  6. for i in range(0,TestNum):
  7. if test_predictions[i]==df.loc[testID[i],u'airline_sentiment']:
  8. accu=accu+1
  9. accus=accus+[1.0*accu/TestNum]
  10. confusionM = confusion_matrix(test_predictions,(df.loc[testID,u'airline_sentiment']))
  11. Gmeans=Gmeans+[pow(((1.0*confusionM[0,0]/(confusionM[1,0]+confusionM[2,0]+confusionM[0,0]))*(1.0*confusionM[1,1]/(confusionM[1,1]+confusionM[2,1]+confusionM[0,1]))*(1.0*confusionM[2,2]/(confusionM[1,2]+confusionM[2,2]+confusionM[0,2]))), 1.0/3)]
  12. train_predictions = predictor.predict(train_regressors)
  13. accu=0
  14. for i in range(0,len(train_targets)):
  15. if train_predictions[i]==train_targets[i]:
  16. accu=accu+1
  17. accus=accus+[1.0*accu/len(train_targets)]
  18. confusionM = confusion_matrix(train_predictions,train_targets)
  19. Gmeans=Gmeans+[pow(((1.0*confusionM[0,0]/(confusionM[1,0]+confusionM[2,0]+confusionM[0,0]))*(1.0*confusionM[1,1]/(confusionM[1,1]+confusionM[2,1]+confusionM[0,1]))*(1.0*confusionM[2,2]/(confusionM[1,2]+confusionM[2,2]+confusionM[0,2]))), 1.0/3)]

In my machine, the performance is

data G-Mean Accuracy
testing set 0.6225 0.7687
training set 0.6297 0.7755

You can set different random seed for multiple runs or use cross validation to evaluate the model.

添加新批注
在作者公开此批注前,只有你和作者可见。
回复批注