DNB and Typing
d4b 33% Sun 07 Apr 2019 04:24:36 PM CEST d4b 33% Sun 07 Apr 2019 04:26:35 PM CEST d4b 56% Sun 07 Apr 2019 04:28:28 PM CEST d4b 61% Sun 07 Apr 2019 04:30:24 PM CEST d4b 28% Sun 07 Apr 2019 04:32:21 PM CEST d4b 44% Sun 07 Apr 2019 04:34:27 PM CEST d4b 22% Sun 07 Apr 2019 04:36:19 PM CEST d4b 39% Sun 07 Apr 2019 04:38:14 PM CEST
“Wherever you are, make sure you’re there.” — Dan Sullivan
Classifying by parts of speech
nltk.download() downloads everything needed.
nltk.word_tokenize('aoethnsu') returns the tokens. From https://firstname.lastname@example.org/tokenization-and-parts-of-speech-pos-tagging-in-pythons-nltk-library-2d30f70af13b. For parts of speech it’s
The tokenizer for twitter works better for URLs (of course). Interestingly it sees URLs as NN. And - this is actually fascinating - smileys get tokenized differently!
EDIT: nltk.tokenize.casual might be just like the above, but better!
EDIT: I have a column with the POS of the tweets! How do I classify it with its varying length? How can I use the particular emojis as another feature?
POS + individual smileys might be enough for it to generalize! TODO test TODO: Maybe first do some much more basic feature engineering with capitalization and other features mentioned here:
Word Count of the documents – total number of words in the documents Character Count of the documents – total number of characters in the documents Average Word Density of the documents – average length of the words used in the documents Puncutation Count in the Complete Essay – total number of punctuation marks in the documents Upper Case Count in the Complete Essay – total number of upper count words in the documents Title Word Count in the Complete Essay – total number of proper case (title) words in the documents Frequency distribution of Part of Speech Tags: Noun Count Verb Count Adjective Count Adverb Count Pronoun Count
textminingonline.com has nice resources on topic which would be very interesting to skim through! Additionally flair is a very interesting library not to reinvent the wheel, even though reinventing the wheel would be the entire point of a bachelor’s thesis.