How to do sentiment analysis

Sentiment analysis is a form of natural language processing (NLP) to determine whether an item of text expresses a positive, negative, or neutral attitude. It is widely used to understand customer feedback, social‑media posts and product reviews at scale. It can also be useful for identifying problem areas for a business and for tracking how sentiments change over time.

A simple way to do sentiment analysis is to assign positive and negative scores to keywords and then sum these scores, to determine the overall sentiment.

For example, score each occurence of ‘amazing’ as +5, each occurence of ‘angry’ as -4, and so on.

This simple approach does not handle all cases (e.g. sarcasm) but it can still yield useful insights and it scales well to large datasets.

You can do sentiment scoring in Easy Data Transform using a Lookup transform. You need to make sure you only match whole words, not parts of words. For example, you only want to match ‘great’ against ‘great’, not against other words containing ‘great’ (such as ‘greatly’ or ‘greatest’). You can do this in Easy Data Transform by matching against the regular expression \bgreat\b, where \b matches a word boundary.

You can easily modify the sentiment scoring dataset to add the regular expression boundary markers to the start and end of keywords, using a Replace transform.

And then Lookup the score where the value in the top dataset contains a regular expression in the bottom dataset.

Finally you need to sum the list of sentiment values using a List transform:

The whole workflow looks like this:

More modern sentiment scoring datasets often include emojis. Easy Data Transform can handle these, just like any text character. You probably don’t need to worry about whole word matching for emojis.

See the video above for more details.

Easy Data Transform can also compute NGram frequency in natural language.

As well as being a useful tool for sentiment sentiment analysis, Easy Data Transform can help with cleaning, filtering, blending, re-shaping and analysing your data. All without coding. Easy Data Transform can easily handle datasets with millions of rows and thousands of columns. And your data never has to leave your computer. Download the free trial for Windows or Mac and try it yourself.