Blogger Sentiment Analysis of The MacBook Air: New Charts in the Works

Nathan Gilliatt recently posted about the “building blocks of social media analysis,” and he concluded that there are three main categories to consider: content sourcing, analytics, and the software application that provides end-user features, such as dashboards, reports, and alerts.

We’ve been focusing of late on the third category and trying out new avenues for presenting our data, so this jumped out at me.

The application software, on the other hand, is what clients see, and it provides many opportunities for evaluation. Is the user interface dated and clunky or sleek and current? How is content presented? What can users do with it? How easy is it to explore the data or create reports?

Regarding our ongoing analysis of the MacBook Air, Jason and I have fed several hundred blog posts into the system and have been playing with the output. We’re trying out new chart formats for the final report we’re building, and like this style a lot:

Green obviously indicates positive sentiment and red negative. This shows how the five major product attributes fare in terms of volume and relative sentiment. Note that features are the most discussed, and there is wide division on how bloggers feel. The number one negative here relates to the battery being fixed and locked into the case.

Aesthetics are a big deal too, but the appearance and feel of the computer are consistently a plus for Apple.

We’re building separate sub-charts for each of the circles displayed here, and will be putting them into our final report. Let me know if you want a copy. Email steve AT parnassusgroup DOT com.

Matt Dickman on the Challenges of Sentiment Analysis

Just ran across this post from Matt Dickman who is Vice President, Digital Marketing at Fleishman-Hillard. His blog “Techno/Marketer” covers a myriad of subjects related to marketing and social media.

His take appears to be identical to ours — that detecting sentiment via traditional NLP is a challenge at best.

One of the most important aspects of online conversations is the sentiment of what the author is saying. Are they positive about you, negative or apathetic? The difference is vitally important, but very hard to determine due to the complexity of language.

We recently spoke with a senior VP at a major PR firm who echoed this. Apparently they have not found a viable alternative to hiring humans. Their next move is to test our process.

Here he describes Twistori.

It’s very cool to watch the service extract the terms and after a few minutes you see how difficult it is to get sentiment right.

We’re arguing that we do get sentiment “right,” and are happy to run sample sets of data through our system for prospective clients. If you’ve got hand scored data, we’d love to show you how we compare.

We’re Now the “Numerati” — According to The Wall Street Journal (and Steven Baker)

The WSJ has just reviewed the book The Numerati By Stephen Baker and I have already ordered my copy. How could I resist when we’re mining the blogoisphere for sentiment and about to test our own home-grown splog detector? Check out this section of the review:

The Numerati are even mining the output of bloggers, those stream-of-consciousness online diarists and self-promoters. “What makes the blog world especially valuable to marketers,” Mr. Baker writes, is “its unfiltered immediacy.” What do consumers think of your new product? What desires are still not satisfied by products of this kind? You can commission a poll or wait for the sales figures to come in . . . or you can read the blogs. Better yet, you can hire Numerati to write programs that will read them for you, since there are now more than 20 million bloggers in the U.S. alone.

…But Adsense has set in motion an ugly arms race online as robot bloggers — clever computer programs — have generated hundreds of thousands of spam blogs, or “splogs.”

A splog, though unreadable, is seeded with words that will attract Google ads. A computer-user may be annoyed at finding himself staring at a screen full of gibberish but click on an ad anyway, allowing the robot blogger to harvest revenue. This sleight of hand has the Numerati hard at work getting their software to distinguish between a blog and a splog. Mr. Baker gives a helpful sketch of the math involved, each blog reduced to a vector in a space of several dozen dimensions.

How Useful is NLP for Sentiment Scoring? Is Brand Monitoring a Realistic Goal?

Matt O’Hern just posted about yet another brand monitoring service (this time from Vocus) who claims to track sentiment.

“Vocus spreads your name to the top bloggers and news sites, but it also offers Sentiment Analysis, a tool that tracks press, positive or negative, regarding your company — in real time.”

After reading the inside story of Monitor 110’s demise, and having developed our own sentiment tagging system, I immediately said to myself, “‘Real time’ huh? Looks like another system where one out of four items is scored wrong…”

Thankfully, the comments contained the real meat of this story — and validated my impressions. First Martin Edic of Techrigy pounced, and injected a dose of reality:

“Be very careful in writing about claims of sentiment analysis. Unless (and this is a big unless) they are using actual humans to read each instance of apparently negative/positive commentary the sentiment is only a very rough indicator. No algorithm, NLP or other machine reading analysis can be very accurate with today’s technology. Machines are simply not good at context semantics, sarcasm, irony, etc.
When I talk to people about our sentiment analysis I’m very careful to make this clear. It is just an indicator until you drill down on a granular level to make the determination yourself…One more thing: my note above and our experience is based on performing sentiment analysis on over 500 million social media search results of all types over the past six months.”

Then no less than Kye Strance, the Director of Product Management at Vocus stepped in:

“NLP and automated sentiment analysis is certainly a useful tool for those looking at trending and real time analysis.”

(emphasis mine)

“We found that people who are looking for up to date analysis in real time, were comfortable with NLP technology that provided 80% accuracy. For those that time is critical and not able to wait until weeks or months end for human analysis, being able to see a trend throughout the day or over any period of time, this is a great approach.

I completely agree with Martin in that if you are looking to send an article straight to your CEO that has been analyzed by an engine thinking it is a positive article, you would certainly want to review it before hand. But I have also seen articles reviewed by humans that should have also been reviewed before being sent as well to save embarrassment.

While it is not able to detect sarcasm and irony, the one thing it can do is be consistent in analysis. A sentiment engine does not show up for work in a bad mood or get tired through out the day, so if you are looking for high level analysis that is consistent, NLP analysis is a great tool to do the major leg work for you.”

My question is what items do you NOT have to review before the CEO reads it? A negative could be neutral or positive, a neutral could be negative or positive etc. etc.

Sounds to me like traditional NLP is what you need for general trends (financial/investing,) but is not that useful for brand monitoring. Considering what happened to Monitor 110, it may be not even that good for seeking alpha…

This is why we think we have a winner with our non-traditional automated text analysis service that:

* Provides human-level (90 percent plus accuracy) scoring.
* Can detect context semantics, sarcasm, and irony.
* Provides low/zero latency (immediate) results 24 hours a day, 7 days a week.
* Never shows up for work in a bad mood or gets tired through out the day.
* Doesn’t require that a VC achieves hurdle rate for their money.