Skip to main content

Software Review Site TrustRadius Has A New Way to Treat Reviews Obtained Through Vendors


Online user reviews are the most powerful marketing technique for influencing purchase decisions. But do they accurately represent the views of most users?

Today, business software review platform TrustRadius is announcing a new way — called trScore — to handle the bias introduced in reviews by users obtained through the vendor of the reviewed software product. The site says more than two million software buyers visit each year to check out its product reviews.

To understand trScore, let’s first look at TrustRadius’ approach.

The site says it authenticates all users through their LinkedIn profiles. It also requires users to answer eight to ten questions about the product, in order to weed out users having no familiarity. Additionally, a staff person reads every review before it is posted, and the site says about three percent of reviews are rejected for not meeting guidelines.

As for the reviews themselves, TrustRadius puts them into two main buckets: independently-sourced reviews and vendor-sourced reviews. (Consider “reviews” to also mean “ratings” in most cases.)

First, independently-sourced reviews:

These include the reviews created by the thousands that register on the site to see the assessments, and who agree to write reviews. Additionally, the site will solicit reviewers through such places as LinkedIn or Quora, based on their interests.

There are also users obtained by TrustRadius from customer lists provided by vendors. But TrustRadius considers these to be independently-sourced, because it only accepts either the full customer list or what it calls “a representative sample,” which it determines by random sampling of a full customer list. TrustRadius then contacts the users, and says it requests unbiased reviews.

Sometimes, TrustRadius is paid by vendors to solicit more reviews, or to “scale out reviews,” so the vendor can use them for marketing. This might mean, for instance, turning 10 reviews into 50. TrustRadius says it always asks for “a fair customer sample” of users and for objective assessments.

These reviews are considered independently-sourced, and the site does not note that it has been paid by the vendor to get them. TrustRadius points out that overall product scores are not dependent on number of reviews, but on their average assessments.

Vendor-Sourced Reviews

Now, vendor-sourced reviews:

Vendor-sourced reviews are ones where the vendor directly approaches its customers, and asks them to write reviews. They’re also supposed to ask them to be objective.

TrustRadius says it counts reviews from vendors that participate in its free review program on the same footing as ones the site has been paid to get.

The problem that TrustRadius is trying to correct, CEO Vinay Bhagat told me, is that vendor-sourced users tend to generate more positive ratings and reviews than users obtained elsewhere. Unsurprisingly, the distribution of highly positive scores is greater for vendor-solicited reviews than for independently-sourced ones.

That selection bias, vp of marketing Bertrand Hazard said via email, has “escalated in the last 12-18 months,” because of a greater emphasis on reviews by vendors and a greater ability for vendors to identify product advocates among customers. The new trScore, he said, is the “action/response we’ve taken.”

Here are two graphs of the score distribution. The first, for vendor-sourced ratings and reviews, shows the number of reviews/ratings (0 to 3500) versus ratings of 1-10, with 10 being the highest:

Distribution of Ratings and Reviews Sourced by Vendors (n=2,733)

And this is the distribution for independently-sourced reviews:

Distribution of Ratings and Reviews Sourced Independently by TrustRadius (n=15,640)The new trScore weights the averages of vendor-solicited reviews, so as to adjust for the more positive bias. More recent reviews and ratings are given more weight, reviews count more than ratings, and independently-sourced reviews count more than vendor-sourced ones.

The main point, Bhagat said, is to get the distribution of high-low scores for vendor-sourced reviews to more closely resemble the distribution of independently-sourced ones.

Since it is correcting for this positive tilt among vendor-supplied reviews, trScore has brought down overall ratings a bit. Here’s the overall rating of social software products before trScore:

  • Agora Pulse [8.9]
  • Expion [8.8]
  • ViralHeat [8.7]
  • Sprinklr [8.1]
  • Shoutlet [8.1]

And after:

  • Expion [8.7]
  • Sprinklr [8.3]
  • ViralHeat [8.1]
  • Agora Pulse [8.1]
  • Shoutlet [7.5]

TrustRadius says that it is only software review site that is making an effort to compensate for vendor-introduced bias, and noted that its leading competitor, G2 Crowd, does not do this.

via Marketing Land


Popular posts from this blog

How to Get SMS Alerts for Gmail via Twitter

How do you get SMS notifications on your mobile phone for important emails in your Gmail? Google doesn’t support text notifications for their email service but Twitter does. If we can figure out a way to connect our Twitter and Gmail accounts, the Gmail notifications can arrive as text on our mobile via Twitter. Let me explain:Twitter allows you to follow any @user via a simple SMS. They provide short codes for all countries (see list) and if you text FOLLOW to this shortcode following by the  username, any tweets from that user will arrive in your phone as text notifications. For instance, if you are in the US, you can tweet FOLLOW labnol to 40404 to get my tweets as text messages. Similarly, users in India can text FOLLOW labnol to 9248948837 to get the tweets via SMS.The short code service of Twitter can act as a Gmail SMS notifier. You create a new Twitter account, set the privacy to private and this account will send a tweet when you get a new email in Gmail. Follow this account …

Another SEO tool drops the word “SEO”

This guest post is by Majestic’s Marketing Director, Dixon Jones, who explains the reasons for their recent name change.
Majestic, the link intelligence database that many SEOs have come to use on a daily basis, has dropped the “SEO” from it’s brand and from its domain name, to become Since most people won’t have used Google’s site migration tool before, here’s what it looks like once you press the “go” button:

In actual fact – there’s a minor bug in the tool. The address change is to the https version of (which GWT makes us register as a separate site) but that message incorrectly omits that. Fortunately, elsewhere in GWT its clear the omission is on Google’s side, not a typo from the SEO. It is most likely that the migration tool was developed before the need for Google to have separate verification codes for http and https versions of the site.
The hidden costs of a name change
There were a few “nay sayers” on Twitter upset that Majestic might be deserting it…

6 types of negative SEO to watch out for

The threat of negative SEO is remote but daunting. How easy is it to for a competitor to ruin your rankings, and how do you protect your site? But before we start, let’s make sure we’re clear on what negative SEO is, and what it definitely isn’t.Negative SEO is a set of activities aimed at lowering a competitor’s rankings in search results. These activities are more often off-page (e.g., building unnatural links to the site or scraping and reposting its content); but in some cases, they may also involve hacking the site and modifying its content.Negative SEO isn’t the most likely explanation for a sudden ranking drop. Before you decide someone may be deliberately hurting your rankings, factor out the more common reasons for ranking drops. You’ll find a comprehensive list here.Negative off-page SEOThis kind of negative SEO targets the site without internally interfering with it. Here are the most common shapes negative off-page SEO can take.Link farmsOne or two spammy links likely won’…