Skip to main content

Software Review Site TrustRadius Has A New Way to Treat Reviews Obtained Through Vendors


Online user reviews are the most powerful marketing technique for influencing purchase decisions. But do they accurately represent the views of most users?

Today, business software review platform TrustRadius is announcing a new way — called trScore — to handle the bias introduced in reviews by users obtained through the vendor of the reviewed software product. The site says more than two million software buyers visit each year to check out its product reviews.

To understand trScore, let’s first look at TrustRadius’ approach.

The site says it authenticates all users through their LinkedIn profiles. It also requires users to answer eight to ten questions about the product, in order to weed out users having no familiarity. Additionally, a staff person reads every review before it is posted, and the site says about three percent of reviews are rejected for not meeting guidelines.

As for the reviews themselves, TrustRadius puts them into two main buckets: independently-sourced reviews and vendor-sourced reviews. (Consider “reviews” to also mean “ratings” in most cases.)

First, independently-sourced reviews:

These include the reviews created by the thousands that register on the site to see the assessments, and who agree to write reviews. Additionally, the site will solicit reviewers through such places as LinkedIn or Quora, based on their interests.

There are also users obtained by TrustRadius from customer lists provided by vendors. But TrustRadius considers these to be independently-sourced, because it only accepts either the full customer list or what it calls “a representative sample,” which it determines by random sampling of a full customer list. TrustRadius then contacts the users, and says it requests unbiased reviews.

Sometimes, TrustRadius is paid by vendors to solicit more reviews, or to “scale out reviews,” so the vendor can use them for marketing. This might mean, for instance, turning 10 reviews into 50. TrustRadius says it always asks for “a fair customer sample” of users and for objective assessments.

These reviews are considered independently-sourced, and the site does not note that it has been paid by the vendor to get them. TrustRadius points out that overall product scores are not dependent on number of reviews, but on their average assessments.

Vendor-Sourced Reviews

Now, vendor-sourced reviews:

Vendor-sourced reviews are ones where the vendor directly approaches its customers, and asks them to write reviews. They’re also supposed to ask them to be objective.

TrustRadius says it counts reviews from vendors that participate in its free review program on the same footing as ones the site has been paid to get.

The problem that TrustRadius is trying to correct, CEO Vinay Bhagat told me, is that vendor-sourced users tend to generate more positive ratings and reviews than users obtained elsewhere. Unsurprisingly, the distribution of highly positive scores is greater for vendor-solicited reviews than for independently-sourced ones.

That selection bias, vp of marketing Bertrand Hazard said via email, has “escalated in the last 12-18 months,” because of a greater emphasis on reviews by vendors and a greater ability for vendors to identify product advocates among customers. The new trScore, he said, is the “action/response we’ve taken.”

Here are two graphs of the score distribution. The first, for vendor-sourced ratings and reviews, shows the number of reviews/ratings (0 to 3500) versus ratings of 1-10, with 10 being the highest:

Distribution of Ratings and Reviews Sourced by Vendors (n=2,733)

And this is the distribution for independently-sourced reviews:

Distribution of Ratings and Reviews Sourced Independently by TrustRadius (n=15,640)The new trScore weights the averages of vendor-solicited reviews, so as to adjust for the more positive bias. More recent reviews and ratings are given more weight, reviews count more than ratings, and independently-sourced reviews count more than vendor-sourced ones.

The main point, Bhagat said, is to get the distribution of high-low scores for vendor-sourced reviews to more closely resemble the distribution of independently-sourced ones.

Since it is correcting for this positive tilt among vendor-supplied reviews, trScore has brought down overall ratings a bit. Here’s the overall rating of social software products before trScore:

  • Agora Pulse [8.9]
  • Expion [8.8]
  • ViralHeat [8.7]
  • Sprinklr [8.1]
  • Shoutlet [8.1]

And after:

  • Expion [8.7]
  • Sprinklr [8.3]
  • ViralHeat [8.1]
  • Agora Pulse [8.1]
  • Shoutlet [7.5]

TrustRadius says that it is only software review site that is making an effort to compensate for vendor-introduced bias, and noted that its leading competitor, G2 Crowd, does not do this.

via Marketing Land


Popular posts from this blog

How to Get SMS Alerts for Gmail via Twitter

How do you get SMS notifications on your mobile phone for important emails in your Gmail? Google doesn’t support text notifications for their email service but Twitter does. If we can figure out a way to connect our Twitter and Gmail accounts, the Gmail notifications can arrive as text on our mobile via Twitter. Let me explain:Twitter allows you to follow any @user via a simple SMS. They provide short codes for all countries (see list) and if you text FOLLOW to this shortcode following by the  username, any tweets from that user will arrive in your phone as text notifications. For instance, if you are in the US, you can tweet FOLLOW labnol to 40404 to get my tweets as text messages. Similarly, users in India can text FOLLOW labnol to 9248948837 to get the tweets via SMS.The short code service of Twitter can act as a Gmail SMS notifier. You create a new Twitter account, set the privacy to private and this account will send a tweet when you get a new email in Gmail. Follow this account …

Instagram Story links get 15-25% swipe-through rates for brands, publishers

Instagram may arrived late as a traffic source for brands and publishers, but it’s already showing early signs of success, driving new visitors to their sites and even outperforming its parent company, Facebook.For years brands, publishers and other have tried to push people from the Facebook-owned photo-and-video-sharing app to their sites. Outside of ads and excepting a recent test with some retailers, Instagram didn’t offer much help to companies looking to use it to drive traffic. So they had to find workarounds. They put links in their Instagram bios. They scrawled short-code URLs onto their pictures. And they typed out links in their captions.Then last month Instagram finally introduced an official alternative to these hacky workarounds: the ability for verified profiles to insert links in their Instagram Stories.Almost a month after the launch, 15% to 25% of the people who see a link in an Instagram Story are swiping on it, according to a handful of brands and publishers that h…

Five great tools to improve PPC ads

Every digital marketer wants to reach the top position on the search engine results. However, if you’ve recently launched a new website or your niche is saturated, starting with paid search ads sounds like a good idea.Strategically created PPC campaigns can drive leads, sales or sign-ups to your websites. You know what? In fact, businesses earn an average of $8 for every dollar they spend on Google Ads.Optimizing PPC campaigns is not easy, but it’s very powerful if you do it properly. Just like SEO, it is essential to conduct extensive keyword research, optimize ad copy, and design high-converting landing pages.Fortunately, there are a lot of effective PPC tools that will help you analyze your competitors’ PPC strategies, figure out tricks in their campaigns, and improve your PPC campaigns.If you are ready to take an evolutionary leap in your PPC advertising, take a look at my list of five amazing tools to save you time, give you crucial insights, and raise money for your business.Fiv…