Not that OT: Reviewing products

[makoti]

I’m wondering what you guys expect in the way of neutrality from sites that do product reviews.
I have a friend who reviews gear for a fairly well known site. He gets the gear for nothing, reviews it, and then…keeps it for personal use. Doesn’t send it back, doesn’t pay for the item to keep it. I'm of the opinion that this, whether mindfully or not, taints the review. I don’t think its possible to be impartial KNOWING they gave you something for free. To me, this isn’t ethical.
It creates a conflict of interest & the review is little more than an infomercial or fake Amazon review. It'd be better, but not perfect, if in the review he (and I assume now all the reviewers on that site) fully disclosed that they will be keeping the item for personal use afterwards, but I never see it.
Maybe it’s because I was raised on Consumer Reports & them only reviewing things they actually paid for, but I can't help but think this is wrong. I can't think of one reputable & respected site that does this. CU still doesn't, and as far as I know, DCRainmaker & GCN both don't. Car & Driver isn’t getting all those cars to keep forever.
What do you think? In this age of everyone with a computer & the internet being an expert on something, is at least the appearance of impartiality no longer important? I know bike mags & manufactures are way too cozy, but I believe even their reviews get the product returned after reviews.

[FlashUNC]

Bruh if you think GCN isn't getting paid placement for some of those reviews, not least of which because the video is flagged by them as "paid placement" in a lot of instances.

[MattTuck]

Just assume that every product review you read or watch is paid for (either directly or indirectly) by the manufacturer, unless it is a review by DC Rainmaker.

[yinzerniner]

Just assume that every product review you read or watch is paid for (either directly or indirectly) by the manufacturer, unless it is a review by DC Rainmaker.

I find the CT reviews pretty unbiased, and they do a good job of perk disclosure.

HOWEVER, if any reviewer gets to keep the product for personal use after reviewing it I think it needs to be disclosed. If they get to keep the item and don't disclose the fact in their reviews it doesn't necessarily mean they're impartial but it creates questions and doubts about their impartiality, which is often worse (ie coverup worse than the crime).

I always thought it was journalistic norms to disclose any benefits and/or connections to the subject that the author might have. Then again, with the advent of the instas, lifestyle / wellness influencers and non - sponsored "sponsored posts" I guess there's no getting the toothpaste back into the tube.

[makoti]

Bruh if you think GCN isn't getting paid placement for some of those reviews, not least of which because the video is flagged by them as "paid placement" in a lot of instances.

And that's ok. It is like reading an article about someone & reading "Full disclosure - Bob is my boss".

[Bentley]

While he does mostly stuff related to triathalon's I find his reviews fairly unbaised. He does not accept gear from the companies that he reviews, he tells you what he uses and I think he does an honest job of reviewing the capabilities/limitations of gear out there. Not sure that everyone out there is as honest

My opinion

Ray

[martl]

Just assume that every product review you read or watch is paid for (either directly or indirectly) by the manufacturer, unless it is a review by DC Rainmaker.
or AvE (https://www.youtube.com/user/arduinoversusevil) or John Cadogan (https://www.youtube.com/user/AutoExpertTV)

[MattTuck]

I find the CT reviews pretty unbiased, and they do a good job of perk disclosure.



Taken today, on the launch of the Topstone carbon. Their review and then two advertisements which I circled. Even if you aren't taking gear or getting other perks, these publications are making money from advertising, and thus are beholden to advertisers for their revenues.

So, the reviews may be fair and accurate, but there will always be that implicit pressure because of the business model. And that is fine, it's only bikes... I don't think we need recusals and conflict of interest disclosures to be a part of reading about a new bike.

On the other hand, if you're really looking for an unbiased opinion, you need to find people that are truly independent, and don't have skin in the game.

[yinzerniner]

Taken today, on the launch of the Topstone carbon. Their review and then two advertisements which I circled. Even if you aren't taking gear or getting other perks, these publications are making money from advertising, and thus are beholden to advertisers for their revenues.

So, the reviews may be fair and accurate, but there will always be that implicit pressure because of the business model. And that is fine, it's only bikes... I don't think we need recusals and conflict of interest disclosures to be a part of reading about a new bike.

On the other hand, if you're really looking for an unbiased opinion, you need to find people that are truly independent, and don't have skin in the game.

We might have different opinions on this but I believe that someone can deliver an unbiased opinion even if they received material benefits from the entity being reviewed. And unfotunately due to the economics of the market (really, economics of ALL markets) it's impossible to keep a publication afloat without taking advertising dollars from those same entities, or other companies that benefit from said entities.

The disclosure of all benefits is a must of course, and that's where I appreciate CT - they always describe how they procured the products they review, or if they've been given other perks as part of the review (hotel, travel, food, etc).

Even DCRainmaker isn't completely immune, as while he doesn't take free items for review nor display ads from the those same manufacturers he DOES make money from affiliate programs where people buy those products.

If anything I might actually trust some online publication reviewers MORE than "independent" reviewers and forumites who supposedly don't have skin in the game but actually have another hidden agenda or beef.

P.S. I was wondering where those ads came from, then remembered I use AdBlock. Bias Solved!

[MattTuck]

Yeah, I had to fire up Microsoft Edge to get the screenshot of the ads! Since I don't see them normally either. But I had a hunch that there would be bike related ads, and it was even more compelling that they were Topstone ads.

You make good points, obviously, and I think we're in 95% agreement.

Eyes wide open when you read/watch these things. They're all intended to shape our perception, so if you understand that, then you're ahead of most of the other people out there.

The good thing is most of this stuff is available for the customer to try as well. I just got word that Pivot is doing a mountain bike demo day in the area later this summer. So, while the reviewers are helpful -- the customer can rely on his/her own impressions when using the actual product as well.

[makoti]

So, the reviews may be fair and accurate, but there will always be that implicit pressure because of the business model.


This is true, of course, and there isn't anything really wrong with it. Or, really, much way around it unless a publication goes full on Consumer Reports. It's the reviews themselves and the reviewer taking product that's bothersome to me. Now, I'm sure that the product being reviewed likely has an ad somewhere in the publication/site, but, like you say - that's the nature of the industry.

[weisan]

I’m wondering what you guys expect in the way of neutrality from sites that do product reviews.

I don't expect neutrality for anything or anyone who is voicing an opinion or expounding their views. But I do expect them to back it up with why they think the way they think. If I find their arguments or rationale weak or baseless, I don't take them seriously...even if they are neutral...so what?!

Neutrality is not the only yardstick in the marketplace of ideas.

[Mark McM]

Even if the reviewers are unbiased, and if the publication takes no advertising dollars, most reviews will naturally be biased, due to the system.

Most reviewers can not afford to purchase all the equipment they review. Instead, the manufacturer will lend the equipment to the reviewer, and when the review is complete, the reviewer will return the equipment. So it may sounds like reviewer receives no material benefit, but here's the catch: If a reviewer produces too many negative reviews, the manufacturers will stop loaning them equipment. Without equipment to review, these reviewers sill simply stop producing reviews. Therefore, only those reviewers who tend to give positive reviews will be able to continue doing reviews.

[MattTuck]

I don't expect neutrality for anything or anyone who is voicing an opinion or expounding their views. But I do expect them to back it up with why they think the way they think. If I find their arguments or rationale weak or baseless, I don't take them seriously...even if they are neutral...so what?!

Neutrality is not the only yardstick in the marketplace of ideas.

I'd find your points more convincing if you weren't so biased against neutrality :banana:

[pasadena]

In today's culture, the goal is to sell-out.
To expect any bicycle review to be unbiased is just unrealistic.

To call any bicycle product reviewer a "journalist" is silly. You can't hold them to a standard that most in the Fourth Estate can't pass.

The only honest reviews I've read for cycling gear is in CyclingTips.

I don't care if they keep product. That is just a by product of what he thinks of his job.
Really, he should donate all of it to cycling programs but that's just my opinion.

[fiamme red]

First CyclingTips post ever sponsored by Walmart: https://cyclingtips.com/2019/06/from-couch-to-kanza-eight-days-notice-200-miles-and-a-bike-from-walmart/. :rolleyes:

[pasadena]

First CyclingTips post ever sponsored by Walmart: https://cyclingtips.com/2019/06/from-couch-to-kanza-eight-days-notice-200-miles-and-a-bike-from-walmart/. :rolleyes:

literally the first words in the CT article

This content is brought to you in partnership with Viathon Bicycles.
Find out more about our sponsored content policies here.

how much more clear do they need to be that this is sponsored content?

[makoti]

So it may sounds like reviewer receives no material benefit, but here's the catch: If a reviewer produces too many negative reviews, the manufacturers will stop loaning them equipment. Without equipment to review, these reviewers sill simply stop producing reviews. Therefore, only those reviewers who tend to give positive reviews will be able to continue doing reviews.

Yes, it's an imperfect system, to be sure. And why would you send stuff to someone who does nothing but trash your stuff? But guys do manage to pull off the balancing act, the better ones, anyway. There are some good reviewers in all fields who manage to do it right. Give credit where is it due & call out a company, if warranted.



Really, he should donate all of it to cycling programs but that's just my opinion.

That was one of my suggestions. Use & donate. Review more, get to try more. Seems like a good deal to me.

[ibis]

When I was sending out bikes, pedals, apparel, etc..to magazines and websites I would let them keep (or pass on to a buddy) everything but complete bikes. Those they had for a predetermined time and had to send back.

Even though I had advertising contracts with some of them (not all) it did not guarantee a favorable review. It was tough to get magazine cover's, and that was always the goal. I was always pitching head-to-head contests with Specialized and Trek and Cannondale and so on. I was always told that bigger ad budgets do not guarantee covers, but the writing was on the wall. I had it out with RBA once over this issue.

The worst part of this process is that once you are known as "that guy" for a particular company...everyone with their cobbled together website would show up at Interbike pretending to be media in order to get free gear.

[Ronsonic]

Back in the day, when nut books were all printed matter, the thing that skewed reviews more than anything else was the fact that most reviewers aspired to jobs in somebody's marketing department and wrote accordingly. Sure, free product that they could keep, sell or at least audition for a year or two was a perk, but the prospect of an industry job was the big deal.

I'm sure the same or similar dynamic still applies now, with the added option of selling themselves as an "influencer." But, I'm old and not sure how all that works out.

[Charles M]

I can tell you that the answer you're asking for is just a bird's nest of speculation (not all off-base).

So many "reputable sights" are straight-up PAY FOR PLAY now that getting a straight answer from a group of forum folks will be nearly impossible (as much uninformed as informed).

I would also say that nearly all (cycling/Moto) reviewers keep product of some form or other. Nearly nobody takes back clothes/shoes/helmets, very few take back wheels, most take back bikes. BUT NOT ALL.

I've been doing this (without Pez covering even my own expenses) for nearly 20 years and have seen a hundred+ late-teen/early 20-something reviewers come and go like mayflies... (from some of the largest publications) Lots with little more industry or manufacturing experience than what they also read in forums.

That's not to say at all that you shouldn't bring up the subject, but I would ask this one question... Before you speculate on the purity of someone's writing, should you not have evidence that what they're saying is inaccurate?

Nobody writes with 100% purity. Nobody.
Even Consumer reports have favorite brands (that have always provided them product and support). People speak to people... The auto and Moto mags get wined and dinned to exceptional levels.

Anyone wanting to troll (and I DO NOT THINK THIS IS THE CASE HERE), regardless of how little they know about the writer or the products, can claim "taint"... It's such an easy stone to throw, and very frankly it can be accurate given how MASSIVELY screwed up the media is now (with major publications prioritizing "volume" and "impact" over accuracy).

I can't speak for anyone but myself (I can't even speak for other reviewers at Pez, because we have a couple of people that really have no business writing reviews given their lack of technical understanding), so I just continue to produce what I think is accurate info about the products I get the chance to handle. I genuinely don't care to try and meet the entire world's moral benchmark because I think it a fool's task.

ALSO, not talking to friends I've had in the industry for a couple of decades just isn't going to happen... But I think it's given me access to a lot more information rather than less. I think it a net benefit.

Put more simply, I do what I can and I hope it gives people a better look at the subject at hand. I also know a few other long term tech guys that are in the same boat.

Makoti, thanks for bringing it up. I don't mind at all and nobody else that reviews kit honestly should mind either...

[Matthew]

If you've got a Baum, IF, or a Firefly in a 55 I will review the heck out of it completely unbiased. I promise to send them back.

[Tickdoc]

Wouldn't it be fun to read of proof that Consumer reports has been in bed with Toyota all along?

Back on topic though, I feel that gear reviews are not near as biased as bike reviews. There are just too many factors regarding bike reviews that make them impossibly unreliable and unbiased. I look back on old RoadBikeAction magazine reviews of bikes I used to lust after and reading them now is almost comical. You can cut and paste all the superlatives and apply them to almost any of the reviews.

You want to know which hiking boots are best? Go to Yellowstone or Yosemite and ask some of the backcountry guides. Those guys aren't paid to wear the gear they use (Usually) and they will tell you straight up what works.

[Charles M]

Tick, One of the few things that I've done for the past 15 years or so is run the same wheels, tires, tire pressures and saddle on everything... Disc Brakes (and now 12 speed and funky front end gearing) toss a monkey wrench in that to some degree, but I at least try and give the platform some repeatability... (now everything runs on 303 and 303 disc).

But even with that, I'm short of the equipment to really dig into flex and Data acquisition... I can do that for moto, but dont have high enough resolution for Bikes.

If you've got a Baum, IF, or a Firefly in a 55 I will review the heck out of it completely unbiased. I promise to send them back.

:) You're a digit too big! Everything here is 54.

[Tickdoc]

:) You're a digit too big! Everything here is 54.

Oh, I can make that work!;)

[19wisconsin64]

delete post

[makoti]

It really isn't that hard, I guess.
From Slowtwitch:

"In way of disclosure, I pestered these folks, stalked them really, to become a Slowtwitch partner. They said yes. Road tubeless, like disc brakes, like stationary training platforms, like electronic shifting, like gravel, among others, are themes that animate me, and are emblematic of where the market is going. This is what you have read here, and will read forward going, partnerships or not. Nevertheless, because of my high praise for this brand it’s important I disclose this. Read more from Schwalbe here if you want to investigate further."

[mhespenheide]

You want to know which hiking boots are best? Go to Yellowstone or Yosemite and ask some of the backcountry guides. Those guys aren't paid to wear the gear they use (Usually) and they will tell you straight up what works.

That's a bad counter-example, though...

As someone who leads backpacking trips as part of my job, I have access to pro deals on some brands of boots and not others. If I choose to buy through that pro deal (typically ~40% off list price), I'm explicitly told to try not to disclose my discount and instead offer whatever positive comments I can on the gear. And, like the aforementioned bike reviewers trying to get into the game, the starting pay for backcountry guides is so little that you basically have to go with the pro deals on gear.

I would guess that that type of game is played in every field.

So we're stuck trying to look for someone like @Clean39T or @Ryun who have the breadth of experience of a dedicated amateur and the means to dedicate himself to trying out a broad variety of bikes in order to be able to offer an unbiased opinion that's actually meaningful.

[FlashUNC]

Any review by its nature is a subjective exercise filled with all sorts of biases both conscious and unconscious.

Just find a reviewer whose sensibility seems to jive with your experiences with gear/movies/music/art/whatever and go off that.