This post describes the motivation for and goals of adding a discussion category labeled “Fact Check”.
The topic of these “fact check” posts refers to vendor marketing or enthusiast discussions where a goal of the vendor / enthusiast is to coax other enthusiasts to accept an argument, usually on a topic directly or indirectly about the purchase or performance of an aftermarket product.
A significant amount of discussion takes place on the topic aftermarket “performance” parts. These parts usually replace stock parts or software, or are add-on parts, that are supposed to improve the performance of the GTI.
This category of posts will be used to analyze the vendor claims, or enthusiast statements, to assess if the argument conclusion is logically supported by the premises or evidence.
Observation of an abundance of material to work with in this manner prompted the idea to add a new category for evaluation.
These posts will be made from the author’s point of view (surprise!) and therefore are an opinion.
Even though the posts will be opinion, an evaluation of the facts and logical support of conclusions will be emphasized in the evaluations.
On the topic of assessing how well “something meets needs”, my opinion is that after spending several years in the test and evaluation profession I have a less common perspective on the subject compared to the average enthusiast, and from what I have observed, the average vendor as well.
Briefly covering what test and evaluation is, it is the process by which a system or components are compared against requirements and specifications through testing. Some activities that are part of test and evaluation are:
- Determining the salient factors to be considered for the specific item to be examined.
- Determining whether test methods will yield valid results.
- Determining if test facilities are adequate for the item.
- Determining if instrumentation is adequate to achieve accurate test data.
- Planning of tests to determine whether an item or system meets requirements.
- Conducting, or overseeing, testing to ensure that a valid test is being accomplished.
- Understanding the relative statistical validity of the test results.
- Reporting on the results of tests.
Or as one FB Group member succinctly put it:
It’s also worth noting that “asking a lot of questions” is a large part of a testing effort.
Another point, test and evaluation isn’t a popularity contest. You might have read this before, “Everybody uses <blank> so it must be good“. Popularity contest right there.
When the process of searching for facts that support conclusions starts to be applied to products, services, or claims that people have a strong attachment to, i.e. something they are selling or have bought or think is right because it is “common sense”, often there is a negative or defensive reaction. (Expect to see some good examples of this!) A phrase I’ve heard used to explain test and evaluation is: “our job is to inform people that their baby is ugly”. (This addresses the situation where testers inform a program manager that the product they have been developing isn’t meeting it’s requirements.)
The T&E process helps to objectively determine how something measures up against requirements, or claims. This post category will serve to look at claims and evaluate how well facts and logic support them.
In the next post I will discuss guidelines that will be applied in these evaluations, such as the FTC Truth In Advertising guidance for vendors.