Test Support Tools - EQT Concerns

Background:

A pair of recent posts documenting the comparison of a catted downpipe and a catless downpipe generated a good amount of criticism from Mr. David Veal.

Normally, a person expressing criticism of test results is not noteworthy enough to warrant making a post about it, but the content of the complaints and source presents an opportunity to take a closer look at the people who make up the aftermarket performance parts and services industry.

Mr. Veal has been an employee of Equilibrium Tuning, a popular supplier of aftermarket parts and services for VW Mk7 consumers.

The business profile shows that Dave served as the “Website and Social Media Manager at Equilibrium Tuning“.

As the face of the company interacting with consumers through social media, reviewing Dave’s comments regarding the test results may give some insight into how helpful he might be at providing consumers with reliable information.

Comment analysis:

Dave from EQT begins with:

scientific: conducted in the manner of science or according to results of investigation by science : practicing or using thorough or systematic methods
Merriam-Webster

Dave from EQT is confusing a software application, a product that is a tool, with something that is a process. A tool can be used in a scientific, or non-scientific way.

I suggested Dave be more factual in his critique:

Dave struggled to provide concrete examples in support of his claims and in the interest of staying on the topic of the test support tools, excerpts from the discussion on this specific topic will be the main subject that is addressed.

The entire exchange is presented at the bottom of this page.

Dave from EQT responded in part with:

Key claims:

Virtual dyno is not a valid measuring tool.
You didn’t actually measure shit.
There is no control or repeatability.
I can compare my real dyno to a virtual dyno and demonstrate how different it is.
I’d trust Dragy over Virtual Dyno.

1) Dave is correct that Virtual Dyno is not a measuring tool. It is a software application that takes data measured by other means and processes that data.

2) Dave claims that nothing was measured. Because Virtual Dyno processes data from a source that takes measurements and Dave has acknowledged that Virtual Dyno was being used, it follows that measurements must have been made. Dave has contradicted himself.

3) Dave claims there is no control or repeatability. A control is an element that remains unchanged or unaffected by other variables. A number of factors were unchanged. For example, the same GTI was used, the same turbocharger, and the same tune. This statement is false. Repeatability refers to how closely the results of successive measurements are of the same measure when carried out under the same conditions of measurement. Dave has failed to indicate what acceptable criteria are which makes the claim meaningless.

4) This was a perfect opportunity for Dave from EQT to demonstrate the difference between his dyno and Virtual Dyno, but he did not.

5) Dave would trust Dragy over Virtual Dyno. Dave is failing to distinguish between a device that records measurements (Dragy) and a software application (Virtual Dyno) that processes measurements to generate an output. These are tools that serve different functions, it’s illogical to trust one over the other since they perform different tasks.

Dave from EQT is not building confidence in his knowledge of these tools or the processes of using them.

Dave’s next reply is broken into two segments with the points to be addressed highlighted in yellow.

Key points:

The Dragy GPS Performance Box is used as an industry standard for performance measurements.
No one cares what Cobb data says.
Refresh rate be damned.
Dragy is usually considered as the golden standard.

1) It seems unlikely to me that the Dragy Performance box would be used as an industry standard. It’s not clear what “industry” Dave was referring to. One would assume it would be the industry he works in, selling aftermarket parts. Again though, Dave is comparing two devices that perform different functions.

2) Dave says “no one considers Cobb results as accurate“. This is puzzling since I had a custom tune made by Equilibrium Tuning a couple of years ago and data logs recorded with the Cobb Accesport were used by EQT for making revisions to the tune. If EQT has determined the Cobb measurements are not accurate that’s an interesting development. Dave also claims that “nor does anyone care what the Cobb data says“. I’m interested in the data the Cobb records because it helps me to assess if the data logging session is meeting my expectations for “repeatability“.

3) Refresh rate be damned. I answered Dave’s concern about trusting the Dragy over the Accessport by mentioning the Dragy has a 10 Hz sampling rate whereas the Cobb is between 20-40 Hz and I preferred using data from the higher sampling rate device. Dave obviously isn’t concerned about the sampling rate of the measurement device.

4) Dragy is considered “the golden standard”. I think Dave meant to use the phrase “it’s the gold standard”, aka a benchmark of the highest quality. I doubt the 10 Hz Dragy is considered the gold standard of automotive performance boxes with alternatives like the VBOX 3i performing at 100 Hz and less than 1 cm position accuracy.

Dave’s belief that Dragy is an industry and gold standard, along with an apparent lack of appreciation for data sampling rates, does not build confidence in his knowledge of data measurement tools.

The second part of his reply is below:

Key points:

Asserting that I have claimed the Virtual Dyno is equivalent to a real dyno.
Virtual dyno doesn’t have an accelerometer. It uses speed and time to calculate acceleration.
I wouldn’t include an outlier in my own testing.

1) I stated that I was using Virtual Dyno to report changes, the same as what physical dynos do. I never stated that Virtual Dyno is the same or equivalent to a real dyno. This is a strawman argument where the person counters an argument with an exaggerated and inaccurate version of that argument.

2) It is correct that Virtual Dyno doesn’t have an accelerometer, it is a software application. It’s unclear from Dave’s statement “simply uses…” if he believes using speed versus time and applying principles of physics to determine acceleration rate is a cause for concern.

3) When taking measurements it is not uncommon to record outliers. It’s important to look into an outlier to understand what caused it. True outliers should always be retained in the dataset because they represent natural variations in the sample. Dave’s policy of discarding outliers is likely to skew the results of any tests he performs. This raises concern about how EQT reports data if Dave has any involvement.

Conclusion:

Equilibrium Tuning is a business that supplies aftermarket parts and services to consumers owning the VW Mk7 GTI, along with other automobiles. David Veal has been the business’s Website and Social Media manager, interacting with consumers through social media platforms.

Dave from EQT provided a critique of a test that compared a catted downpipe with a catless downpipe.

Dave At EQT’s comments suggest he does not have a good understanding of the differences between scientific processes and the tools that can be used to support an experiment. Despite working for an automotive aftermarket supplier Dave was unable to present any evidence in support of his claims about potential shortcomings of the test.

Returning to the question that was posed at the beginning of the post:

“As the face of the company interacting with consumers through social media, reviewing Dave’s comments regarding the test results may give some insight into how helpful he might be at providing consumers with reliable information.”

My opinion is that Dave Veal may provide consumers with unreliable information if it is of a technical nature. By extension, consumers seeking information from EQT who communicate with Dave may receive unreliable information from the company. When consumers hear a fellow enthusiast state; “my tuner says…“, the person receiving the information ought to pause and consider that “my tuner” could be referring to someone like Dave.