Background:
A couple of days I ago I made a post to facilitate a discussion with Diego Arias who publishes content about MQB cars on YouTube.
Diego made a claim in the comments section of one of his videos that “bench flow tests are BS“.
Diego also commented that the information I have provided should be viewed as likely to be misleading or untrue because many times data from others have contradicted my findings.
Last weekend he agreed to substantiate these claims in the comments sections of that post, but four days have now passed and he has not responded, so I’m assuming that discussion won’t be happening.
At the same time that exchange was going on, I came across another person with similar concerns about the testing in another comments section of a different video.
Like Diego, Adam asserts there is data from “other people” that doesn’t match my findings which I supposedly dismiss. Second, Adam disparages the flow bench tests.
I also offered Adam the opportunity to present evidence in the Monday post, but he has not responded either.
This post will briefly address each of the topics that were brought up, and then look at a more significant issue.
Analysis:
Addressing the second point first, I get the impression that there is a growing body of enthusiasts discussing parts testing that misunderstand what a flow bench does.
To shed some light on what a flow bench is good for, here’s what Wikipedia has to say about the device:
It’s not just Wikipedia sayings a flow bench is good at measuring airflow. Here are some names you may recognize using one of these devices.
Industry:
Eventuri stock intake flow test
Mishimoto Mk7 intake flow test
Wagner Tuning Flow Test
Audi Performance & Racing Mk7 Intake Flow Test
S&B Filters Intake Flow Test
do88 Intake Flow Test
How?
How do these automotive enthusiasts get it soooo wrong?
It isn’t just these two that I’ve encountered lacking an understanding of what a flow bench excels at. I highlighted these two people because they’re the most recent examples, and I was hoping I might get some insight from their responses, but over the past few months there has been a number of similar comments from others.
Data Issue:
The other complaint, the notion that I dismiss data presented by other people, can only be explained as a willful ignorance of the innumerable times I have asked people to explain what data they have to support claims.
As an example, when I mentioned to Diego that I would make the post where we would fact-check his claims and review his evidence he replied with the statement:
“Go ahead no problem it’s not that deep”
In the exchange below Diego appears to believe that he can accomplish this demonstration of evidence on the spot. I have highlighted a couple of sentences to illustrate why this is not the case.
The red-underlined claim is not supported by any evidence.
The orange-underlined claim is a short story that is supposed to be a specific example of the first claim, but there is no evidence provided to demonstrate the claim is true. (I know he used numbers in this sentence, but that does not make this good data).
The green-underlined claim also has no evidence to support it, although I agree with him.
With this statement, we can deduce with logic that it is correct. Driving a car introduces more input factors than a flow bench, thus increasing the difficulty of making accurate and repeatable measurements. This raises the chance that measuring airflow on a car would differ from measures made with a flow bench.
The question for me is, which method is more accurate and repeatable for measuring airflow? Using a flow bench (see Wikipedia and Industry sections above) or driving an automobile?
The latter involves measuring indirect systems responses, accounting for interactions and confounding variables, and deducing if the controlled input factor is responsible for the change in the response variable of interest. This assumes the person conducting the test has sufficient knowledge of testing procedures to not introduce process errors and replicates the experiment enough times to address experimental noise.
Back to my point above about good data, there is no evidence presented by Diego to support his claims.
On the matter that I purportedly dismiss data that other owners have, this statement omits the fact that not all data is good data.
Poor data will lead to poor conclusions, and that is something I strive to avoid.
When other owners have good data I will factor it into my conclusions, as I did with the Burger Motorsport IC Analysis which was based entirely on other owners’ data.
Irony:
The irony of Diego’s comments is that he states that when he installed the Blaze intake on his car some of these operating values supposedly improved, suggesting the intake improved airflow on his car.
My flow tests of the Blaze intake showed that when replacing an intake setup that uses an inlet elbow smaller than the 55.7 mm Blaze inlet flange, which is just about every elbow I know of other than the DBV2 TIP (55.3 mm), the airflow increased.
Diego’s argument is that the flow test is BS and should not be believed, even though his own measures, if accurate, validate the flow bench measurements.
So why are Adam and Diego claiming there are “many times” that “multiple users” are getting results that don’t agree with the flow bench?
That would make their position: the Blaze intake decreases airflow.
Bigger picture:
I selected these two guys comments because they’re the most recent instances of me encountering these issues, and they both tidily addressed two of the most egregious misunderstandings that I have been reading lately.
It’s clear to me that these “issues” they’ve got came from what “somebody else said“. This begs the question of where this misinformation originated from.
It’s a shame that the community gets so defensive when challenged about their claims but can’t provide hard data to prove themselves. Many would go straight to a manufacturer’s/ tuner’s site and read off the data as gospel. But when you go out of your way to prove/disprove that, you get shunned. I appreciate the time and research you put into the research and testing you perform. Completely unbiased compared the claims of others because they said so or someone else said.
Thanks! In the end, not taking the time to think about the information being presented is likely to lead somebody to buy a part that’s not what they wanted, wasting time and money, before they eventually stumble upon what they were after in the first place.
Yep – it’s a thing and it’s unfortunate. I’ve had much the same response on some testing I did on many of the popular “performance” changes you can do with VCDS/OBDEleven that are passed around the Internet…testing/data presented are trumped by “I can feel it bro.”