Back in the day when attending live, face-to-face conferences was “a thing,” I always looked forward to the breaks when I could join my peers around that long, huge table put out by the hotel offering a variety of Danishes, fruit, and of course, coffee. I am sure those days will return soon, and with it, the equivalent of the “paper or plastic” dilemma, that is, should I pour my coffee into a ceramic mug, or a paper cup?

The argument for the ceramic cup is that it creates no byproduct waste; it is reusable.  On the other hand, potentially harmful soaps and other chemicals are used to sterilize it, along with water, a precious resource. The effect of this process on the environment is not something we consider as we approach the table. On the other hand, the paper cup consumed natural resources in its creation, and it will consume additional resources in its destruction, even if recyclable. What is the better choice? 

It would be great if quality managers were faced with binary choices like this, because not much time would be needed to research the best path (in fact the paper cup is NOT the better choice, because the plastic coating inside the cup renders it non-recyclable; now you know!).  Fortunately, with the benefit of a hypothesis test, you can take a seemingly out of control process and answer the question, “can we improve our approach, or not?”

By now you are thinking to abandon this article, just as you did the stats book that you sold back to the college bookstore the day after you completed your statistics class. But you should not be intimidated by the hypothesis test. The underlying principles of the test are simple and learning to master its use will be a powerful addition to your portfolio of quality tools.

In a Student’s t-test, we are comparing two sets of data and asking the question, “is there a difference?” The test compares the mean and variance of the two datasets and renders a binary decision. We go into the exercise with the assumption of “no difference,” aka the null hypothesis. We then specify a risk level, our tolerance for accepting a difference when none was there—or concluding that there was no difference when in reality it was staring right at us.

The term “Student” refers to the inventor of the test, William Sealey Gosset, who devised the test while working for Guinness Brewery at a time when the company required publishing under pen names, hence Gosset posed as a “student” of statistical theory. Gosset used the tool to determine the properties of barley that would produce a higher quality stout.

There are a number of uses for the student’s t-test:

• To compare the performance of a workaround versus the SOP
• To evaluate the impact of one independent variable on overall performance
• To determine if an intervention makes a significant impact on an output measure

A quality director was struggling to improve cycle time to produce a time-sensitive product. Creation of the product included the use of a pneumatic tube system to communicate requirements to separate departments. Conventional wisdom was that the tube would accelerate production, because it moved faster than humans. The binary question to answer became: “is the tube producing faster results than simply walking instructions to the next station?”

When comparing the tube process to having staff physically take the instructions to the next department, the t-test revealed a significant difference: using the tube was slower! Apparently, so many tubes were being used at one time for both urgent and nonurgent issues that the high-priority work was essentially stripped of its rank.

Pivoting to a service example, a hospital was witnessing a large number of patients leaving their emergency room (ER) before being seen by a physician. The “left without being seen” rate increased significantly during flu season. This was not only contrary to their mission to serve the community; it also had a financial affect since a high percentage of inpatients enter from the ER.

The bottleneck was found to be the process of finding an inpatient bed so that patients could be moved out of the ER. A software system was used by housekeeping staff to determine which rooms to clean, so the question became, “should the ER staff use the same software to expedite transfer to rooms once they’ve been readied?” A comparison of performance after trialing the new process demonstrated a reduction in cycle time of less than 10%. While not compelling to the naked eye, the t-test revealed that the improvement was statistically significant, and over the course of a year, could generate huge profits. The t-test gave the organization the confidence to implement the measure after the hospital CFO projected $650,000 in net contribution as a result of the change.

This final point is perhaps one of the most important when it comes to advocating for change in your organization. We often get only one chance to make an impression on the decision maker. It’s one thing to see your role as a change agent, however complaining about process performance without the benefit of a t-test is tantamount to “a lot of stirring, but no gravy.” Leverage the test!