€10’000 down the drain (the costs of anecdotes!)
INCONCLUSIVE? WTF?!! I had finally succeeded being a freeloader at the section of Sensory and Consumer science at the University of Copenhagen by getting 6 samples evaluated using Sensory Descriptive Analysis using the professional panel at the university. The street price of such a project is around €10,000 so blowing the chance of getting good research data was devastating! If we had just chosen our samples more wisely (bigger differences between samples) this would not have happened. I had an AMAZING opportunity to have my theories proven and I blew it! During the 5 years prior to this project, I had been teaching hundreds of students on my roasting classes at London School of Coffee as well as consulting several coffee roaster startups. I had a lot of claims about how the roasting curves affected the flavour of the coffee and all my students and clients where very convinced about my claims. At the end of each class they could taste it themselves and thereby verified my theories – I thought… So I was pretty convinced that the samples I provided the research project were systematically different when it comes to flavour and I was just ready for getting this proven objectively, scientifically in a written report. But according to the statistics, the professional panel found nothing! They could not even taste any difference at all between the samples.
This was the first time in my career that I had a sobering experience of how small differences in the curves have no significant effect on the flavour on the coffees! After this disappointing experience with mapping out flavour aspects of roast modulations, I decided to map out flavour differences between samples with clear and drastic differences first, and only if we succeeded getting data with clear between-sample differentiation we could move on to the more subtle aspects of roasting. The first well documented project led to our first scientific publication with the title “Common roasting defects in coffee: Aroma composition, sensory characterization and consumer perception” published in the Elsevier Journal “Food Quality and Preference”. And after mapping out extreme differences (roasting defects) successfully we moved to the next step of mapping out the more subtle and less obvious modulations and according to our data this is Development Time (time after 1st crack to End of Roast) which is published in “The Effect of Roast Development Time Modulations on the Sensory Profile…”. If you want to get an overview of all our research on sensory evaluation of roast profile modulation you can get it FOR FREE here: “Roasting Conditions and Coffee Flavor: A Multi-Study Empirical Investigation“
There is something seriously wrong with our global coffee roasting education approach
As explicated in CoffeeMind’s podcast series CoffeeScience for CoffeePreneurs we think there is something seriously wrong with how coffee roasting education and sensory skills education has been developed and conducted worldwide that last 50 years or so. Despite that fact that the ‘quality’ of a theory has been analyzed and explicitly described all the way back to Plato and revised and improved several times with very specific and [better] easily understood principles (Occam’s razor, Empericim, Critical Rationalism, Bayesianism, research design, probability theory and statistics) that can be used to create useful and correct theories and distinguish these from bad theories (called Sophism by Plato, Pseudo Science by Popper and Bro-science in a more humoristic modern term often found in the fitness community when people discuss biochemistry and physiology in the locker room without any correct application of any of the concepts anyway..)
What we have identified is a set of common assumptions that violates scientific principles directly but nonetheless has been dominating only to confuse and waste time for both teachers and students around the world:
- To claim that a certain parameter value (for roasting duration of any phase of the roast, ratio calculations or specific shapes of curves or for sensory a linear quality score such as the SCA cupping form) provides a universal sweet spot that is the ‘correct’ taste of the coffee
- Obsession with small details of process parameters. Small causes have small consequences so small detail differences are often not perceivable on the cupping table (such as small variations on the bean probe curve, individual organic acids in coffee..)
So it seems like education aligned with science would not assume any particular sensory sweet spot nor look at small details of processes but keep the big picture in mind.
As a consequence, many students feel that they are entering a vast unorganized flat land with thousands of partial theories with no overall logic and many of them are contradicting each other. We need to find some methods to make it simpler, clearer and more relevant. If you have not heard the podcast yet, you have to stop what you are doing (which is reading this) and click this link: Coffee Science for CoffeePreneurs by CoffeeMind. Ok, good. Now you know about Occam’s Razor and how striving for simplicity in use of concepts and explanations is important and with this simple approach A LOT of the concepts and approaches in coffee roasting education will by itself melt away like snow in the sun! The next tool for finding the concepts and claims that will be fruitful, relevant and useful will be to critically ask for every claim that is presented to you: Do I expect the claim to provide flavour differences above sensory threshold? Claims about small differences are typically just good stories and DO NOT translate into relevant sensory differences in the cup just as we saw with my above research project failure! The third thing to be aware of here is that there is another very typical mistake made by many educators and consultants which is that they play around with small curve differences, drum speed, airflow and other subtle differences and present you with a set of cups where you can clearly taste the difference and you choose a favorite and double check that they indeed did perform the experiments with either small curve differences, drum speed, airflow or whatever they focused on, but if you took our your coffee color reader, you would find that the samples had different colors! This is called a ‘confounder’ in research design and is the typical error that is introduced in a data set if you don’t remember to keep everything else equal (if you have not heard the podcast yet, at least listen to episode 1 where this principle is explained)
Applying the above principles to shave away irrelevant concepts and sensory differences which are below the sensory threshold, we are starting to build a high-level framework which can be used as a map to stay on solid ground when understanding the different aspects of roasting. This framework shows us what to focus on for education and business process design. Even if such a big picture framework is somewhat a simplification it is best to learn this by heart before exhausting yourself with details and getting lost. A confused mind does not make decisions – or at least not the right decisions. This is aligned with science where the simplest model with the highest explanation power is preferred. A similar approach like this is popularized as the Pareto Principle where you are encouraged to look for the roughly 20% of causes that are responsible for roughly 80% of consequences in a complex system. Everybody who has gotten advice from an experienced person wil have been advised to drop the focus on a lot of things that you have been focusing on for a while and to only focus on the really important things! It is also know as the effect of “diminishing returns” where you only get a big gain from dealing with something to some level and if you over-engage you get a smaller and smaller return and often people overobsess about something that is not even a gain as it is below sensory threshold.
So how would the big picture of building a coffee roasting business with focus on all the educational technicalities involved? With almost 20 years roasting and teaching (more than 1500+ individuals, 30+ startups worldwide and scientific research projects across lean startup methodologies, Sensory Descriptive analysis and Consumer Science) the following picture of the necessary and sufficient (!) competencies you need and we call this blueprint ‘The Perfect Coffee Roastery’.
If there is one painpoint (??) I have observed for Coffee Roastery startups over the years it would be confusion and feeling of being overwhelmed when it comes to finding necessary and sufficient skills and knowledge to execute the CoffeePreneurial dream of creating a coffee roastery. Often they feel that there is no end to the details you should know about and execute while roasting.
The mission in CoffeeMind is to help Coffee Roastery startups to succeed by only spending time on processes and methods that are explicitly relevant for the business they are trying to create (Your Perfect Coffee Roastery) which has prooven to be relevant and useful for complete beginners but also experienced coffee roasters who has had their mind cluttered by decades of accumulated anecdotes in the global coffee community.