Social media, even LinkedIn, is a particularly good petri dish for studying popular trends and fads in training. A good way to practice critical thinking is to test ourselves on these and see if they add up! So, we picked a typical instance – something that had garnered a lot of agreement – and decided to see if we could determine what made the argument sound (or not). In the spirit of practitioners thinking aloud and learning together, here’s our experiment…
Context:
The event shared took place in the USA. A young guy is about to take his driving test. The account shared pertains to his experience of the driving lessons. This experience is then used to draw conclusions for learning design in general. This drew a lot of hearty agreement and the comment thread took the presented conclusions further with barely any hesitation.
Some background:
Someone enrolled for driving lessons. He was learning through the formal lessons, as well as from many practice sessions with his parents during which they gave him feedback.
He found his instructor’s style too complicated, and started turning to YouTube video tutorials instead. He praised the videos saying they were really short and simple. Particularly the one on parallel parking was praised for being less than 5 minutes long. This was something the author picked up on and echoed approval of.
Conclusions put forth:
- What he did is unique to his generation. If young folks (i.e., generation whichever-applies-for-the-time) don’t get what they’re looking for from a teacher, they will find the information through other means.
- Learning using such video tutorials plus feedback and practice are a guaranteed, winning strategy every time.
- This incident has lessons for learning designers on how we design learning henceforth.
Who posted this and where don’t matter at all for the purpose here. This kind of argument is quite popular and very common. We’ve all likely seen a similar form of argument made several times before so it also seems quite unfair to single out anyone! Instead, here’s what we’d like to focus on: how does confirmation bias insidiously undermine reasoning and skew conclusions? What really is the damage done, if any?
Why is this interesting?
We know that in every worthwhile murder mystery, the sleuth must investigate every possibility when searching for the truth. It’s wrong to fixate on one person as a suspect without also verifying everyone else’s alibi and eliminating them as suspects. The other requirement is to search for corroborating evidence and to study all the available evidence, adjusting our theory of the crime as dictated by the evidence.
These are exactly the two facets of rigorous thinking ignored in the argument. There are possibilities that are ruled out with no good reason offered for the rejection. And, there is a selective narrative of elements, as though only one interpretation is possible. This one interpretation is presented as The Design Truth.
If there’s a single line summary I have to offer on what is most problematic in the whole thing, it would be ignoring several possibilities to pick just one that suits a narrative we have already subscribed to. In other words, a confirmation bias is in play.
What do we learn from it?
Our field is characterized by conflicting explanations offered by several authorities, perhaps more so than many other fields. Nobody can as yet tell us one infallible and complete version of how people learn, or how to teach well. We often have to look for guidance in a nebulous mass of conflicting perspectives, contradictory study conclusions or inconclusive findings and seize on whatever seems to make intuitive sense or seems to be most convenient for the purpose at hand.
That’s why many of our arguments seem particularly prone to confirmation bias – and their Achilles’ heel seems to be in the form of evaluating if an explanation accounts for everything, and if it is the only explanation that makes sense.
The breakdown
(Of course, I’m not claiming that this is every single reason to reject the conclusions presented in this argument! It’s more a simple bunch of considerations to start thinking about.)
Check of conclusion 1:
- What he did is unique to his generation. If young folks (i.e., generation whichever-applies-for-the-time) don’t get what they’re looking for from a teacher, they will find the information through other means.
But this is actually behavior we’ve known and resorted to from long before the internet. We’ve asked questions (of the teacher and others) to clarify our understanding and seek more information on a particular point. We’ve done supplementary learning. We’ve asked peers to tutor us. This is in no way something to do with the learner being from a particular generation – mere occurrence is not unique correlation.
Malcolm Knowles similarly tried to pin some behaviors as typical of adults when learning – and subsequently walked back his claims when he was correctly challenged. (You wouldn’t suspect this however, given how the term of ‘andragogy’ still took on a life of its own!)
Like Knowles, this author had no basis for an age-based characterization of the learning behavior, but since they generally subscribe to the idea, they went ahead and seized on that as the explanation.
Check of conclusion 2:
- Learning using such video tutorials plus feedback and practice are a guaranteed, winning strategy every time.
Here’s the bulk of where the reasoning seems messed up. Given the anecdote, ‘video tutorials’ seems to refer to specifically learning from YouTube videos. Well, it looks like the fashionable practice of bashing on schools just got extended to one-on-one personalized teaching too. When we simply stand back and see if that fits with everything else that we know about teaching and learning, for example, Bloom’s writing on mastery learning or Keller on personalized learning or the reality that we consider student-teacher ratio in evaluating schools… hmm, that doesn’t add up at all. It tells us that perhaps our conclusion can do with more checking.
Feedback and practice are presented as being outside of the coaching that an instructor would provide. That doesn’t make any sense either. Such a compartmentalization is likely from considering the medium of YouTube videos alone. (Because, what, does the instructor sit blindfolded next to the student and keep mum?!) This tells us that such an assertion is at best built on a hastily assumed, inadequate understanding of what an instructor does. Any conclusions drawn from this foundation, such as about the instructor’s performance, are suspect. And then to extrapolate to all instructors and this as a mode of teaching…? Big ouch, much no.
Do we really need to lay out why claims of ‘guaranteed’ success are dubious and snake-oil-ish? I really think we can declare that self-evident and continue.
Check of conclusion 3:
- This incident has lessons for learning designers on how we design learning henceforth.
This conclusion is already rendered dubious, because the problems demonstrated in conclusions 1 and 2 have made the legitimacy of the entire argument questionable.
And, there are several other overlooked elements to explore.
What considerations have been ignored?
- The learner may not have known the first thing about what makes for effective teaching, given that he was looking for only brevity and simplicity (i.e., something that wouldn’t need as much effort from him to learn). To evaluate teaching on solely this basis is inadequate. What if brevity was achieved by taking sloppy shortcuts? What if simplicity came at the cost of essential knowledge? Without knowing the details of the curriculum no conclusions can be drawn about the caliber of teaching and the methods used. Yet the argument rushes to do exactly that.
- The real-life instructor may have been bad, and the teacher in the video may have been good. Then that’s a difference between a good and bad teacher. How is it warranted to jump to the conclusion that it’s entirely to do with the superiority of the second medium (online videos)?
- Neither the learner nor the person making the argument factored in that the video was used a lot as a revision tool. There was prior teaching, rigorous practice and feedback to support a targeted level of learning outcome. Of course, the teaching work to be done in a review is far less than the teaching work to be done to initiate the student into proper performance of the task. Who takes the same time to summarize as to do the initial teaching? How can the two be blindly compared?
- Social platforms make the access to variety far easier. How is that not factored in? Why is it seen as a function of the medium making for a superior teaching strategy than that the platform offers the ability to search more easily for a better teacher?
- The experience of one person does not represent a universal truth. You cannot so grossly generalize and extrapolate to say that you will immediately make pedagogical decisions for all adult learning you design, based on that one learner’s experience or feedback because it must be true for everyone.
Which is your favorite example of a ‘terrible but popular’ kind of fallacy?! Let us know in the comments!
Written by Mridula R., Principal Learning Consultant @ Learnnovators
