Tag: education

Oct. 26, 2026

In Part 1, I compared shallow and deep understanding Levels 1 and 2 of Bloom’s Taxonomy (Knowledge and Comprehension, respectively, to use the original terminology).

In Part 2, I did the same for Levels 3 and 4 (Application and Analysis).

In this final installment, I do the same for Levels 5 and 6 (Evaluation and Synthesis).

Note that the names of Bloom’s levels, as well as the order, are different depending on which version you’re looking at, but it doesn’t really matter.

Level 5 (or 6): Evaluation

Activities relating to using concepts to make judgments.

Continuing with the example of the arithmetic mean, shallow understanding could be demonstrated by reliance on heuristics and decision rules.

For example, when dealing with employee salaries, one might assume that the median is necessarily a superior representation of a “typical value,” because salaries tend to be positively skewed.

But, as with all rules of this nature, it’s not so simple…

Deeper understanding would take into account the nature of the particular situation or problem.

For example, suppose the organization has narrow salary bands due to limitations by law or agreements with labor unions. If so, then the salaries might not be skewed much or at all! In that case, the median becomes no better than the mean (or possibly worse, depending on what you’re doing with it).

It should not always be assumed that a variable must be distributed in a characteristic way (even if it usually is).

Additionally, even if there truly IS positive skew, that doesn’t necessarily eliminate the relevance of the mean.

For example, suppose you are analyzing compensation for a commission-based sales team. Imagine that most sales representatives on the team have comparable pay, but a few all-stars substantially increase their commissions year-after-year.

• If you rely solely on the median to represent average team performance, this trend could be completely concealed.
• But relying solely on the mean could lead to a different incorrect interpretation—that everyone on the team is improving, when it’s really just a few.

Here’s a quick demonstration of this using R…

> # Creating a toy dataset
> typical=c(80,84,86,88,90,92)
> stars=c(150,170)
> (df=data.frame(id=101:108,
+                year1=c(typical,stars),
+                year2=c(typical,stars*2),
+                year3=c(typical,stars*4)))
   id year1 year2 year3
1 101    80    80    80
2 102    84    84    84
3 103    86    86    86
4 104    88    88    88
5 105    90    90    90
6 106    92    92    92
7 107   150   300   600
8 108   170   340   680
> # The totals are increasing year after year...
> colSums(df[2:4])
year1 year2 year3 
  840  1160  1800 
 
> # But the median does not reflect this!
> (medians=apply(df[2:4],2,median))
year1 year2 year3 
   89    89    89 
 
> # The mean DOES reflect this...
> # ...but it's biased by the all-stars!
> (means=apply(df[2:4],2,mean))
year1 year2 year3 
  105   145   225 

The above demonstration shows that the median gives a more “typical” experience, whereas the mean reflects the overall group pay more accurately. That said, the latter is easily misunderstood; it paints a rosier picture than reality, because the change is only driven by a couple high performers.

With a distribution this small, the solution might be simply to view all scores simultaneously, in a table or graph.

For any number of scores, you could also combine the two types of averages to get a fuller picture…

> # Ratio of mean/median year over year...
> data.frame(medians,means,ratio=round(means/medians,2))
      medians means ratio
year1      89   105  1.18
year2      89   145  1.63
year3      89   225  2.53

The above table makes it clear that median pay is constant, while mean pay is rising (which suggests some subset of team members are getting increasingly higher pay).

Again, what all this illustrates is that simplistic rules and heuristics like “Don’t use the mean when the distribution is skewed” are the domain of shallow understanding.

Deeper understanding allows for considering what numbers are truly most relevant for a particular situation.

Level 6 (or 5): Creation (Synthesis)

Activities relating to combining concepts to create superordinate ideas or broader systems.

Shallow understanding could be demonstrated by creating an HR analytics dashboard that includes various summary statistics for relevant variables, including means.

Now, that’s not necessarily shallow. But it doesn’t inherently convey depth, either.

With basic knowledge of key terms and formulas, as well as familiarity with coding or a platform like Tableau, this can be accomplished even by someone who has no idea whether the mean is relevant for the variables and situation at hand.

One example of a task requiring deeper understanding is designing a job applicant scoring system that involves averaging ratings across a panel of interviewers, where it is known that one interviewer gives systematically harsher ratings than the others.

At first blush, you might think that the more sophisticated approach would involve some kind of correction, like a trimmed mean. (In fact, I know a guy with a PhD who once suggested this solution to this exact problem.)

This solution seems deeper, in that it doesn’t rely on canned statistical functions, and it takes into account the influence of skew…

But in a typical selection system, the overarching goal is to rank-order applicants and make job offers to those highest-ranked.

Critically, if a single rater is systematically more severe than other raters, this does NOT change the rank-order of applicants. (Assuming the same panel rates all interviewees.)

This is because systematically lenient or severe ratings are like adding or subtracting a constant (on average) to each score, which “moves them” all in the same way, and therefore has no effect on rank.

Consequently, you don’t need to make any adjustment at all! You can preserve all the information that the panel members provided about the applicants—and therefore avoid inflating average scores—while still getting the same applicant rankings.

Here’s a quick demonstration in R …

> # Creating a dataset of job applicant ratings
> (ratings=data.frame(
+   Candidate=c('Applicant A','Applicant B','Applicant C','Applicant D'),
+   Rater1=c(4,3,5,2),
+   Rater2=c(5,3,4,2),
+   Rater3=c(4,4,5,1),
+   Rater4_harsh=c(2,1,3,0)))
    Candidate Rater1 Rater2 Rater3 Rater4_harsh
1 Applicant A      4      5      4            2
2 Applicant B      3      3      4            1
3 Applicant C      5      4      5            3
4 Applicant D      2      2      1            0
> # Rater 4 is systematically harsher than others
> # Rank-ordering applicants
> rank(rowMeans(ratings[,2:5]))
[1] 3 2 4 1
> # Rankings without the harsh rater
> rank(rowMeans(ratings[,2:4]))
[1] 3 2 4 1
> # The rankings are unchanged...

This shows that corrections like trimming extreme scores (in this case, literally eliminating the harsh rater’s scores from the dataset) have no effect on the actual applicant rankings.

And I can tell you from experience that this is far from obvious to most people, including many who have taken multiple statistics courses.

Now, this is not to say that corrections would never be appropriate. The point is just that choosing the right solution requires deep conceptual understanding of how averages like the arithmetic mean work.

Real-world applications require greater depth of understanding than mere definitions, formulas, and canned code.

All right, that’s all six levels. Hopefully I’ve made it painfully clear that depth of understanding does NOT necessarily rise along with successive levels of Bloom’s Taxonomy.

A reasonable question at this point might be, “Okay, so exactly how deep should we expect a given student’s understanding to extend?”

And I guess the answer must be individualized to the learner, instructor, training curriculum, university, and/or other context.

More generally, How deep does anyone’s understanding need to extend?

As I’ve mentioned previously, it’s relatively easy to identify shallow understanding, as well as to distinguish shallow from deep…

But it’s never clear how deep deep can be. How far down the rabbit hole can you go? Does it ever bottom out?

I don’t know. For now, I’ll keep digging.

Oct. 25, 2026

Part 1 compared shallow and deep understanding for the first two levels of Bloom’s Taxonomy. This post covers the next two levels, again using the mean (average) as the example.

Level 3: Application

Activities relating to using concepts to solve novel problems.

Shallow understanding at this level would involve simply plugging numbers into an algorithm to produce the mean.

For example, I might use the following R code to find the mean for the values (1, 2, 3, 4, 5, 6).

> mean(1:6)
[1] 3.5

Clearly, this shows one can apply some superficial concept of the mean (e.g., “it’s an important number I can get with the mean() function”) to solve novel problems without necessarily having any idea what it actually means.

A contrived problem like this could be made trickier to test deeper understanding.

For example, suppose that a class of 24 students has an average exam score of 85, and a separate class of 31 students has an average score of 82. What’s the average score across the two classes?

Without grasping the underlying concept, the first thought might be to average the averages: (85 + 82)/2 = 83.5.

But this is WRONG.

As mentioned last time, the arithmetic mean can be conceptualized as an equal share per unit. In other words, if we totaled all the scores in the distribution, then cut the total into equal parts, the size of each equal part would be the mean.

But the combined classes do NOT have 2 scores, so that’s the wrong denominator; the combined count is 24 + 31 = 55. We need to account for that in our calculation.

(There’s a quick formula to solve for a weighted mean, but going straight to that from memory would again be missing the point here. That approach could also be employed formulaically, without really understanding what’s going on at a conceptual level.)

If we had all the individual scores, we could just ignore the individual class means and calculate the overall. But since we only have summary data, we’ll have to reverse-engineer it…

The mean for one class is found with M₁ = Σx₁/n₁. That means that we could multiply the mean times the count to get total: Σx₁ = M₁n₁. We can do this for both groups… Σx₁ = 85 ⋅ 24 = 2040 and Σx₂ = 82 ⋅ 31 = 2542.

Now we can find the total sum and the total count for the two groups combined: Combined Mean = (Σx₁ + Σx₂) / (n₁ + n₂) = (2040 + 2542) / (24 + 31) ≈ 83.31.

In fact, the process we just walked through can be formalized as Weighted Mean = (M₁n₁ + M₂n₂) / (n₁ + n₂). This is the quick formula I mentioned previously, but, again, simply memorizing it would bypass the underlying intuition.

If someone genuinely understands the concept of the mean at a deep level, then they could figure this out on their own, without needing to be taught the formula separately.

Level 4: Analysis

Activities relating to breaking concepts down into sub-components.

Shallow understanding, at the most basic level, would involve simply recognizing and grasping the two components of the mean formula:

1. The sum of the values, Σx
2. The number of values, n

It might follow from this that outlying values can drag the mean around, which might constitute deeper learning. But there is also a version of such “understanding” that is merely a memorized rule: The mean is dragged in the direction of outliers.

If someone told me that, I’d be tempted to ask, “Why? What makes it work that way? Does this happen in some lawful way?”

And those are harder questions to answer…

Deeper understanding could come from examining the formula more closely…

The formula Σx/n could be rewritten as x₁/n + x₂/n + … + xₙ/n. This provides an alternative interpretation of the mean as a weighted sum of scores, where each score gets a weight of 1/n (meaning each score counts equally).

As such, each score contributes xᵢ/n to the mean. That is, every score contributes in proportion to its own value (but the influence is scaled by the total number of scores).

This is why every score (including outlying values) “pulls the mean” in its own direction. The further a given score, xᵢ, lies from the other values, the more it changes the mean from what it would be without xᵢ.

To illustrate this, let’s see what would happen if we changed the value of a single score…

Increasing a given score by Δ increases the total sum of the distribution by exactly Δ. Therefore, taking the weight (1/n) into account, the mean must increase by Δ/n.

For example, consider this set of scores: (0, 1, 2, 3, 4). If we increase that first value from 0 to 500 (i.e., Δ = 500), it will increase the total sum by 500, and it will increase the mean by Δ/n = 500/5 = 100.

Here’s a quick demonstration in R…

> # Create a distribution with 5 scores...
> x=c(0,1,2,3,4)
> sum(x)
[1] 10
> mean(x)
[1] 2
> # Increase the first value by 500
> x[1]=x[1]+500
> # Now the sum has increased by 500...
> sum(x)
[1] 510
> # And the mean has increased by 500/5 = 100...
> mean(x)
[1] 102

(We would also see the opposite pull downward if Δ were negative.)

But, by the same token, this also demonstrates how the extent of any single value’s influence is controlled by the number of scores.

In the above example (n = 5), the Δ for that first score changed the mean dramatically. But if we have 50,000 scores, the same increase has almost no effect…

> # Create a distribution with 50k scores...
> set.seed(123)
> x=sample(1:5,50000,T)
> sum(x)
[1] 150075
> mean(x)
[1] 3.0015
> # Increase the first value by 500
> x[1]=x[1]+500
> # Now the sum has increased by 500...
> sum(x)
[1] 150575
> # But the mean has only increased by 500/50000 = .01...
> mean(x)
[1] 3.0115

In this way, all values in the distribution “drag the mean around,” but this influence is most apparent when (1) a given score is further away from the others and (2) there are relatively few datapoints in the distribution.

And this is just one example of how breaking the concept down into component parts could lead to a deeper understanding—in this case through rearranging the formula to get a different intuition.

Okay, once again, this ended up being longer than expected. Final installment tomorrow…

Oct. 24, 2026

In a previous post, I gave examples of how a person could understand the mean (average) across all six levels of Bloom’s Taxonomy.

But I later argued that Bloom only gives us categories of activities relating to knowledge, not necessarily levels of increasing complexity or depth.

To more clearly illustrate what I mean, below are examples of both shallow and deeper understanding at each level of Bloom…

Level 1: Remembering (Knowledge)

Activities relating to identifying and recalling concepts.

Sticking with the example of the mean (average)…

Shallow understanding can be demonstrated by writing out a basic version of the formula, Σx/n, or saying something like, “Add up the parts, then divide by the number of parts.”

While this is accurate, a person could write or utter these words without necessarily being able to clarify what any of it actually means.

Deeper understanding in this level might constitute memorizing a network of concepts within which the mean sits.

For example, you might know that this is only one type of average among many, including…

• Mean
• Median
• Mode

In fact, it’s not even the only mean out there. This category includes…

• Arithmetic mean (what people usually mean by “average”)
• Geometric mean
• Harmonic mean

At this level of Bloom, deeper understanding could include the ability to identify and recall definitions and formulas for each of these and others.

It could also include knowing different or more complex variations of the formula, which type of average is most appropriate for different types of analytic situations, and so on…

Ultimately, however, all these terms, as well as their definitions and formulas, are just words, which are merely labels for concepts.

As such, while it would be difficult to imagine someone having deep understanding of these concepts without knowing the relevant terms, that knowledge alone is still not sufficient to guarantee comprehension.

Imagine you memorized a brief passage in a foreign language by writing and speaking it daily…

At some point, you’d have it down pat. But unless you ever bothered to translate it, you’d never know what any of it meant!

In the same way, a person could memorize every relevant term and definition in a given subject domain until they could recite it all at will, and that still wouldn’t guarantee that any of it meant anything to them.

Level 2: Understanding (Comprehension)

Activities relating to explaining concepts.

Shallow understanding here might be demonstrated by an explanation like, “The mean is a typical or average score in a distribution.”

This is true, but it’s incredibly vague. It seems to overlook the existence of other types of averages (see previous section). There’s also no apparent realization that certain factors (e.g., outliers, distribution shape…) could affect how well it typifies the distribution.

Two common candidate explanations for deeper understanding would be…

1. The mean as the balancing point of the distribution; half the “weight” lies below it, and half lies above it. This is why the sum of deviations from the mean is always zero.
2. The mean as the equal share per unit; if we totaled the scores, then cut the total into equal parts (one for each observation in the distribution), the value of each part would be the mean.

This is usually how I introduce the concept to students.

But we could go even deeper.

Perhaps the deepest (?) level of any mathematical or statistical concept is the algorithmic level.

Here’s the idea: If you truly understand a concept, you should be able to create your own algorithm from scratch that reproduces the idea. (Doesn’t really matter whether it’s unique or not, so long as you generate it yourself.)

One way to take the arithmetic mean to this level of understanding is to realize it’s only one case of a more general process. There are more technical versions of this, but here’s the basic generalization…

Suppose you perform some operation (e.g., addition, multiplication) on a set of values, resulting in a total effect. For example, summing all the values results in a total; multiplying all the values results in a product; et cetera.

A mean is a single value that could replace each score in the distribution without changing the result of that operation.

Consider summation (adding things up)…

If our set of scores is [1, 2, 3, 4], the result of summation would be 1 + 2 + 3 + 4 = 10. The arithmetic mean, Σx/n = 2.5, is the only value that could replace each score in that operation, while still producing the same result: 2.5 + 2. 5 + 2. 5 + 2.5 = 10.

Or consider multiplication…

The product of multiplying all the values in that same distribution is 1 ⋅ 2 ⋅ 3 ⋅ 4 = 24. Again, the geometric mean, ⁿ√(Πx) ≈ 2.2134, is the only value that could replace each score in that operation, while producing the same result: 2.2134 ⋅ 2.2134 ⋅ 2.2134 ⋅ 2.2134 ≈ 24.

What makes this explanation “deeper”?

To me, it’s deeper in that it represents a more general principle about how things work. The mean is not (just) an arbitrary concept that people use by convention; it fits into the bigger picture of how math works.

Based on this, it becomes apparent that there are infinite number of possible means that we could define.

This allows for understanding an entire class of calculations with a single concept. For example, someone who understands the above explanation is in a far better position to next grasp the harmonic mean and various other means.

Viewed through this lens, the arithmetic mean is only one case of a single value that can represent a distribution of scores in a particular context. It’s not some magical statistic that we should always default to when summarizing a set of scores. (This has critical implications for other Bloom levels, like application, creation, etc.).

Another takeaway from this discussion is that deeper understanding can often be acquired by exploring different ways of explaining the same concept.

The more multidimensional your understanding of a concept, the more your mental representation of it begins to transcend any particular set of symbols on a page.

And it doesn’t have to stop here.

Yet another alternative explanation could be through a regression lens:

Using an ordinary least squares approach, the arithmetic mean is the predicted value of the distribution when no additional variables are taken into consideration (i.e., no predictors; intercept-only model).

By “predicted value,” I mean one possible “best guess” for what a randomly selected score from the distribution would be.

And on and on it goes…

Okay, this post is already longer than I expected it to be. More next time…