"Stearns Method" Critique

The following critique illustrates a methodology for preliminary evaluation of a proposed method for PWD racing. I have performed this evaluation manually on one example of the Stearns Method - 28 Car Schedule as published on the web on 1/20/96. (The site has since been "reorganized" and the referenced page sent into oblivion.) This 13 round chart is based on an estimated 1 hour racing time. This is not a complete evaluation, involving simulation of many repititions of chart application, and is, therefore, subject to possible error in the preliminary evaluation method.

Criteria

Evaluation Criterion 1. "Is it fun?"

Does everyone get lots of racing in a reasonably short period of time?

Evaluation Criterion 2. "Is it fair?"

Can the method can be applied in a fair manner, i.e. in a way that no factors other than the Cub, the Car, and random chance affect the outcome of the races?

Here are some aspects of potential unfairness that can be avoided easily:

Chance: Cubs are assigned racer identification numbers (chart numbers) in other than a purely random fashion, e.g. by assignment "in order" at sign-in,
Judging: Heat finishes are judged in an inequitable manner,
Manipulation: External influences on the race, e.g. the starter ocassionally adjusts cars on the starting line,
Auditability: Scoring is not auditable, e.g. not done "in the open" for all to see.

Evaluation Criterion 3. "Is it accurate?"

Some sources of inaccuracy in PWD racing:

Competition assignment: Racing against a "different" distribution of the competition.
Lane assignment: Racing in the "fast lane" or in the "slow lane" more frequently than the competition.
Elimination resulting from "unfortunate" pairing.
Elimination resulting from "unfortunate" lane assignment.

None of these are "fairness" issues. They are measures of the racing environment into which the individual Cub may be inserted "by pure chance."

So how does this example of Stearns Method "stack up"?

Fun:

Yes!

Stearns Method has few peers in this measure. The method adapts readily to maximize use of the lanes and time available to provide racing for all the Cubs present.

Fairness:

Can be run fairly.

Chance: This can be done easily. For instance, have the Cubs draw their race chart numbers "from a hat."
Judging: Select finish line judges that have no known reason for bias. Have the finish line visible to as many people as possible.
Manipulation: Require (and enforce) that only the Cub may touch his car during the conduct of the race.
Auditability: Keep score on a paper chart or overhead projection transparency so that everyone can see what heat place was assigned to each car and what points were assigned.

All of these can be accomplished rather easily for Stearns Method Charts.

Accuracy:

Improvable.

Competition assignment: In the case in point, each car races in 13 heats. In each of those heats the car is matched against 3 other cars. Thus, in the course of his 13 heats, each contestant will be matched against 39 others. The best that can be accomplished is that he will race 15 competitors once and the other 12 competitiors twice, and no competitor more than twice. (That the underlying mathematics will allow construction of a chart in which this is true is a conjecture which I believe to be true.)

In the example, on average, each car faced 1.85 (nearly 2) other cars 3 times, and 2 cars (#3 and #11) actually raced against each other 4 times! To the extent that these can be avoided, the Method, as implemented, is improvable.

Anomolies such as these contribute to inaccuracy. They are "glaring" when "critical" cars are involved.

For instance, suppose that the cars #3, #11, and #10 (in that order) are objectively the three fastest cars in the competition. Further assume that the lanes are equal.

Observed pairings: #3-#11 (4 times) #3-#10 (one time) and #10-#11 (one time).

The scoring method suggested in the documentation accompanying the software gives 1 point for finishing first. An alternative awards points awarded 4-3-2-1 for placing 1-2-3-4. Assuming that heat results conform to the objective relationships between the cars, Car #3 will amass 52 points (13 1st place finishes), Car #11 will amass 48 points (9 1st place and 4 2nd place finishes, and Car #10 will total 50 points (11 first place and 2 2nd place finishes.

In this example, by either scoring procedure, this Stearns Method chart determines that second place be awarded to the 3rd (objectively) fastest car. The effect could be observed anywhere in the results... it is simply most glaring in a case like this.
Lane assignment: The 13 Heats spread over 4 lanes will average 3.25 races per lane. The best that can be accomplished in this case is for each Car to race 4 times in one lane and 3 times in each of the other lanes.

What is actually shown in the referenced chart is that after that each car races at least once in each lane, the program seems to quit trying to provide lane equity. 7 Cars raced once in a line; 21 cars raced twice in a lane, 11 Cars raced 5 times in a lane and 5 cars raced 6 times in a lane.

How would you like your car to race nearly half its races in the slowest lane on the track?

It is fair if everyone had the same chance to draw that chart number. It is inaccurate because it probably causes your car to place much worse than it should.
Elimination resulting from "unfortunate" pairing -- Not an issue with Stearns Method.
Elimination resulting from "unfortunate" lane assignment -- Not an issue with Stearns Method.

The point is not that Stearns Method is a poor way to run a Pinewood Derby Competition. It is, in fact, one of the best and most fun ways to run a PWD to be found in the literature.

Rather, the point is that the method is improvable.

In its defense, minimizing pairing and lane assignment anomolies is not a part of the Stearns Method design criteria (at least as described in the literature which accompanied the software.) The stated criteria are (1) racing at least once in each lane and (2) racing against as many other opponents as possible. Criterion 1 is important if each car will race a number of times approximately equal to the number of lanes on the track. Criterion 2 is important if there are fewer possible pairings than actual opponents.

Challenge:

My challenge to the "world class game theorists" and "computer hacks" is this:

Devise a program which will balance lane assignments and opponent assignments within theoretical limits, while retaining the scheduling flexibility for which the Stearns Method has become renouned.

Watch out... there are some "critical values" which will require all your wits and resourcefulness to overcome.

Entries