With Empirical Handicapping schemes, where the handicaps set are based on observations of actual racing, there is the problem of separating the two principal elements which constitute the performance of a boat.
These elements are:
a. the part of performance that can be attributed to the skills of the boat designer, and
b. the part that can be attributed to the skills of the people preparing and racing the boat.
The former we will call the Designer Element (DE) and the latter Crew Skill Factor (CSF).
c. In practice there is a third element which is random variations in the application of CSF.
Personal Handicaps are an estimate of the combined effect of the Designer and Crew elements (with the random element eliminated) and therefore they represent a measure of the normal performance of the competitive unit (boat and crew combined).
Personal Handicaps are not used in conventional handicap racing because in sailing the normal objective of handicaps is to eliminate the differences between the boats (ie the Design Elements) so that the best prepared and raced boat normally wins regardless of the design speed potential of the boat they sail.
Personal Handicaps can be used in racing with the objective that the winner of a race is the crew that most improves on their own previous normal performance, no matter how good, bad or indifferent that was. This can give encouragement to the less skilled crews.
Because of the objective of conventional handicap racing, it is the design element of performance that we wish to assess and handicap. In order to do that we have to eliminate the random element (which can be done by using many observations) and the crew element (which requires judgement).
The principles of this method are based on two assumptions:
1 that Personal Handicaps (PH) for competitive units can be estimated objectively by analysing the finish times in a series of races.
2 that most racers know (even if only intuitively) their position in the pecking order of racing skills within their club, even though different designs of boats are being raced, and that the subjectivity of the individual judgements can be reduced using many mutual assessments.
The method is based on the RYA Portsmouth Yardstick Scheme (PYS) which is the principal empirical handicapping scheme in UK (and has daughter schemes throughout the world from Australia to USA). In PYS the Portsmouth Number (PN) or handicap represents the times (in unspecified but common units) for each boat to sail an unspecified, but common course, when prepared and raced by crews of equal skills.
PYS Corrected times are calculated from elapsed times by dividing the latter by the PN and multiplying by 1000.
This method will transfer to any Time-On-Time handicap scheme – provided the corrected times are calculated either by multiplying or dividing the elapsed times by the handicap (there may be a multiplying constant – as in PYS).
There are four stages – each the subject of a separate section.
1 (Section 2) Estimation of Personal Handicaps and “In-Class-CSFs”. “In-Class-CSF” is the relationship between the performance of each class member and the best in the class.
2 (Section 3) Peer Ranking Evaluation using the opinions of each of the racers as to their positions in the skills pecking order. The objective is to identify racers believed by their peers to be o equal racing skills.
3 (Section 4) "Merging" the “In-Class-CSFs” and peer-ranked-equal pairs to bring the individual class “In-Class-CSFs” on to a common basis. Then to use the commonised “In-Class-CSFs” with the Personal Handicaps to estimate Proto-PNs which are explained in the section.
4 (Section 5) Normalising the Proto-PNs so that the Portsmouth Numbers produced are compatible with the national PNs published in YR2 – the PYS booklet.
Evaluation Periods should be as long as possible so as to incorporate as many races as possible. Every boat should have completed at least five races for their PH to be valid – the more the better.
It is important that an Evaluation Period ends shortly before Yardstick Returns are due at the end of September.
Analysis of races (including level class races) should be done in the order that the races happened and should include level class races. The rest of the process is done around the end of the Evaluation Period.
In-Class-CSFs are calculated after the last race analysis is complete.
Peer Assessment forms should be issued for return to the Yardstick Officer at the time of the last race.
Peer Ranking and the remaining stages are done following the last race analysis and in time to complete the club’s Yardstick Return before the end of September.
The instructions normally require a whole number as the final result. However in order to achieve that accuracy of calculation it is desirable that intermediate calculations be made to two decimal places. This is simple with an electronic calculator and paper and a computer spread sheet will handle it automatically no matter what its display shows.
The method used is based on YR2 Specimen Race Zero which analyses race finish times simply. However there are some refinements.
The prime differences are:
We use the Specimen Race Zero form, and where appropriate the same step numbers.
1 Enter boats and crews in the order in which they finish in columns a, b, c, d (entering the crew in column a) and the elapsed times E in seconds in column g
2 Enter in column e as N the Personal Handicap (PH) calculated from the previous race or if there has been no previous race) the boat’s PN. If this is the beginning of a second or later Evaluation Period then the final PH from the previous period should be used.
3 Enter in column f the PN status of the boat: PY, SY, RN, CN, TN.
4 Using the formula C=E*1000/N calculate and enter the corrected times in column h
5 skip – the order of the corrected times is not relevant to this analysis.
6 Enter in column j the Cs of all boats for which this is the second or later race. (If all the Ns are PNs enter all the Cs)
7 Calculate and enter in
column j below the Cs, the average (ACT) and
standard deviation (SD)* of all the Cs in column j
(* If you do not have the facility to calculate SD skip to
step 9.)
8 Calculate HI = ACT + SD and LO= ACT – SD and enter in column j below SD
9 Bracket all the Cs
in column j with values greater than HI or lower than LO
and calculate the average of the unbracketed Cs.* This is Race
Mean Time (RMT)and is entered below LO
in column j.
*If you do not have the facility to calculate SD, then as long
as you have at least ten boats times, (otherwise RMT = ACT)
bracket the highest one-sixth and lowest one-sixth of the Cs.
10 Using the formula P=E*1000/RMT calculate to the nearest whole number and enter in column l a P for all units. These are the performances of each unit in this race based on the handicap (N) used and RMT.
11 Using the formula PH= (3*N+P)/4 calculate the new PH for each unit and enter in column s. This is used for analysis of the next race in which the unit competes.
That completes the analysis of a single race. All races must be analysed in the order in which they happen using as N the PH from the competitive unit’s immediately previous race. Note that it can be applied to level class races as well as handicap races.
At the end of the Evaluation Period In-Class-CSFs are calculated using the following additional steps are performed using a new form.
13 On a new plain form list all the competitive units (boat & crew), which have completed at least one race, in the order of popularity of class and within class in order of personal handicaps (best – lowest - first)
14 Enter in columns a, b, c, d each competitive unit’s details and in column e its personal handicap PH - all copied from the last race analysis sheet on which the unit appears.
15 Calculate for each unit and enter in column g an In-Class-CSF by dividing the unit’s Personal Handicap (PH) by the PH of the best in class. If this is done correctly the best in class will have an In-Class-CSF of 1 and the others will be greater than 1 and in ascending order.
This form is now ready for the actions described in Section 4
1 Issue each crew with a Peer Evaluation Record Form (sample form at end) which gives full instructions for completion.
The form lists all the competitive units by class stating the class sail number and crew. One boat may appear more than once because she has raced with different crews and similarly one crew may appear more than once if it has sailed more than one boat.
2 Each crew now completes the form by entering against each unit a letter from this table. The letter chosen is that against the statement which they believe most closely represents the relationship between the skills (in preparation and racing) of the crew being evaluated and the crew doing the evaluation. Crews are not required to evaluate crews of which they consider they have insufficient experience. They mark their own unit “D”
The crew marked is normally:
A Very much better than me
B Significantly better than me
C Slightly better than me
D Much the same as me
E Slightly worse than me
F Significantly worse than me
G Very much worse than me
3 When the Peer Evaluation Record Forms are returned to the
Yardstick Officer, he uses an extended version of the same form to record the
evaluations made. He uses one column for each evaluating crew and enters in
place of the evaluations A to G the corresponding numbers 1 to 7.
4 When all the evaluations have been entered the Yardstick Officer averages the numerical evaluations from each crew (averages the rows). This should be done to one decimal place for clubs with up to sixty competing boat units and to two decimal places for bigger clubs. All the averages should lie between 1 and 7. These averages are the Peer Ranking Scores of the crews
5 The Yardstick Officer sorts the Crews and their Peer Ranking Scores into numerical order of the scores. This will bring crews together who are in the opinion of their peers much the same as each other in the skills of preparing and racing their boats.
This information is used with the In-Class-CSFs and Personal Handicaps to calculate Proto-PNs (Section 4)
The objective of this section is first to merge the classes to a common In-Class-CSF base. This is done by considering a base class (usually the most popular, but it might be the class with the largest number of racers judged as good or best) together with each other class in turn (working from the next most popular to the least popular). The In-Class-CSFs in the second class are adjusted to be compatible with those of the base class using the Peer Ranking Scores. These two classes are merged to form a new base class and the process repeated until all boats are merged.
You can use an extended version of the form used for calculating the In-Class-CSFs.
1 Search the ordered Peer Ranking list for pairs of units (one in the base class and one in the second class) considered equal ie having the same Peer Ranking Score.
2 For each pair divide the In-Class-CSF of the unit in the base class (now called Base-Class-CSF) by the In-Class-CSF of the other. If there is more than one pair judged equal, then average the quotients.
3 Multiply all the In-Class-CSFs in the second class by the average quotient and enter the adjusted In-Class-CSFs in column i. These adjusted In-Class-CSFs are now compatible with those of the Base Class. The In-Class-CSFs of the base class should be copied to column h too. Please note that this process may create new Base-Class-CSFs that are less than 1. This is acceptable because the best in one class is not necessarily as good as the best in another class.
4 Treat both classes as the new base class.
5 Repeat the process with the next class, entering the new Base-Class-CSFs in the next column (j…), until all classes are merged into the base class.
It is possible than some classes, or more likely individual boats, will remain unmerged because they have no unit in the base class judged as of equal skills in the Peer Ranking Score list. These units need to be merged by interpolation.
6 Treating an individual boat unit: In the Peer Ranking Score list consider the units that are already in the base class and rank immediately above and below the boat being considered. Calculate the Base-Class-CSF of the unit thus.
B = A+ (C-A) * (Y-X)/(Z-X)
Where:
A, B, C are the Base-Class-CSFs of the unit below, the unit being considered and the unit above,
X, Y, Z are the Peer Ranking Scores of the unit below, the unit being considered and the unit above.
(Note this is the only time that the actual values of the Peer Ranking Scores are used in calculations.)
7 For an unmerged class we first calculate, for all units in the class, a temporary Base-Class-CSF as for an individual unit (see step 6). These temporary Base-Class-CSFs are divided by the corresponding In-Class-CSFs and the resulting quotients are averaged. Final Base-Class-CSFs are calculated by multiplying all the In-Class-CSFs in the class by this average quotient.
Now all units are in the merged Base Class and have compatible Base-Class-CSFs.
8 Divide each unit’s PH by its Base-Class-CSF and enter in column f. Within each class the number should be the same. This is the Proto-PN for the class.
Proto-PNs can be used for club racing provided no boat joins the races that has not been evaluated as above. The ratios between pairs of Proto-PNs, as evaluated, are correct but their absolute individual numerical values are not compatible with PNs from elsewhere, particularly from YR2. This is because, at one stage in the process, we assumed (implicitly) that the best crew in each class had a CSF of 1. This would imply perfection of preparation and racing of their boat – which clearly is not possible.
It is necessary to adjust the absolute values of the Proto-PNs to be compatible with YR2 PNs by normalising.
Proto-PNs have the right relationship one to another but not the right individual values for general use. They do indicate the ratios between the expected performances of pairs of boats, which is what is expected of the published PNs. Normalising adjusts the individual values to fit with the values in YR2.
Within PYS each published PN is assigned a status based on the volume of the data in the RYA database and its consistency. The three statuses relevant are:
PY – Primary Yardstick classes whose PNs are well attested by returns from many clubs over many years
SY – Secondary Yardsticks classes whose PNs are well attested by returns from many clubs over many years but not as well as PY classes. Development classes which have consistent returns are included here and never given Primary status.
RN – Recorded Numbers which have sufficient data to publish a PN but not of quality or quantity to justify SY or PY status.
For normalising we consider only the classes which have the highest status. So, if the club has one or more PY classes we consider them only, if no PY but one or more SY then we use them, and only if there are no PY and SY classes do we use the RN classes.
We will assume the club has PY classes (The procedure is the same if SY or RN classes have to be used.)
1 Divide the PN of each PY class by the corresponding Proto-PN and enter in column g. Average the quotients. The average is the normalising factor.
2 Multiply all the Proto-PNs (regardless of status) by the normalising factor and enter in column h.
The result of this final adjustment is Portsmouth Numbers which are compatible with the PNs published in YR2 and can be used for racing as a Club Number (CN) and equally importantly can be entered on Yardstick Return forms as Recommended Numbers..
We have used an objective method to calculate Personal Handicaps (eliminating the random element present in each race) and the relationships within each class (In-Class-CSFs).
We have utilised many subjective mutual evaluations of relative skills to minimise the subjectivity of the Peer Ranking Scores.
We have used equal Peer Ranking Scores to adjust the relationships within each class to be compatible with the other classes.
Using the compatible relationships
we have calculated Proto-PNs and adjusted them by normalising
to be compatible with PNs from outwith the club – in
particular the PNs published by RYA in YR2.
XYZ SC - Crew Relative Performance Grading Form -200X
last race Crew Class SailNo grade last race Crew Class SailNo grade
Please enter, in the grade box, your estimate of the normal relative performance of that crew to you.
Please ignore the speed potential of the boats you sail - what we want is the relationships between how well they prepare, tune and sail their boats with how well you prepare, tune and sail your boat.
People and boats appear more than once because of movement between boats - relative performance may not be the same in a different boat!
Relationship Grades
A - Very much better than me
B - Significantly better than me
C - A bit better than me
D - About the same as me
E - A bit worse than me
F - Significantly worse than me
G - Very much worse than me
Please mark your own entry with an "D"
If you have insufficient experience of a crew, then leave that grade box blank.
Please correct any spelling mistakes!