Brass Band Index - Introduction & Methodology
In 2015 I began implementing a rating system for Brass Bands. Brass Bands have a tradition of competing throughout the year in various competitions and so this provides a good opportunity for developing a ranking system.
A few variations of ranking systems have been developed for Brass Bands over time, with the most popular being the 4BarsRest system which has it's methodology outlined here.
Whilst this is a popular system and one that seems to work intuitively it suffers from the same problems as all of the ranking systems that I have seen thus far. This problem is that points are accrued over time and so it hugely favors bands that compete often. This is the main issue that the BBI (brass band index) overcomes.
Brass Band Index – Methodology
The method used to is based on the classic Elo Rating method originally designed for chess (first implemented in 1960) and is still used (although slightly modified) to this day.
You can read about the Elo method on it’s wiki page here.
Each band has a rating and when it competes against another band there is an expected outcome based upon the difference between the two bands ratings.
After a contest, the winning band will be given some of the losing bands points. The number of points transferred is based upon how likely or unlikely that result was. Thus, if a band with a low rating beats a bands with a high rating then they will gain a lot of points. This means that over time, the rating of each band will converge towards its “true rating”.
Initial Rating Values
For new bands, we have nothing to base their rating value on. Over time the rating will converge towards the ‘true rating’, however it is beneficial to start with a good estimate in order to obtain their true rating faster.
When new bands are entered into a contest they are given the average rating of the bands within that contest. Originally, all new bands were given a rating of 1000 but this new method has provided a small increase in accuracy.
K-Value
The amount of points that are transferred from a ‘losing’ band to a ‘winning’ band is a number between 0 and K based upon how unlikely that result was. The choice of K affects how quickly a bands rating can change and both small and large k values have advantages and disadvantages.
A large K-value means that the ratings can ‘keep up’ with bands that are quickly improving (or getting worse). However it also means that the ratings are more susceptible to freak results (which can happen a lot in musical competition!).
A number of k-values have been tested and a value which provides the most predictive power is used. K is currently set at 24 at the time of writing.
Adjustment for Contest Draw
An interesting part of optimising the Brass Band Index rating system has been investigating the effect that draw position seems to have on results.
Almost 20,000 individual band contest results were analysed for their position within the draw (as a %, where 0% is drawn first and 100% is drawn last). A significant correlation was found between draw order and the bands chance of finishing in the top half of the results.
There is approximately a 54% chance of a top half position for bands drawn last and 44% chance for bands drawn first, with a roughly linear relationship.
This change is approximately equal to a 45 rating point disadvantage if you are drawn first and up to a 25 rating point advantage if you are drawn last. This is slightly less than, but comparable to, the home/away advantage seen in football, which is usually considered to be around 90 Elo points.
This effect has been taken into account when calculating ratings and gives a slight improvement in predictive power.
Comments