Friday, 16 September 2016

MODEST IMPROVEMENT

Back in February, a season preamble post noted the tipping calculations were altered with the aim of increasing accuracy on three measures.

And with a few weeks to play out of the 2016 season, now is as good a time as any to just check back and see if there were any improvements.

The three key measures targeted were for improvements in
- match results tipping accuracy,
- tipped margin error reduction, and
- improved Monash competition score.

In all three measures, the new system has delivered improvement.
THREE MEASURES COMPARISON
MEASUREOLDNEWΔΔ%
CORRECT TIPS14014110.7%
MAE (Margin Error)30.630.4-0.2-0.7%
MONASH SCORE17561773171.0%
MONASH POSITION17th14th3

A small improvement, but forward progression none the less.
Yes the red negatives for MAE indicate a better performance as you want to be as close to zero as possible with this measure.
Also added in is the Monash Position data, which is not critical, and hence shaded grey.

Reducing MAE is also linked to improving the Monash score, as the Monash score is built around margin accuracy, with points awarded to margin differentials separated into brackets.

Comparing those brackets under the two systems yields some improvements as well.
MARGIN BRACKET COUNTS
01-67-1213-1819-2425-3031+
OLD1272326131993
NEW3272126151892
Δ2--2-2-1-1

With a prima facie look at that table, you could make a case for the two more 'exact' tipped games as having come from the immediate 2 brackets above the zero point, as well as the 2 more results in the 19-24 point bracket as having come from the 25-30 and 31+ point brackets.

It will need further investigation to determine exactly if that's the case. Or maybe its just best left alone.


Finally, the changes made to the algorithm also impacts a teams rating, and with the multi-step taper employed under the new system, the changes had different impacts at different stages of the year.
But as the table below shows, over the space of the year (to date), these were minimal once accumulated.
CURRENT (NEW) SYSTEMOLD SYSTEM
RANKINGSvs Old SystemRANKINGS
TEAMPTSΔPTSΔRKTEAMPTS
1Adelaide14105+11Sydney/SMFC1407
2Sydney/SMFC14092-12Adelaide1405
3GWS13914-3GWS1387
4Geelong13263+24Hawthorn1327
5W Bulldogs/FFC13250-5W Bulldogs/FFC1325
6Hawthorn1323-4-26Geelong1323
7West Coast12083-7West Coast1205
8Port Adelaide1182-4-8Port Adelaide1186
9North Melbourne11751-9North Melbourne1174
10Collingwood1096-3-10Collingwood1099
11Melbourne10223-11Melbourne1020
12St Kilda10173-12St Kilda1015
13Richmond972-6-13Richmond977
14Fremantle902-4-14Fremantle906
15Carlton8861-15Carlton885
16Gold Coast8563-16Gold Coast854
17Essendon693-3-17Essendon696
18Brisbane Lions629-1-18Brisbane Lions630


The differences above are really quite minor, but yet yielded a 1% improvement.
Clearly a new and perhaps more radically modified algorithm will need to be developed.

No comments:

Post a Comment