Glicko2 Ratings

Announcements and discussion about community-run events.
Post Reply
TTTPPP
Posts: 100
Joined: Wed Sep 05, 2018 8:25 pm

Glicko2 Ratings

Post by TTTPPP »

To seed players for RAGL S11 we combined the history of games played in RAGL and on the Ladder to create a data set of about 3400 1v1 games covering almost five years.

We used the Glicko2 algorithm with a one week period and the default parameters to process these and to assign a rating for each player. As with the existing OpenRA Ladder we rank the players using three standard deviations below their expected ranking (i.e. LadderRating = GlickoRating - 3 * RD). This ensures that a new player who has some lucky wins is unlikely to be above a perennial player of higher quality. The Glicko2 algorithm increases the RD (or error) for a player's rating if they are inactive for a long time - this results in a slow decline in the ladder position of inactive players.

Overall the results seem pretty reasonable, but it's worth bearing in mind that there are a few limitations of this process.

Firstly the RAGL data was really hard to clean. Occasionally it was very hard to tell who played in a game (the in-game names and player abbreviations are frequently misleading). Additionally there are replay files which do no contain information about who actually won the game, and so these games were excluded from the data set. More recent results are much more accurate due to the use of authenticated forum accounts.

Another limitation is that the RAGL divisions act as cliques in the data set. The victor of the first game in Recruits S01 was awarded the same rating as the victor of the first game in Masters S01. This effect broadly disappears as time goes on, since a large number of the players are consistent between seasons and some players get promoted/relegated. More recently the OpenRA Ladder has meant that players from all divisions have played against each other, and this has pretty much removed the issue.

More generally any attempt to represent a player with a single number will have flaws. The rating system might determine that player A is worse than player B, but their playstyles may result in B winning most of the head-to-heads.

With those caveats out of the way then here is the full table from just before the start of RAGL S11.

Code: Select all

Pos  Player           Rating Pld  Won
  1, i like men      , 1886,  96,  84
  2, goat            , 1866, 101,  90
  3, sup             , 1864,  95,  74
  4, Kav             , 1847, 211, 184
  5, morkel          , 1708, 261, 200
  6, Clockwork       , 1689, 179, 133
  7, Lorrydriver     , 1688,  61,  56
  8, Punsho          , 1675, 178, 116
  9, ZxGanon         , 1663, 131, 100
 10, dang_shot       , 1606, 152, 109
 11, Mr Cloudy       , 1559, 330, 214
 12, barf_openra     , 1551, 147, 104
 13, WhoCares        , 1534, 284, 169
 14, Pinkthoth       , 1531, 196, 145
 15, SinJul          , 1526, 266, 174
 16, JuiceBox        , 1523,  32,  25
 17, bµg             , 1514, 486, 263
 18, OMnom           , 1508,  34,  32
 19, Orb             , 1502, 168, 115
 20, Dassault        , 1499,  29,  22
 21, AMHOL           , 1489, 111,  66
 22, Death-Sentence  , 1477,  15,  12
 23, Blackened       , 1475, 131,  76
 24, Misery          , 1471,  12,  11
 25, Goremented      , 1467, 718, 379
 26, Sotsch          , 1464,  39,  30
 27, KSK_Nico        , 1453, 139,  82
 28, Widow           , 1452, 176, 105
 29, Smitty          , 1444,  84,  58
 30, eskimo          , 1436,  72,  51
 31, JustSomeGuy     , 1434,  42,  25
 32, kazu.           , 1434,  88,  70
 33, donb            , 1427, 112,  67
 34, anjew           , 1425, 209,  86
 35, Yara            , 1409,  30,  28
 36, Eugenator       , 1403, 275, 158
 37, Upps            , 1395, 190,  78
 38, Medium Tank     , 1379,  40,  31
 39, Iaigel          , 1377,  19,  13
 40, Antarctica      , 1375,  46,  36
 41, TermX           , 1372, 156,  71
 42, .1              , 1369, 128,  69
 43, f0rk            , 1369, 276, 122
 44, Snizzle         , 1340,  27,  14
 45, biasao          , 1325,  50,  28
 46, Gatekeeper      , 1320,  92,  52
 47, FiveAces        , 1320, 203, 103
 48, weeman          , 1289,  21,  18
 49, maceman         , 1289, 207, 104
 50, toiletbreakbrb  , 1280, 166,  80
 51, bullE           , 1274,  23,  17
 52, 8.9             , 1267,  74,  26
 53, OzzyOuzo        , 1261,  32,  26
 54, maths           , 1257,  74,  42
 55, jvk             , 1254,  18,  16
 56, SoScared        , 1243,  65,  40
 57, Sigil           , 1236,  64,  34
 58, JackoDerp       , 1231,  45,  20
 59, WaRRyKeR        , 1230,  42,  27
 60, Polaris         , 1229,  11,   9
 61, TiTo            , 1229, 467, 150
 62, luckyluke       , 1224, 164,  71
 63, ROCKhardFISTnips, 1224,  48,  24
 64, Kaution         , 1224, 159,  43
 65, Metalcallous    , 1219,  22,  16
 66, Gajcus          , 1218,  21,  17
 67, FRenzy          , 1214,  14,  12
 68, Ekanim          , 1211, 121,  47
 69, Dodder          , 1207, 192,  73
 70, Digital Jihad   , 1202,  18,  14
 71, Jur             , 1194, 107,  58
 72, Doomsday        , 1186,  44,  25
 73, Mo              , 1184,  87,  35
 74, Dying Fetus     , 1179,  18,  13
 75, Raishiwi        , 1173,  24,  14
 76, netnazgul       , 1171, 127,  54
 77, kyrylo          , 1168,  14,  12
 78, BigBadBain      , 1161,   4,   4
 79, mechANIC        , 1152,  89,  54
 80, BigSmoke        , 1151,  29,  19
 81, Mentats         , 1140,  32,  18
 82, Kernel Panic    , 1137, 147,  56
 83, Dualwarhead     , 1131,  42,  27
 84, poop            , 1128,  62,  19
 85, Han             , 1124,  60,  34
 86, Lord of War     , 1121,  18,  11
 87, klaas           , 1121,  44,  20
 88, fris            , 1121,  70,  29
 89, DVoid           , 1115,  24,  14
 90, Steffen123      , 1111,   3,   3
 91, Chewey          , 1106,  81,  40
 92, Abarrat         , 1101,  18,  12
 93, Murto the Ray   , 1094,  15,  11
 94, Duke Bones      , 1093,  80,  34
 95, Stinky Silo     , 1092,  10,   5
 96, vvoj            , 1085,  16,  10
 97, y2?             , 1080,  16,   8
 98, Cymbal          , 1068,  10,   9
 99, t               , 1064,  67,  32
100, Mr_Biro         , 1063,   6,   4
101, Parsely         , 1061,   5,   3
102, SarahSicaria    , 1055,   3,   3
103, Flecken         , 1046,  18,  11
104, Anykeyich       , 1036,  32,  14
105, jreed           , 1026,  12,   8
106, guigz           , 1024,   4,   3
107, MicroBit        , 1016,  21,  13
108, PieChild        , 1015,  48,  23
109, yellow (theRaffy, 1014,  28,  11
110, noobmapmaker    , 1013,  71,  26
111, SeaCucumber     , 1013,  12,   9
112, abcdefg30       , 1012,  67,  30
113, lucassss        , 1011,  48,  19
114, doggo the pupper, 1010,   2,   2
115, lundiz          , 1004,  48,  23
116, KOYK_GR         , 1002,  25,   8
117, quarksplitter   ,  997,  22,   6
118, _Bo             ,  995,  26,  11
119, avro            ,  987,   2,   2
120, Bamsi           ,  982,  16,   8
121, Odourless       ,  982,   7,   4
122, s3kit0          ,  981,   2,   2
123, Printer         ,  975,  12,   5
124, Hermann G'      ,  975,  15,   9
125, Craig Christ    ,  970,  30,  10
126, porenut         ,  961,  24,   9
127, blood           ,  961,  19,   9
128, LukaMile        ,  957,  17,   7
129, At0mic          ,  956,   6,   3
130, SirCake         ,  955,  99,  42
131, Zaqzorn         ,  951,  95,  42
132, TTTPPP          ,  949,  80,  26
133, Ohmk Torque     ,  935,  20,   6
134, jjkramok        ,  927,  44,  14
135, SlickSloth      ,  925,  29,  10
136, worldpeace      ,  918,  50,  11
137, Fahrrad         ,  915, 146,  30
138, 3.Lucian        ,  913,  21,   9
139, Rusty           ,  901,   3,   2
140, dragunoff       ,  898,  86,  14
141, GoldenHippo     ,  887,  14,   7
142, manta151        ,  879,   2,   2
143, PrinzPi         ,  862,  14,   5
144, Jamesasthenia   ,  860,  10,   4
145, Fenix           ,  859,  16,   7
146, spetsnaz84      ,  857, 151,  32
147, asam            ,  856,  11,   4
148, Handel          ,  855,   8,   3
149, Wippie          ,  854,  32,  17
150, iljaco          ,  848,  12,   4
151, Tomminho        ,  846,   6,   3
152, Aggr3ss1vePanda ,  843,   4,   3
153, NethIafin       ,  841,  22,  10
154, xy              ,  838,  12,   6
155, Comrat          ,  819,  16,   4
156, raymundo        ,  816,  23,   8
157, Fruttielicious  ,  810,  14,   5
158, Daz             ,  809,  18,   6
159, IronScion       ,  805,  19,   3
160, Henry_the_Slav  ,  803,  20,   7
161, Cupko           ,  803,   1,   1
162, Michigan        ,  801,   2,   1
163, LaKulta         ,  801,  10,   3
164, Tailix Killa Men,  800, 111,  24
165, PallyRoy        ,  800,   9,   6
166, EUR5,-          ,  799,   1,   1
167, Insomnic        ,  793,   4,   2
168, Declan          ,  792,   8,   3
169, tompollon       ,  791,   1,   1
170, Batfire         ,  790,   1,   1
171, SnidlWhutz      ,  788,   1,   1
172, lonestar        ,  785,   5,   5
173, Nilhall         ,  784,  64,  16
174, corrode         ,  783,  20,   6
175, Underware       ,  781,   1,   1
176, neveragain      ,  780,   4,   2
177, Oafa            ,  778,  12,   4
178, Sisna22         ,  777,   1,   1
179, wismar007       ,  776,   4,   0
180, std::nullptr_t  ,  776,   5,   2
181, Commander_Konka ,  774,   1,   1
182, Kalma           ,  771,   1,   1
183, Testosterone Rex,  770,  44,  11
184, Greasyudders    ,  769,  14,   5
185, DrLuke          ,  758,   5,   3
186, NumericPad      ,  756,   5,   2
187, GDave           ,  754,   2,   2
188, Kuba            ,  749,  52,   9
189, ass_pounder_4000,  747,  18,   4
190, Kingster        ,  744,   3,   1
191, Icesinithol     ,  740,  25,   7
192, merimursu       ,  739,  24,   6
193, Strelkov        ,  737,  14,   5
194, Super_Newbie    ,  731,   3,   1
195, DomoTheRussian  ,  730,   5,   2
196, Bibodo          ,  726,   3,   1
197, AmericanBlunt_  ,  719,  10,   3
198, copla201        ,  718,  48,   5
199, Nukin'          ,  715,  14,   2
200, porcukor        ,  710,   6,   1
201, MR C            ,  707, 110,  10
202, hexagon         ,  705,   6,   0
203, Chrono          ,  701,   3,   1
204, BlueJay         ,  701,   6,   2
205, Aaron_Lloyd     ,  699,  12,   4
206, Chin0           ,  699,   6,   3
207, Mees            ,  694,   5,   2
208, DaniwA          ,  693,   4,   1
209, Federico        ,  683,  10,   1
210, Kallen          ,  682,   4,   2
211, NEGAN           ,  677,   5,   1
212, crash_dash      ,  676,   8,   3
213, Tommo_86        ,  674,   2,   1
214, Xahid           ,  674,   2,   0
215, hamb            ,  673,  48,  13
216, TasteOfIndia    ,  672,   3,   0
217, jkalmar         ,  671,  21,   3
218, beuk            ,  664,  26,   6
219, Eleiko          ,  656,   2,   0
220, Daniel          ,  654,   3,   0
221, shavkat         ,  651,   6,   0
222, Mint Spokesman  ,  644,  20,   6
223, bete            ,  629,  13,   1
224, gokserg         ,  626,   2,   0
225, skarockoi       ,  623,   2,   0
226, zDig            ,  613,  10,   1
227, Therapist       ,  610,   4,   1
228, Dr. Schmock     ,  604,   7,   1
229, N/a             ,  603,   7,   0
230, wtigga          ,  602,   1,   0
231, stoutfire       ,  601,   3,   0
232, stereofield_    ,  601,   2,   0
233, Green Giant     ,  600,   4,   0
234, Antares         ,  598,  20,   3
235, SunTzu          ,  597,   1,   0
236, igorbushko      ,  597,   1,   0
237, Bobuc?<         ,  596,   2,   0
238, King Curtis     ,  596,   1,   0
239, 4mfie           ,  596,   1,   0
240, flaker          ,  595,   1,   0
241, maeshi          ,  594,   1,   0
242, Timber          ,  587,   3,   0
243, johnnyboy       ,  582,   2,   0
244, Teew            ,  582,   2,   0
245, jirik           ,  581,  18,   1
246, bobbyd          ,  581,  10,   0
247, PeterRusselClark,  576,   1,   0
248, Ballistic       ,  574,   1,   0
249, nvrnight        ,  568,   1,   0
250, kwibus          ,  564,   1,   0
251, Astolfo_Chan    ,  563,   1,   0
252, x3oo            ,  560,  10,   0
253, Elon_musk       ,  559,   1,   0
254, zx5             ,  557,   5,   0
255, Kasajohn        ,  556,   1,   0
256, R.Sarah.L       ,  556,   1,   0
257, Oaked.ru        ,  556,   4,   1
258, akpella         ,  554,   1,   0
259, valdurs         ,  549,   1,   0
260, Major Kusanagi M,  539,   5,   0
261, Chaosgrunt22    ,  534,  12,   2
262, General Zico    ,  533,   1,   0
263, CottonTheButton ,  530,   4,   0
264, Skillless       ,  530,   6,   0
265, Tuborgnamozgu   ,  528,   8,   0
266, jaymanxyz       ,  527,  34,   6
267, Biomecaman      ,  526,   4,   0
268, kalessin        ,  524,   1,   0
269, Kadirhan        ,  523,   1,   0
270, Freki           ,  514,   1,   1
271, MegaTank        ,  514,   1,   1
272, pgrwoo          ,  513,   1,   0
273, timl            ,  512,   3,   0
274, ZorN230         ,  509,   1,   0
275, xiaoRen         ,  504,   5,   1
276, MOLLMANIA       ,  499,   2,   0
277, fragment        ,  498,   4,   1
278, Onexygor        ,  496,   1,   0
279, noniXXnarreZZ   ,  492,   3,   0
280, Rimmel          ,  491,   1,   0
281, Beebz           ,  490,   3,   0
282, shpooky         ,  487,   2,   2
283, DodoBirb        ,  482,  16,   1
284, hmma            ,  481,   1,   0
285, Borgmo          ,  474,   1,   0
286, LordMani        ,  470,   8,   0
287, chaircollector  ,  469,   2,   0
288, ActionMan       ,  467,   2,   0
289, doppiogaming    ,  467,   1,   0
290, ISeeMangos      ,  466,   2,   0
291, Elmir           ,  466,   1,   0
292, Drixiart        ,  465,   1,   0
293, Mad_Max         ,  463,   1,   0
294, ThunderStruck115,  460,   7,   0
295, Socan           ,  460,   2,   0
296, alden_the_23rd  ,  457,   1,   0
297, snotty otty     ,  454,   2,   0
298, boncis          ,  453,   1,   0
299, potato          ,  452,  47,   6
300, CatMan3e3       ,  451,   1,   0
301, KlimaOpa        ,  449,   1,   0
302, Zhall           ,  446,   1,   0
303, DoDoCat         ,  445,  26,   0
304, squidgeny       ,  444,  18,   0
305, daniel40392     ,  443,   1,   0
306, ioverthoughtthis,  443,  32,   3
307, feelynx         ,  438,   1,   0
308, JoanotColom     ,  437,   1,   0
309, Card Ninja      ,  437,   1,   0
310, zoidyberg       ,  434,   2,   0
311, Amit            ,  423,   7,   0
312, fittsaft        ,  416,   1,   0
313, DaolDnaKcol     ,  414,  17,   0
314, Strateeg        ,  413,   8,   0
315, Ikke            ,  410,   2,   0
316, matts_mum       ,  410,   2,   0
317, Balloon         ,  410,  18,   3
318, MrZuerg         ,  403,   4,   0
319, vcxz09876       ,  403,   1,   0
320, Orangenaffe42   ,  402,   1,   0
321, Submarine       ,  401,   4,   0
322, Zimmer          ,  400,   4,   0
323, Karina          ,  392,   7,   0
324, bone|superr0x   ,  388,   6,   0
325, Pikica          ,  383,   1,   0
326, McGriff         ,  381,   2,   0
327, Julle           ,  371,   4,   0
328, manu200         ,  367,  11,   1
329, flamewh33l      ,  366,  28,   2
330, gargraval       ,  361,   1,   0
331, 0x0             ,  361,   1,   0
332, WillVaz         ,  356,   1,   0
333, Barxx           ,  352,   2,   0
334, Snake           ,  346,   9,   0
335, malboo          ,  344,  12,   2
336, Follicle        ,  343,   6,   0
337, Spy             ,  338,   4,   0
338, Fundahl         ,  338,   2,   0
339, FasTSlacK       ,  334,  10,   0
340, disruptor       ,  331,   1,   0
341, Snufflegrunt    ,  331,  16,   0
342, Fin             ,  304,   2,   0
343, Beef            ,  299,   8,   0
344, Karthus         ,  293,   2,   0
345, General         ,  293,  14,   0
346, Petit Dejeuner  ,  291,   6,   0
347, KillaB          ,  283,   4,   0
348, ConcaveCats     ,  235,  14,   0
349, chrisf          ,  217,  18,   0
350, chouchani       ,  200,   1,   0
351, boriken1171     ,  182,   1,   0
352, Sneeky-Bandit   ,  117,   1,   0
353, HeejinSon       ,   89,   1,   0
354, 9H0DxF          ,  -99,   1,   0
To get an idea of how the rating system works then I drew a graph showing the ratings for the winners of every season of RAGL and also those players who have been recently top of the rankings for some time.
topPlayersOverTime.png
topPlayersOverTime.png (67.95 KiB) Viewed 497 times
A few things stood out for me. Firstly FiveAces' line gives a pretty good indication of where each season of RAGL happened. This is because he's played in every season and has barely touched the ladder.

Secondly Barf has never been top of the table, despite winning RAGL Seasons 5 and 6. Lorry's performance in the earlier seasons was so convincing that he continued to top the table without playing any games.

Post Reply