New Integrated Rating System For All NATF Sanction Tournaments & Leagues A Deeper Dive of the Collins Rating System
Toronto, ON: May 29, 2019, to follow up on our announcement yesterday regarding the Collins Rating System, we wanted to dive a bit deeper into the algorithm and give some context to the system in order to better explain the reasoning behind some of the rankings moving forward.
First, this new system is not a living breathing entity yet. We debated pushing back the entire release until it was fully baked-out on AxeScores, but with a sense of urgency already present around it, we decided to release this list as it fits with the participation of all players' careers thus far up until May 17, 2019.
The live system will be rolling out in the next few weeks on AxeScores and will be a searchable, self-updating piece of software related to the active matches happening across the NATF at all times.
Doubled Accounts Our team has been working since AxeScores began to merge all duplicate accounts. We continue to focus on this and acknowledge that some of these are still appearing in this list. If you are one of the people that sees themselves duplicated, please reach out to us so that we can remedy that matter more quickly. Be aware that if you use a different email in a new season, that will create a new account as your email is your UserID#. If you need to change the email on your main account to keep your records, please email us to make that change for you so all records can remain on a single account.
What we have also learned is that a few people have deliberately started second accounts and have been actively trying to keep them separate so that one account in one league doesn't drag down their stats in another. We will be tracking these accounts down over time and merging them, so be ready for that if you are one of those people.
Ideation Behind The System We based the Collins Rating System on the ELO system; a very well known mathematical system used in chess to track career performance. We then modified it and added or adapted additional characteristics to turn it into a system that works for the sport of axe throwing.
We had debated having the rating system exist in a rolling, 12-month long period, similar to tennis. In a system like this, a player would simply have any points they won 13 months prior disappear as their year progressed to be replaced by the points (or lack thereof) they won in their current season. The downside of that system is that it causes drastic shifts in ratings as each month passes and penalizes players for skipping seasons or tournaments.
In a rolling, tournament circuit-based system, every win rewards a player points of a set value based on the rating of the tournament, regardless of the opponent. So those that play more, have more points. This is another part of the system that we didn't like; we believe that simply throwing MORE, does not make a thrower the BEST, and we are always trying to be mindful of how axe throwing businesses need to operate and the flexibility our throwers need in order to be able to compete while still taking care of the other responsibilities in their life.
We're still evaluating the appropriate timeline for the inactivity of an account before being removed from the list. Furthermore, all of our throwers are a historical list including every axe thrower with more than one season recorded in AxeScores. Since all of their data informs the system, we felt that it was important to include them in this first pass at the rankings.
How the Ratings are Calculated A player's Collins Rating is a distillation of their career's performance. It's worth noting that this algorithm is unbiased; It is a mathematical equation based on the Collins rating of the competing players in a given match with a multiplier for higher stakes matches. It only cares for the caliber of opponent played versus the amount of rounds won or lost against that player. That caliber is determined by a few factors:
Every player's career starts at a score of 1450. Based on their wins and losses, this number will go up and down after each match played.
Since new throwers receive only a hypothetical ranking when starting out in a league, their first 28 opponents across any and all leagues they are participating in will not see any change in their own ranking during matches played against the said rookie. This is to prevent any drastic swing in ratings while the algorithm determines the proper number for a rookie's Collins Rating.
The system doesn't simply award points for playing/beating anyone at any time. It instead applies a very complicated equation that awards a varying amount of points for different levels of victories and the swing factor in between.
The amount of points awarded for a victory are based on rounds won and the opponents rank.
If you beat someone with a higher rating than yourself, you will be awarded more points than they would if they were to beat you.
If you were to beat someone rated much lower than you, you may receive very few points or, if that person rated far, far lower, you may not receive any points at all.
If there is an extreme discrepancy between rankings, such as a lower rated player winning a single round of a match against a much, much higher rated player, this may be enough to cause the higher rated player to lose a handful of points even if they won the match itself.
In the same sense, if two players that are incredibly close in rating that have an incredibly close match will not receive as many points for winning a round as a major underdog would.
This equation also applies a multiplier based on the level of competition, increasing the swing factor between wins and loses. This means that playoffs in the league will have larger point totals awarded or subtracted from players ratings based on wins and losses. Higher profile tournaments, Regionals, and the NATC will all have different multipliers for this swing factor.
*I've included some examples in a simple spreadsheet. If you're interested in examples of the math used, please feel free to take a look.
Other Considerations A tricky part of operating axe throwing venues across the NATF is to manage leagues while at the same time managing our league members schedules. An important piece of the puzzle is allowing for an acceptable level of flexibility so that competitors are able to participate around their other life commitments, while still making sure that they do not simply ignore their commitment to the league and never show up, leaving their opponents without a match to play.
Because of this, different skill levels of leagues are not possible in most markets. In some of those markets, where there are enough members on a given night that we could identify one league as intermediate and one an expert, when asked, those throwers did not want to be relegated from their league group based on their performance. There were many reasons for this response including wanting to throw in the group that their friends were in, wanting to stay in a tougher league so that they could get better, or wanting to stay in a league with a few dominant throwers because the day they toppled the reigning champ would be that much sweeter after months of failed attempts.
We decided that we did not want to force throwers out of certain league groups based on skill level but instead, let the throwers themselves understand how the rating system works and decide for themselves what is most important to them. Not every one that competes in the league wants to be number one and we want to have space where everyone can participate but we can still see personal improvement based on the criteria that the rating system uses to determine that rate.
This system lets all matches still have an applicable worth in the global rank as it awards a varying amount of points for wins against different levels of opponents.
IE. If you are the single best player in a league for a long time and there is no one that really competes with you, you won't get as many CRS points for the wins in that league.
It doesn't mean that you won't still do well in the rating system, and it doesn't mean that you won't make the NATC (champion points within your league grouping is still the main way for qualification throughout the year), but it does encourage those at the top of the pack to try to find higher level opponents while greatly rewarding those in the lower ranks for beating much higher ranked players (You can see this type of reward in almost all sports, underdogs are rewarded for incredible play)
Moving Forward Several things we're still evaluating are a) a decay system for points from inactive players, and b) the ability to add additional points based on high scores. This would mean that 75's and 81's would be rewarded as additional points to the rating system so that those that are throwing at a top level, but that might not be able to take down the higher level competition, can get additional points as they improve their averages.
Another key piece that is coming down the pipeline is the addition of the Tournament software for AxeScores which will allow all tournaments at NATF venues to have wins and loses applied to the Collins Rating System. This will allow another environment for competitors to seek out higher competition at events like Chopper's Turnpike Strike and other venue based one of the tournaments at all NATF member locations.
Please keep in mind, this is our first pass at the system but we are confident in the logic behind it and are excited for the doors it opens for competition in the future.
If you have some additional input after understanding the thought process behind the system outlined above or if you'd like to provide constructive feedback, I'm more than happy to answer any emails at this address.
Hope you all have an excellent week and that I've been able to provide added clarity to the inner workings of the new CRS. We are very excited about this release and can't wait to share it with you as a living and breathing system on AxeScores in the coming weeks.
As always, I'm available for any comments and concerns.