Hi, I tried going through this to figure what is happening, but its too much words.

Is there a rating system up and running? Even if in draft? I tried http://www.vdiplomacy.com/halloffame_elo.php but I do not have privileges to view. Is this because it is still in process?

Oli had a working system. But, there were a lot of different opinions on the "best" system. I hope he adds his system to the site, and continues to pursue the other systems that were proposed.

LARGE UPDATE (and one question):

The longer I try to make one complete formula for a complete game the worse the results.
The problem is not to calculate a %-score, the problem is to define what this % means. Is 50% a 50% chance to hit the final SCs? Or the number of SCs on the board? The GR does calculate 2 different %, one for winning (hitting the target SC-count) and one for not getting the final SC-count, but it failes to distribute the points in a fair way (beating 4 100-points player as a 20-point-player gets the same as beating them as a 500 player). And how do we relate the own rating to all the others? Build a diff? But beating a 1800-player and a 1200 player is clearly not the same as beating a 1500-player. Same for the SC-scores...

Because of all these uncertainties I will use 2 different ratings for now.

1. I call this "normalized" DPoint system. At the end of each game you put 10% of your rating in the pot, and the pot will be re-distributed like the DPoints.
(Funny: The end result of this is very similar to the much more complex GR).

2. My proposed way of splitting each game in many 1on1 matches (because this is the only way the Elo-rating makes any sense).
I will score win > draw > survive > resigned/defeated in a pure 1-0.5-0 system. If there are more than 1 survived I will check the SC-difffs instead of scoring 0.5 to each player.
I do not plan to change this based on pot-type, as you should not be able to put your rating on stake. If you managed a 15SC survived in a classic game it shows you are a good player, even if someone else got all DPoints in a WTA. (You will get a 0-1 loss against the winner, but a very good % to the rest of the survivours.)

I tried to build in a time-factor that shrinks over time. (Example: It will start as 24 and go down by 1 each month. Ratings will be calculated on your actual score at the time of completion.) but this didn't work that well, because your first games usually weight much more than the later games. And this system did put all new players at the top and older players where left behind too fast. Any other ideas on this?

For the large maps I will need to change the value so it does not weight more than a certain ammount of players. Because in such large maps it's unlikely that you played each other player (think of a chaos gama). But I still need input how to scale down.

Question oli: will the "survive" score always be less than 0.5? For example , in a classic game with a 18 sc solo and a 13 sc and a 3 1 sc survives, what would the 15sc player score against the 1 sc players assuming everyone is the same rate? If they all drew, they would gain 4x0.5 for 2 right?

Damn didnt mean to hit post.

Carrying on, the 13time sc survive would score a 0good against the winner and ? Against the survives.

I think that it's great that you're putting so much effort into developing a player rating syste, and clearly there are so many opinions that you'll never make everyone happy. I definitely like your mention of the stress on a win>draw>survive>resign basis. But I think that you do a disservice to everyone here when you take the position that "If you managed a 15SC survived in a classic game it shows you are a good player, even if someone else got all DPoints in a WTA." A 15 supply center survive to a solo is NOT a good result. It's a loss just like the other fiver players. If you want to awared the actual survival wiith a marginal score to recognize an absence of elimination, that makes sense. But it you give that player " a very good % to the rest of the survivours" then at some point you are encouraging him to throw the solo. A 15 SC survive to a solo is a loss just like everyone else. A solo is total victory. The game is only ended there to avoid the needless futility of playing on from that point.

I don't think that the reduced score over time works well either. This is open to manipulation by the players that look to drag on games by failling to submit ready orders on retreats with no available retreats and disbands that they are going to NMR on anyway. It is also vulnerable to the stall and delay tactics used by players who are in trouble on the board, and suddenly start asking for extends and pauses in an effort to sap interest and momentum in games. The length of a game should really have no relation to the score. Finishing faster does not equate to a better game or a better achievement. Fast game are more often a result of NMRs , abandonments or thrown gamens. These occurences are regrettable realities, but if you have the scoring sytem favoring them, there will be even more of them. There are too many factors, such as game turn schedule, player roster strength, and the human behavior factors cited above.

And while I may be bringing this up at the 11th hour (and at the risk of angering the Guaroz clan into a frenzy) I thinkt hat something needs to be done to punish NMRs and people who resign (quit) from games. These actions are incredibly frustrating and disrespectfult ot he community. There's should be some sanction in place, even if it's only a marginal penalty to distingues this behavior as less that a player who nobly plays out his position, but is eliminated in the process. Players who NMR and quit games do not deserve the respect and protection being afforded to them presently. And a scoring system that punishes this behavior would ultimately result in fewer such results.

@drano:
Your example: Each player should be able to score a 3 (6 x 0.5)
Pl:1 won => score:6.00 (1+1+1+1+1+1)-> diff= +3.00
Pl2: 13SC => score:4,79 (0+0.93+0.93+0.93+1+1) -> diff 1,79
Pl3: 1SC => score:3,07 (0+0.07+0.5+0.5+1+1) -> diff 0.07
Pl4: 1SC => score:3,07 (0+0.07+0.5+0.5+1+1) -> diff 0.07
Pl5: 1SC => score:3,07 (0+0.07+0.5+0.5+1+1) -> diff 0.07
Pl6: 0SC => score:0.50 (0+0+0+0+0+0.5) -> diff -2.50
Pl7: 0SC => score:0.50 (0+0+0+0+0+0.5) -> diff -2.50

Now can multiply this with a factor of 20 (this is not set in stone) and would get:
Pl1: +60
Pl2: +36
Pl3: +1
Pl4: +1
Pl5: +1
Pl6: -50
Pl7: -50

@RUFFHAUS:
1.You are right. Maybe I should think about the WTA again. But this would only change the handling of the survived. They should get a win against all defeated anyways and a draw against each other.
2. Reduced score: Maybe we do not need a reduced score, but a system for the HoF to show only the more active players. Usually Elo-Ratings tend to create problems with the top-players not playing anymore to put their good rating at risk. That's why we will never go away with the points system. This gives an incentive to play even if you might loose some rating-points.
3. NMRs: My plan is to score a 0 against all active players if your country is taken over by another player.

Oli -

The problem I see with the example I gave is that the survive is worth more than the draw. Assuming a 17-14-1-1 draw instead of the 18-13-1-1 survive, the draw would be worth (0.5+0.5+0.5+0.5+1+1) = 4, which is less than the 4.79 of a survive. This is exactly what we should strive to avoid as a survive is clearly NOT a better result than a draw.

In regards to your response to RUFFHAUS, I have to disagree. Survives in a WTA should NOT get a win over defeated people. If you survive in a WTA, you lost, just as much as a person who was defeated. I agree that in PPSC they should get a "win" versus defeated players, but in WTA, the entire point is that the winner took everything, meaning everyone else loses.

@drano:
But that's true in many PPSC games:
If you play the same game with bet of 10 you have:
Pl1: no-draw=+27 -> draw=+4 (diff: -23)
Pl2: no-draw=+17 -> draw=+4 (diff: -13)
Pl3: no-draw=-8 -> draw=+4 (diff: +12)
Pl4: no-draw=-8 -> draw=+4 (diff: +12)
Pl5: no-draw=-8 -> draw=+4 (diff: +12)
Pl6: no-draw=-10 -> draw=-10 (diff: 0)
Pl7: no-draw=-10 -> draw=-10 (diff: 0)

Any ideas how to scale down the value of large player games?
Can someone with many different games tell numbers how many of their opponents they usually actively play?

That's precisely my point Oli. Why is a survive better than a draw?

Suppose there's 2 players:

Player 1 - 0 wins, 100 draws, 0 survives, 0 defeats
Player 2 - 0 wins, 0 draws, 100 survives, 0 defeats

Assuming equal quality opponents, we would all probably agree player 1 is the "better" player as they have never lost a single game. However, in this scoring system, player 2 could be ranked higher despite LOSING every single game they were in. How can we possibly put this in the scoring system?

If you have many SCs surely a draw is worth less than a survive in a PPSC game. If you managed to get 100 draws with players much worse than you it's obvious that these draws are worth less than a very good SC count, because you played PPSC.
In this way I think in a WTA game the winner should score 1 for each opponent, and all opponents should score 0.5 with each other survived or not.

Oli - If you note, I said same quality opponents. Therefore the question is, what is better? 100 draws (you have never lost) or 100 survives (you have lost every single game)?

I agree with your second statement. The only issue I see there is that in a 34 player game, theoretically players should score 16.5 (33x0.5). If someone solos, everyone still scores 16 (32x0.5). Does that really make sense that they still are almost at the theoretical score?

If all opponents are equal, and you give up a 14SC 2nd place for a draw (in a PPSC) you should really get a lower score than a survive, because a 14SC survive is really better than a 5 way draw.
A WTA is a totally different matter.

Thats exactly the reason many people think WTA is more to the real game.

In PPSC a 14SC 2nd place is a really good finish. And because it's a really good finish in the current system it should be a really good finish in the new system too as long as they exists besides each other.

Of course 100 survives can be much better than 100 draws in PPSC games. Thats the point of PPSC (and the reason for the hate it gets for this by experienced players) In PPSC it's hard to define loosing. Many times a survive is worth nearly the same as winning, because you have a similar SC count.

It's important that the new system does not work in opposition to the old system, because this would cause some player playing for points on the old system and have really different reasons than the players playing for the new system in the exact same game.

Then if that's the case Oli, you need 2 ratings. One for WTA games and one for PPSC. Otherwise the win>draw>survive>defeat mentality is all screwed up. You'll only be encouraging people to LOSE the game to get a better score. And considering the standard setting on this site is PPSC, this is going to skew the stats greatly I would guess.

^ was in reply to the second message above.

It does not screw the stats.
A 14SC-survive is better than a 5-way draw (in a PPSC) all the time. Changing this would screw the stats even more.

I was referring to stats as a whole Oli. When PPSC and WTA games get combined.

I would say that a draw is always better than a survive, for the simple reason that otherwise you get games thrown just to finish on more SCs, which is not in the spirit of the game. Solo>Draw>Survive>Defeat, no? So a draw should always award more points than a survive. Perhaps, (using system 1), if the points were split in a solo situation so 75% of the points went to the soloer and the rest were divided out between the other players? (Maybe a little less if that skews the results too much, but you get what I mean). Then a draw is pretty much always worth more than a survive, but a survive is still worth more than a defeat. This would solve the thrown games situation.

However, how to adapt this to system 2, I don't know...

Not in a PPSC.
All that counts in a PPSC is the SC-count. And throwing a 14SC for a 5way draw is plain stupid. In the new system the same as in the old.

And yes. I'll do stats for global, per variant, per pot-type and per press-type (and all possible combinations)

I think it's just a definition thing.
In PPSC the point of the game is to get as many SCs as you can, in WTA the spirit of the game is to win. Thats just the way it always has been. And its dangerous to put a WTA rating style on a PPSC game.

Hey, chiming in here again..

I've got to say again, good work Oli and thanks for sticking with this and trying to get us a better system. I know we're all gonna have objections here and there to various parts, but I hope I speak for all that we are looking forward to anything that's an improvement over what we have and we thank you for all the hard work! We're all opinionated and just trying to make sure we get the best ranking system(s) for the community.

I agree with both sides of the PPSC/WTA debate here in part, and thus have felt all along that we need multiple systems to cover the variety of play styles/game types/etc. we have here. The more this debate goes on, i don't think we can have just one ranking system that gives us solid math on who is the best player here. There are too many play styles and too many variants for one single overarching ranking system to be completely effective (and accepted).

Thus I'm going to second what someone above mentioned (can't remember who as I go back and look). I propose separating (or at least an option to separate) PPSC standings and WTA standings (a completely different game and truest imo to the W/D/L results that ELO is designed for and the game of diplomacy as well for that matter). In PPSC, maximizing your centers in a survive is not a loss or equivalent to other smaller survives, but is debatable whether it should still be less than a draw, even though there are a number of players here who don't play that way. I personally feel it should still be less than a draw, but it is a popular set of victory conditions and not everyone played this way, so there will be much objection from those players if we change the scoring to the way we feel the game should have been played. In that case I feel that if we want to change things to how they should have been, we should start over fresh with those standings.

Also, am I missing something or are we no longer able to see the experimental rankings? http://www.vdiplomacy.com/halloffame_elo.php doesn't work for me anymore.


Oli,

To specifically answer your question about high player count games and number of players that we interact with on those games (as WWIV, and to a lesser extent Known World 901 and Modern II and other large maps account for almost half of my games), I would say that I at a minimum talk to everyone on my continent and all my neighbors and their neighbors, and have significant relations with probably 1/2 to 3/4ths of those through the early game. Mid game, I'm usually interacting significantly with another 4-8 players I didn't interact with in the early game, and if the game makes it all the way to what I would usually call the end game (though most WWIV games don't make it that far in my experience), I'm usually talking and interacting with everyone left. So out of 35 on a WWIV, I'd say I interact with anywhere from 15-25 of those players in a significant way, even though I often try to talk to most everyone at the beginning. I think this estimate is high though as I interact a lot more and a lot farther out than most on large map.

On a side note, for players like me who play much more of these high player count games, I'd like to interject once again that a high center count performance, with a high number of draws and large survives repeatedly on WWIV games, appears (from stats I've seen at least) to be 'penalized' or less statistically significant in comparison to people who play a lot of 1v1 games. Once again, I'm not arguing for changing the overall stats, but that we really need standings per variant (and I'm also now recommending per game type WTA and PPSC, in addition to the 5% or 10% bet of skill that Devonian? proposed a while back). If in the end, the overall standings on this site are biased toward smaller games (as a natural result of the more explicit win/draw/loss nature of smaller games) than the larger games, that is fine by me (mostly because I don't know how to address it), but I definitely feel it won't be a valid answer to reduce or scale down the score adjustment for large games based on approximately how many people on average a player would really interact with.

I'm not sure how to address the relevancy of established players vs newcomers who show up immediately and high in the rankings without having challenged very many of the experienced crowd. Would having an overall 'all time' ranking that it takes a while to get listed on (say 3 months?) and a monthly 'rolling' ranking of top 100 or top 200 or something be a way to do this?

"And yes. I'll do stats for global, per variant, per pot-type and per press-type (and all possible combinations) "

Why do I feel like I've been ninja'd... :(

Thanks!!

"Not in a PPSC.
All that counts in a PPSC is the SC-count. And throwing a 14SC for a 5way draw is plain stupid. In the new system the same as in the old."

I think this is the problem with PPSC. A draw should always be valued more highly than a survive, for the reason that a draw is a type of win, whereas a survive is a form of loss.

WTA doesn't solve this either, because surviving with 15 SCs is better than a defeat. You've still lost, because you haven't managed to force a draw, but you've done much better than the people who have been defeated or survived with less SCs.

This is why I think the system needs to be based on Solo>Draw>Survive>Defeat. This is a sort of compromise between PPSC (Solo>Good Survive>Draw>Bad Survive>Defeat) and WTA (Solo>Draw>Survive/Defeat), and I think it makes more sense than either of those systems.

I just think that the algorithm's coefficients must be different with respect to pot type.
The concept behind WTA and PPSC is completely different, and so it is (meant to be) different the play style.

There's not an usable compromise.

+1 Captainmeme

If only PPSC count in the PPSC format, then why does the game end when someone wins and, say, not after a certain number of seasons, for example (disregarding custom settings)? In my opinion, a draw in either format should be worth substantially more than a survive, as a survive is a loss in either format. And no one should desire a loss over a share of a win.

In WTA surviving with 15 SC is not better than a defeat, because it's WTA.

If you mess too much with the system it will lead to even more randomness.
THe Elo really just work with 2-players, compare their performance and give a weight to the outcome to change the result. Big weight-> players will reach their destiny-score fast, low weight-> players will reach their destiny-score slow, but we won't have these spikes in the ranking-lists.

Because it mimics the DPoints much better and does not require the player to choose between the 2 systems I will use the system I proposed before. If you play for maximum DPoints you also play for a maximum ranking-score.

PPSC:
I will score win > draw > survive > resigned/defeated in a pure 1-0.5-0 system. If there are more than 1 survived I will check the SC-difffs instead of scoring 0.5 to each player.
This will in some cases (and there are only few) favour a survive over a draw, but it does this the same way like the point-system and this is a feature of the PPSC-system thats not going to change.

WTA:
I will score 1-0 against everybody to the winner and 0.5 to everybody else.

There will be no penalty for older games.
There will be a global rating and a rating for each PotType (Each starting at 1000)

And we will see how this will work.

For limiting the importance of really big games I will need more input.