@Decima, you bring up a number of interesting concerns. Several of them seem to be quite interrelated and some may stem from misconceptions about the proposed rating system, so I'll try to explain the philosophy of the rating system a bit more first before attempting to address these concerns.
The high-level idea of the rating system, and other Elo-like systems, is to make adjustments to players' ratings based on whether they have met, exceeded, or fell short of their expected performance, which is somehow estimated based on their prior ratings. The Elo system does a good job of this for two-player games, since it's straightforward to say that a higher ranked player should perform better than a lower ranked player and the only question is to come up with a model that quantifies the relation between prior rating differential and expected performance. In particular, the Elo system uses an exponential model for characterizing expected performance, which is a reasonable approximation with some empirical evidence of its validity. Overall, this system tends to work well for these two player games, since it will make smaller adjustments when the strongly expected occurs (a much higher ranked players wins) while making larger and opposite adjustments when the less expected occurs (a much weaker player wins). This approach tends to result in a self-correcting behavior that adaptively converges the ratings toward some true underlying measure of relative skill.
Extending the concept of the Elo-rating system to games involving more than two players is tricky, since the multiplayer dynamic raises questions about how one should compute the expected performance, as it is difficult to develop a model that can estimate the expected performances given multiple players of differing skills. For example, suppose you have seven players and their *true skill* is given by the relative ratings 1400, 1200, 1140, 1250, 1460, 1050, and 1320. Is the first player under-performing if he takes a 3-way draw? Should who he drew with/against matter? Toward resolving the difficulty of answering these questions, Oli proposed a rather clever heuristic for making adjustments in these multiplayer games. Instead of attempting to adjust each player's rating based on some difficult to compute expected performance against the rest of group, the proposed system breaks it down into a series of pairwise adjustments.
Thus, the philosophy is simplified: each pair is adjusted based on the difference between their relative actual performance and their relative pairwise expected performance. Calculating the relative pairwise expected performance is more straightforward, since one can follow the philosophy that the better ranked player is expected to perform better and we can apply the exponential model to quantify this. Calculating the relative actual performance is a matter of a looking at if one player has done better that the other in the game. When comparing a winner versus a loser, or a drawer versus a loser, it's reasonable to say that the winner/drawer convincingly performed better than the loser. When considering a pair that has both drawn, one could argue that they have both performed equally well. However, for such pairs, the adjustment would have an equalizing effect, bringing the two players' ratings closer together (effectively penalizing the higher ranked player for sharing in a draw with a lower ranked player). This Draw/Draw equalizing effect could even cause players to sometimes lose rating while drawing. Also, I believe that a draw/draw comparison is somewhat less convincing at indicating that both played equally well, than a draw/loss comparison would indicate that the drawer had played better than the loser. Hence, due to both of these effects, I proposed that we introduce a factor (of around 0.2) that lessens the relative impact of the draw/draw adjustments versus draw/loss adjustments, making it less likely for a player to lose rating in a draw, and observing that a draw/draw may be less indicative of relative skill than a draw/loss comparison.
The final type of comparison for WTA games is loss/loss, which I believe has almost no indication of the relative skill between two players. Two players of vastly different skill can both lose in the same game for various reasons, but it does not indicate that they have performed equally well. For example, one player may have just played bad enough to lose, while another may have spectacularly failed possibly even by throwing the solo. Further, if one were to make loss/loss pair adjustments on the premise that the two players performed equally well, this would illogically penalize stronger players for losing alongside weaker players. In fact, making loss/loss adjustments could even lead to losing players with weaker prior rating actually gaining rating for losing.
Some have proposed that perhaps some information could be garnered out of loss/loss comparisons by considering other factors, such as how long a player lasted in a game and whether they survived to the end. I think that considering simple measures to differentiate between different types of loss, such as looking number of years before defeat or differentiating survivors, are fundamentally flawed. Consider comparing a player that was defeated early on simply because he was ganged up on, another player that survived longer by ganging up on the first but inadvertently threw the game out of balance setting a up solo opportunity for someone else while eventually being defeated, and a third player that survived to the end by disregarding the WTA objective of the game and instead simply played for a (meaningless) strong second while giving up an easy solo. A naive system that simply considers survives to be better than defeats, might actually encourage players to go for the strong second if they are lower ranked and can earn rating over the defeated group despite giving up a solo. To accurately compare different types of loss (survives and defeats), an ideal rating system could not simply rely on length of survival in determining relative performance of losers. In general this would be a very difficult and subjective problem that would have to look into the specific game dynamics and essentially try to assign blame for the solo. This would not be a straightforward task and perhaps beyond the realm of a practical, automated rating system.
I'll address your specific concerns one by one:
1) Discarding some pairwise adjustments
> Quote: "we collect 7X6=42 pairings results and get rid of 6X5=30 of them. The major part of information is lost. The greater size the game, the greater the % of information lost.
I find difficulties in understanding this partial matchup approach: every game we collect n*(n-1) pairings data and then, depending on the outcome type, we get rid of a (great) part of them."
> Response: Some pairwise adjustments are discard or deemphasized since they contain weaker or misleading information. The loss/loss adjustment (explained above) is an example of this. Keeping loss/loss adjustments could even lead to very strange results, such as weaker ranked losers gaining rating points simply for losing alongside good players, while punishing those better players needlessly.
2) Expected result should depended on all players
> Quote: "In a given game, the Expected Result of user “A” should be depending on
- Rating of “A”
- Rating of the other users in game (classic for fixing ideas, B,C,D,E,F,G)"
> Response: By making pairwise adjustments, the implicit overall expected result does depend on all other players in some cases. For example, as you noted, the winner is adjusted with respect to each other player. From the perspective of a loser, certain pairwise adjustments are dropped (as explained above) due to the difficulty of quantifying loss/loss comparisons. This point also seems related to issue 5 below, which I discuss further.
3) Expected result should not depend on the game outcome
> Quote: "The Expected Result should not be depending on the game outcome. I mean, once the joining list is filled, the “minus” part of my net score should be already defined, while the “plus” part of it will be defined at the game end."
> Response: The overall expected result for a player is not directly calculated, but rather arises implicitly from the pairwise adjustment heuristic. The adjustments do however depend on the specifics of the game outcome. For example, a 3-way draw between 3 strong players over 4 weaker players, results in little gain/loss for that game, since that result is not very surprising. Also, there is a difference between a weak player taking a 3-way draw with two strong players against 4 weak players, versus a situation where that weak player took a 3-way draw with two other weak players against two strong players and two other weak players. From the perspective of that weak player, it's a 3-way draw either way, but having defeated strong players versus going along with them should make a difference.
4) Possibility of abusing the system
> Quote: "We have to avoid that a player finds more convenient an outcome because the user involved is C instead of D.
My driving force must be solely my own result, a solo or a small size draw, regardless of whom is involved for points reasons."
> Response: In general, with games that are played non-anonymously, with some sort of rating system in the mix, players can abuse the system to bring stronger players down while hoping to bring themselves up. I think this is more of an argument that games should generally be anonymous to avoid ratings meta-gaming. Not taking into account who you win/lose against, or who you draw with or against, would create inaccuracies in the ratings.
5) Consideration of the strength of the field in a loss
> Quote: "Moreover, the algorithm does not to take into account the “power” of B,C,D,E,F in calculating the adjustment of user A:
On the same example of above, I will lose the same amounts of points regardless B,C,D,E,F being a bunch of beginners or a bunch of Diplomats."
> Response: Taking into the account of the skill of the other losers when calculating the adjustment of a players loss would seem to presuppose that a loss could be blamed on the general weakness of the field, suggesting that the solo should be discounted since he had a weaker field to manipulate. From the perspective of the winner, the solo is already discounted through the pairwise adjustments with the weaker players. From the perspective of a strongly rated loser, that feels cheated by having played with weaker fellow losers that he blames for giving up a solo, I would have to argue that he had just as much opportunity to take advantage of that weaker field but failed to do so.
In a different vein, I do think that a feature that should be added at some point is to adjust the impact of adjustments based on the presence of provisionally rated players (players that have not played many games and hence have an inaccurate rating).
Best Diplomacy Website
)
)
)