Kaner,

That would be my preference. Fubar has had 20 games and Rinascimento has had 45. They represent 65 out of 6483 games played. At around 1% of games played, they shouldn't make a significant impact. If they do, we could address it at a later time.

Well. Games played:
Classic - Germany vs Italy 764
Classic - France vs Austria 758
Classic - England* Vs Turkey 603
Classic - Frankland Vs Juggernaut 439
Fall of the American Empire: Civil War! 342
Classic - Pick your countries 322
Duo 243
Lepanto
Classic - Germany vs Russia 174

TOTAL: 3833 games played.

If you take out from 6483 the 1v1 games, you'd have 2650 games left for the remaining 74 variants, that is an average of 36 games per variant. Rinascimento is above average, Fubar is below, but not completely meaningless.

Further, there are tons of unbalanced maps:

http://www.vdiplomacy.com/stats.php?variantID=45
Paraguay looks advantaged

http://www.vdiplomacy.com/stats.php?variantID=12
China ELIMINATED 46 out of 72 games

http://www.vdiplomacy.com/stats.php?variantID=46
Greece won a third of games, eliminated only 3 times out of 30

http://www.vdiplomacy.com/stats.php?variantID=83
Morocco won half of games, NEVER eliminated

http://www.vdiplomacy.com/stats.php?variantID=37
NSW ELIMINATED 21 out of 29 games

http://www.vdiplomacy.com/stats.php?variantID=47
Cathol. ELIMINATED 6 out of 8 games

http://www.vdiplomacy.com/stats.php?variantID=77
Huron ELIMINATED 13 out of 16 games

and more...

So I believe that imbalancing of maps IS an IMPORTANT issue.
When you draw up Huron in "Indians", you know it'll be a miracle if you survive.
And if you draw up Morocco in "Africa", half the job is done...
Who your opponents are, how strong they're supposed to be, doesn't look so important on unbalanced maps.

Like I mentioned earlier...

The quick and easy way to account for such possible imbalances is to normalize score adjustments using the stats for the countries in the variants. This normalization process can be applied independent of any calculation of score change based on relative player skills, thus you can present both scores and see the difference it would make.

Leif,

The problem with normalizing is that the sample is not significant enough and a lot of the differences are due to the players, not the map.

For example, in Migrane, Beta has NEVER been eliminated, but but Sigma has been ELIMINATED 86% of the time! How do you normalize that?

In pure, you are almost 3 times more likely to be eliminated as Italy than as Russia.
And Germany solo's as often as Austria, France, Italy, Russia, and Turkey, COMBINED!

In Rat Wars, Shirt tails solo NINE TIMES more often than Hell Cats.

Not enough games played? Ok then how about DUO, with 243 games...

Red has been eliminated 17% more often than Green. It also has about 7% fewer solo's.

In order to normalize, we would need thousands of games played, or at least more games with equal players, not a few dozens games with vastly unequal players.

leif, I didn't see your proposal to have a normalized rating and a non- normalized rating at the same time. That might work.

So 2 extra ratings on the profile page?

Ghost Rating: GhR
Normalized Ghost Rating: nGhR

I think we shouldn't care about the corner-cases of normalization.
It's much better than the different pot-sizes and if more Migrane games get played the stats will change too and if not much games will played it doesn't matter anyways.

In a fully balanced game world, a fair system would look like this:

for a 1v1 game you get 1 point. If you draw you get 0,5 point
for any games with more players you get "number of players" x 1 point for a win.
If you draw you should get "number of players" x 1 point divided by "drawing players".
So how could survivals be rewarded, the lowest draw score would always be 1 point, (1 x "number of players" divided by "drawing players") . A survival should receive that. In 1v1 games, only the winner gets the point.

It is also really important to not penalise eliminations, otherwise you may get a lot of boring games, which drag on forever.

It is a simple system and will deliver quite realistic results.

For unbalanced maps you could give bonusses based on the comparison of starting supply centers... BUT, I think these are special cases and most maps should be fairly balanced. (And if you draw Egypt in WWIV, well sorry). So i am against giving special points for real or perceived unbalances...

Another point would be to reward you more for winning against players with more points. While this may work in MMOG, it would really impact negatively on dipping, as it would incentivise metagaming.

Oh and if you dont like fractions of points, then multiply all calculations with 100 or 1000.

Why dont we make a list of the non 1v1 games and discuss them one by one? And through that find the most agreeable formula.
For example. Chaos - 34 players - equal number of starting centers..
A solo should get 34 (according to Illum's formula)
- I do not know the other formulas-

Cypeg is right, we need to discuss it one by one, as some winning conditions are so low that they would also need to be taken into account.

The benchmark for full points is 50%+1 centers for winning. If it is only 25% it should be the relative value in comparion to the benchmark.

@iLLuM: I'd like to add the rating of the opponent in the equation.
Playing against a high ranked player should be more rewarding than playing a rookie.
(example: If I play 10 2-player games against rookies I would get 1000 easy points, if I win 10 games against a top-ten player I only get 1000 too).

Illum, one more reason why seeking for an Elo/Ghost – rating system instead of a point – rating system
is to contain metagaming and multigaming effects on ranking.

@all
We’ll never be able to take into account every variant unbalance in a proper manner. We’ll always have this side vulnerable to any kind of method critics. We’d better bear this in mind.
I applaud to Devonian’s vision. The most pragmatic, the most feasible, the most simple, the most not-needing-adjustments-in-future.

We have to keep it reasonably simple, else:
How would you handle stats? In most cases stats are unreliable due to the variant sample size.
How would you handle gunboats vs full press?
How would you handle “flawless” games vs games affected by non-playing users?
How would you handle public games vs private games?
How would you handle standard games vs Special Rules Games?
How would you handle games with standard win conditions vs games with altered win conditions?
A proper answer to those questions does not exist. Or even if exists, it will be found after endless discussions.
The current point system does not care about those questions, and, to be honest, I’ve seen a very small % of the vDip community taking care about them.

My vote is for 2 parallel ranking systems:
- The current ranking system (and point-bet system)
- Elo - Ghostrating system.

__________________________________________________________________
Ps: a side note about stats.
I’ve read tens of times in many threads about that Egypt stuff in WWIV. If I read again that playing Egypt is a pain in WWIV due to stats… I am going to vomit.
It’s premature to write such a consideration. WWIV is a 35 players variant. The current sample size, 27, is even lower than 35.
For instance, try to write some considerations about a classic variant after a 6 games played sample stat, or about a duel variant after 2 games… you’ll find them to be poor of meaning.

Can the pointn system be tested by applying it to at least the top ten variants and of course standard. or just applied to Standard to start with.
or be applied to all variants individually?

Elo ratings are nice for a 1v1, but here we are in a 1vN situation and it is diplomacy not just gunboat.
I would prefer to have a simple and mostly right system. Absolute accuracy can only be achieved with high complexity and then you may get other problems (not enough data for example).

I did a player analysis for the Bourse 901 game on the basis of the statistics on vdip and it pretty much develops as predicted. But my main point was to decide if a player will survive.
I based it on country performance and player statistics. I looked for points that looked significant and for all analysed player's prospects, I manually discarded crap data.

For example, winning a chaos gunboat game is noteworthy, but the real clue, is that player likes risky moves... and so on.

Conquest of Nations Rating - CoN rating is calculated at the end of each game. Players receive a base number of points for solo wins, draws, and controlled territories. Those points are then adjusted based on the statistical difficulty of the country they controlled. Finally, players have their points increased or decreased based on their rating compared to that of the other players in the game.

That's Conquest of Nations scoring system which I was a player of for a long time

It seems to me that no matter what system of scoring players' performance you use, there are going to be quirks that fail to provide a true measure of talent, skill, results, and so forth. Some of the very best examples of Diplomacy play occur in games where a player does not necessarily "succeed". And to capture these intangible moments with a point calue is nearly impossible.

There are some good ideas being kicked around though that might cover the basics, and then when applied over time, and against other in the community, a reasonable assessment could be made.

One aspect that I think would really help would be a way to give respect to the strength of opponents/particiapants.

I do think Decima raises a good point (which I have made before as well) that it's important not to get mired down in statistics with small sampling sizes. The WW4 Egypt example is a classic case in point. Likewise one any new scoring system is in place, it may take a good year to begin to get any idea of an accurate measure. Frankly I think the whole suggestion that any one nation on any of these games is necessarily stronger than the other is misguided. Ultimately it comes dow to how you play the cards that you are dealt.

+1 ILLuM's scoring system fused with the rating of the players one is playing against. And I agree, no scoring system will be perfectly accurate, it just needs to capture the most essential elements, such as the difference in the number of players one is playing against and the strength of the opposing players.

How could we hande CDs and take-overs? It's not quite fair to give a bad rating to someone that took over a dying nation.

For takeovers, consider no penalty for an elimination and (maybe) only 25% of the normal points for all other outcomes?

Multiplayer Elo ratings (or ghostratings which appear to be essentially the same with a different k-factor?) are much, much better than what we have now. D points or even wins/draws/survives/defeats, just don't cut it for what the community seems to be looking for. I'm all for this and fully support this system being implemented.


My point is not that normalization should be used instead.
I am simply stating that normalization of the score awarded for victories/defeats based on country performance could be carried alongside the above calculations without changing the original calculation. It is an additional step added after the above calculation and this easily added or removed (or reported separately as I suggest for comparison). I would further propose that the effect of the normalization could itself be weighted based on one or both of the statistical significance (http://en.wikipedia.org/wiki/Statistical_significance) of the sample size and the effect size (http://en.wikipedia.org/wiki/Effect_size) of the given performance imbalance, real or imagined. Thus variants with few games played would have the normalization effect for their performance numbers skewed towards zero (as the expected result is should be near the pure average at that point), but in variants with many more games played, the statistics that do matter would begin to have more effect.

The overall effect normalization would have on game play (admittedly over a long period of time) would end up being a desired effect in that it would counter stigmas that get associated with poor performing countries (egypt in WWIV being my case in point, as I likewise agree that it's performance is more due to stigmas and poor representation than imbalance in position, and failed to explain this adequately above). Normalization would thus work in a negative feedback manner to flatten the performance curves in spite of any perceived imbalances while working to correct for true imbalances. Obviously for symmetric maps these normalizations would be unnecessary and would not apply, country performance there is obviously pure chance.

Playing as morocco in Africa would thus net you fewer points if you solo'ed (though your score would still go up, but a fraction less than if you had solo'd as Mali for instance), while soloing as the French or Sienna in Rinascimento would net you significantly more points than as Venezia or Napoli. This encourages people to try the lower performing countries with more hope of success as the ratings increase would be greater, and thus the risk of playing them less.

The other attribute of note for this method is that the performance ratings for each country are already in place and this normalization process could be applied without individually tailoring for each and every variant. Thus if we wanted to do anything additional to correct for the fact that there are imbalances in some or all of our games, this is the only way that really makes sense without being purely arbitrary.

Great post Leif,

I agree with you, it is a great system, but it would have to be very dynamic and the rules would have to be maintained adjusted and so on. If it works, it is what we all want. The license to really claim to be the best diplomacy player.

My proposal is a simplistic approach, it is low tech, robust and easy to implement. But reading your ideas on the subject it might lack ambition.

Oli,

"How could we hande CDs and take-overs? It's not quite fair to give a bad rating to someone that took over a dying nation."

My system has no penalty for taking over cd's, only for cd'ing.

The skill differences are built into the "pool of skill-points", so, a person CD'ing would get nothing from the pool.

As for take overs, it could be handled in one of 4 ways: The person taking over could either be allowed to take over at no cost, full cost, at a reduced cost, or you could apply the CD'ed persons skill-points toward the cost to enter.

I like the last option the best. It would work this way:

If a noob (100 skill points) drops out and who added 4.44444 to the pool, then CD's, and a person with twice the skill (200 skill points) takes over, they would add the difference. Normally, they would have submited 8.88888 to enter the game, but since 4.44444 was already added, they would just add 4.44444 more. The pool is good, and is again equal to the current skill level of all players.

If the replacing player was less skilled than the CD'ing player, the player would be able to join without adding to the pool, but the pool would be reduced by the difference to keep the pool equal to the current skill level of the current players.


Leif,

I also agree that both systems side by side would be nice. (With the current point system remaining in place, to regulate the potential number games a person can play)

Agreed, points are an excellent currency for metering the number and, to some extent, the caliber of games a player can join, and are an excellent anti-metagaming tool which if I understand is what they are/were designed to counter. They should by all means be kept around.

Decima,

Thank you for supporting my vision for the rating system. I continue to hold the opinion that all maps are imbalanced, that it is simply part of the game (which I like), that the players have a bigger impact on the win/draw/survive rate of any imbalances in the countries, and that the sample size is too small.

However, the reason I am agreeing to have a normalized system side by side, is because I do not think everyone will agree that ignoring the differences is fair. If I there can be an elo/ghost rating system, I will be very very happy. If we need to have a normalized system side by side, I can live with that.

Great proposal Leif. I like that as well.

Imbalance is all in the eye of the beholder. Many Diplomacy players commonly assume that Italy is a terrible draw on the standard map. It's not. It's all in how you play the game. You may play England better than Italy, but it's not necessarily better. Some variants may gove slight advantages to certain nation, but these same advantages can just as easily make you a target. In the end, it's always about how you play.

And, if we were to accomodate for imbalance, then that would throw off any new variants, right?

As long as imbalance accomodation is update in the code then it should be fine:

Say when you play the game it is determined that Italy is the weakest power on the map, but a month after the game ends it is determined that Austria is. Then your score is updated to reflect that Austria is the weakest power.