Design the 'perfect' matchmaking system

[deeFive]

i have been thinking about this (im doing a shooter at uni) and i think the following ideas may be good:

Player ranking - this should be able to go up and down (a little like the jap fighting games handle it), so the players rank is a closer repesention of the players skill and should take into account how good they are with each weapon, tools, perks, time spent in the game, errors (stuff like blowing your self up, TK'ing, running out in the open for no reason), heath handicap and aim handicap. this should be show as part of the players profile (it may be a good idea to provide high level players with cool DLC to reward time sepnt in game, stuff like one off titles, weapon skins ect). players should only earn XP form players of a simular rank to stop people praying on new players.

Player Squads - 3 to 16 players -these would allow small groups of players to form small squads for team modes that are kept as a unit whilst 3 or more members are online and also can go up and down in rank (which is simular to how the single player rank works but rated on your squad) i.e yoiu and your peeps that are in your xbl-party as a squad when you "squad up" you will be entered time to squad based games first then team games if none are avalable. you will always be kept in that squad untill they "un squad". players can be part of more than one squad but can only play in one squad at a time. squads can also earn squad items.

Player Teams/Clans - These are for larger goups of players and bigger gamestyles again can go upand down in rank, but rewards and bounses are given to clans with new players who go up in rank with them(for training them so to speak). players can only play for one clan at a time but players free to leave clans at any time, clans can earn clan items and extra xp against other clans in other game modes.

I would have a total of arround 25 levels in rank with 20 being the lowest and 1 being top and the final 5 ranks have titles with unique (not unlockable) titles given to high level players as a reward for winning.

Players should also be able to quickly bring a list of players who are close to them in rank and join there game to help people level up and play people of the same rank fast (this shouldbe a short cut from the title screen)

not sure that im going to have tme to get this in to my project but its the start of what i would like long term!

let me know what you think

 

[patsu]

There is a training ground in Halo 3 where only newbs (allegedly ) can play for the first few levels. Also MW2 restricts the games you are allowed to play in and opens up other modes as you advance, but you are still in those modes with 5 star generals. the learning curve is certainly not a "game quitting" experience but a challenge as much for not knowing the maps as the difference in levels.

Yes, you have dedicated "newbie" Warzone in KZ2 too. Even 5 star generals who are out of touch, like me, can join. Unfortunately, match making is not enforced (So high level people can hop in as well).

I'd say drop in/out is a good thing in general. At the moment, not too sure about gimping new players' capability yet.

 

[Tap In]

another thing to think about was something that Red Faction Guerrilla used and talked about (although they said it is under the hood in most online games) and that is "Dynamic Handicapping".

Each match regardless of skill level there was a palpable (yet subtle) shift in momentum which often kept most of the games incredibly even. It makes for some of the most exciting matched games i have ever played. often coming down to the last instant or event. what they did according to devs, was to shift the balance depending on how well each team was performing to weigh in favor of the losing team just enough to have a chance to recover and make it a game. Primarily, as i understand it, made dieing slightly harder and killing slightly easier for the losers as well as possibly more favorable respawns.

I found it intriguing and the even added game modes with HC OFF as some people cried bloody murder when they found out about it. I OTOH, thought it was implemented brilliantly.

oh and RF:G has an excellent MM system as well (although sadly the community is nearly dead -3rdPS don't get love.

 

[Scott_Arm]

I think you need some kind of rank for each player, and when you get matched into a game the average rank on each team should be the same, with a bell-curve deviation.

The way you assign a rank to each player should be based on score/minute, kill/death, win/loss and not solely on accumulated stats like hours played or experience gained. What you want is a representation of skill or effectiveness, not time and persistence. Experience does mean something, but it would be reflected in ratios like score per minute or kill to death. I think you'd assume the lowest rank to start and let players elevate themselves above that. There would obviously be a period of time before your rank level settled.

Join in progress, drop-in, drop-out, parties, clans.

 

[specwarGP2]

matchmaking should be rank and after a certain rank then by kdr. that way noobs get to play with noobs for the most part and if they still suck after a certain rank then they play against other sucky people.

Parties rank and kdr should be averaged for the all the members of the party for a single rank and kdr number. that way they can stay together even with auto balancing.

 

[Acert93]

I guess people don't have Halo 3? Because the lack of discussion of its match making indicates such.

It is pretty simple: newb area for people with few rounds and no rank (sometimes you will be against "new" layers under new GTs, but oh well, it is restricted by GT so not a huge issue). Each ranked area has a ladder: Win a game go up a rank, lose a game go down a rank. With 50 ranks it works very well. Uneven teams are balanced by an accumulation, so if your team has a 42, 26, 23, and 7 ranked player (for TDM) it will try to find a team with a rank of about 98 and try to get similar "pairing" of ranks.

The system in general works very well. I found once you get out of the first 5 or so ranks things get pretty "right" in terms of how good people are. I have seen 10 rank players have a blast but the system scales where the top 45-50 players are the elites playing the elites.

 

[Tap In]

I guess people don't have Halo 3? Because the lack of discussion of its match making indicates such.

It is pretty simple: newb area for people with few rounds and no rank (sometimes you will be against "new" layers under new GTs, but oh well, it is restricted by GT so not a huge issue). Each ranked area has a ladder: Win a game go up a rank, lose a game go down a rank. With 50 ranks it works very well. Uneven teams are balanced by an accumulation, so if your team has a 42, 26, 23, and 7 ranked player (for TDM) it will try to find a team with a rank of about 98 and try to get similar "pairing" of ranks.

The system in general works very well. I found once you get out of the first 5 or so ranks things get pretty "right" in terms of how good people are. I have seen 10 rank players have a blast but the system scales where the top 45-50 players are the elites playing the elites.

oh yea.... that is the best ranking system out there as far as who your competition is

 

[h0munkulus]

wait a second... the most important point is missing in this discussion!

matchmaking is an inferior system to create multiplayer games/sessions which was introduced to mask the lack of dedicated servers in a console environment!

i sure hope next generation console networks (next gen xbox live/psn) will feature a working interface to integrate external dedicated servers with low cost/effort!

if you compare the classical PC multiplayer configuration of dedicated servers with players using a server browser to find an adequate server/game with the console matchmaking system you really get a lopsided result!

advantages of dedicated servers/server browser system:
- play to search ratio. you spend a lot more time playing, lot less time searching games. usually if you find a pleasing game-server you do not have to change the server again for the entire playing session. but if you are limited to a matchmaking system after every round/game ended you have to go through the whole matchmaking process again.

- latency. plain and simple. dedicated server properly hosted provide a better plattform for playing than user hosted games. not to mention that dedicated server's are a lot more stable than user hosted games. no host change midgame, no conncection problems.

- transparency and freedom of choice. what you see is what you get. you see the ping of the server and the players before and after joining. you know map and gamemode as well as how many players are on a given server. you don't find yourself playing a map or a gamemode you don't like because you could not specify these options in the matchmaking parameters.

- familiarity. after playing a game with dedicated servers for a few weeks you have your list of servers you like. you will see familiar players and experience a more or less stable level of play.

- admins. on dedicated servers you can have admins that kick all the scum that messes up your playing experience. good bye flamers, teamkillers, cheaters, lamers, kiddies, ... make your pick.

- fairness. the leavers problem is a lot less severe. if you're playing on a good server a substitute will join shortly. also team strength is a lot easier to balance. instead creating an arbitrary balance based on some ranking system before the game starts you can balance the teams during the game with actual data of how good the players are doing.

- customisability. a community with the possibilities of dedicated servers will often find great ways to customize the gaming experience.


now what does the console matchmaking systems have going for them? honestly, only 1 thing. the illusion of matching players with similar skill...

you wanna know which system is actually matching players with similar skill a lot more reliably ?
you guessed right. a dedicated server system with server browsers. it's rather simple actually. if someone plays on a server he or she feels over matched this person will just join a new server. rather naturally the servers with players closest to your skill will be most fun to play. you add those servers to your favorites and will find yourself playing among players with similar skill in no time. make no mistake, there will always be someone on top and someone on the bottom of the scoring board, that's the way things work. but you will find servers with players close to your skill without some arbitrary ranking system...

to close this post. you can obviously include dedicated servers in a match making system. and you can have a server browser for user hosted games. i just have no better description for the system I'm talking about. so please don't start nitpicking about the terms. i think everyone with some mp gaming experience with pc and consoles knows what i'm talking about.

but of course feel free to disagree with my opinion

 

[tha_con]

Skill balancing and ranking are pretty worthless for matchmaking. There is no way to accurately measure skills other than stats, and stats can be accumulated and improved over time, without any real dramatic increase in "skill".

I do think that it should be stat based, however, with an equal "balance" on each time if that is a viable option.

i.e. if you're playing TDM with 16 players, put players on teams according to their KDR. i.e. Team 1 has 4 people with a 1.0 KDR, and 4 people with a .5 KDR (hypotehtically). Team 2 would be similar.

The major kink in matchmaking's "armor" is modern party systems, which make it even more difficult to do based on skill / stats / etc.

This is also why I think ranked play (for leader boards) should be for solo play PERIOD. I also believe that there should be new methods of ranked play for teams, i.e. in order to play ranked TDM you need a team of 8 individuals to compete in a 16 player match. This way you're always playing against another party of 8 people. Obviously this could be tweaked for ranked 16, ranked 10, ranked 8, etc, to match requirements for smaller parties, each having individual leader boards.

 

[deeFive]

skill ranking can be very useful for matchmaking the issue is how you measure the stats of players in games other than fall back to KDR as the main measure of how well someone is doing, in the project that we are doing for uni we have look at this subject and feel that we should try to approach it in the same way that opta do for football stats.

i don't have the details on stats current systems track but they seem to be milestone based, in fact i think there is a paper on trueskill on the Microsoft research site, we are planning to finalize the system tommorw but we want to look at things like:

how well does the player know the map/s

how well does the player know the weapons

how well does the player know their equipment/perks/tools

is the player using holes/exploits in the game system

how well is the player shooting

where is the player shooting its targets

how is the player dying

and KDR....

but we also feel that this needs to be teamed with the ability to be demoted in rank after a spell of bad play, we meet tommorw so i will post what we decide.

 

[Phil]

IMHO, I think the perfect matchmaking system doesn't need to be too complicated or do anything significant. Matchmaking main goal and first priority should be to find players with the best possible connection to another. Without this, lag will dominate the game which is bad for all players.

If we assume this is handled well, then the other factors that are important is creating a level playing field for all players. That means taking care in not placing new players against very experienced ones. If you have a basic ranking system, I think that would take care of it. You would definately want to have a player with about 10 hours of playtime play against someone the same as well, as they will roughly have played a similar amount and have a similar amount of experience with the maps.

Once the system has established a full game of say, 10vs10 players, the system should analyse each players performance and efficiency. For this, I'd either use a K/D ratio based on the last 50 games (not life-time K/D) or if possible, the K/D of the last game with the same group of players. Then split up the best players in opposing teams. This way, if you end up with a game where group A absolutely dominates group B, in the next game, lot of players of group A will be split into group B to make it a more level playing field. All players within this entire session should be within a similar rank though.

 

[Cheezdoodles]

The biggest problem with matchmaking is imo to use the correct values.

If by rank, you people mean experience level\how long youve played the game, rank has no correlation with skill whatsoever (atleast for FPS). This is obviously because even thought map experience is obviously a good thing, there are natural upper bounds for all players out there. If you naturally suck at fps games your still going to be naturally sucky at fps games even after youve played for 1 year non stop. Of course, your going to be slightly better than before, but your still bad. Rank gives no indication of anything outside of how LONG you play, not how GOOD you play. I can agree with matching players that are NEW to the game together, but outside of that its a completely useless rating.


A proper match making system should aside from matching people that have good connections with each other, should match skill. Skill is incredibly hard to measure.

KDR is far from perfect. Campers can get higher KDR than their skill actually imply, some people playstyles causes a lot of deaths, but can still be elite.

Win\loss ratio for team based games is completely without any value whatsoever in terms of how skilled you are. Kills per minute and score per minute could be useful metrics but again, just like the KDR, far from perfect.

Now that i think about it, wouldn't any such matching be completely useless in the long run?

Lets say that we somehow managed to find a metric that measures skill level. If we start matching by skill, all the elite players will be playing eachother, and all the bad players will be playing eachother.

Now, these elite players cannot all continue to have good KDR's and what not, the best elite players will have good KDR's, but most of them will reach normal or even sub-par levels, as they are playing only with the best player out there. Meanwhile, in the noobs group, some noobs will be better than the other noobs, thus leading to abnormally high "skill" ranking.

Thus, next time match making happends, the best noobs will play elite players, and obviously get raped, while the worst elite players will play noobs and have a lot of fun.

 

[Shifty Geezer]

Some good points. I'm glad I started the thread!

Regards the worth of ranking players, I see it like any sport. You need a tiered system of sorts. In the UK football league, there are loads of tiers, matching similarly capable teams against each other. There's no point putting Woking up against any Premiereship teams because they'd be constantly slaughtered and it wouldn't be a fun game for either team, unless the Premiereship team is remarkably shallow and would enjoy a completely one-sided fight.

A good tiered system will ensure a team/player is playing similarly capable people so that they are neither bored or hasseling the less capable, nor being decimated, but they should be receiving sufficient challenge to push their abilities.

An alternative option would be to handicap players. How about everyone plays against everyone (except noobs get some quiet training grounds) but you get a damage/resistance bonus against players depending on their rank? eg. A level 1 poor player will be able to take more damage and deal more damage against a top-tier level 10 player, levelling the playing-field against them. Their inability to aim well would be balanced by dealing more damage when they do hit, and they won't have to worry so much about fumbling with controls when they can take more damage. Whereas the same top-tier player would take and deal equal damage to a similarly 10 Ranked player. Visuals cues would inform players what they can expect from their weapons so they know in an engagement if they will be doing lesser damage and adjust play accordingly. Such a system seems to go completely against the status quo, which is to empower the top players to make them even more capable, so I don't know how well it'd be received, but I think anyone wanting a fair game and a challenge wouldn't mind.

Another option I've discussed with friends is not to balance the players but to offer significant rewards so a poor player gets satisfication from persevering. In the case of U2, if I've been killed 10 times by the same Rank 65 player, having shot (at) him lots without a kill, when I do finally get my revenge, a big monetary bonus will leave me pleased I stuck with it rather quitting out to a less frustrating game.

 

[Cornsnake]

An alternative option would be to handicap players. How about everyone plays against everyone (except noobs get some quiet training grounds) but you get a damage/resistance bonus against players depending on their rank? eg. A level 1 poor player will be able to take more damage and deal more damage against a top-tier level 10 player, levelling the playing-field against them. Their inability to aim well would be balanced by dealing more damage when they do hit, and they won't have to worry so much about fumbling with controls when they can take more damage. Whereas the same top-tier player would take and deal equal damage to a similarly 10 Ranked player. Visuals cues would inform players what they can expect from their weapons so they know in an engagement if they will be doing lesser damage and adjust play accordingly. Such a system seems to go completely against the status quo, which is to empower the top players to make them even more capable, so I don't know how well it'd be received, but I think anyone wanting a fair game and a challenge wouldn't mind.

Another option I've discussed with friends is not to balance the players but to offer significant rewards so a poor player gets satisfication from persevering. In the case of U2, if I've been killed 10 times by the same Rank 65 player, having shot (at) him lots without a kill, when I do finally get my revenge, a big monetary bonus will leave me pleased I stuck with it rather quitting out to a less frustrating game.

I wouldn't call that fair. So your playing against better players, but you do manage to make some kills, do you want to cheapen that accompaniment by having the game compensate for you weaknesses, or crippling somebody else? I'd feel cheated out of my own achievements, and no amount of bonus is going to make up for it.

IMO a good online game should start with a decent party system. (Because wanting to play with friends is the fundamental idea behind online games Guerrilla Games. ) After that a good matchmaking system should sort things out. How that works probably depends on the kind of game. K/D ratio might not everywhere, but it's good enough for just deathmatch. Score Per Minute might be better for objectives based games, although that still might not for accurate enough for some games.

MAG works well thanks to shear numbers. 128 random players versus another 128 random players kind of evens itself out.

 

[Tap In]

Some good points. I'm glad I started the thread!
An alternative option would be to handicap players. How about everyone plays against everyone (except noobs get some quiet training grounds) but you get a damage/resistance bonus against players depending on their rank? eg. A level 1 poor player will be able to take more damage and deal more damage against a top-tier level 10 player, levelling the playing-field against them. Their inability to aim well would be balanced by dealing more damage when they do hit, and they won't have to worry so much about fumbling with controls when they can take more damage. Whereas the same top-tier player would take and deal equal damage to a similarly 10 Ranked player. Visuals cues would inform players what they can expect from their weapons so they know in an engagement if they will be doing lesser damage and adjust play accordingly. Such a system seems to go completely against the status quo, which is to empower the top players to make them even more capable, so I don't know how well it'd be received, but I think anyone wanting a fair game and a challenge wouldn't mind.
.

http://forum.beyond3d.com/showpost.php?p=1383885&postcount=23

 

[Cheezdoodles]

Im fine with having a match making system that connects me with people that have good connections to me, and i would even be fine with a skill based system,(althought, the more i think about it, the less i like it. I like owning people online. Being constantly matched with good people could possibly deteoriate my enjoyment. I dont know, as i rarely encounter elite people online). Im fine with people that are new to the game being grouped together.

What im not fine with at all, is Shifty's suggestion of handicapping players. If this ever becomes the norm, i will stop gaming completely. Why on earth should you penalize a player for being good at your game?

I dont want a "communist world" where every player has 1.0 KDR and all matches are even.
I like playing games where my own skill can determine the outcome of a game.

I would get incredibly frustated if i would get killed because some player is ranked lower than me, and has some increased damage modifier.

Why should he kill me if he is worse??? Why should he get rewarded for being bad? If he doesn't like loosing, he should either rethink his approach to the game, or play a different game.

I understand that for some of you that are really really crap at FPS games, its frustrating to continously keep on loosing, but if one looks at the other side. The really good players, how frustrating do you think it would be for them, if suddenly 50% of the encounters that you normally would have recieved kills from, now suddenly results in deaths, NOT because somebody did something better than you, but because they where worse??

How about everyone plays against everyone (except noobs get some quiet training grounds) but you get a damage/resistance bonus against players depending on their rank? eg. A level 1 poor player will be able to take more damage and deal more damage against a top-tier level 10 player, levelling the playing-field against them.

It would be as much as fun, as if you played CR9 for a round of soccer, and since he is better than you, he has to play with 50kg steel boots. Fun for you, but sucks for the person in the boots.

A top tier player getting killed by a lvl 1 player because the lvl 1 player has some crazy bonus system, would just piss off all the good gamers.

 

[NRP]

Absolutely NO to handicapping. Terrible suggestion that would absolutely eviscerate any kind of competitive play.

@ostepop
Halo 3 implements some type of skill-based matchmaking system, which seems somewhat effective at matching you against players that give you a challenge. It's not perfect because it can be exploited, but I guess it works for the most part. The idea is that, after some playtime, such a system minimizes the difference between the "good" noobs and the "crappy" elite players.

 

[Tap In]

Absolutely NO to handicapping. Terrible suggestion that would absolutely eviscerate any kind of competitive play.

according to Red Faction :G devs it is not only already used in their game but they know of others who have it under the hood. FTR, RF:G was one of the most fun online and competitive experiences I have ever had.

 

[h0munkulus]

after reading my first post in this thread again, i think i struck the wrong tone. i really wasn't going for a troll post or anything. it's not always easy to find the right words as an non-native English speaker.

it seems to me that match-making really is accepted as a desirable way to create multiplayer matches. i honestly find that odd and hard to understand. coming from a PC background i found it a lot harder to adjust to the way console multiplayer works (matchmaking) than the change in controls (gamepad vs mouse/keyboard).

i truly believe in the arguments in my first post and think a lot of console games would be better of working towards an easy to use server browser and dedicated servers than trying to find the "perfect" matchmaking system...

most of the arguments are quite clear (more time playing less searching, complete freedom of choice, and so on) but i think one point deserves some discussion:

- a server based system balances out the games a lot better than the arbitrary ranking systems in matchmaking systems!

it has a lot to with what Shifty Geezer said about tiered systems. every league from the bottom up till the premiere league has a first and a last placed team. if a team is on the last place after a season it will be relegated in an minor league and the best teams are promoted.

in a very natural way this works with the same principles in a dedicated server system. without any rules and rankings to force the issue. if someone has no fun playing on a server because he is not able to compete with other players he is going to find a new server. because the servers you have the most fun are most likely the ones with players that are similar in skill and even personality people are coming back to these servers.

this is one of the main reasons why games like counter-strike are so long-living. of course the game itself is good. but believe me if it would have been released with a match-making system that would have matched players in 5on5 teams against each other which would have played one map in a best of 5 mode the game would have never reached the same success. never.

players that striven for these 5on5 matches created leagues and founded teams. there they got real competitive games.

matchmaking tries to force this teaming and tries to create this "public competitive experience". i really think this is the wrong way. consoles should go back to the roots (multiplayer gaming roots). let public multiplayer be public but support people that want to play competitively with the option to create leagues and tournaments ingame!

 

[Shifty Geezer]

What im not fine with at all, is Shifty's suggestion of handicapping players. If this ever becomes the norm, i will stop gaming completely. Why on earth should you penalize a player for being good at your game?

For the same reason faster horses are handicapped. But I think you're imagining a complete levelling, rather than a more moderate rebalancing. So instead of a noob losing 30 deaths to no kills, they lose 20 deaths to 10 kills and feel they at least were contributing and had fun doing so. I'd have thought the best players would appreciated bragging rights to a huge handicapping too. "I've got a quarter your health and I still beat you!"

I understand that for some of you that are really really crap at FPS games, its frustrating to continously keep on loosing, but if one looks at the other side...

The idea is to look at online gaming from every side and come up with a system whereby everyone is having fun, not just either the bad players or the good ones. I'm not really, really crap, often coming near the top in rounds of games I've learnt. I could often by chief bigwig in a Warhawk round once I got the hang of it, but not with ace pilots to fight against. The addition of an awesome player renders me useless and turns the game from fun to annoying. That's not why I play computer games! The current system favours the best players almost to exclusion. Clearly though, player balancing doesn't go down well, although Tap In's post (I did see your first post but didn't appreciate what they meant by Dynamic Handicapping and its application as 'shifting momentum') suggests it's in effect unknown to players and making a positive contribution. It'd be good to get more info on this!

after reading my first post in this thread again, i think i struck the wrong tone. i really wasn't going for a troll post or anything. it's not always easy to find the right words as an non-native English speaker.

Your English is fine (although your capilisation isn't.

in a very natural way this works with the same principles in a dedicated server system. without any rules and rankings to force the issue. if someone has no fun playing on a server because he is not able to compete with other players he is going to find a new server.

What if there's no server at your ability though? Warhawk had server selection. First experience was miserable as there were no rank caps. They introduced noob servers which meant we had fun, until we hit rank 5 or whatever it was and had to venture out into the big, wide world, and then got slaughtered. You can set up your own level capped games but then you need to wait for players to choose your server, and with hundreds available it wasn't fun with only 4 people in attendance.

If you're going server based, you need a way to manage games to player requirements and still populate them. Matchmaking ensures games are full, but falls down with balancing. Hence, IMO, the need for a new model (though not handicapping if we're to keep Ostepop!).