Welcome to the Networking and Multiplayer forums!

We're very lucky, because this forum is one of the most on-topic forums, and one of the highest signal-to-noise forums on all of gamedev.net. Let's keep it this way! For example, if you ask something, but get no replies, please don't bump your thread. If you get no answer until the thread falls to the bottom of the first page, more bumps won't help. Re-formulating the question might, but please don't re-post the same question unless it's a very different formulation showing you've come to a deeper understanding of the problem since last time.

Some questions, while they are on-topic, do tend to come up again and again. I'll just give you the answers, and you don't have to go through all the trouble post and have to wait for an answer -- they're right here!

Q0) I have this dream to start the most uber MMORPG evar but I don't know whether I can write the networking part in javascript. Would Visual Basic be better?
Q1) I want to start learning about networking. Where do I go?
Q2) Should I use DirectPlay or WinSock?
Q3) Should I use TCP or UDP?
Q4) If TCP makes sure the data gets to me without me needing to worry about it, why would I want to use UDP, which makes no such guarantees?
Q5) Is UDP faster than TCP?
Q6) I'm sending data with TCP, but my ping times are insanely high! It seems the data is slowed down somwhere on the way. What's wrong?
Q7) Is there an easy wrapper on top of UDP which will give me reliable packets for packets that need it?
Q8) What are some other networking libraries that often get mentioned in the forum?
Q9) Why can't I host a game when I'm behind a firewall or DSL router?
Q10) Should I spawn a thread per connection in my game code?
Q11) Should I use blocking sockets, non-blocking sockets, or asynchronous sockets?
Q12) What should I put in my network packets, and how often should I send them?
Q13) Why can't my FD_SET be bigger than 64 on Windows, and 1024 on UNIX?
Q14) I'm trying to send two packets over TCP, but it seems like they arrive "merged" or one packet disappears on the receiving end. What's going on?
Q15) How do I send a struct using send() or sendto()? They only take char*!
Q16) How do I interpolate my character based on server lag, and still keep it in sync with the server?
Q17) Is a peer-to-peer MMORPG a good idea? Wouldn't that save a lot on server bandwidth?
Q18) How do I get the IP address of the machine I'm running on?
Q19) No, really. I want to use TCP for my MMORPG.
Q20) I need a server that will stay up longer than my cable modem, but typical web hosts won't let me run my own server program. What are my options?
Q21) How does my client find the server it wants to talk to?
Q22) Why do I get an error when trying to open a new socket soon after I've closed a socket on the same port?
Q23) Do I need to send 4 bytes to send a float for position or heading in my entity state update packets?
Q24) How do I find a server for my game? Currently, I'm hard-coding an IP address, but that won't work when I ship it to others.
Q25) How do I synchronize the clocks between the client and the server?
Q26) How can I see what's going on on my network? Is there a good networking debugging tool?
Q27) I'm using client prediction by simulating objects client-side, but update the entity positions to the "true" positions when I receive updates from the server. This causes snapping in the position on-screen. How can I hide this snapping?
Q28) What does the canonical chat server code look like?
Q29) Is it possible to write a MMO server that can possibly handle a hundred simultaneous connected clients in about four hours?
Q30) I don't want to have to buy servers for all my players, so I'd like to use peer-to-peer for my 8-player games. Is that a good or bad idea?
Q31) What would the interface for an easy-to-use networking library look like?
Q32) Project Darkstar claims in their FAQ to solve all my problems, and scale to however big my game becomes. Can that really be done?
Q33) If I want to set up a large multiplayer game server with support for things like forums and VoIP, what kind of servers do I need?
Q34) I'm following a tutorial for {some library or engine} but it {crashes/doesn't work}; what should I do

Q0) I have this dream to start the most uber MMORPG evar but I don't know whether I can write the networking part in javascript. Would Visual Basic be better?
A0) The best place to start is here.
After you have assimilated the full knowledge of that highly relevant article, you're ready to move on to answer 1 below, or a link such as this.

Q1) I want to start learning about networking. Where do I go?
A1) First, make sure that you understand pointers, structures, unions, and how a compiler will lay out data in memory. For C# this means understanding the StructLayout attribute and the use of unsafe pointers. For Python, this means understanding the "struct" package. For C/C++, this means understanding void*, char* and generally how "struct" and "union" are implemented. Then, you can start worrying about networking, typically using sockets.
Try Beej's Networking Tutorials. You could also try the intro section right here on this site.
Also try the WinSock FAQ for sockets programming, especially as relates to Windows, and the GameDev.net WinSock tutorial.
Last, the MSDN section on WinSock has a decent overview section, and good reference documentation.
If you want to know what to do about networking in games, specifically, check Gaffer's tutorialsthe answer to question 12 below.

Q2) Should I use DirectPlay or WinSock?
A2) As of Summer Update 2004, Microsoft has put DirectPlay into "sunset" mode, where they will not develop the API further. Microsoft now recommends you use WinSock for networking, until the re-cast X-box Live! technology is rolled out for PCs, probably around Longhorn time-frame. If you're on Linux, MacOS, or some other platform like that, life is easier: you only have regular sockets to work with. Luckily, sockets and WinSock code can look very similar if you stick to the Berkeley sockets functions of WinSock, and avoid WSA functions; you can write quite portable programs with a little bit of thought.

Remember to include <winsock2.h> before including <windows.h> in your win32 program, or compile errors will ensue. Also, always start your WinSock program with WSAStartup( MAKEWORD(2,2), ... ) to get version 2.2 of WinSock, as the older versions have bugs and limitations that you don't want to deal with. Finally, you need to link with ws2_32.lib on Win32 when using WinSock.

Q3) Should I use TCP or UDP?
A3) There are at least four different kinds of games as far as networking is concerned: Turn-based games, Real Time Strategy games, Role Playing Games, and Action Games. Briefly, if you're turn-based, you should go with TCP because it's simpler, unless you need peer-to-peer NAT punch-through. For RTS games, you're usually best off using TCP, although UDP can make sense in extreme cases. For role playing games, the story is less clear -- action-based RPGs with lots of kinetics, like City of Heroes, use UDP, whereas slower RPGs and MUDs often stay with TCP. For action-based games like first-person shooters or racers, you should absolutely use UDP.

Q4) If TCP makes sure the data gets to me without me needing to worry about it, why would I want to use UDP, which makes no such guarantees?
A4) TCP is a stream-based protocol. What goes in one end, will come out the other end, in order, with no duplication or packet drops. This is very convenient when correctness is paramount. However, to make this guarantee, TCP must suspend all packet delivery to the receiving end, if one packet is lost, until that packet can be detected as lost and re-sent. This may cause hick-ups of several seconds for a single packet that's lost!

Q5) Is UDP faster than TCP?
A5) UDP uses slightly smaller headers per packet than TCP, which may give you marginally better throughput on narrow channels (such as modems). UDP will also deliver any received packet as it is received, without waiting to re-order packets, so if latency is more important than correctness of the underlying channel, then UDP will introduce less jitter in your game, and gameplay will be smoother.

Q6) I'm sending data with TCP, but my ping times are insanely high! It seems the data is slowed down somwhere on the way. What's wrong?
A6) When you send data on a TCP connection, the network packet typically implements something called Nagle's algorithm. This will make the system not immediately send data if you don't fill up an MTU's worth of data in a single write; instead it will sit and wait for some time (200 milliseconds is typical), hoping that you will write more data that it can coalesce into a single packet. You turn OFF this delay by turning ON the TCP_NODELAY socket option.

Q7) Is there an easy wrapper on top of UDP which will give me reliable packets for packets that need it?
A7) I hear Enet is pretty good, although the documentation is pretty sparse.

Q8) What are some other networking libraries that often get mentioned in the forum?
A8) Low-level "sockets portability" libraries include:
Enet
HawkNL
PLib/Medusa
SDL_net
lidgren network library (for C#)
Boost ASIO (asynchronous I/O)

High-level "we give you structure" libraries include:
ClanLib
Net-Z/Eterna
Nevrax/NEL
Massiv
OpenPlay
OpenSkies
OpenTNL
RakNet
ReplicaNet
ZoidCom
Etwork

If you want to make an MMORPG, there are some libraries as well:

PlaneShift is open source to the core (including art).
Nevrax/NeL is open source, but tools/art is commercial (The Saga of Ryzom).
RealmCrafter comes with tools, starter art, and a small game; it's $55.
Torque plus RPG Kit at $249 total make for an RPG toolkit.
Massiv (hasn't been updated since 2004).

There are also several commercial libraries that provide more than just networking, such as Forterra Systems OLIVE, the Multiverse platform, Sun Project Darkstar, etc, but those are too extensive in scope to easily fit in the "multiplayer" forum.

Note that, with all of those libraries, scaling to a large number of players is as much derived from how you design your game, as from any specific technology (or lack thereof).

Mention (or the absense of mention) of a specific library does not mean gamedev.net or the moderator endorses (or counter-endorses) that library. The most commonly mentioned libraries are Enet, RakNet and OpenTNL, although your mileage may vary.

Q9) Why can't I host a game when I'm behind a firewall or DSL router?
A9) You're probably suffering from the Network Address Translation in the firewall/router. There's a number of documents describing how to set up an introducer system to punch through NAT and allow you to host from behind a firewall. Try this web page or wait for my upcoming article in Game Programming Gems 5 (out spring 2005).

Q10) Should I spawn a thread per connection in my game code?
A10) The short answer is "likely no." The longer answer is that, if you're using UDP, you should only need a single socket, from which you can read a packet, process a packet, repeat. If you're using TCP, you will need a socket per player, but the players are likely sharing and affecting a single world state inside the program, which needs to be protected by locking; thus, the extra threads would just end up blocking on locks anyway, so using select() and reading out what's there, then processing it, is recommended. Some users find "co-operative threads" or fibers to be a nice middle ground, because it gives some of the advantages of real threads (a stack per context) without the locking overhead (because yield is explicit).

Q11) Should I use blocking sockets, non-blocking sockets, or asynchronous sockets?
A11) On UNIX (at least Linux), asynchronous sockets are not generally available, although you can arrange to get a signal when "any fd" is ready (FIOASYNC). If you're on windows, and you're prepared to deal with the significant management overhead, they can lead to good performance. However, using non-blocking sockets in concordance with select() is usually not only the easiest to implement correctly, but will also perform well -- I recommend not doing anything fancier unless a profiler tells you to (i e, until oprofile tells you you're spending 10% of your CPU time inside select()).

Blocking sockets makes it much harder to write a real-time game, because, well, they block! If you're writing a file transfer program or something like that, especially if it's point-to-point (only a single connection), blocking TCP sockets are very easy to work with, assuming you can live within the constraints of that model.

If you are Windows-only, a high-performance alternative is Overlapped I/O on sockets.

Last, there's a good write-up of TCP/socket performance, with some comparisions of Unix and Windows (although mostly Unix-based), that's recommended for people worried about throughput.

Q12) What should I put in my network packets, and how often should I send them?
A12) That depends on your goals and your game.
For an FPS style game, you might want to read about how Quake III, Unreal, Source/HalfLife and Tribes does it. (If that link doesn't work, try webarchive.org, although that's very slow.
For a turn-based game, the most important thing is to make sure that players stay in sync; bandwidth and latency doesn't play into it as much.
For a real-time strategy game, bandwidth compression becomes paramount, because there is often a large number of units moving around. A good article (known as "1,500 Archers") is on GamaSutra.com.
For data compression approaches, Jon Blow wrote a series in Game Developer: here.
There's also this thread on the forum about RPG and RTS command/packet encoding that might be helpful.
For MMORPGs, the typical solution is to send the position & state of some entity (possibly with delta compression and periodic baselines), every so often, where you send it more often the closer the player is to the entity. If you have a good link describing this, please let me know!
Other recommended articles include BookOfHook GameDesign Wiki and JakeWorld networking articles.

Q13) Why can't my FD_SET be bigger than 64 on Windows, and 1024 on UNIX?
The implementation of FD_SET on Windows is a big array of socket values, and adding, testing, and removing sockets is linear in number of sockets in the set. Luckily, modern caches make this not be as terrible as you might first think, but hard-coding a solution that loops over the FD_SET specifically to extract ready sockets is probably a good idea. Windows does this because its sockets are not low integer file descriptors.
The implementation on UNIX is a bit mask, where you set bit N if you want to examine file descriptor N.
Both of these implementations can easily be extended if you declare your own file set structure compatible with the operating system structure, but bigger. Most system headers will also allow you to define FD_SETSIZE on the command line, to make for a bigger set size without re-declaring the structure.

Q14) I'm trying to send two packets over TCP, but it seems like they arrive "merged" or one packet disappears on the receiving end. What's going on?
TCP is a stream protocol. Everything you push in one end will arrive, in order, out the other end. However, the "boundaries" you establish by calling send() on one end are not preserved in the stream. The TCP implementation may split the data such that multiple send() calls get merged into one recv() call, or so that a single, large send() gets received through multiple recv() calls.
To construct message semantics on TCP, each time you send a message, you have to precede it with the length of the message, as a short or a long (use htons() or htonl() to make it cross-platform). Then write that many bytes of message. On the receiving side, you'd first read out the length (a short or long) and then that many bytes of message data, re-constituting each message sent from the other end.

Q15) How do I send a struct using send() or sendto()?
The socket send functions may be prototyped using char * in some older headers, but they really take just a byte pointer and a size. You send a struct the same way you would write it to a file. To send and receive a player position update, some simple code might look something like this:

  struct PlayerPosPacket {
    unsigned short code;
    unsigned short player;
    float pos_x, pos_y, pos_z;
    float vel_x, vel_y, vel_z;
    float heading;
  };

  PlayerPosPacket packet;
  packet.code = PLAYER_POS;
  packet.player = thePlayerId;
  ...
  int r = sendto( sock, (char const *)&packet, sizeof(packet), 0, (sockaddr *)&dst_addr, sizeof(dst_addr) );
  if( r < 0 ) { error(sock); }

  union {
    PlayerPosPacket playerPos;
    SomeOtherKindOfPacket someOther;
  } recvPacket;

  int r = recvfrom( sock, (char *)&recvPacket, sizeof( recvPacket ), 0, (sockaddr *)&src_addr, sizeof(src_addr) );
  if( r < 0 ) { error(sock); }
  /* assume all packet types start with unsigned short code; */
  switch( recvPacket.playerPos.code ) {
    case PLAYER_POS:
      if( r != sizeof(recvPacket.playerPos) ) { formatError(); }
      do_playerPos( recvPacket.playerPos, src_addr );
      break;
    case SOME_OTHER:
      if( r != sizeof(recvPacket.someOther) ) { formatError(); }
      do_someOther( recvPacket.someOther, src_addr );
      break;
    ...
  }

Q16) How do I interpolate my character based on server lag, and still keep it in sync with the server?
There's a great description in the Zen of Networked Character Physics thread.

Q17) Is a peer-to-peer MMORPG a good idea? Wouldn't that save a lot on server bandwidth?
First, server bandwidth isn't actually the first, or even second cost for an operating MMORPG anymore. Content production, customer service, and other people-related costs are usually much higher.

Second, the two main technical challenges of peer-to-peer MMORPGs are hard to solve in a sufficiently satisfactory manner for current game designs:
- How to secure the game state data such that a user could not hack his client, or data files, or attach to the client with a machine monitor when running, and change the simulation to his advantage. This is, technically, not possible without a secure-computing platform like the much-maligned Palladium.
- How to find the right other peers to interact with with sufficiently low latency, and update this mapping when players come and go.

For now, peer-to-peer works best for transfer of large bulk items where you can easily verify correctness, such as content downloads and patches. If you're thinking about posting a thread regarding your great new peer-to-peer MMORPG design, make SURE you've written some code, and actually tested it on at least three collaborating machines, before you start. Else most people will likely just sigh at your unfettered newbie optimism :-)

There's some existing published research, for example that collected into the VAST project. You should check up on this, too, if you're interested in P2P MMOGs.

Q18) How do I get the IP address of the machine I'm running on?
Typically, it's a bad idea to get your own IP address, because it's not certain that the address is visible to people outside your router. Also, many machines have more than one IP address (each interface has one in a machine). Last, not all ISPs or routers have DNS/reverse DNS set up such that gethostname() followed by gethostbyname() will give you the right answer.

To use command-line tools to list your addresses on Windows, use "ipconfig". On Linux, use "ifconfig". On BSD, use "ifconfig" but you probably have to be root.

In a browser, you can go to www.whatismyip.com; this will show your EXTERNAL IP address (that of your router), not the INTERNAL IP address (that of your machine) if you're using a NAT firewall.

To programmatically determine the CORRECT address to use, you have to bounce a query off a server that's available on the public internet, and look at what address it returns back. This will work correctly for situations where you're behind NAT (routers/firewalls).

If you want to know the list of all interface addresses, say for selectively binding to only some interface, then you need to use OS-specific system calls; i e you need to do whatever ipconfig/ifconfig does, which involves the network device API on Windows, and walking /proc on Linux.

Q19) No, really. I want to use TCP for my MMORPG.
Sure, it can be done. Check out this long-running and productive thread for a large variety of viewpoints and suggestions.

Q20) I need a server that will stay up longer than my cable modem, but typical web hosts won't let me run my own server program. What are my options?
Assuming you don't have friends or a school that'll allow you to run your internet server process, you have four basic options:
1) Self-managed Hosting
2) Virtualized Hosting
3) Server Co-location
4) Roll Your Own

1) Self-managed hosting is like web hosting, except you actually lease a full machine, with some amount of network bandwidth included. You get root access to the machine (or Administrator, for Windows), and can install whatever you want. You typically get access to some web page that will allow you to reboot your machine if necessary. If you screw it up, you have to pay the hosting company to re-image the machine (which will lose your files, so have back-ups!). Places include ServerBeach, 1-and-1, ServerMatrix, Aplus.net and a host of others. Do some research on their customer service and availability, because it varies wildly. Prices start at around $60/month, with everything from 100 GB to 2000 GB network transfer per month included. Watch out for overage charges, which is how some of these places make their real money!

2) Virtualized Hosting. This is like self-managed hosting, but you can usually pay for only the time you use, and you may be able to lease only a fraction of a server. Often, these types of server plans suffer from the virtualization used, such that they don't work well for real-time servers like you would use for a first-person shooter, but for a turn-based game like poker, it would work fine.
Examples include the Amazon Compute Cloud (which lets you rent a virtual Linux or Windows server by the hour), and the Google App Engine, which lets you host a Python application written to the Google App Engine API on a virtualized server cloud. Other similar solutions are offered by web hosting companies. One option that's affordable is to set up a 250 MB virtual Linux slice with a hoster like DreamHost. $10/month for the web hosting, and $25/month for the virtual slice makes for $35/month for a permanent server. (If you decide to go that route, you can use promo code "L33TN3SS" to get $50 off the first year, and a free domain registration).

3) Server Co-location is the next step up. You buy the hardware, and lease some space in a rack or cage at a well-connected facility. You get physical access to the facility, and they provide you with what's basically an Ethernet cable wired to the Internet. You agree on some level of service an bandwidth, which will determine the price. Typically, traffic is measured over 5 minute intervals, and you'll pay for the amount of traffic used by the 95th percentile measurement each month. Prices start out higher than self-managed plans, just because of the physical set-up, but when you have ten servers or more, co-location starts looking pretty attractive (assuming you live within reach of a co-lo hosting center, of course). Typical costs for a set of servers runs four digits per month.

4) Roll Your Own: Buy the hardware. Buy the management and control infrastructure. Buy the cooling. Get a wider power cable from your utility company. Fight city zoning ordinances. Take out a mortgage on your house to afford the up-front cost of DS-3 service and up (45 Mbps). Find an internet service provider that will agree to route your traffic. Congratulations! You just spent months building your own data center, and you're now in business! Expect to pay five digits to build a sufficiently heavy-duty set-up.

Q21) How does my client find the server it wants to talk to?
There are several ways of finding the server, depending on where the client runs with regards to the server.

1) Some clients allow the user to type in an IP address, or a server name to use with DNS. These then get saved to a settings file for easy access later.

2) Some clients have a hard-coded list of servers/IP addresses. This is very similar to option 1).

3) Some clients have a hard-coded name of a "master server" which will list the actual available servers. Typically, this hard-coded name will be text form (not IP form) so that the master server can be moved (or DNS load balanced) when necessary, without patching the clients. The client will connect to the master server, get a list of available actual servers to talk to, and let the player choose (or choose for the player).

4) For LAN play, the most common thing to do is to use UDP broadcast on a known port, perhaps once every few seconds. Start the packet with some known signature (a few special bytes) so you can tell your own packets from someone else's packet if there happens to be a collision. Make sure to turn on the broadcast option on the socket before attempting to broadcast. Broadcasts only work within a current subnet; they will not be routed to the greater internet.

5) If you want a pretty good implementation of option 4), that also happens to be open source, AND is the technology underlying the iPod/iTunes device discovery, then look no further than the mDNS package from Apple. It's pretty easy to use, and works pretty well for local LAN service discovery. mDNS has some optimizations to reduce network traffic, too.

Q22) Why do I get an error when trying to open a new socket soon after I've closed a socket on the same port?
This happens in TCP, because the implementation will wait for a while to allow any late, missed fragments to arrive (even though nobody will read them). This is to avoid confusing fragments from a previous connection with fragments from a new connection.

To turn off this behavior, turn on the SO_REUSEADDR socket option as soon as you've created the socket; this will allow future sockets to re-bind to the same port without waiting for the timeout to expire.

Q23) Do I need to send 4 bytes to send a float for position or heading in my entity state update packets?
To pack a floating point number of a known range into a fixed number of bits, do something like:

unsigned int pack_float( float v, float minVal, float maxVal, int bits ) {

  assert( bits < 32 );

  assert( v >= minVal && v < maxVal );

  return (unsigned int)((1U<<bits) * (v-minVal) / (maxVal-minVal));

}

float unpack_float( unsigned int val, float minVal, float maxVal, int bits ) {

  assert( bits < 32 );

  assert( val < (1U<<bits) );

  return val * (maxVal-minVal) / (1U<<bits) + minVal;

}

Q24) How do I find a server for my game? Currently, I'm hard-coding an IP address, but that won't work when I ship it to others.
If your game is "MMO" and needs a specific server back-end, register a domain name for your game, and look up your login server using a name such as "login.instance1.yourgame.com".

If your game is networked over the internet, but uses user-hosted servers, then you need a matchmaking/lobbying/introduction service. The service itself lives at a known DNS name; hosts and clients connect to this server to register/query the database of active hosted games. You may also be interested in NAT punch-through in this case; see previous questions in this FAQ.

If your game is networked over LAN, then you need to use some kind of broadcast announcement. More information can be found in this article on LAN matchmaking.

Q25) How do I synchronize the clocks between the client and the server?
In general, you don't need to perfectly synchronize the clocks; only the correct ordering of events is necessary, and typically events are said to happen at regular intervals ("steps" or "heartbeats"). As long as heartbeat X happens on all machines before heartbeat X+N happens on any machine, for some upper bound N (that does not have to be 1) you are doing fine.

There is a classic article by Lamport on the subject for those who want an in-depth analysis.

Q26) How can I see what's going on on my network? Is there a good networking debugging tool?
Most users on the forum recommend the free Ethereal packet sniffer (which has recently changed names and sites to WireShark).
Check out WireShark.org or the older Ethereal.com web site for downloads.

Q27) I'm using client prediction by simulating objects client-side, but update the entity positions to the "true" positions when I receive updates from the server. This causes snapping in the position on-screen. How can I hide this snapping?

Option 1: Use a forward extrapolating algorithm, such as Entity Position Interpolation Code.

Option 2: You can use the simulated position, and only if you receive a server timestamp that's singificantly off where the simulation says, will you start sliding the object over. When you're in the mode of sliding the object, you immediately set the simulated position to the server update position, but you set the display position to the old position; you then set a timer that goes over about half a second, where you draw the object at the old position in the beginning, and at the new simulated position (which updates) at the end.

objectSimulationStep() {

  updateSimulation();

  if (interpolateTimer > 0) {

    displayPosition = lerp(simulationPosition, rememberedPosition, interpolateTimer / InterpolateTime);

    interpolateTimer -= deltaT;

  }

  else {

    displayPosition = simulationPosition;

  }

}



objectUpdate(position, velocity) {

  simulationVelocity = velocity;

  if ((position - simulationPosition).length() > DeltaPos) {

    rememberedPosition = simulationPosition;

    simulationPosition = position;

    interpolateTimer = InterpolateTime;

  }

}

Q28) What does the canonical chat server code look like?

A: This code uses TCP, and uses some well-known behaviors about send(), recv() and select() that you should study.

forever:

  select()

  if the accepting socket is readable:

    if maximum number of clients is not reached:

      accept a new client and allocate a new client structure

      go back to select

    else:

      accept a new socket

      immediately close it

  for each readable socket from select:

    recv() on the socket, and push that data onto the end of the client queue

    if recv() returns 0, the client disconnected:

      remove the client structure and close the socket

  for each writable socket from select:

    send() on the socket from the client ougoing queue

    remove actually sent data from queue

  for each client (that received data, or all clients)

    check whether there's a full message at the front of the queue

      if so, deliver that message to the incoming queue of the client, and remove from the incoming queue

Q29) Is it possible to write a MMO server in about four hours?

A:Yes.

(Not entirely in jest)

Q30) I don't want to have to buy servers for all my players, so I'd like to use peer-to-peer for my 8-player games. Is that a good or bad idea?

A: Note that client/server doesn't generally mean that you need to pay for a central server (other than lobby server, which you need in either case).

In most cilent/server games, one player will "host" a game, and other players will "join." That player will then be the "server" and the other players the "clients." This means that everyone send their commands to the host, and the host sends a copy of everyone's commands to everyone.

In a true peer-to-peer topology, everybody sends their commands to everybody else, so no one machine is the "host" or "server."

Let's consider the amount of bandwidth used, with some assumptions:

State/command record/framing per player per tick: 24 bytes.
Protocol framing overhead per packet (clock etc): 8 bytes.
UDP overhead per packet: 28 bytes.
Send rate: 20 packets per second.

When I say "server" I really mean "hosting client."

Client/server client:
Send: 20 * (28 + 24 * 1 + 8): 1200 bytes/sec
Receive: 20 * (28 + 24 * 7 + 8): 4080 bytes/sec

Client/server server:
Send: 20 * 7 * (28 + 24 * 7 + 8): 28560 bytes/sec
Receive: 20 * 7 * (28 + 24 * 1 + 8): 8400 bytes/sec

Peer/peer node:
Send: 20 * 7 * (28 + 24 * 1 + 8): 8400 bytes/sec
Receive: 20 * 7 * (28 + 24 * 1 + 8): 8400 bytes/sec

You will note that every node will send a lot less data when the nodes are clients. This is important because upload is typically a lot more limited than download on DSL/cable connections.
You will also note that overall network bandwidth consumed is a lot less in the client/server case.
For peer/peer nodes, because everyone sends an equal amount, the uplink rate of the slowest peer will determine quality of gaming.
Finally, you will note that the poor server will receive as much as any peer/peer node, but it will send more than that. Thus, in C/S, the upload rate of the server will determine quality of gaming.

It is generally easier to find one node with decent upload bandwidth (to act as the host), than to make sure that everyone has sufficient upload in the peer/peer situation.

Finally, if you worry about NAT punch-through or firewall port forwarding (which you generally want to), then EACH peer node needs to have a port forwarded, or have a firewall/router that allows NAT punch-through. As soon as one node doesn't support that, that node cannot play. With a client/server approach (user hosting), then only the hosting/serving user needs a port forwarded, and/or NAT punch-through to work.

No matter what you do, you won't get away from needing a lobby server, and generally you will want this lobby server to also do NAT punch-through introduction, which means that basing the lobby server on a "free web host" (or any web-server based solution) won't actually work. In the end, if you want to run a good quality networked game, you'll need to keep some kind of machine always online, running a process that's specific to your game.

Q31) What would the interface for an easy-to-use networking library look like?

See this blog article about a networking library interface by Jon Watte.

Q32) Project Darkstar claims in their FAQ to solve all my problems, and scale to however big my game becomes. Can that really be done?

PDS (Project DarkStar) is designed to scale a transactional game back-end like the rules engine for EverQuest or WoW using an easy-to-program transactional Java object update model. To achieve this, you lose some of the convenience you may know from the SQL world. For example, to implment a high score table that can be sorted by various criteria, you would have to code it explicitly up front, and you would add additional commit load to the server each time a highscore was updated for each kind of sort criteria that the highscore might partake in. This is because there are no ad-hoc indexed queries. Another key to scaling with Darkstar is to make the client authoritative over quickly changing transient state, such as precise location. PDS is not designed to solve the problems inherent in doing server-side authoritative physics, or doing lag compensation for action games over the internet.

Also of note: A lot of the overview documentation and marketing material talks about the design intent of PDS, not what the current code actually implements. If you want to develop servers in Java and base a project on PDS (or any other library, for that matter), you should carefully examine what it can do for you, compared to what other libraries can do.

Q33) If I want to set up a large multiplayer game server with support for things like forums and VoIP, what kind of servers do I need?

You generally need a web server for marketing, an application server of some sort for the game, and a database server for persistently storing state.
You probably also need the typical data center set-up of load balancers, monitor servers, firewalls/reverse-NAT, managed switches, etc.

I'd recommend a few of these, or the equivalent from one of the competitors:
PowerEdge Blades
PowerEdge Blade Enclosures
Storage Servers (I'd probably go all SSD these days)

You also need to wire it all up:
ServerIron Load Balancers
Arista Switches
(I'm sure Cisco or HP or whomever would sell you equivalents, too)

OS software:
Ubuntu Lucid Lynx (10.04 LTS)
or, if you're a Windows guy:
Windows Server 2008 R2 SP1

Pick a hosting location and bandwidth provider, develop your game, and you're good to go!

Or you might want to take a chance on "the cloud." It's not so great for real-time apps just yet, but it's a lot easier to get started with than your own data center.
Try:
Amazon Cloud Servers

For VoIP, you can license something like TeamSpeak or go with a provider like Vivox.

For forums, there are a zillion tools available, although if you want RuneScape or WoW scale forums, you want one of the top end names (vBulletin, IP.board, UBB.threads, etc).

For application server, one will likely come with your game engine. If not, then it gets tricky; simple systems like node.js or Python Twisted don't scale well across threads, but scale great if you just create many smaller instances on a single multi-core host. I've had terrible experiences with larger, integrated stacks like Project DarkStar or BigWorld.

For web server, Apache comes with Ubuntu, and IIScomes with Windows Server.

For database, you probably want MySQL as the default (or SQL Server on Windows) -- although don't forget, MongoDB is Web Scale!

Good luck on your project! (Don't forget to buy a pager so that you'll get notifications when something breaks)

This thread talks about how to learn network programming. Yes, you really need to take all that time and follow all those steps.