Upcoming Events
Unite 2010
11/10 - 11/12 @ Montréal, Canada

GDC China
12/5 - 12/7 @ Shanghai, China

Asia Game Show 2010
12/24 - 12/27  

GDC 2011
2/28 - 3/4 @ San Francisco, CA

More events...
Quick Stats
116 people currently visiting GDNet.
2406 articles in the reference section.

Help us fight cancer!
Join SETI Team GDNet!
Link to us Events 4 Gamers
Intel sponsors gamedev.net search:

KJam - a new build tool for game development


Introduction

KJam is a build tool - like make. It is designed to support very large software projects, especially project which require building more than just code but data as well. It is also designed to be extremely fast.

I am a game developer. After my last game project I went looking for a build tool which was better than the tools I was using, Gnu Make and MSBuild (the build tool that comes with MSVC). The best tool that I could find was something called Jam, developed by the same people who develop Perforce. The great thing about Jam is that it is simple, powerful and extremely fast.

After working with Jam for some time, I started seeing things that I wished Jam could do. Specifically:

  • Batched Building: MSVC can compiled multiple source files into multiple object files in a single call to the compiler, much faster than it if you compile the files one at a time. But few build tools support that, and those that do are very slow.
  • Network Builds: Many game development projects require that data be compiled in addition to code. For example pre-computation of visibilty data, environmant maps, shadow maps and the like. I needed a build tool which could distribute these expensive build tasks over a distributed network of build machines. ( Like Incredibuild does with MSVC source files - but for data ).

Luckily the sources for Jam are available, and I started adding these features. But the more work I did to Jam the more I realized that things could be done so much better. I didn't intend to, but I ended up working on a completely new version of Jam for more than a year. I call it KJam.

KJam has several major benefits over Jam and other build tools:

  • Fast : KJam is the fastest build tool I have tested. KJam can scan a large codebase and build a dependency graph more than 20 times faster than MSBuild and Gnu Make. Through the use of parallel execution, and batched building it will build source files under MSVC significantly faster than other tools. In some cases KJam can reduce full build times by more than 3x. These accelerated build times are possible on multi-processor workstations
  • Easy: KJam's build language is easy to use, and designed to support projects which target multiple build environments. Build files are split into rules and actions . Rules describe how files are related to each other, what dependencies exist and where the files can be found. The rules can be completely platform independent. The actions describe exactly what build steps need to performed on a given target platform to build. Targeting a new platform normally means writing a new set of actions. The rules remain unchanged. This is a great idea take directly from Jam.
  • Robust: KJam is designed to support large projects. For extremely large projects, determining what targets need to be rebuilt can take minutes. Every compile becomes a chore. KJam can reduce this scanning time by a factor of 20 or more.
  • Distributed: KJam has a distributed build mode. KJam servers can be set up to run in the background on workstations all over the office, spreading the build load to multiple workstations, resulting in dramatic reductions in build times even above what is already possible in non-network mode.
  • Portable: By default KJam runs all build commands in its own internal sh-like shell. This means that developers have a single shell environment no matter what platform they are on. This is mostly to the benefit of Windows programmer who would otherwise be stuck using Window's extremely limited cmd.exe shell. With the built-in shell many build actions are completely portable across platforms.

KJam is currently still in development, but it is stable and useful enough that it is already used in production by several major game development studios for their next generation projects. A version of KJam is available for download free for use by non-commercial projects. Commercial developers are encouraged to contact us and join our beta program. Eventually, when KJam is perfect we plan to sell a commercial version. For right now we just want to make it easy for people to start using KJam and to give us feedback on how well it worked for them. Comments from our users currently drive most of the development of KJam.





Getting Started with KJam


Contents
  Introduction
  Getting Started with KJam
  Managing Sub-Projects

  Printable version
  Discuss this article