Introduction

Welcome to this article on using XML technologies for, what you might consider as, a relatively trivial part of a software library. Whilst this is a game development website (and everyone seems to love writing engines these days!) the content discussed herein should be of general use to any substantial software program and/or library – be it a game engine, a rolling demo, an API or just a helper library.

Taking a step back, this all started when I made a post "Is this possible? If so, how...?" [1] in the Gamedev.net Web Development forum [2] wondering if there were ways that I could improve the existing event logging system in a recent commercial game project. At the time I was exporting HTML directly from my logging system which resulted in files that were nice to view, but impossible to search or sort in any way. The project team each focus on a specific development area so the ability to filter log files is important since the logs themselves can often be several megabytes in size and contain many thousand entries. With help from Gamedev.net forum members I was able to take my existing knowledge of XML and apply it directly to this problem domain. Drawing the original inspiration from a simple example provided by Oli (evolutional), I was able to research this topic further and discovered a relatively simple yet powerful solution to my problem using XML with XSLT technologies. The resulting system was not only a very powerful system for the other developers, our testers, and myself but the most surprising thing was that the system came almost for free. The solution I am about to present requires no expensive technologies, no excessively steep learning curve and is relatively easy to deploy to the rest of the development team.

This article is a discussion of the original thread that serves as a demonstration in how you can use these technologies in your own projects. I personally had a big grin on my face when I realized the power these simple technologies yielded, so I am confident that you will be able to utilize them to enhance the logging system in your projects, whether it is an educational/learning project, an Indie game or even a full-scale commercial venture.

Why bother?

By "logging" I don't mean cutting trees down and making them into supplies for B&Q – I mean a common feature in software engineering that helps you diagnose problems (or lack thereof) in your application.

Log files typically consist of a long list of events in chronological order – indicating the status of variables, the results of decisions, warnings of potential problems and error messages when things go well and truly wrong. For any substantially complicated system, a log file can be invaluable in analysing various aspects of your applications runtime characteristics. Also, as a physical file written to disk (as opposed to writing to a console output) it can easily be attached to an email bug report or stored in an archive for later comparison.

So, why invest in this? Depending on your programming language of choice it's very simple to open up a text file and stream out a line of information each time something noteworthy happens. What is the point in spending valuable development time in making something more complex?

This method is okay for basic debugging – but as a piece of software becomes bigger and more complex, sifting through thousands (or millions) of lines of trivial output not only takes time, but is tedious and prone to user error. Conversely, a structured and well-understood format that contains as much information as possible can become a lot more powerful. If you think about it, given a suitably large set of data it's often easier to work out which elements are not useful and remove them, than to work out which elements are useful. By using a structured data storage method this becomes simple by tagging all the elements with filterable values/properties.

There are three main ways that you could implement a rich logging strategy:

1. Use proprietary file formats, algorithms and technologies
For example, running the output of your program to a relational database on a web server. This is exceptionally powerful if you combine it with SQL queries and a web-based front end. Unfortunately, this isn't easy to set up or maintain.

2. Use your own file formats, tools and algorithms
Design your own text file format that includes/formats all the information you could ever need. Then write some tools that take a given file and generate a report accordingly. This is about as powerful as you choose to make it – but that's the drawback, you have to write and maintain it.

3. Use open standard file formats and the readily available tools
With an open file format, and standardized tools you aren't necessarily linked to a given platform, software vendor or particular set of tools. Not only have other people done a lot of the work for you, you will often have the choice as to whose tools (and how) you use them.

Why use XML?

eXtensible Mark up Language [3] is the latest in a long line of similar technologies (e.g. GML and SGML [4] ) spanning several decades of modern computing, and is considered as an open standard (ratified by the ISO) governed by the W3 Consortium [5].

It does run the risk of being the current "in thing", but so far it's reasonably well supported across the majority of platforms available – both new and old. Also, the key consideration in its support across platforms is that lots of the tools are free!

If you're currently reading this document online, there is a good chance that the very same program (your web browser) can also display XML data, both natively in a hierarchical view direct from the file, or via a custom output style (XSLT). Also, the actual files (extra to that generated, at runtime, by your software) can all be developed using a text editor – better tools exist, but there is no requirement to use them. For the purpose of this article, I wrote my HTML and XSL files using the editor built into Visual Studio 2002 (my current development IDE).

Basic requirements and background

Before I get stuck in with explaining the finer points of how to apply XML technologies to software logging there is some background and foundation work to get through.

It is out of the scope of this article to explain all of the technologies from scratch – everything is pretty easy to learn, and I'd say it's even possible for any moderately intelligent individual to pick up everything they need from this article alone.

In order to generate a powerful logging tool it is not necessary to understand all of the technologies fully – there are some parts that just aren't that important. For example, one assumption (that may be frowned upon by some!) is that we will be completely skipping validation with DTDs or schema. This assumption is based on the idea that a computer program will be writing the source XML, and provided the code is written correctly the output should also be correct. However, if you desire this extra validation or wish to do further work on the XML, it is advised that you look into creating your own DTD for your logfiles.

As a quick summary, competency in the following 4 areas is necessary:

1. XML
   • Basic XML tag definitions/rules
   • Not the DTD/Schema validation
   • Read through the information at [3] and [6]
2. HTML
   • Basic HTML document structure
   • Text formatting (bold, underline, italic, fonts)
   • How to construct tables
   • Read through the description at [7] and the tutorials at [8]
3. 3. XSLT
   • Basic structure and usage of XSLT to produce an HTML output
   • Use of parameters
   • Conditional branching ('choose' and 'if' statements)
   • Read through the description at [9] and the tutorials at [10]
4. 4. JavaScript
   • How to execute JavaScript from within an HTML context
   • Basic JavaScript function definitions and variable declarations
   • Read through the information at [11] and the tutorials at [12]

If you feel you're fairly proficient on all of these, you can skip to the next section, otherwise I suggest you read the recommended links and have a go at this article later.

Now that I've mentioned all of the core technologies that will be used in this article, it's necessary to mention a few of the necessary tools.

As previously stated, the required tools are all free and readily available. All you really need is a text editor and a web browser. Given that we're going to be integrating this directly with the software, I assume you already have that set up and working, it may well be that you can use your development tool suite.

Using Microsoft's Visual Studio 2002 (and later) it is possible to have your HTML page to be viewed in the browser component (where MSDN usually pops up), then another tab with the HTML source code, another tab with the XML source (for reference) and a final tab with the XSLT code. Changing something in one of these tabs, saving changes and then refreshing the original browser pane worked extremely well for me.

A simpler set of tools would be to use Notepad (or equivalent under Unix/Linux) and a browser such as FireFox [13].

During the course of developing this technology, I came across a variety of problems with cross browser compatibility (a.k.a. making it run on Mozilla and Internet Explorer) and it is quite easy to write a very clever logging system and have it fail to operate on a different browser. This article has been tested with FireFox and Internet Explorer 6 under WindowsXP SP2 and appears to work fine. One small caveat is that under SP2, Internet Explorer can get quite picky (with respect to security considerations) about what it will and won't allow – so you can end up having the browser refusing to view your own files!!

Basic requirements and background