Extremely Simple Scripting Engine
by Cyberdrek

Introduction

Hi, I'm Cyberdrek from the boards. Some of you have seen me answer some posts in the forum. Ok, I have an attitude and I'm sometimes hard on people but I'm not all bad, the proof is this article that I'm giving the community about how I first wrote my scripting engine. Keep in mind that this engine was my first attempt at creating a scripting language and was actually pretty limited. At the time of writing, I'm still not sure if I'll give out any source code since the engine is proprietary code for HSSL (Headhunter Soft Scripting Language). Even though the new version of the engine has some major changes in it, most of the code still fits the bill. Enough for the intro, let's get to the planning.

Planning

I wanted to create a scripting engine that would allow me to manipulate certain aspect of my game that I was writing at the time so I started thinking about how I could implement it. At the time, I wanted to create an engine that would work like the one used by Sierra to create all their games. Having read a bit about compilers and how they work, I knew I needed to tokenize my input and that I'd need an interpreter (also known as a VM) to execute the code. I thought about it for a long time and decided to go with an OOP approach since it would help keep the code clean. NOTE: I used STL to create the engine so if you don't know much about it, grab a couple of STL tuts and go through them. You don't need to know much, just vectors and strings.

Language

Here's usually the fun part since everybody has his vision of how a scripting language should look. My vision was pretty simple. I wanted simple instructions (about 5 chars max) and I wanted syntax a bit like ASM. With that said, on a side note, I don't use any parsers to check the syntax in this version of the engine, even though the tokenizer is in a class called CParser.

Last Words Before We Start

Before we start, I have a couple of other things to clear up. At first, I had created the engine as standard header and source files. Once I had tested the engine and made sure it was working correctly, I moved on to putting it in a DLL file. What I'm trying to say is, don't be scared if you see __declspec(dllexport) before the definition of a function. Also, note that all header files are kept to a bare minimum; in other words, one class definition per header, and the same for source files. I've taken the liberty to encompass the engine in a namespace called HSScript. After trying both ways of using STL, I felt more comfortable without the command "using namespace std" so I don't use it and scope everything that's related to a certain namespace. It just makes it easier, IMHO, to follow the code. 

Let's Begin

As I stated earlier, I started out by designing my HSSL class as follow.

namespace HSScript
{
   class HSSL
   {
   private:
      CParser parser;
      // ** HSSL version for reference     
      std::string version;
      std::string copyright;
      int     count;
      int     varInt[ 10 ];
      float   varFloat[ 10 ];
      int     fPtr;
      int     iPtr;

      void handleTokens( const std::vector<std::string> &instruction );

   public:
      HSSL();
      ~HSSL();
      
      std::string getCopyright( void ) const;
      std::string getVersion( void ) const;

      int getCount( void );
      void resetCount( void );
      
      void setInt( int index, int value );
      int getInt( int index );

      void setFloat( int index, float value );
      float getFloat( int index );
      
      int execute( const std::string &filename );
   };
}

Up to now, as you can see, it's just plain and simple definitions. Notice also that the handleTokens function is declared as private. And that I declare an instance of CParser into the declarations of the HSSL class. This is just an old habit. I know I could've done it differently but hey, every coder has his way of coding. I could have made, what I refer to as, a class function ( ie: A class that only defines the operator() but I wanted it to be simple to modify later on ).

With that header done, I knew I needed another one in order to put the CParser class and defined it like so:

namespace HSScript
{
   class CParser
   {
   public:
      CParser(){};
      std::vector<std::string> tokenize( const std::string &codeVec );
   };
}

Once more, nothing complicated here. An extremely simple class with only one function( in later versions, it holds a bit more stuff ). All that is left for use to do is simply write the code in source files. So here we go. The first file I created was the HSSL source file. Keep in mind that the engine was simple at that time. The handleTokens function is a bit long but that's been corrected and changed in the version we currently use.

const std::string RETCHAR = "_";

namespace HSScript
   { 
   // temporary debug function to print the values on screen.
   // ** Remove once the rest of implementation is done **
   __declspec(dllexport) void print_str( const std::string &s )
      {
      std::cout << s << std::endl;
      };

   __declspec(dllexport) void
   HSSL::handleTokens( const std::vector<std::string> &instruction )
      {
      if ( instruction[ 0 ] == "prn" )
         {
         std::cout << instruction[ 1 ].data();
         if ( instruction[ 2 ] == RETCHAR )
            {
            std::cout << std::endl;
            }
         else
            {
            std::cout << " ";
            }
         }
        
      if ( instruction[ 0 ] == "prnf" )
         {
         std::cout << getFloat( atoi( instruction[ 1 ].data() ) );
         if ( instruction[ 2 ] == RETCHAR )
            {
            std::cout << std::endl;
            }
         else
            {
            std::cout << " ";
            }
         }

      if ( instruction[ 0 ] == "prni" )
         {
         std::cout << getInt( atoi( instruction[ 1 ].data() ) );
         if ( instruction[ 2 ] == RETCHAR )
            {
            std::cout << std::endl;
            }
         else
            {
            std::cout << " ";
            }
         }

      if ( instruction[ 0 ] == "prnbrk" )
         {
         std::cout << std::endl;
         }

      if ( instruction[ 0 ] == "movi" )
         {
         setInt( atoi( instruction[ 1 ].data() ),
                 atoi( instruction[ 2 ].data() ) );
         }

      if ( instruction[ 0 ] == "movf" )
         {
         setFloat( atoi( instruction[ 1 ].data() ) ,
                   atof( instruction[ 2 ].data() ) );
         }

      if ( instruction[ 0 ] == "fptr" )
         {
         fPtr = atoi( instruction[ 1 ].data() );
         }

      if ( instruction[ 0 ] == "iptr" )
         {
         iPtr = atoi( instruction[ 1 ].data() );
         }

      if ( instruction[ 0 ] == "addii" )
         {
         setInt( iPtr, ( getInt( atoi( instruction[ 1 ].data() ) ) +
                         getInt( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "addff" )
         {
         setFloat( fPtr, ( getFloat( atoi( instruction[ 1 ].data() ) ) +
                           getFloat( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "addif" )
         {
         setFloat( fPtr, ( getInt( atoi( instruction[ 1 ].data() ) ) +
                           getFloat( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "addfi" ) 
         {
         setFloat( fPtr, ( getFloat( atoi( instruction[ 1 ].data() ) ) +
                           getInt( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "subfi" )
         {
         setFloat( fPtr, ( getFloat( atoi( instruction[ 1 ].data() ) ) -
                           getInt( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "subii" )
         {
         setInt( iPtr, ( getInt( atoi( instruction[ 1 ].data() ) ) -
                         getInt( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "subif" )
         {
         setFloat( fPtr, ( getInt( atoi( instruction[ 1 ].data() ) ) -
                           getFloat( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "subff" )
         {
         setFloat( fPtr, ( getFloat( atoi( instruction[ 1 ].data() ) ) -
                           getFloat( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "mulii" )
         {
         setInt( iPtr, ( getInt( atoi( instruction[ 1 ].data() ) ) *
                         getInt( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "mulfi" )
         {
         setFloat( fPtr, ( getFloat( atoi( instruction[ 1 ].data() ) ) *
                           getInt( atoi( instruction[ 2 ].data() ) ) ) );
         }

       if ( instruction[ 0 ] == "mulif" )
         {
         setFloat( fPtr, ( getInt( atoi( instruction[ 1 ].data() ) ) *
                           getFloat( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "mulff" )
         {
         setFloat( fPtr, ( getFloat( atoi( instruction[ 1 ].data() ) ) *
                           getFloat( atoi( instruction[ 2 ].data() ) ) ) );
         }

      if ( instruction[ 0 ] == "%" ) 
         {
         // skip this instruction since % is for comments;
         }
      };

   __declspec(dllexport) HSSL::HSSL()
      {
         version = "1.0";
         copyright = "(C)opyright 2002 Headhunter Soft";

         //varInt = new int( 10 );
         //varFloat = new float( 10 );
         fPtr = 0;
         iPtr = 0;

         for ( int j = 0; j < 10; j++ )
            {
            varInt[ j ] = varFloat[ j ] = 0;
            }

         count = 0;
      };

   __declspec(dllexport) HSSL::~HSSL()
      {};

   __declspec(dllexport) std::string HSSL::getCopyright( void ) const
      {
      return( copyright );             
      };

   __declspec(dllexport) std::string HSSL::getVersion( void ) const
      {
      return( version );
      };

   __declspec(dllexport) int HSSL::getCount( void )
      {
      return( count );
      };

   __declspec(dllexport) void HSSL::resetCount( void )
      {
      count = 0;
      };
      
   __declspec(dllexport) void HSSL::setInt( int index, int value )
      {
      varInt[ index ] = value;
      };

   __declspec(dllexport) int HSSL::getInt( int index )
      {
      return( varInt[ index ] );
      };

   __declspec(dllexport) void HSSL::setFloat( int index, float value )
      {
      varFloat[ index ] = value;
      };

   __declspec(dllexport) float HSSL::getFloat( int index )
      {
      return( varFloat[ index ] );
      };
    
   __declspec(dllexport) int HSSL::execute( const std::string &filename )
      {
      std::vector<std::string> tokenizedStr;
      std::string buffer;

      // get a filestream and open the file
      std::ifstream scriptFile;
      scriptFile.open( filename.c_str() );

      // read, tokenize and execute each line of the file
      while( !scriptFile.eof() )
         {
         std::getline( scriptFile, buffer );
         tokenizedStr = parser.tokenize( buffer );
         handleTokens( tokenizedStr );
            
         // Clears the tokenizedStr for the next element.
         tokenizedStr.clear();
         count++;
         }
        
      return( 0 );
      };
   }

Explanation: Hmm.. Looks complicated doesn't it. When I created the engine, I didn't want to go into any complicated stuff. So the handleTokens function basically run through a series of if statement to check the script function name and does whatever is appropriate. Now, here is a description of what is what in that code. Instruction[0] is the script function name; Instruction[1] is the first parameter before the ","; and Instruction[2] is the instruction after the comma until the end of the line. I've also had to be a bit creative about variables. You simply have 2 types. Either integers or floats and you only have 10 of each. The iPtr and fPtr variables store the location in the integer and float array, respectively, where you want to store the value. I've had to use the old atoi and atof functions as I didn't know how to convert from string to integer or float. As a matter of fact, I still don't know. Anyhow, I'll keep that for later. After the handleTokens function, I have my getters and setter for the variable arrays. Notice I also have a getCount and setCount function. These aren't used in this version of the engine but they are used in later versions. Before I forget, in this version, there are no loop functions at all. No while, no for, no do. The last function in the file and the one we are most interested in is execute. This function takes a filename as parameter. It then proceeds to open the file. From there, it reads a line, treats it using parser.tokenize and then takes the returned vector and passes it to handleTokens to execute the code. Note: notice that I have a global constant set for the return character definition. This is for the PRN function, if that character is met in the script file, it will add a "\n" to the cout statement if not, it won't change lines. You can also use the PRN BREAK statement which works fine also but it needs to be a separate PRN. Now with that said, we only have one last source to check out and that's the CParser code. Here it comes:

namespace HSScript
   {
   __declspec(dllexport) std::vector<std::string>
   HSScript::CParser::tokenize( const std::string &codeVec )
      {
      size_t spaceAt = 0,
             commaAt = 0,
             commentAt = 0;

      std::vector<std::string> retString;

      spaceAt = codeVec.find( " " );
      commaAt = codeVec.find( "," );
      commentAt = codeVec.find( "%" );
        
      if( commentAt > codeVec.size() )
         {
         retString.push_back( codeVec.substr( 0, spaceAt ) );
         }
      else
         {
         return( 0 );
         }

      if ( spaceAt < codeVec.size() )
         {
         if ( commaAt < codeVec.size() )
            {
            retString.push_back(codeVec.substr( spaceAt+1,
                                   ((commaAt)-(spaceAt+1)) ) );
            retString.push_back( codeVec.substr( commaAt+1,
                                   codeVec.size() ) );
            }
         else
            {
            retString.push_back( codeVec.substr( spaceAt+1,
                                   codeVec.size() ) );
            }
         }
         
      return( retString );
      };
   }

The way this works is simple. It takes the string that we've just read in, and checks for the first space. It then copies the part of the string before that space into a vector<string> and moves on to search for the first comma, adding the part before the comma as another element in the vector. Finally, it takes the part after the comma and creates a last element in the vector. Finally, it returns that new vector.

Finally, I had to test the engine by creating a small program that would use the engine's DLL and lib to make sure it worked fine. Here's that file for you:

int main( int argc, char *argv[] )
{
   std::string HSSfile = "test.hss";
   
   HSScript::HSSL hssl;

   hssl.execute( HSSfile );

   std::cout << std::endl;
   std::cout << "\t\t\t** Variables **" << std::endl;
   std::cout << std::endl;
        
   for ( int i = 0; i < 10; i++ )
      {
      std::cout << "varInt[" << i << "] = "
                << hssl.getInt( i ) << "\t\t\t\t";
      std::cout << "varFloat[" << i << "] = "
                << hssl.getFloat( i ) << std::endl;
      }
  
   return( 0 );
}

The program above does nothing more than create an instance of HSSL, execute a script file called test.hss. The test.hss file will be displayed in a couple of seconds. Notice the use of main and not winmain, there is a reason for that. The reason is that I simply wanted to test the file quickly without having to create a window, and create a function to ouput text, so a console app was the best choice. Let us now move on to the test.hss file.

prn HSSL v1.0 is nice,_

movi 4,110
movi 5,223
movf 3,20.42

iptr 7
addii 4,5

prn i(7)=
prni 7,_

iptr 8
subii 7, 4

prn i(8)=
prni 8,_

fptr 6
addfi 3,5

% prints the value of the 
% 6th float variable and then inserts a return
prn f(6)=
prnf 6
prnbrk

prn i(4)=
prni 4,_

fptr 9
subff 6,3

prn f(9)=
prnf 9,_

End Notes & Conclusion

The script above simply prints a couple of things to the screen, stores values in the variable arrays and then does some math with those values. Here's a brief description of the commands:

prn = standard print
prni = prints the value located at x in the int array
prnf = prints the value located at x in the float array
prnbrk = breaks the text line( \n )

As for the rest of the functions, here's a brief descriptions of the last two characters in the name:
ii = first and second parameter are int
if = first param is int, second is float
fi = first param is float, second is int
ff = first and second param are float

The first three characters in the function names should tell you what they do. Now, I'm taking the liberty of adding the list of script functions available in this version of the scripting engine for you at the end.

In conclusion, this engine was version 1.00Beta( even though that's not the value that the version variable returns ). Now, for most of you, this engine will just be a piece of junk but to me, it was a first attempt at programming a script engine and I was also learning STL at the time so it was a first of firsts. And it has grown and gotten way better since then. It's now more of a Game Scripting Language rather than just a scripting language. None the less, I still like to mess around with that version once in a while. And it did teach me how to use some parts of STL. The latest version, HSGL 3.0 Beta, should be available for licensing sometime along the line of the beginning of 2003.

Commands For Script Language:

Comment Operator: