Some more RnD on scripting languages. This is the 3rd part of the series, the first 2 parts [1] and [2]. I generally do all the RnD on this when I take time away from working on the current game or I am generally bored or stuck on a bug somewhere. This time I focused on Python again and also tried something with Squirrel. Both are very different languages and each one has different challenges to overcome. I also took a dig at something I haven’t touch before and that is C#, yes, as a scripting language.
Python:
Python is general is very good as an extension language. You could bind all your code with python and then use only python to code your entire game. Python is clearly designed as heavy duty application development language and embedding it is not what is recommended. While you could very well do so, I found it to be rater clunky and non-intuitive, that’s not to say it can’t be done. So I focused primarily on “extending” and not “embedding” while working with python. There is no real advantage or disadvantage to either method, but I rather want to carry the O2 engine in the direction of “extending” too keep the engine accessible to people who have no, or limited knowledge of C++.
This time I focused on Boost.Python more than SWIG. I haven’t benchmarked anything as yet, but I guess any overhead via bindings would mean a small (, or minuscule) decrease in speed. That doesn’t really bother me since most of the heavy duty calculation stuff will be done in the C++ libraries anyways. However what bothers me is the maintenance and scalability of the binding code. Any binding code is difficult and uninteresting work. It is something that I know programmers will hate to do and it is something that will have to be done time and again as and when the engine scales. Another thing is time. It takes ridiculously large amount of time to write large portions of binding code. That is where I think Boost.Python edges out slightly over SWIG, hello to the Py++ bindings code generator!
Apparently the code generator parses C++ code to GCC-XML and then uses the XML to generate the bindings. It seems too good to be true, but that is what it does. To be fair I have only tried a very small program with it and I have no clue about larger programs. I might try something larger with Py++ this weekend, again, maybe. However, larger projects like PythonOgre already successfully use it so I don’t think it will be too much of a problem with the O2Engine. Check out the next in the series for that.
Squirrel:
Squirrel is a lightweight language, much lighter than Python and supports all modern programming features. “Squirrel is a high level imperative/OO programming language, designed to be a powerful scripting tool that fits in the size, memory bandwidth, and real-time requirements of applications like games.” I experimented a lot with squirrel for the past couple of weeks. The language is cool, there are no two opinions to that. It is all what they say it is and more. It tastes like an embeddable language but I am sure it will do an equally good job at extension. In someways it is a lot better than python. For one it is designed to be embeddable while python is not. It has a much smaller memory footprint and, even though I have no numbers to prove, it must be faster than, or at least equal in speed to python.
The big negative part of this language is it’s bindings. Seeing the binding code will make you cringe! It is hopelessly complicated, almost impossible to understand at times. Binding even a small piece of code with about 5 classes took me over and hour, and those classes were not even extensive. I hate to think what will happen while integrating a heavily templatized game engine! For some reason the bindings have to be integrated into classes themselves and even if you did bypass that, it is not clean at all. A lot of glue code needs to be put in, too much I think. While there are binding systems like SQPlus and jkBind, bindings in Squirrel are non trivial. jkBind is particularly interesting and easier than SQPlus, but even then it misses out on functionality that is critical to extending any language. Scaling any such binding system on a large live project having multiple developers will be a nightmare.
If the binding systems were improved, Squirrel would have been my choice hands down. Having said that, I am currently not discounting any language outright. Choosing a scripting language can be tricky and my choice could very have a significant consequence in the future.
C#: (& Mono)
Ah! So I finally got around to having a go at this language. I have only tried a couple of things thus far but I am already impressed. First, C# is by no means a scripting language. OK to be clear I have never used C# before, just had some fleeting glimpses at it, but never worked on any project involving C#. It is a full fledge high level language and is very very powerful, but I bet you already knew that. Extending C++ code using the Microsoft compiler is easy, maybe not as trivial as Py++. One thing I can say for certain is C# is faster than Python. I am not so sure about Squirrel but I bet it must be a little bit faster to that as well. I won’t go into too many details just as yet because I still want to do some more testing and extending before I make some claims. So more on C# later.
In my next part of this series I will try and post some timing values. Timing values will allow for better comparisons of languages and binding systems. I will also try and post some binding code. That will just give a better overview of the task and effort required to have a binding for a language.
I’ve used pygccxml for parsing c++ code, it was a lot of fun to use 🙂 though installing GCC-XML was a bit painful.
Looking forward to the next in the series.
Was it for bindings or was the project something different? Py++ does depend on pygccxml.
I used the GCC-XML installer they have with the Py++ project downloads. It’s a little bit unconventional install script I agree but the main thing is you will have to do this once. Then maybe use a build script to just generate python bindings on a nightly build or something like that. Beats writing binding code every time, not to mention the bugs that may be introduced by the programmer.
I am not sure if it would as easy as that, but I kinda hope it will be. Need to do some more research on that.
It wasn’t for bindings. I was experimenting with automatically injecting source code for function logging, easier debugging etc for an embedded systems project.
You should probably generate the python bindings as part of your build process – re-generate them every time the related files are modified.
For your game how do you maintain the build scripts for different platforms? Do they use the same tool chain? When using different tool chains one commonly violates the DRY principle. I came across http://www.ociweb.com/products/mpc which can generate build files for various tool chains, but haven’t been able to spend any time on it.
Working with Makefiles can be a nightmare. I used to work with them when I was in Geometric working on a cross-platform Qt related project, so I pretty much cringe when anyone mentions Makefiles. We worked with Visual Studio and had to be extra careful so as not the break the build. The project also had some Java and Python bindings done via SWIG, but I don’t exactly know how it was handled in the build script. The client had kept that information confidential. Too bad :(.
The engine and the game is currently being built on Windows platform using the express edition of Visual C++ and, no I don’t have a build system as yet. For now it’s click and build ;). I am however confident that the system can be ported across platforms without a lot of effort. I was thinking of SCons (http://www.scons.org/) for the job, but I haven’t used it as yet. MPC seems to be an interesting concept, hadn’t seen it before.
The idea of having a tool to generate bindings that can integrate with a build system seems almost too good to be true. I am not saying it’s not possible but I am a bit skeptical. Mind you I haven’t done enough research on Py++.
I am a fun of Squirrel:. Contrary, I found Squirrel: easy to embed using SQPlus than any other, it might be ‘couse I spent some time on it.
Embedding goes something like;
/C++ code:/—————————————-
#include “sqplus.h”
#pragma comment(lib, “sqstdlib.lib”)
#pragma comment(lib, “squirrel.lib”)
#pragma comment(lib, “sqplus.lib”)
class Test
{
public:
void CalledFromScript(const char* s,long k){
print (s);
}
};
void main()
{
SQClassDef(“Test”).func(&Test::CalledFromScript,”
CalledFromScript”);
SquirrelVM::Init();
SquirrelObject test = SquirrelVM::CompileScript(_SC(“test.sq”));
SquirrelVM::RunScript(test);
SquirrelFunction ScriptFunction(“ScriptFunction”);
Test theT;
ScriptFunction(&theT, “pass_back”);
SquirrelVM::Shutdown();
}
/test.sq SQ code:/—————————————-
function ScriptFunction(t, str)
{
t.CalledFromScript(str, 100);
}
@Marius
Squirrel is a good embeddable language and it will do a great job at exporting simple classes and functions. I am still experimenting it, quite a bit actually, and most of the times it does get the job done. However I am not so sure if embedding it in a full production project with large amount of classes having a lot of interdependencies is all that trivial. I can give you an example, our engine heavily depends on classes, class factories and interfaces/abstract classes. Relationships between existing classes must be preserved and carried over from the engine to anything that has to be built on top (eg. a game). By design the engine structure is abstracted to such a degree that any future game development via the engine will involve inheritance/aggregation/composition across the embeddable language and C++. It’s not to say that Squirrel can’t achieve that. I know it can, it’s just that there is a lot of hard work involved and at the end of the day it’s code; code that has to be maintained by someone. When I talk of Py++ and GCC-XML I am talking about offloading some of the work that a programmer would otherwise do to an automatic program, or at least a partially automatic one that will probably require only a fraction of effort needed to embed squirrel. To be fair though I have found Squirrel to be better and more intuitive than Python and I stand by my statement that had something like Py++ existed for Squirrel, it would have been my choice hands down!