Holy cr*p! It’s been like 40 days since I have posted anything! That’s probably the longest the blog has gone silent. Well, that’s partly because I have been working on a new game and yes a new game engine. The new game will not be a 3D game but a 2D game. No, I haven’t abandoned the O2 game engine, to the contrary the O2 Game engine has had quite a few updates to it. So why another game engine?
Long long ago (, maybe not far far away) when I was designing/prototyping the O2 engine, I had plans for it to be a 2D and a 3D game engine. Overambitious? Yeah, maybe. But tell me who hasn’t had overambitious design plans? Anyway, during that time a lot of design went into the 2D part. I even went ahead and created a mock prototype of 2D game builder UI. In any case, that was soon put on the back burner since we started in earnest on the 3D game. The design however, lingered in the archives, until I had the time to revisit it after the Doofus game was completed. Instead of letting the design rot, I decided to peruse and finish the 2D part.
A couple of problems immediately arose when I had a re-looked. The O2 engine is designed to be a 3D engine and a lot of abstraction has been made to ensure that the engine does 3D properly and efficiently. That’s not really required for 2D. 2D is a lot less complicated than 3D and most of the sections of the 3D game engine felt over-engineered when considered in the context of 2D. Most of the book-keeping required for 3D is down right unnecessary for 2D. In the end the two designs were a significant overlap, so I decided to rehash the 2D concept using similar design but start over from the engine core. Basically meaning, a rewrite of a significant section of the game engine code. To my surprise the process was completed far sooner than anticipated, and that too without significant regressions. Yes the new engine uses a lot of same code-base as the O2 engine, but the engine itself is very different from the 3D engine.
Next came the game. There was an old prototyped 2D game (more details soon) that was lying with me for ages. Most of the game was polished up and ported to the new engine (lesson: Never throw away your old experimental code; ever. ), and that’s about it. I am pretty happy with the results of my 2D experiment thus far. Completing a game is a complicated process. It requires a lot of things to come together and work correctly. True, 2D is lot easier than 3D, but there is still work to be done finishing the game and I hate to give out a release date just yet (, because of the fear of being proved abjectly wrong yet again ) .
As you probably already know, the first Doofus game was released a few weeks back, and though I haven’t fully shaken off the lethargic hangover that accompanied the game release; I am slowly coercing myself to get back to daily chores. After every project one must look back and take a note of hits and misses of the project development cycle (Causal Analysis). That’s what I have been doing for the past few days, and though there have been lots of those (hits and misses), one of the important lesson I have learned from the whole experience of making the game is the importance of data-driven programming.
Most of the tedious and time consuming activities during development turned out to be simple manipulation and ordering of code. Placing scene-nodes, structuring the scene-graph, ordering render batches, grouping shadowing objects, and the list could go on. Things like these took up ridiculous amounts of time. In the game all this is done through code — when this could easily have been solved by a more data driven approach. I must admit there were times during development of the project when I yearned (almost cried) for a more flexible system. The problem was a lack of script driven programming, or should I say, a lack of a more data-driven system on top of the engine. Everything with the engine is currently done in C++, except maybe the configuration files. I hate it when you have to recompile 20-30 files just because you had to move a local variable to a class member. This does happen, especially for classes that have a long list of data variables and/or the ones that operate and integrate game data. We had initially planned on integrating a script and data-driven system into the engine, but that was later push down the priority of things to be done — and ultimately pushed out from the release due to lack of time and resources. Through not verbatim, that’s probably the story with every dev project out there that has to keep up with at least some sort of deadline.
In data-driven programming, data controls the flow of the program. Data is not just used to define an object state but also used to control the program flow, sometimes externally. Data-driven programming means writing generic code that consumes data and modifies it’s behavior depending on the structure of that data. The concept of data driven programming is not new. UNIX utilities have traditionally been data-driven and many applications use data driven designs. A web browser is a good example of a data driven application. It consumes a web page and modifies it’s behavior based on the structure of the web-page code, in this case HTML. Markups are again an excellent examples of data that encodes structure within itself (XML).
Well, there is a lot to take away from the current project. The engine will be made more data driven in the future, that’s for sure. The game also allowed me to pinpoint exact areas where a data-driven approach is needed. Many of those places wern’t immediately obvious during design.
A very Happy New Year to all. A bit belated I know, but I was kinda busy doing nothing. Well, not really. Yeah, I have been taking time off, but I was also busy with other activities, most importantly, marketing of the game.
So what’s 2008 been like? Well for me it was pretty uninteresting. Not, that I didn’t enjoy it, it’s just there was precious little in the way of what I like to do best; research. Most of 2008 was spent on fixing bugs, play testing, hardware testing, level creation and solving some insanely complicated issues, issues that shouldn’t have been there in the first place and some unavoidable circumstantial problems, that shouldn’t have been there in the first place. Most of the coding that was done was also equally uninteresting. Majority of the time was spent on getting thing working right with gameplay and design. Not the most pleasurable of things I must say, at least not for me. That said, a lot of ground work has been done w.r.t the engine, most of which will not have to be repeated for sometime to come. So that’s a big positive, something I can take away from 2008 as being extremely productive.
Having said that, the biggest hit of the year for me is of-course the release of the game; which took far more time than I had initially anticipated. True, it turned out OK (great ) given the budget, time and resource constraints, but I would have liked to do more. Maybe all that was missed in this one can quickly be added to the next one. A Causal Analysis is due, however I would like to hold on to that a bit longer. At least till we finish up with the final marketing parts which I am currently focusing on. A part of last year was also spent in starting 3D Logic Software and there are a lot of things that had to be done before we went online. Unfortunately they accounted in a pretty big delay for the launch of the game.
On the tech front, 2008 has been equally low. Very little interesting developments. Most of things that happened were evolutionary rather than revolutionary. On the OS front XP still rules and will probably do so in 2009 as well. However, the year belonged to the underdog Apple. Both their OS and their products have gained significant market share and will probably continue to do so in 2009. Linux has always been interesting and 2009 will be no different. Linux grows from strength to strength in some areas and remains the same in others. If anything I am looking forward to Linux in 2009, some interesting developments on the horizon.
In 2008 we saw a resurgence of the GPU battles with ATI throwing in some impressive technology, and that’s good thing. For the first time I am an owner of an ATI card (HD 4850) and though NVIDA held on to the top spot (barely), ATI was close behind and even edging out in front at times during the year. Then again we can’t forget general purpose computing on the GPU. The year has been interesting for GPU and GPGPU. Powerful cards with supercomputing capability were unveiled and this year will see more power being packed into cards as the GPU titans clash with better with more powerful weapons at their disposal. Oh, let’s not to forget Intel here. Intel finally unveiled Larrabee, so you very well could have another titan arising in those battles.
Personal wish list for 2009.
- Intel comes around to finally putting a proper on-board GPU with at least good hardware T&L and releases moderately good drivers.
- Microsoft releases DirectX 11 for XP along with Vista and Windows 7.
- OpenGL spec gets a overha….. well forget it!
- Linux gets a single package-management/installer system that everyone across the board adopts, and most importantly is easy to use and deploy.
- The economic downturn ends.
- All people in the world become sane and killing of innocent people stops completely.
That all for now,
Once again a Happy New Year.
For the last week I have been involved in rather uninteresting activity. Well, I have been literally throwing the game on all possible hardware configs hoping it will run. All of this (yes, again) to find out how the game fares when exposed to different hardware configurations. Well it may seem like this activity is rather mundane, then let me assure you — it is. Well, not entirely . It takes some effort to get a game to scale seamlessly to all kinds of hardware and currently I am enduring all the pain of crappy drivers and broken functionality, which, should I say, underscores some of the major headaches in real-time graphics development. It’s not like you can throw the game with it’s peak setting ON and expect it to run on a crappy Intel on-board graphic cards. Such a thing will just end in a disaster. The game must scale to different kinds of hardware and in our case especially so; that too seamlessly and effectively.
Doofus 3D is uniquely placed. It doesn’t aim to be a top-line, hardware intensive, hard-core gamer only, triple A (AAA) title. Neither is it a 2D game capable of running flawlessly under software rasterized graphics on your grandma’s old school PC. It is geared more towards intermediate level hardware. Hardware that most people have on their work laptops and home desktops. This effectively means an extremely wide range of hardware to cater to, and that in turn means scaling the game’s software paths (internally) based on a *lot* of underlying factors. Assuming a player to have a specific functionality available on his hardware setup can be catastrophic and disastrous. Such assumptions could mean a total failure of the game on a machine and could mean a potential loss of a buyer in the end.
While drawing up specs of Doofus 3D we were especially careful not to go overboard with graphics galore. Even with careful planning, there was significant feature creep, and with each new feature that was added, new countermeasures had to be put in place so that the game would scale to lower-end hardware. Not everything was straight forward, but we still did manage to push it through. If you have been following my blog for some time now, you would know that this is not the first time I am into such activity. I (personally) run such tests after each beta (feature addition/ feature freeze) of the game. That is probably why we haven’t faced too many problems this time around.
Under Doofus 3D we followed a process that is a bit different from traditional software development. Every beta under this game project was actually a feature complete runnable version of the game. Before or between any beta, every release was an internal alpha version. A beta meant, “A set of features is complete enough to be tested”. After each beta, each feature was tested on various hardware setups. Something like an iterative method of software development, but not quite. I would say, a process tailored specifically for our project and more specifically for our situation given our limitations.
Doofus 3D runs on most middle rung hardware without too much problems. It will run on on-board graphics cards too, but I find Intel on-board graphics to be an abomination. Hopeless hardware support for 3D graphics and equally crappy driver support! Enough reason for the engine to scale the game to run on a low setting when it detects an Intel graphics card. The situation with NVIDIA and ATI cards is a lot better with ATI’s low end cards (,assuming the price point, ) to be consistently outperforming NVIDIA cards. That said, NVIDA has the most stable hardware and drivers and most settings work uniformly across cards and driver setups, though there can be problems there as well. ATI’s drivers can be buggy at times and in case of OpenGL can be totally broken. Fortunately the O2 Engine and the Doofus Game can use either Direct3D or OpenGL as rendering APIs. For any high end or for that matter even for most mid-range graphics cards, Doofus 3D is not a problem at all.
Whew! The game has officially (finally) hit the last code-freeze today, actually yesterday but I tagged the repository today so it’s officially code freeze today. All the levels have been done and most of the internal (alpha) testing is complete. Yeah, the blog has been silent for a while; but that’s to be understood, I have been really hard pressed to finish this on time, yet I overshot my (mostly self-imposed) deadline by a good 15 days In part due to unforseen circumstances, in part due to my own faults, but that’s the way it goes (generally ). I am really glad it’s finally done and all but the most trivial issues remain. There is some artwork left though, I still have to finish up the initial screens and some sound-tracks and some sound-effects need to be incoroprated into the game as well. Some parts of the demo version also need to be finished up. However, I am going to hold on to that till I release the RC for one final bout of testing, that way I have a good feedback on which levels to include in the demo and which to hold off for the full version.
Update: Doofus Longears – Get ‘em Gems has been released and can be found on www.doofuslongears.com
Whew, finally found some time to update the blog. I have been frantically working on putting final polish to the game, business related activities, tweaking graphics, ironing out small glitches in gameplay, play-testing levels, and the list goes on!
My major headache was the background. There were a lot of people who had complained about the background not being up to the mark. So I decided to paint a brand new background from scratch. It was a hell of a lot difficult though. Doofus 3D being a cartoon game, I wanted to have a flamboyant background (, rich and colorful with a distinct cartoon touch). However it’s not quite that simple, it’s not as easy as firing up good ol’ GIMP and just having a go at painting any ordinary scene. Since you are painting for a sky-box you really have to be a lot more careful and lot more sensitive about how to handle depth in your scene, plus you have to paint for a full panoramic view. A lot of experimentation went into this one, believe me! Lot’s of hits and misses later, and after studing some other skyboxes this is what I ended up with.
Sorry, but I have been a little busy for the past two weeks. Too busy to blog I guess. I have been aiming for a code freeze on the Doofus game and it’s been hard work getting all the bugs and issues in. I’m going for the final push this time around to get at least the coding issues out of the way. The good thing is there isn’t too much left on the coding side, so I may be able to push out another beta by next week. Hopefully it will be the last and final beta before (; wait, don’t get your hopes up) at least one release candidate before Doofus goes gold. I guess there are still a sizable amount to levels to be completed.
Unlike release cycles of other software, Doofus game release cycles are a little bit different. I devised a new method after we initially found the old method to be rather monolithic for this particular project, and because of obvious constraints we have as a small team (unavailability of testers at specific times and things of that nature). Traditionally, you have a set of alpha releases of a product where each alpha release is tested in-house by both the developers and/or testers. Bugs are filed for specific releases and fixed during bug-fixing stage, whenever that maybe (, generally differs from project to project). Beta releases are pushed out when alpha releases get stable enough for “general consumption”. Beta releases are generally widely accepted as “almost complete” versions of the product. So a beta release often signifies a “feature-freeze” of the product. A bug fixed beta release can become a Release Candidate if the dev team feels confident enough which eventually turns Gold when everyone is confident enough.
In the case of the Doofus game, things are a little bit different. A beta release signifies a “feature freeze” for “a particular set of” features. Let me explain. When we started developing the game, the O2 Engine was the first to come (, before we started on the actual game code). The name “O2 engine” comes from the repository branch of my older game that was never released because it had too many flaws! I guess a lot was carried over including primitive libraries and some design decisions and implementations. Anyways, since the new project was a bit complex and our testing team small and working part-time, we decided to have specific release milestones having only limited set of completely complete features. When I say completely complete I mean “feature frozen”. Each beta release addressed different features. The first was for engine integration with geometry. The level structure was finalized and resource management was put in place. The first release looked really ugly because the renderer was partially finished.
The second beta addressed collision systems, basic gameplay things like triggers activators and integrations with third-party modules and libraries. The third was for rendering sub-systems, when those screenshots were posted. This release, the fourth, will be for AI (NPC) and Physics and that marks the end of the game features. The beta still has to go through a bug fixing stage before I am confident enough to even look at a RC, but it does mark and end to any major code modifications to the game code. Many would say the betas are actually alphas, but there are 2 reasons I call them beta releases. a) They are feature freeze releases. No features are added or removed to the already tested features. b) Our testers are no full-fledged project member so white-box testing responsibility falls on the dev team, mostly. That said, the beta testers are not just kids beating at the keyboard and have been instrumental in testing the product.
I guess this release has got me a bit exited, and I am working on the website/s at the same time. I have actually started on quite a few blog posts in the past week, but haven’t had the time to polish and/or finish them yet. Maybe this week will see more posts on the blog, I hope.
Just before logging off of the Gutsy, I tried running the 0.0.5 beta version of Doofus 3D via Wine and “Voilà”, it actually ran! On Linux, with no tweaks of any kind, no nothing! Obviously only with the OpenGL driver, Wine has near absent DirectX support so it was only OpenGL, but still I was like blown away. I didn’t even expect it to work and I must say I could see no performance issues. Almost the same speed as Windows. I ran most of the unit tests and managed to finish the first 7 levels without any problems (, didn’t try any more but I am sure they will work just as well). To be frank I didn’t expect the game to run, and I mean run at all, let alone with the same speed as Windows.
I had no idea Wine had no DirectX support. I guess it must be very difficult to port the DirectX APIs (, but who’s complaining when the engine can run on OpenGL just as well) . The engine logs show Wine hooks in the native OpenGL driver on Linux, so I guess there is no visible performance loss while running the game using OpenGL renderer via Wine.
Update (Dec 17th, 2007): After reading Dan’s comments and some experimentation I could in fact run Doofus 3D via the DirectX renderer. However, the DirectX renderer will not run out of the box and there might be other legal issues associated with it, which are as yet unclear. You can read the entire comments here. I like to thank Dan for pointing out the error in the above post, Wine does have DirectX 9 support.
Some more RnD on scripting languages. This is the 3rd part of the series, the first 2 parts  and . I generally do all the RnD on this when I take time away from working on the current game or I am generally bored or stuck on a bug somewhere. This time I focused on Python again and also tried something with Squirrel. Both are very different languages and each one has different challenges to overcome. I also took a dig at something I haven’t touch before and that is C#, yes, as a scripting language.
Python is general is very good as an extension language. You could bind all your code with python and then use only python to code your entire game. Python is clearly designed as heavy duty application development language and embedding it is not what is recommended. While you could very well do so, I found it to be rater clunky and non-intuitive, that’s not to say it can’t be done. So I focused primarily on “extending” and not “embedding” while working with python. There is no real advantage or disadvantage to either method, but I rather want to carry the O2 engine in the direction of “extending” too keep the engine accessible to people who have no, or limited knowledge of C++.
This time I focused on Boost.Python more than SWIG. I haven’t benchmarked anything as yet, but I guess any overhead via bindings would mean a small (, or minuscule) decrease in speed. That doesn’t really bother me since most of the heavy duty calculation stuff will be done in the C++ libraries anyways. However what bothers me is the maintenance and scalability of the binding code. Any binding code is difficult and uninteresting work. It is something that I know programmers will hate to do and it is something that will have to be done time and again as and when the engine scales. Another thing is time. It takes ridiculously large amount of time to write large portions of binding code. That is where I think Boost.Python edges out slightly over SWIG, hello to the Py++ bindings code generator!
Apparently the code generator parses C++ code to GCC-XML and then uses the XML to generate the bindings. It seems too good to be true, but that is what it does. To be fair I have only tried a very small program with it and I have no clue about larger programs. I might try something larger with Py++ this weekend, again, maybe. However, larger projects like PythonOgre already successfully use it so I don’t think it will be too much of a problem with the O2Engine. Check out the next in the series for that.
Squirrel is a lightweight language, much lighter than Python and supports all modern programming features. “Squirrel is a high level imperative/OO programming language, designed to be a powerful scripting tool that fits in the size, memory bandwidth, and real-time requirements of applications like games.” I experimented a lot with squirrel for the past couple of weeks. The language is cool, there are no two opinions to that. It is all what they say it is and more. It tastes like an embeddable language but I am sure it will do an equally good job at extension. In someways it is a lot better than python. For one it is designed to be embeddable while python is not. It has a much smaller memory footprint and, even though I have no numbers to prove, it must be faster than, or at least equal in speed to python.
The big negative part of this language is it’s bindings. Seeing the binding code will make you cringe! It is hopelessly complicated, almost impossible to understand at times. Binding even a small piece of code with about 5 classes took me over and hour, and those classes were not even extensive. I hate to think what will happen while integrating a heavily templatized game engine! For some reason the bindings have to be integrated into classes themselves and even if you did bypass that, it is not clean at all. A lot of glue code needs to be put in, too much I think. While there are binding systems like SQPlus and jkBind, bindings in Squirrel are non trivial. jkBind is particularly interesting and easier than SQPlus, but even then it misses out on functionality that is critical to extending any language. Scaling any such binding system on a large live project having multiple developers will be a nightmare.
If the binding systems were improved, Squirrel would have been my choice hands down. Having said that, I am currently not discounting any language outright. Choosing a scripting language can be tricky and my choice could very have a significant consequence in the future.
C#: (& Mono)
Ah! So I finally got around to having a go at this language. I have only tried a couple of things thus far but I am already impressed. First, C# is by no means a scripting language. OK to be clear I have never used C# before, just had some fleeting glimpses at it, but never worked on any project involving C#. It is a full fledge high level language and is very very powerful, but I bet you already knew that. Extending C++ code using the Microsoft compiler is easy, maybe not as trivial as Py++. One thing I can say for certain is C# is faster than Python. I am not so sure about Squirrel but I bet it must be a little bit faster to that as well. I won’t go into too many details just as yet because I still want to do some more testing and extending before I make some claims. So more on C# later.
In my next part of this series I will try and post some timing values. Timing values will allow for better comparisons of languages and binding systems. I will also try and post some binding code. That will just give a better overview of the task and effort required to have a binding for a language.