Wednesday 27 October 2010

Quick and dirty scripting with Python

Quick and dirty scripting with Python


I came home from work today and, instead of taking a nap (something that would have come in really handy) or playing Mass Effect 2, I started coding a couple of scripts to help me organising some files that I've got in several Micro SD cards.

Keep in mind that I've omitted the imports, "const-like" vars declarations and the if __name__=="__main__" statement for the sake of brevity.

The first one just lists the files and subdirectories from a given path, and then it saves it to a text file. Python also provides the function os.walk (or the deprecated os.path.walk), but I got it into my head that I wanted to indent the files according to the depth from the root path and the easiest way to accomplish this was using os.listdir, which worked as a charm =)

def extract_dir_files(arg, dirname):

    dir_str= "%s/%s/\n" % (" "*arg[0],os.path.split(dirname)[1])

    arg[1].write(dir_str)

#os.listdir just returns a list containing all files and directories #contained within the path 'dirname'
    for item in os.listdir(dirname):
        new_path = os.path.join(dirname,item)
        args = [arg[0]+1,arg[1]]
        if os.path.isdir(new_path):

#The child directories will make recursive calls
#until the root_path directory has been completely #parsed
            extract_dir_files(args,new_path)
        else:
            name,ext = os.path.splitext(item)
            #do not write out the files having one of the
#extensions specified in IGNORE_EXT
            if ext not in IGNORE_EXT:
                file_str = "%s-%s\n" % (" "*(arg[0]+1),item)
                arg[1].write(file_str)
                print(file_str)    


def main():
    root_path = os.path.join(DRIVE_LETTER,ROOT_PATH)
    with open(OUTPUT_FILE, 'a') as f:
        f.write("*"*PAD_NUM+"\n")
#args[0] will hold the value of the current depth in the dir. #hierarchy; args[1], the file object
        args = [0,f]
        extract_dir_files(args,root_path)
        f.write("*"*PAD_NUM+"\n")




The second one gets the HTML contents of a given URL which are parsed to a DOM-like object. After that, all that it's left is to extract the data of interest and do whatever you want. In my case, I decided to write them to a *.CSV file for future use. I used the library available at http://www.boddie.org.uk/python/libxml2dom.html

It was pretty easy to use, and it allowed for DOM-like processing of HTML pages, which aren't generally well-formed. This causes complaints and exceptions when using better known Python DOM libraries.

def main():
#URL returns a list of links, of which I'll only be
#interested in their text values.
#FROM and TO are the
values I want to pass to the query
#as GET parameters.

    with contextlib.closing(urllib.urlopen(URL%(FROM,TO))) as url:
        encoding = url.headers['Content-type'].split('charset=')[1]
        contents = url.read().decode(encoding).encode('utf-8')
    doc = libxml2dom.parseString(contents,html=1)
    links = doc.getElementsByTagName("a")
    results = []
    for l in links:
        release = l.childNodes[0].nodeValue
        #...then do whatever you want with it :P



Tuesday 12 October 2010

Once upon a night

First, I believe that apologies are in order for the delay in updating. I finally got a job in a Barcelona-based studio, and I've been happily working there as a junior programmer since August :)

The rest of my daytime (and, more often than not, night time as well T_T) up until last Tuesday has been devoted to our master's final project delivery, which was presented that day on Caixaforum. 

First, a Konami exec came and gave a talk about the upcoming release of Castlevania: Lords of Shadow, developed by Mercury Steam, a Madrid-based studio, and then the 5 teams showcased their projects.

  • The creature. A full 2.5d platformer game, featuring 20 levels. Your mission will be to help a creature being complete again by getting back some parts of its body, scattered around several areas.
  • Wyverns Assault. A funny, retro-style beat 'em up game where you're a huge dragon fighting hordes of soldiers and destructing everything :-D
  • Undertown. A graphically amazing action game a la God of War, made by just an artist and a programmer. The main character must fight the Disgusting (Asquerosos, in Spanish: that's their real name) and other monsters so he can have his girlfriend back.
  • Ghosty Town. This game has a truly original concept where you're put in the place of a ghost that must scare the hell out of all the villagers of a small town in order to conquer it.

You should check them all (although some of them still lack a proper downloadable release, ours being one of them -_-U), there's a lot of work put into each and everyone of them, and the results have been really good imho.

And yes...I mentioned there were 5 teams, but only 4 projects are listed above.

So now, enter Once upon a night!



The game puts you in the place of Any, a girl (it wasn't always like that, though ^_^U) that's been trapped in her dreams by a malfunctioning magical clock. To escape and wake up again, she must find a set of gears, springs and other pieces to make it work properly again. These gears are supposed to be similar gameplay-wise to the stars on Mario 64, to name a very obvious example, as it was one of the most important references we took. To retrieve them, she'll have to solve the puzzles scattered throughout the two parallel worlds of dreams and nightmares that shape a level.

I performed tasks as a designer (well, the whole team was, actually, which is something that I now deem a terrible mistake) and, mainly, as a programmer.

Since our work tends to slip unnoticed, I'll compensate by listing the main features I wrote as a programmer. I might talk a bit more about some specific features in the future if I'm in the mood or anyone is interested. I'm sure that lots of stupid errors will be found, but that will allow everybody, specially myself, to learn from those mistakes.

- Base application engine skeleton. It consisted of a main Application class that would contain the remaining subsystems, encapsulated as independent (or that was the goal) modules to handle audio, graphics, AI, etc. After initialization, it just starts the game loop and handles input and state management, the game's subsystems being the ones in charge for updating the world and their attributes.
       
- A stack-based FSM for the game state manager. I had previously used a simpler approach for my previous games, based on IDs and an abstract factory to provide concrete implementations of the states. The stack approach has worked quite well despite an embarrassing error I detected not much time ago concerning state changes -_-U

- Event manager. A template-based approach that allowed any subsystem or the game world itself to subscribe/unsubscribe and react to specific events triggered by the animation engine, the physics subsystem, etc. Events were posted to a priority queue and dispatched at a fixed point of the game update call.

- Game entities management...partially (it was more of a joint task). This is kind of a long story: during preproduction and the earlier parts of the production stage, I had been reading several books and articles on the internet about component-based architectures and data driven design, and thought it could be nice, as it would allow to avoid deeply nested class hierarchies, deadly diamonds, and such. More importantly, we would have a flexible architecture to make it possible to define new entities without changing much code, or modifying their attributes' values without recompiling the whole project.

However, I was unable to communicate the concepts and new philosophy to my fellow programmers, and besides there were conflicts with the level loading module, so we came up with an alternate, middle-of-the-road approach that, honestly, could have been much better.

I still won't desist, though, and sooner or later I'll come up with some tests to compare the implementation, usage and ease to understand that type of architecture as opposed to the classical inheritance-based ones.

- Scripting engine. We used LUA and luabind as an interface between that scripting language and C++ to implement the following features:
* FSM-based AI-controlled entities. I believe we could have made entities a lot more script dependent, allowing them to react to state changes (for instance, playing sounds, changing animations and such) inside LUA code instead of delegating to the main C++ code, but it was
decided to leave it at that.

* Triggers. We had some entities that could act as triggers when another entity (typically, Any, but it wasn't the only one) entered/exit/was at the volume defined by the trigger. Through LUA, we could define a pair of functions condition/action, so that the logic subsystem could check if a set of additional conditions had been fulfilled (besides the enter/exit/presence checks, of course) and, in that case, execute the action function. This provided triggers with some interesting flexibility.

* Cutscenes. The in-game cutscenes in our game were also coded in LUA, taking advantage of the coroutine feature LUA provides (this was a bit difficult to understand at first). We had made available a set of exported functions (move an entity, change an animation, show a message box, etc), and then all we had to do was define the script to launch the different actions sequentially.

- Audio engine. It was based on FMOD and a specialisation of Ogre3D's ResourceManager, that made it possible to define sound resources as scripts that could configure several properties: looping, 3D, intensity thresholds, etc.


- User interface: To tell the truth, the user interface is actually split into three different components:
- The actual GUI subsystem, used for good old buttons, combos, etc., widgets. For these ones I went with CEGUI, an open source GUI library that already came bundled with the Ogre release we used (1.6.5). The more familiar that I got with it, the more powerful I've been finding it, so we could have used it instead of overlays in some other places, but then it was too late ^^U

Skinning a GUI layout, however, has been hell (well, I managed to come with a new skin on two days under a lot of time pressure, but still, it was a nightmare)

- Ogre's overlays. Mainly used for the HUD, and for almost the whole development cycle, the pause and game over screens (they got replaced with the third approach in the end). Ogre provides a specification for overlay scripts, but I find it a bit messy when you want to do some more complex positioning.

- Ogre's Rectangle2D class. Even simpler approach, consisting on creating a quad and applying a material to it. The intro, game over and loading screens are examples of this.


- Several graphical features.
- Animation engine. We adapted some code of an animation blender found on Ogre3D's wiki, adding the possibility to incorporate partial blending using an array of weighted bones. Unfortunately, it wasn't used at it fullest, due to time constraints and bugs.

- Initial approach to incorporating soft shadows, that unfortunately didn't make the cut. They were working, but then, when the change world functionality you've seen on the video was incorporated, it stopped functioning. The world change being a key feature of the game, it took precedence and dynamic shadows were removed until a later stage in the game.

- Some special effects. Nothing fancy, though. Some of them were reused from existing open source code, and then incorporated to the game:
    - The intermittent tint effect shown every time Any gets hit. This one was pretty easy to implement by using a pixel shader that would apply a tint whose intensity would vary according to a sine-like function.
    - The flashlight halo effect, combining texture and node animation  with a projective decal. The particles were incorporated at a later stage by another fellow programmer.
    - A lens flare, but it triggered on very rigid constraints, so it had to be tweaked a bit.

- Gameplay programming. I implemented the behaviours for several entities in our game.
- Any.
- The tripollos in the world of dreams.
- The scared plants (those weird plants that hide when Any gets close).
- The diamond trees. They work like the multiple-coin boxes from the Mario platformers: you may hit them for a set amount of time and obtain diamonds.
- Egg nests: These ones could drop an item -or a tripollo- from a varying choice of weighted.
- The story book item.
- An initial version of the world change item.
- Signposts.


As you can see, there were quite a few things to do @_@. Despite all this,  to be completely honest, I'm not entirely satisfied with the end result, despite the efforts the whole team has put onto it. Maybe it's just that I'm too hard on myself, but I believe that some degree of ambition and self-criticism should be a must for anybody that aims at developing games professionally, and I think that our game still has a long way to go until it can reach the status of 'acceptable'. The experience, however, has been really valuable, either as an example or as a counterexample.

Last, I would like to recommend a book that I found particularly useful during all this time (it's not the only one at all, but it was the one I've enjoyed the most):

Game Engine Programming, by Jason Gregory. I discovered it while I was looking for some algorithms on game loop synchronization, or maybe for entity managers and event handling. This book is amazing: exhaustive (it covers such things as animation engine programming, physics or entity management) and at the time I found it quite easy to read.