Saturday, September 26, 2009

How to Compile PyQt4 for Python 2.6 AMD64 (64 bit) on Windows x64

PyQt 4.5.4 for Python 2.5 amd64
PyQt 4.5.4 for Python 2.6 amd64

Recently I've written an interface for the first time with PyQt4 in Linux for one of our pipeline related scripts. It was working very well under Linux, until I decided to check how it is going to look under Windows x64.

Well, it didn't take too long for me to recognize that there is no PyQt4 for Python 2.5 amd64. So I decided to compile my own. By the way you will see that I started for Python 2.5 but compiled for Python 2.6, it is just because, in a panic mode I've installed Maya 2010 to see if the package I found for PyQt4 for Python 2.6 amd64 for Windows x64 will work under. But no it wasn't ( actually it was, but I didn't know that I should add the path that contains the Qt DLLs to environment path ). So I dived in to compiling PyQt4 for Python2.6 amd64 for Windows x64.

Here you can find the How-To of that compilation...

You need these packages and programs:

MS Visual Studio 2008
Python 2.6 amd64
Qt 4.5.2 SDK for Windows
Sip 4.8.2 Source
PyQt4 4.5.4 Source
Nullsoft scriptable install system

Lets start:

  1. Install Python 2.6.2 amd64:
    • install it to C:\Python2.6
    • if you have already installed it somewhere else, in the rest of this "how-to" use that path instead of C:\Python26

  2. compile sip:
    • download sip-4.8.2.zip
    • unzip sip-4.8.2.zip to C:\sip-4.8.2
    • open up the Visual Studio 2008 x64 Win64 Command Prompt from start menu/programs/etc.
    • run the commands below
    • cd C:\sip-4.8.2
    • C:\Python26\python configure.py -p win32-msvc2008
    • nmake
    • nmake install
    • open the python interpreter and check the module by using:

      from sip import *
      print SIP_VERSION_STR


      if it doesn't give any error messages about a DLL or something, it is installed correctly

  3. compile Qt4:
    • download Qt SDK for Windows
    • install it to C:\Qt\4.5.2 not to 2009.03
    • delete the two tmp files: (they were causing problems in the middle of the compile process, very annoying to start again)
      C:\Qt\4.5.2\qt\src\3rdparty\webkit\WebCore\tmp\moc\{debug,release}_shared/mocinclude.tmp)
    • open up the Visual Studio 2008 x64 Win64 Command Prompt
    • run the commands below
    • cd C:\Qt\4.5.2\qt
    • set QTDIR=C:\Qt\4.5.2\qt
    • set PATH=%PATH%;C:\Qt\4.5.2\bin
    • configure -opensource -platform win32-msvc2008
    • enter "y" for the question
    • nmake ( and go to have a lunch or dinner, seriously, it took 3 hours in my computer, it will be around on yours too )
    • nmake install
    • congratulations you have built Qt4 x64 for Windows x64
      (NOTE 1: you can create a batch file that runs the 'configure' then the 'nmake' and then the 'nmake install' commands, but when you run this batch file wait for 1 seconds to the question in configure step then leave :) )
      (NOTE 2: because we didn't specified a -prefix option in configure, it will compile it over the source directories )

  4. compile PyQt4:
    • download PyQt4 4.5.4 Source
    • unzip the PyQt4 source to C:\PyQt-win-gpl-4.5.4
    • open Visual Studio 2008 x64 Win64 Command Prompt
    • run the commands below
    • set QTDIR=C:\Qt\4.5.2\qt
    • set Path=%PATH%;C:\Qt\4.5.2\qt\bin\
    • cd C:\PyQt-win-gpl-4.5.4
    • C:\Python26\python configure.py -w
    • enter "yes" to the question
    • nmake ( again it takes some time )
    • nmake install
    • now you should be able to use PyQt4, but if there is an error message saying:

      Python 2.6.2 (r262:71605, Apr 14 2009, 22:5+:60) [MSC v.1500 64 bit (AMD64)] on win32
      Type "help", "copyright", "credits" or license" for more information.
      >>> from PyQt4 import QtCore
      Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      ImportError: DLL load failed: The specified module could not be found
      >>>


      you need to add the folder that contains Qt DLLs to the environment path variables, by using the Control Panel -> System -> Advanced -> Environment Variables

  5. Nullsoft installer to make installable binary package:
    • download and install Nullsoft installers latest version
    • right click C:\PyQt-win-gpl-4.5.4\PyQt.nsi and select "Compile NSIS Script"
      ( unfortunatelly the script gives a lot of errors, you need to copy/paste some files and create folders by your self according to the complains of the compiler )


for Python 2.5, compiling PyQt4 for Python 2.5 x64 is quite similar, you just need to use vs2005 and its x64 command prompt compiler with the nearly the same commands, just replace win32-msvc2008's with win32-msvc2005, that's all...

EDIT 1:

for Python 2.6 version of PyQt4 you need to install Microsoft Visual C++ 2008 SP1 Redistributable Package (x64) also

for Python 2.5 version of PyQt4 Microsoft Visual C++ 2005 SP1 Redistributable Package (x64) also

EDIT 2:

I've compiled the Python 2.5 amd64 version of PyQt4 for Windows x64... Download (the Phonon Multimedia Framework is only supported by python 2.6 an later)

Thursday, September 24, 2009

Python Function Decorators 2: Decorating a Class Method

I think I need to make things a little bit clearer with the code shown in previous post...

The Decoration

Decorating a function enables you to call other code before the original function is called, or you can completely change the functions code. As I've red somewhere it is best equated to preprocessor macros or template libraries in C++.

In my previous post, I've decorated my class methods by adding @CachedMethod at the definition of the class method. This is one of the two ways of defining decorators. When I do that the Python interpreter decorates the unbound function, which means, it is decorated whenever I've imported the module not when I've created the instance object. This leads us to a little but important difference that can easily be neglected, so be careful to not to miss what you intended to do. I mean, when I've first written the CachedMethod class, it was caching the data in the Class not in the instance object. So every instance of that class was trying to use the same cached data, which in my situation was a wrong behaviour, though it is may desirable in some other cases...

By decorating my functions with a class (you can decorate them with functions too), I'm practically turning them in to classes. So instead of being a method of a class, they become a class belonging to another class.

The key to decorate a Class method in a way to be able to reach the instance object is to store the instance object in the __get__ method of the decorator and then use it in the __call__ method.

The order of execution of the CachedMethod classes methods are like that: when you import your module the functions are decorated, and the __init__ of the CachedMethod runs for all of the class methods you have decorated. Then when you create your object nothing happens, but when you call a decorated method, first the __get__ method runs, than the __call__ method. It is a good practice to add print() commands to the decorators method to understand what is going on.

The __get__ method has 3 arguments self, instance and class. Self, as regular, is the CachedMethod instance, instance is the instance of which the decorated function belongs to, and the class is the super that the instance is derived from.

You can change the settings of the arguments of __call__ method, but I think the best settings is a couple of *args, **keys together ( def __call__(self, *args, **keys) ) . So you can get any type of arguments setup passed to your original function and use them in your decorator. Other than this setup you can use the same argument count and order in your original function, but I don't recommend that, because you are restricted to decorate only the methods that has the same types of arguments.

CachedMethod Explained

Here are the brief explanation of what is going on in the methods of CachedMethod class.

__init__ : I store unbound method and the name of the unbound method in class variables, to let my self to call the original function later on.

__get__ : as far as I know, this is the only place that I can get the instance object. I store the instance object in self._obj and check if it already has its attributes set up. If not I create the attributes by using setattr commands

__call__ : I've all the data I need to have, so I do the caching by calling the original function if the self._data is empty or the time delta is bigger than the self._maxTimeDelta, otherwise I return the cached data. To get the data from the instance appropriately I use getattr, and I evaluate the attribute name on the fly by using self._name + '_data' for example.

Thats all... Hope it helps you to understand what is going on. And if not, there are some nice tutorials here and here is a nice function decorator library in Python.org

Thursday, September 17, 2009

Python Function Decorators: Caching a Class Methods Output

I've been writing a class and I've noticed that one of its methods may could create a lot of overhead to our file server when it is used often. And its result was not changing so often. So trying to get all the data over and over again is something I should avoid.

I've decided to create some kind of caching system to overcome this problem. I've changed the function so it was caching its own result, and it was not trying to get the data over and over again before some definite time is passes. It was working very nice. But later on I've noticed several other functions those had to have this functionality.

So I've decided to use a decorator that decorates the functions with that ability. Here is the decorator:
#######################################################################
class CachedMethod(object):
    """caches the result of a class method inside the instance
    """
    
    
    
    #----------------------------------------------------------------------
    def __init__(self, method):
        # record the unbound-method and the name
        #print "running __init__ in CachcedMethod"
        
        if not isinstance( method, property ):
            self._method = method
            self._name = method.__name__
            self._isProperty = False
        else:
            self._method = method.fget
            self._name = method.fget.__name__
            self._isProperty = True
        
        self._obj = None
    
    
    
    #----------------------------------------------------------------------
    def __get__(self, inst, cls):
        """use __get__ to get the instance object
        and create the attributes
        
        if it is  a property call __call__
        """
        #print "running __get__ in CachcedMethod"
        
        self._obj = inst
        
        if not hasattr( self._obj, self._name + "._data"):
            #print "creating the data"
            setattr ( self._obj, self._name + "._data", None )
            setattr ( self._obj, self._name + "._lastQueryTime", 0 )
            setattr ( self._obj, self._name + "._maxTimeDelta", 60 )
            #print "finished creating data"
        
        if self._isProperty:
            return self.__call__()
        else:
            return self
    
    
    
    #----------------------------------------------------------------------
    def __call__(self, *args, **kwargs):
        """
        """
        #print "running __call__ in CachedMethod"
        
        delta = time.time() - getattr( self._obj, self._name + "._lastQueryTime" )
        
        if delta > getattr(self._obj, self._name + "._maxTimeDelta") or getattr(self._obj, self._name + "._data" ) == None:
            # call the function and store the result as a cache
            #print "caching the data"
            data = self._method(self._obj, *args, **kwargs )
            setattr( self._obj, self._name + "._data", data )
            
            # zero the time
            lastQueryTime = time.time()
            setattr( self._obj, self._name + "._lastQueryTime", time.time() )
        #else:
            #print "returning the cached data"
        
        return getattr( self._obj, self._name + "._data" )
    
    
    
    #----------------------------------------------------------------------
    def __repr__(self):
        """Return the function's representation
        """
        objectsRepr = str(self._obj)
        objectsName = objectsRepr.split(' ')[0].split('.')[-1]
        cachedObjectsRepr = ''
        return cachedObjectsRepr


You can use that anywhere you want, but beware that the caching is time dependent, not input dependent.