To help debug, you can try reading the raw mouse events from `/dev/input/mice`.  This python script, from [][1], can help interpret the raw data.  I've modified it slightly below:

<!-- language: lang-py -->

    mouse = file('/dev/input/mice')  
    while True:  
        status, dx, dy = tuple(ord(c) for c in  
        def to_signed(n):  
            return n - ((0x80 & n) << 1)  
        dx = to_signed(dx)  
        dy = to_signed(dy)  
        print "%#02x %d %d" % (status, dx, dy)

The next time your computer "freezes", run this script as root.  It will print out x and y values for the _change_ in the mouse position as input on your USB port.  If these values correspond to what the pointer does on the screen, then it is likely you have a problem with your USB ports, motherboard, or a driver.  If these values correspond to what the physical mouse is doing, then the problem is somewhere in the software.