I've read so many answers on here about how to do Ray Picking, that I thought I had the idea of it down. But when I try to implement it in my game, I get garbage.

I'm working with LWJGL.

Here's the code:

public static Ray getPick(int mouseX, int mouseY){
    glPushMatrix();

    //Setting up the Mouse Clip
    Vector4f mouseClip = new Vector4f((float)mouseX * 2 / 960f - 1, 1 - (float)mouseY * 2 / 640f ,0 ,1);

    //Loading Matrices
    FloatBuffer modMatrix = BufferUtils.createFloatBuffer(16);
    FloatBuffer projMatrix = BufferUtils.createFloatBuffer(16);
    glGetFloat(GL_MODELVIEW_MATRIX, modMatrix);
    glGetFloat(GL_PROJECTION_MATRIX, projMatrix);

    //Assigning Matrices
    Matrix4f proj = new Matrix4f();
    Matrix4f model = new Matrix4f();
    model.load(modMatrix);
    proj.load(projMatrix);

    //Multiplying the Projection Matrix by the Model View Matrix
    Matrix4f tempView = new Matrix4f();
    Matrix4f.mul(proj, model, tempView);
    tempView.invert();

    //Getting the Camera Position in World Space.  The 4th Column of the Model View Matrix.
    model.invert();
    Point cameraPos = new Point(model.m30, model.m31, model.m32);

    //Theoretically getting the vector the Picking Ray goes
    Vector4f rayVector = new Vector4f();
    Matrix4f.transform(tempView, mouseClip, rayVector);
    rayVector.translate((float)-cameraPos.getX(),(float) -cameraPos.getY(),(float) -cameraPos.getZ(), 0f);
    rayVector.normalise();
    glPopMatrix();

    //This Basically Spits out a value that changes as the Camera moves.
    //When the Mouse moves, the values change around 0.001 points from screen edge to edge.
    System.out.format("Vector: %f %f %f%n", rayVector.x, rayVector.y, rayVector.z);

    //return new Ray(cameraPos, rayVector);
    return null;
}

I don't really know why this isn't working. I was hoping some more experienced eyes might be able to help me out.

I can get the camera position like a champ, it's the vector the rays going in that I can't seem to get right.

Thanks.