I'm really struggling to get a strong grasp on how I should be handling collision response in a game engine I'm building around a 3D ball physics concept. Think Monkey Ball as an example of the type of gameplay.

I am currently using sphere-to-sphere broad phase, then AABB to OBB testing (the final test I am using right now is one that checks if one of the 8 OBB points crosses the planes of the object it is testing against). This seems to work pretty well, and I am getting back:

Plane that object is colliding against (with a point on the plane, the plane's normal, and the exact point of intersection.

I've tried what feels like dozens of different high-level strategies for handling these collisions, without any real success. I think my biggest problem is understanding how to handle collisions against walls in the x-y axes (left/right, front/back), which I want to have elasticity, and the ground (z-axis) where I want an elastic reaction if the ball drops down, but then for it to eventually normalize and be kept "on the ground" (not go into the ground, but also not continue bouncing). Without kluging something together, I'm positive there is a good way to handle this, my theories just aren't getting me all the way there.

For physics modeling and movement, I am trying to use a Euler based setup with each object maintaining a position (and destination position prior to collision detection), a velocity (which is added onto the position to determine the destination position), and an acceleration (which I use to store any player input being put on the ball, as well as gravity in the z coord).

Starting from when I detect a collision, what is a good way to approach the response to get the expected behavior in all cases?

Thanks in advance to anyone taking the time to assist... I am grateful for any pointers, and happy to post any additional info or code if it is useful.

UPDATE

Based on Steve H's and eBusiness' responses below, I have adapted my collision response to what makes a lot more sense now. It was close to right before, but I didn't have all the right pieces together at the right time! I have one problem left to solve, and that is what is causing the floor collision to hit every frame. Here's the collision response code I have now for the ball, then I'll describe the last bit I'm still struggling to understand.

// if we are moving in the direction of the plane (against the normal)...
if (m_velocity.dot(intersection.plane.normal) <= 0.0f)
{
    float dampeningForce = 1.8f; // eventually create this value based on mass and acceleration

    // Calculate the projection velocity
    PVRTVec3 actingVelocity = m_velocity.project(intersection.plane.normal);

    m_velocity -= actingVelocity * dampeningForce;
}
// Clamp z-velocity to zero if we are within a certain threshold
// -- NOTE: this was an experimental idea I had to solve the "jitter" bug I'll describe below
float diff = 0.2f - abs(m_velocity.z);
if (diff > 0.0f && diff <=  0.2f)
{
    m_velocity.z = 0.0f;
}

// Take this object to its new destination position based on...
// -- our pre-collision position + vector to the collision point + our new velocity after collision * time
// -- remaining after the collision to finish the movement
m_destPosition = m_position + intersection.diff + (m_velocity * intersection.tRemaining * GAMESTATE->dt);

The above snippet is run after a collision is detected on the ball (collider) with a collidee (floor in this case). With a dampening force of 1.8f, the ball's reflected "upward" velocity will eventually be overcome by gravity, so the ball will essentially be stuck on the floor. THIS is the problem I have now... the collision code is running every frame (since the ball's z-velocity is constantly pushing it a collision with the floor below it). The ball is not technically stuck, I can move it around still, but the movement is really goofy because the velocity and position keep getting affected adversely by the above snippet.

I was experimenting with an idea to clamp the z-velocity to zero if it was "close to zero", but this didn't do what I think... probably because the very next frame the ball gets a new gravity acceleration applied to its velocity regardless (which I think is good, right?).

Collisions with walls are as they used to be and work very well. It's just this last bit of "stickiness" to deal with. The camera is constantly jittering up and down by extremely small fractions too when the ball is "at rest".

I'll keep playing with it... I like puzzles like this, especially when I think I'm close. Any final ideas on what I could be doing wrong here?

UPDATE 2

Good news - I discovered I should be subtracting the intersection.diff from the m_position (position prior to collision). The intersection.diff is my calculation of the difference in the vector of position to destPosition from the intersection point to the position. In this case, adding it was causing my ball to always go "up" just a little bit, causing the jitter. By subtracting it, and moving that clamper for the velocity.z when close to zero to being above the dot product (and changing the test from <= 0 to < 0), I now have the following:

// Clamp z-velocity to zero if we are within a certain threshold
float diff = 0.2f - abs(m_velocity.z);
if (diff > 0.0f && diff <=  0.2f)
{
    m_velocity.z = 0.0f;
}

// if we are moving in the direction of the plane (against the normal)...
float dotprod = m_velocity.dot(intersection.plane.normal);
if (dotprod < 0.0f)
{
    float dampeningForce = 1.8f; // eventually create this value based on mass and acceleration?

    // Calculate the projection velocity
    PVRTVec3 actingVelocity = m_velocity.project(intersection.plane.normal);

    m_velocity -= actingVelocity * dampeningForce;
}

// Take this object to its new destination position based on...
// -- our pre-collision position + vector to the collision point + our new velocity after collision * time
// -- remaining after the collision to finish the movement
m_destPosition = m_position - intersection.diff + (m_velocity * intersection.tRemaining * GAMESTATE->dt);
UpdateWorldMatrix(m_destWorldMatrix, m_destOBB, m_destPosition, false);

This is MUCH better. No jitter, and the ball now "rests" at the floor, while still bouncing off the floor and walls. The ONLY thing left is that the ball is now virtually "stuck". He can move but at a much slower rate, likely because the else of my dot product test is only letting the ball move at a rate multiplied against the tRemaining... I think this is a better solution than I had previously, but still somehow not the right idea. BTW, I'm trying to journal my progress through this problem for anyone else with a similar situation - hopefully it will serve as some help, as many similar posts have for me over the years.