
That was the next thing I was going to say. Are you using the normals of the planes?
Hmm. If the normals all point outwards, I'm not sure why the algo is failing when you look down. Quite odd.
So you have surrounded your characters with planes that approximate their shape? Very good idea. In fact, I might steal it.
:D
I guess it would not make sense to try to detect every collision with every poly nor would it be efficient.
I'm really lost as to why the camera or view vector would cause collisions to not register. Are you also rotating the planes that you are using for collision detection? Perhaps you are rotating them too far which would cause the normal test to fail. Give me some more info on exactly how you are testing for hits on the walls  I think I understand but I want to make sure.

well at least part of the problem was the fact that the collisions failed when your move length was less than EPSILON units

So basically you are using the normals as 'feelers'? Then you do a 'pointinplane' test to see if the normal (extended a bit) hit any walls?

Umm, not exactly. Here's the algorithm. Note this is described in Frank Puig's tutorials in the code dump at gametutorials.com. Frank helped me a lot by uploading me his projects before posting them on GT, and just plain helping me when I was stuck.
The algorithm:
float startfraction = 1.0f;
float endfraction = 1.0f;
Loop through every single plane of brush.
For every plane:
If the startdistance and enddistance are both positive cancel the tests (the move doesn't hit the brush if it's completely in front of any one of the planes)
If the startdistance and enddistance are both negative skip this plane but continue checking the brush for it may intersect another plane of the brush
Otherwise your move spans one of the brush planes and potentially hits the geometry represented by the planes. To find the exact intersection and to determine if it actually hits the geometry (and doesn't just hit an infinite plane) do the following:
If the startdistance is on the positive side of the plane calculate the fraction of the intersection against the plane, and if the startfraction is less than the intersect fraction assign fraction to startdistance
If the startdistance is on the negative side of the plane calculate the fraction of the intersection against the plane, and if the endfraction is greater than the intersect fraction assign the intersect fraction to the endfraction
At the end of checking this brush if startfraction is > 1.0f, and if startfraction is less than endfration then you know you hit the geometry and is a valid intersection. Then you go on to check to see if this fraction is the closest fraction of the entire leaf (each leaf usually has many brushes, so you only want to keep the closest brush intersection for the leaf you are in)

If you've already got at least the BSP file then download this, it's a bunch of balls that bounce around the world