I'm ashamed to say this but, I don't have the slightest clue. :(
Here's is a doublecone: File:DoubleCone.png - Wikipedia, the free encyclopedia
In this instance the up arrow is the Z-axis.
To transform a point (x, y, z) into a clyinder, you use (x/z, y/z, z) although you can then ignore the z and just treat it as (x/z, y/z)
However, thinking about it a little more, you're probably better off just skipping that step altogether and using the whole point in cone test. This has the advantage of not dividing by zero.
So instead just check for x*x + y*y < z*z for each point after having applied the transformation matrix.
Seems like I will be using the circle based one from my first post because we will be pushing for other features first. Thanks alot though, guys. Maybe I'll come back again after it's less hectic here. I'll make a new thread instead of using this one though, so don't worry about pushing up old junks. :P
Again, thank you very much.