There are different ways to build hierarchies around geometry data. The basic idea for bounding volume hierarchies is to build a hierarchy of axis aligned bounding boxes (AABB).
Maybe look at some of the NVIDIA GTC or SIGGRAPH presentations on OptiX.
Here is the newest from last Monday: http://www.ustream.tv/recorded/51247222
In OptiX you actually provide the program which calculates the individual AABB around your arbitrary geometric primitive yourself.
A ray tracer traverses the acceleration structure (AS) hierarchy to quickly arrive at a node which points to the actual geometry and does intersection tests with any geometric primitive in that AS node. It doesn’t need to be a single geometric primitive, there could be multiple, and vice versa the same geometric primitive might appear in multiple AS nodes.
For example think of a very long and thin triangle which lies diagonally in space. The AABB around that would have a big volume and most of that volume it is empty. (Standard issue with hair geometry.) It would be more efficient to split that box into two or more overlapping boxes so that they each contain only a part of that triangle, but both together contain the whole triangle.
Now most of the empty space in the original AABB would not be considered for intersections anymore => faster rendering. Mind this is 3D, halving the edge sizes of a box is an eighth of the volume.
But when hitting that triangle in the overlapping region contained in both AABBs, both would register a potential hit at the same point. Only one of them would reach the closest hit program though. That’s all.
Now some acceleration structures do that, some don’t. E.g. BVH doesn’t split, SBVH and TRBVH do.
This actually happens, for example when you use the any_hit program of the shadow ray to color and attenuate your shadows from transparent materials in a Whitted ray tracer.
For global illumination ray tracers this is normally not a problem because the visibility test of a light done with a shadow ray is a boolean decision no matter if objects between surface and light are transparent or not. The applied light transport algorithm is taking care of colored shadows and caustics.
Identifying the front side of a triangle is normally done with a dot product between the ray’s direction and the geometric normal of the primitive.
If the triangles in your model all have consistent winding, let’s say counter-clockwise in a a right-handed coordinate system, then a triangle front face is described by the triangle vertices order (v0, v1, v2) and the face normal can be calculated at runtime like this:
const float3 geometricNormal = normalize(cross(v1 - v0, v2 - v0));
// ray.direction and geometric normal not in the same hemisphere means looking at a front face:
const bool isFrontFace = (dot(ray.direction, geometricNormal) < 0.0f);
If the model triangles are not wound consistently, you would either need to make it consistent (recommended) or you would need to provide this geometric normal yourself indicating which side you consider to be the front face.