Multiple material On GeometryInstance ?

It seems that we could give more than one material to a GeometryInstance. (rtGeometryInstanceSetMaterialCount)

But how to decide which one to take effect while rendering?

You can use a materialbuffer, which maps primitives to material ids, e.g.:

rtBuffer<int> materialBuffer; 

RT_PROGRAM void intersectionProgram(int primitiveIndex)
{
...
rtReportIntersection(materialBuffer[primitiveIndex]);
...
}