Difficult to speak in generalities without seeing actual code, to determine the precise access patterns.
Thrust is built on top of CUDA C/C++ (at least, if you are using the CUDA backend). So if you are talking about a thrust tuple, I would liken it to a structure – there is no concept of a tuple in CUDA C/C++ (even if there were, we would have to understand how tuple access decomposes into fundamental read/write transactions - for that understanding it is sufficient to liken it to a structure).
We must ask, then, what does that structure look like, and how exactly is it accessed?
suppose I have a struct like this:
struct my_struct{
int x;
int y;
int z;
};
Let’s say I have an array (AoS) of those:
my_struct A[1024];
Now what does the thread code look like? By your statements (" … each one of n-tuples that I’m working with is always accessed in entirety… ") I presume you mean some (CUDA thread) code like this:
my_struct my_tuple = A[idx];
what access patterns will be generated as a result of that? First of all, since this particular tuple happens to be 96 bits, there is no way for the CUDA compiler to resolve that into a single access.(*) It must do at best(assuming packed storage) a 64-bit access followed by a 32-bit access. In your packed storage case:
x1y1z1x2y2z2x3y3z3...
1 1 2 1 1 2 1 1 2...
we would have a result pattern where adjacent threads in the warp reading adjacent tuples would access first the elements marked by 1, then, later, in another read transaction, the elements marked by 2. So this is less than ideal efficiency - the profiler would indicate this.
If your tuple happened to be exactly 32, 64, or 128 bits in length, and you’re feeling lucky (betting that the compiler will figure out that this structure access can be combined into an elementary read) then you would get ideal, efficient access across the warp. The way to remove the luck would be to take the extra step of creating a union with int4 or some other construct in the struct that makes it painfully obvious to the compiler that you intend to load it all in a single transaction.
But barring the above specific cases, every other tuple configuration will result in inefficient access, to some degree. So AoS is risky, at best. In the general case, it simply cannot, and will not guarantee efficient access - pretty much no matter what you do in thread code.
If we convert to SoA, even if you intend to use the entire “structure” together, we can always, trivially, easily enforce perfectly optimal access, for any struct/tuple configuration:
struct my_struct2 {
int x1[1024];
float x2[1024];
double x3[1024];
bool x4[1024];
char x5[1024];
}
struct my_tuple {
int x1;
float x2;
double x3;
bool x4;
char x5;
}
my_struct2 A;
access code:
my_tuple t;
t.x1 = A.x1[idx];
t.x2 = A.x2[idx];
t.x3 = A.x3[idx];
t.x4 = A.x4[idx];
t.x5 = A.x5[idx];
// then process t...
this will always be perfectly efficient access, for fundamental types up to 128 bits, and assuming SoA storage:
x1x1x1…x2x2x2…x3x3x3…
which of course is implied in the SoA definition:
my_struct2 A;
And, beautifully, magically, the thrust zip_iterator concept takes the above messy thread access code in the SoA case, and turns it into something clean, and easy to understand from the programmers view, almost as if we were back to our clean looking AoS access code:
my_struct my_tuple = A[idx];
I think in general, using generic terms like “tuple” and “iterator” and “predicate” don’t provide enough specificity to understand this clearly. It’s necessary at least to get to the C code level, and perhaps even to the SASS code level, to understand what is efficient and what isn’t.
(*) http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#device-memory-accesses