varying variables in a struct

I am currently adding layout locations for all my varying variables between shader stages.
The first thing I noticed was a noticeable drop in performance compared to not using layout locations.
When checking the assembly it looked like layout locations generated more
ATTRIB fragment_attrib = { fragment.attrib[0…7] };
compared to not using it.
This is the only real difference I could find.

Since then I switched everything to an input and output struct between stages and performance improved again.
There is one more thing that looks odd to me though.

The following struct uses 8 locations in the fragment stage, when it would perfectly fit into 5 locations for interpolation.

struct in_varyings {
vec2 var0;
float var1;
float var2;
vec3 var3;
vec3 var4;
vec3 var5;
vec3 var6;
vec4 var7;
};

So my question is, are varyings not packed when used with layout locations? Do I have to manually pack everything into vec4s and unpack it again in the next stage to maximize performance?