I've been getting interested in SIMD programming on CPU using SSE recently, but I'm a complete noob on the subject. I found this article describing how to make an efficient float3 type using the recent __vectorcall convention, and it made me wonder how best to deal with the wasted space inherent in using a 128-bit type to represent vectors smaller than 4.
For a typical use case, you might want to transform a bunch of positions and normals on the CPU in order to do a batched draw call. When transmitting the data over to the GPU, though, ideally you'd want this as tightly packed arrays of float3, either interleaved or in separate buffers.
It seems like there are two options here:
A - store the data tightly packed, and unpack and repack to and from SSE-friendly format when you need to manipulate it, taking the CPU performance hit, or
B - store the data loosely packed on 16-byte boundaries, SSE it directly, and take the bandwidth and VRAM hit when sending to GPU.
Which of these options would be preferred in a real-world 3D engine, or under what circumstances would you prefer one to the other?