Skip to content

Last updated：2026-10-12

C++ SIMD Vectorization

I have recently been reading content related to vectorization, and it looked a bit overwhelming, so I am taking this opportunity to learn high-performance SIMD programming.

SIMD stands for single-instruction multiple-data: one instruction, multiple data.

In traditional computer architectures, a CPU can process only one data element at a time. However, many tasks involve performing the same operation on large amounts of data, such as addition, multiplication, or logical operations on all elements of an array. SIMD programming provides the CPU with dedicated instruction sets, enabling it to perform the same operation on multiple data elements simultaneously.

This way of processing is particularly well suited to computation over data such as vectors, matrices, images, audio, and video.

The commonly used ones today are SSE, SSE2, AVX128, AVX256, and AVX512.

In this section we will briefly study some AVX512 operations; there are many operations in total, and only a few are introduced here.

1. Terminology

The first question is: SIMD code looks quite different from the code we usually write — full of underscores and names that are completely incomprehensible. How should we understand it?

For example:

  • _mm512_set1_ps

Broadcast single-precision (32-bit) floating-point value a to all elements of dst.

  • _mm512_set1_epi32

Broadcast 32-bit integer a to all elements of dst.

So I found the following table:

Abbreviation Full Name C/C++ Equivalent
ps packed single-precision float
ph packed half-precision None*
pd packed double-precision double
pch packed half-precision complex None*
pi8 packed 8-bit integer int8_t
pi16 packed 16-bit integer int16_t
pi32 packed 32-bit integer int32_t
epi8 extended packed 8-bit integer int8_t
epi16 extended packed 16-bit integer int16_t
epi32 extended packed 32-bit integer int32_t
epi64 extended packed 64-bit integer int64_t
epi64x extended packed 64-bit integer int64_t

https://stackoverflow.com/questions/70911872/what-are-the-names-and-meanings-of-the-intrinsic-vector-element-types-like-epi6

Another example:

_mm512_mask_load_ps

_mm512_mask_loadu_ps

The u stands for unordered, indicating an unaligned load: when loading data from memory with _mm512_mask_loadu_ps, no alignment requirement is imposed on the memory address, whereas _mm512_mask_load_ps requires 64-byte alignment.

By studying with this kind of cross-reference, you can very quickly figure out what each interface means.

For the related APIs, see the Intel Intrinsics Guide.

https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html

2. Practical Examples

2.1 Finding the Minimum

With 512 bits we can store sixteen 32-bit floats.

static inline rf_512 load(float* ptr) { return _mm512_loadu_ps(ptr); }
float a[width] = {1.0, 2.0,  3.0,  22.0, 5.0,  17.0, 9.0,  8.0,
                  9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0};
float b[width] = {3.3, 6.2,   5.3,   4.4,   5.5,   6.6,   7.7,  8.8,
                  9.9, 10.10, 21.11, 12.12, 13.13, 14.14, 15.0, 16.16};
rf_512 data = minimum(load(a), load(b));

So we can quickly get:

1 2 3 4.4 5 6.6 7.7 8 9 10 11 12 13 14 15 16

2.2 Quickly Shuffling the Data Order

For the input data

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

we can quickly get:

16.16 2 14.14 13.13 5 6 7 8 9 10 11 12 13 14 15 16

The corresponding implementation:

static inline rf_512 permutexvar(ri_512 idx, rf_512 src) {
  return _mm512_permutexvar_ps(idx, src);
}
/*
raw data: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
shuffle data: 16.16 2 14.14 13.13 5 6 7 8 9 10 11 12 13 14 15 16
*/
void print_permutexvar_mask() {
  float a[width] = {1.0, 2.0,  3.0,  4.0,  5.0,  6.0,  7.0,  8.0,
                    9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0};
  float b[width] = {1.1, 2.2,   3.3,   4.4,   5.5,   6.6,   7.7,  8.8,
                    9.9, 10.10, 11.11, 12.12, 13.13, 14.14, 15.0, 16.16};
  mask_type mask = make_bit_mask<1, 0, 1, 1, 0>();
  int idx_array[width] = {15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0};
  ri_512 idx = _mm512_loadu_si512((__m512i*)idx_array);
  rf_512 result = permutexvar_mask(load(a), mask, idx, load(b));
  float result_arr[width];
  store(result, result_arr);
  std::cout << "raw data: ";
  for (int i = 0; i < width; i++) {
    std::cout << a[i] << " ";
  }
  std::cout << std::endl;

  std::cout << "shuffle data: ";
  for (int i = 0; i < width; i++) {
    std::cout << result_arr[i] << " ";
  }
  std::cout << std::endl;
}

2.3 Rotation

For an array:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

We can rotate it to get:

4 5 6 7 8 9 10 11 12 13 14 15 16 1 2 3

or

14 15 16 1 2 3 4 5 6 7 8 9 10 11 12 13

That is, rotate down or rotate up.

Here, we can implement it like this:

rf_512 rotateGeneral(float* arr, int s) {
  int idx_array[width];
  for (int i = 0; i < width; i++) {
    idx_array[i] = (i + s) % width;
  }
  ri_512 idx = _mm512_loadu_si512((__m512i*)idx_array);
  rf_512 result = permutexvar(idx, load(arr));
  return result;
}

And so on — there are other examples too. You can see that by using SIMD we can implement some very interesting algorithms and speed up the processing of arrays and bulk data.

I will continue learning SIMD going forward — let's keep at it together~

Note: the complete examples for this section have been published in the Knowledge Planet, thanks~

Comments