Do these results demonstrate hardware accelerated crypto is enabled?

Question

I currently use a Pi 2 as my OpenVPN server at home. I bought a Pi 3 to see if it was much faster, especially because it should have hardware accelerated AES crypto (hardware crypto features are included in the ARM Cortex-A53, I believe).

I think I've succeeded in compiling and installing the cryptodev-linux kernel module, and I think I've managed to build a working OpenSSL which uses the cryptodev engine.

But the results of speed tests are confusing. Do they look right? How come the (hopefully) accelerated version has such short times, if it's supposed to run for 3 seconds?

First, the stock openssl, as provided in the latest Raspbian Jessie Lite:

pi@raspberrypi3:~ $ openssl speed -evp aes-256-cbc
Doing aes-256-cbc for 3s on 16 size blocks: 5543752 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 64 size blocks: 1629278 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 424968 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 107249 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 8192 size blocks: 13438 aes-256-cbc's in 3.00s
OpenSSL 1.0.1k 8 Jan 2015
built on: Tue Mar  1 16:38:12 2016
options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial)    blowfish(ptr)
compiler: -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wa,--noexecstack -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-cbc      29566.68k    34757.93k    36263.94k    36607.66k    36694.70k

Next, the compiled openssl, with cryptodev engine enabled:

pi@raspberrypi3:~ $ /usr/local/ssl/bin/openssl speed -evp aes-256-cbc
Doing aes-256-cbc for 3s on 16 size blocks: 596490 aes-256-cbc's in 0.25s
Doing aes-256-cbc for 3s on 64 size blocks: 424524 aes-256-cbc's in 0.24s
Doing aes-256-cbc for 3s on 256 size blocks: 200017 aes-256-cbc's in 0.09s
Doing aes-256-cbc for 3s on 1024 size blocks: 64655 aes-256-cbc's in 0.05s
Doing aes-256-cbc for 3s on 8192 size blocks: 8710 aes-256-cbc's in 0.00s
OpenSSL 1.0.2g  1 Mar 2016
built on: reproducible build, date unspecified
options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) idea(int) blowfish(ptr)
compiler: gcc -I. -I.. -I../include  -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -march=armv7-a -Wa,--noexecstack -O3 -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-cbc      38175.36k   113206.40k   568937.24k  1324134.40k         infk

I'm relatively new to the compilation of kernels and modules etc... so it's been a steep learning curve trying to get this working - and I'm still not sure if I've succeeded or not!

Answer 1

Looks to me that you certainly have managed to hand off the encryption to the cryptodev kernel system.

The results you give indicate this as the "time" measured is the time spent by openssl encrypting the data. As an example, for encrypting the 1024 byte blocks, only 0.05 seconds of the 3 seconds run time was spent in the openssl binary, the remainder (mostly) was in the cryptodev kernel space.

The 'numbers' calculated are the throughput per second of openssl process cpu time, this does not mean that your throughput is actually 1.3GBps @1024 byte frames, sorry!

If you want to compare actual throughput, include the "-elapsed" argument to the speed test - e.g.:

openssl speed -evp aes-256-cbc -elapsed

Then, the results you get will be for 3 kernel+openssl seconds, rather than openssl only process cpu seconds.

By the way, although your results do demonstrate that you have handed off the hard work to the cryptodev successfully (nicely done!) - it doesn't guarantee that the hw acceleration is actually being used. I don't know how to check that - sorry!

Answer 2

The good news is that you've successfully enabled offloading of cryptographic operations to the kernel. DTick's answer explains how the numbers show this very well so I won't repeat the explanation here.

The bad news is that this is not useful because the Raspberry Pi does not have a hardware cryptographic accelerator. So you're using the software implementation in the kernel instead of the software implementation in OpenSSL. There's no particular reason why one would be faster than the other.

Some Cortex-A53 processors have acceleration for AES and SHA-2. It's an optional feature of the ARMv8 architecture. But this feature was present, you wouldn't need to offload to the kernel. The ARMv8 acceleration doesn't come from an accelerator device that only the kernel can access, like on some older Arm boards. It comes from special-purpose CPU instructions that any process can access (as long as the feature is enabled by the kernel, but Linux should take care of that at boot time). To take advantage of ARMv8 cryptographic extensions, you only need to ensure that your cryptographic library is compiled with the relevant support, you don't need any other software.

The Raspberry Pi 3 does not have the optional ARMv8 cryptography acceleration. The best performance you can get is using SIMD instructions which aren't specifically designed for cryptography.