I'm less familiar with C or C++, but depending on endianness and alignment you may be able to cast one array type into another, otherwise you'll have to resort to iterating with endian swapping functions or bit shifting. Overall that math is something that C is very good at, so as Joan mentioned, it's not likely to be significant. See this question for more information on the subject.
I'm most familiar with using the spidev module interface (although in python, not C), which is portable between different linux platforms (RasPi, BeagleBone, etc), and handles a lot of the particulars for you like chip selects and can take advantage of platform specific features like DMA. Reading the full 8kB is definitely the simplest method, and I can't think any reason to avoid it if you're using a fully-featured linux system. If it's a rolling buffer you may be able to read and process smaller chunks, but that would depend on other parts of your project that I'm not familiar enough with to decide on.
For your use case it looks like you'll need to write two bytes for the "CMD_HDR" that includes start address, read/write bit, and some padding bits; followed by reading the data you want. You could try something like the following (inspired by spidev_fdx.c (spidev full-duplex) example code (adjust to actual C/C++ and to your needs).
#include <sys/ioctl.h>
#include <linux/spi/spidev.h>
int fd;
unsigned char buf[8192]; // Nice if this is page aligned
unsigned char cmd[2];
struct spi_ioc_txr xfer[3]; //struct is 32 bytes x3
int status;
fd = open('/dev/spidev<bus>.<device>', O_RDWR)
// handle error if fd < 0
memset - buf, cmd, xfer
cmd[0] = address >> 4;
cmd[1] = (address << 4) & 0xff;
if(read_command) {
cmd[1] |= 1 << 3;
} // else write command
xfer[0].tx_buf = (unsigned long)cmd;
xfer[0].len = 2
xfer[0].speed_hz = 20000000; // 20,000,000 20MHz
xfer[1].rx_buf = (unsigned long)buf;
xfer[1].len = 4096;
xfer[1].speed_hz = 20000000;
xfer[2].rx_buf = (unsigned long)(buf + 4096);
xfer[2].len = 4096;
xfer[2].speed_hz = 20000000;
status = ioctl(fd, SPI_IOC_MESSAGE(3), xfer);
// error if < 0
// data now in xfer as bytes
One caveat is that you need to (re)load the spidev module with a larger that default buffer to handle transfers with a total size above 4096 bytes (each direction counted separately).
$ sudo rmmod spidev
$ sudo modprobe spidev bufsiz=8192
or to add it permanently (following reboot):
$ echo "options spidev bufsiz=8192" | sudo tee /etc/modprobe.d/spidev.conf
Configuration of the interface (max speed, SPI mode, lsb/msb first, etc) are done with ioctl calls on the file descriptor, more ioctl usage examples are shown in spidev_test.c