I have a text processing project that is currently Bash, GNU Parallel and other command-line tools. It's getting moved to a C++ queue type design. It is about 14,000 files each day, plus performing math and making models. I have a 60gb backlog. The format is RAR archives and then inside are CSV files.
I'm trying to decide on an 8 node Raspberry Pi 3B+ cluster (about $500) or buying a used Dual Xeon (12 core) desktop (about $800, HP Z820).
The RPi cluster will help ensure my code scales well. However, I don't quite know how speeds compare.
8 RPi's x 4 cores each = 32 cores 2 x Xeon (12 cores) = 24 cores.
The RPI's are 1.4GHz 64-bit quad-core ARMv8 CPU, 1 GB RAM.
The dual Xeon is 2.5ghz each, 64gb RAM.
How can I accurately think about the cores and speeds of each to decide what to do? This project will get larger within the next year and there will be more files to parse, manipulate, etc.