I think others here have done a good job answering:
Is it even worth switching from ARMv7 to ARMv8
Various software cited to require ARMv8 includes CockroachDB, MongoDB (if accessing >2 GB), the Dolphin emulator, and OpenMW. However, likewise you'll lose out on other features or programs that may be less-supported with a 64-bit kernel (e.g. gaming with brcmEGL + dispmanx).
Why is it not working?
It seems you've skipped the key step of actually copying in the 64-bit kernel and its modules. That eLinux article may also be out-of-date as the default Raspbian Stretch bootloader will automatically boot kernel8.img if present.
If you aren't building a 64-bit kernel from source, you can download a working version from the original author of this article, here on GitHub (Pi 3B only). His image is based on Debian arm64, and if you intend to use this kernel with Raspbian make sure to copy in
/lib/modules/ from the Pi64 image. You may also download and install such pre-configured Raspbian images:
Back to part of your other question:
(taking into account I'm willing to compile some C++ programs in the future).
If it's your intent to use your Pi 3 to develop and test both armhf and arm64 applications, you might as well run Pi64 directly. Unlike Raspbian, it has support for MultiArch and thus doesn't require running 64-bit containers. Outside of the Raspbian/Debian ecosystem, other 64-bit distros include Ubuntu Server, Gentoo, openSUSE, and Fedora (see others' comments).