I think others here have done a good job answering:
Is it even worth switching from ARMv7 to ARMv8
One thing to add coming up lately is that Firefox 57 Quantum only works as arm64. Another: Recently the /r/raspberry_pi subreddit highlighted significant speedups specific to elliptic curve cryptography. However, likewise you'll lose out on other features or programs that may be less-supported with a 64-bit kernel (e.g. gaming with brcmEGL + dispmanx).
Why is it not working?
It seems you've skipped the key step of actually copying in the 64-bit kernel and its modules. That eLinux article may also be out-of-date as the default Raspbian Stretch bootloader will automatically boot kernel8.img if present.
If you aren't building a 64-bit kernel from source, you can download a working version from the original author of this article, here on GitHub. His image is based on Debian arm64, but if you intend to use this kernel with Raspbian make sure to copy in
/lib/modules/ from the Pi64 image. Some Raspbian functionality will most likely break, so back up your existing 32-bit kernel configuration prior.
Back to part of your other question:
(taking into account I'm willing to compile some C++ programs in the future).
If it's your intent to use your Pi 3 to develop and test both armhf and arm64 applications, you might as well run Pi64 directly. Unlike Raspbian, it has support for MultiArch out of the box without package conflicts. Outside of the Raspbian/Debian ecosystem, other 64-bit distros include Gentoo, openSUSE, and Fedora (see others' comments).