Avoid frezing/optimize Raspberry Pi 3 is under heavy load

Question

I am trying to run some prediction on pictures with Keras with pre-trained model vgg16. Here is my code:

    from keras.applications import VGG16
    from PIL import Image
    from keras.applications import VGG19
    from keras.applications import imagenet_utils
    from keras.applications.inception_v3 import preprocess_input
    from keras.preprocessing.image import img_to_array
    from keras.preprocessing.image import load_img
    import numpy as np

    inputShape = (224, 224)
    preprocess = imagenet_utils.preprocess_input

    image = load_img("/home/pi/Pictures/cam_test/test.jpg", 
    target_size=inputShape)                                                              

    image = img_to_array(image)
    image = np.expand_dims(image, axis=0)
    print(image.shape)
    image = preprocess(image)
    print("image")
    # Loads arhitecture, weights, optimizer etc.
    model = load_model('keras/src/vgg16_model.h5')
    print("Prediction")
    preds = model.predict(image)
    print(preds)
    P = imagenet_utils.decode_predictions(preds)
    for (i, (imagenetID, label, prob)) in enumerate(P[0]):
         print("{}. {}: {:.2f}%".format(i + 1, label, prob * 100))

I have installed all needed packages and I am using Python 3.4.2.

My hardware specifications are Raspberry Pi 3 with:

• 1GB RAM
• 4× ARM Cortex-A53, 1.2GHz (I clocked all CPU processors to 1.2GHz with command cpufreq-set)
• Added 1GB of swap with ZRAM (instructions here)
• Added 2GB of swap by changing this file (/etc/dphys-swapfile -> line CONF_SWAPSIZE=100 to CONF_SWAPSIZE=2048)
• Changed swappines from 60 to 80 in file /etc/sysctl.conf (because I have more swap than RAM, I sacrifice speed for more virtual RAM)
• Disabled oom-killer and overcommitting in file /etc/sysctl.conf (now programs cannot allocate more space that actually is free)
• OS: raspbian jessie

Now my question is can I optimise my Raspberry Pi even more (add more CPU power or RAM)?

Because now OS freezes or kills my python process even if I run this command:

    sudo nice -n -20 sudo python3 vgg16_keras.py

Any links or suggestions will be highly appreciated.