This behaviour has very little to do with the hardware. Yes it is true that using the GPU speeds up rendering of encoded video or rendering 3D graphics but they must follow a strict standard. The GPU is not a round robin solution that magically makes everything work.
The core problem is the compression settings used with the encoder for your file. If it was encoded by a noob or used custom settings to try and get the best compression settings, this is the consequence, slow seek times and artefacts.
Bluray files follow industry standard profiles so that any decoding hardware that follow these standards can quickly deal with user requests. We are talking about 15mbs video rate with i-keyframes every second. This takes up a lot of space but gives you the best quality of video and the fastest seek response with no artefacts. (5~10GB files)
When you acquire a video file encoded by somebody that needs smaller files because they are in a remote place, they use a custom profiles, usually achieved by reduces i-keyframes in the video, to reduce file size but keep a fairly decent image quality. Something like 2~15 seconds apart will reduce the file size up to 40% without loosing video quality, impressive!!??
When the software reads the video from start to end without interruption, its fine, but when you seek on video that has i-keyframes very far apart, this is what happens. Lets assume i-keyframes are 5 seconds apart.
- Seek to 6 seconds
- Decoder looks for keyframe before seek time, found at 5 seconds.
- If its an i-keyframe, read data from file between 5 seconds to 10 seconds, as RAW video in to video buffer. Now start to play from 6 seconds from memory and continue to buffer from file as normal.
- If its not an i-keyframe keep on searching backwards until we can load an render the entire part of the video.
This in memory process can take some time to process the video file.
The artefacts that you experience are caused by unusual framesizes, weird framerates or really big keyframe spaces. If you see this is means the video file has its keyframes very widely spread apart and the player and hardware was not able to fully load the segment of video into memory to chop it up. So it loaded the best P or B frame and started playing from there. Sometimes strange framesizes cause corruption during splitting of video in memory and these artefacts are produced.
There are so many variables that cause really bad video experience- but its mostly because people encode video without actually understanding how to do it properly, using bad examples, or built in high compression profiles that are designed for continious streaming, like webcam feeds, TV feeds but are used for stored video instead !
The video decoders on the Pi are designed to decode video and work best with video containers that strictly conform to specified standards or streaming profiles where you cannot seek.