CPU speed changes by a multiply of 2 for short durations

Question

I'm using a raspberry pi and I need really fast performance from my CPU for a certain process.

To achieve that, I added isolcpus=3 to my kernel boot parameters, to isolate the core for this process only.

From looking at /proc/interrupts, it seems that this core irqs are also minimal (after isolation).

Now, I'm running this code on the isolated CPU (taskset -p 8 PID):

#define XX asm(" add     r3, r3, #1");
for (i=0; i< 100000; i++) {
    clock_gettime(CLOCK_REALTIME, &start);
    startc=clock();
    XX XX XX XX XX XX XX XX ... (about 5000 times)
    endc=clock();
    clock_gettime(CLOCK_REALTIME, &end);
    timespec_diff(&start, &end, &diff);
    printf("%d) Time: %d, Cyles: %d\n", i, diff.tv_nsec, endc-startc);
}

(Compiling with -O0 for no optimization, objdump -d shows what is expected)

The output i see is this for about 95% of the output lines:

300019) Time: 15938, Cyles: 13,
300020) Time: 15729, Cyles: 13,
300021) Time: 15677, Cyles: 13,
300022) Time: 15938, Cyles: 14,
300023) Time: 16406, Cyles: 14,
300024) Time: 15677, Cyles: 13,

But rarely, there are short durations where i get this:

339555) Time: 15521, Cyles: 13,
339556) Time: 15989, Cyles: 14,
339557) Time: 16198, Cyles: 14,
339558) Time: 21250, Cyles: 17,
339559) Time: 31875, Cyles: 27,
339560) Time: 31094, Cyles: 26,
339561) Time: 31823, Cyles: 27,
... (all 26-27 cycles)
341470) Time: 31666, Cyles: 27,
341471) Time: 22552, Cyles: 19,
341472) Time: 15781, Cyles: 13,
341473) Time: 15833, Cyles: 13,

Notice its almost exactly double the time. (Not completely sure how its 26 cycles though)

The tasks running on core 3 according to ps are:

PID   TID CLS RTPRIO  NI PRI PSR %CPU STAT WCHAN          COMMAND
3     3 TS       -   0  19   0  0.0 S    -              ksoftirqd/0
22    22 TS       -   0  19   3  0.0 S    -              cpuhp/3
23    23 FF      99   - 139   3  0.0 S    -              migration/3
24    24 TS       -   0  19   3  0.0 S    -              ksoftirqd/3
25    25 TS       -   0  19   3  0.0 S    -              kworker/3:0
26    26 TS       - -20  39   3  0.0 S<   -              kworker/3:0H
1305  1305 TS       -   0  19   3  0.0 S    -              kworker/3:1
1349  1349 TS       - -20  39   3  0.0 S<   -              kworker/3:1H
1677  1677 TS       -   0  19   3  127 R+   -              a.out

(Not sure what 127% cpu means.)

Both speeds are good enough for me, I just need it to be consistent. Does anyone have an idea why could this happen?

Is there any other option I can create a loop with consistent runtime?

Answer 1

Interesting question - something is competing for CPU resources.

1. have you looked at cpu temperature - is it throttling due to heat?

   pi@marvin:~$ sudo vcgencmd measure_temp

   temp=47.1'C

from memory, it will start to throttle @ 85 deg C.

2. What else is going on - have you checked "top" (insert favourite cpu / process monitoring tool here). The kernel will not dedicate a core just to your single process. Previous answer makes a good suggestion re cache impact from other processes.

Another approach, if you need a constant loop time, have you considered using the ualarm() function and signals? (see https://stackoverflow.com/questions/32734895/timer-interrupt-on-raspberry-pi-under-linux for a non-ideal example.) Hint - don't do much in your interrupt handler, just set a flag and detect this in your main loop where you need the real magic to happen.

Answer 2

I don't know what is happening but don't forget that you still share resources with the other cores.

It can well be that another core is causing replacement of your core code in the L2 cache. Maybe a sudden usage of the DDR by the GPU or it could be a sequence of DDR refresh cycles.

Is there any other option I can create a loop with consistent runtime?

Well the dumbest but simplest method is to have an 'busy wait' at the end. You have to find out about the longest time it take. Now check the time at the end of your loop. If it is shorter then the 'longest' time, you just wait before jumping back to the start of the loop.