2

I have a code written in Python that takes about 9% of the CPU when ran. I want it to use more so it can run faster. To solve this problem, I tried multiprocessing, however, before writing a code for it, I ran the same program in two different terminal which created two different Python processes. However, instead of each taking 9% of the CPU for a total of about 18% of CPU, each take about 5% for a total of 10% CPU. How can I change that and have each process take 9% of CPU or have just one program take >20% of CPU?

Thank you!

    import spidev, time, os

# opens SPI bus
spi = spidev.SpiDev()
spi.open(0, 0)

#spi1 = spidev.SpiDev()
#spi1.open(0, 1)

ch0 = 0
#ch1 = 1

now = time.time() * 1000
data = open("test_plot.dat", "w")
#data1 = open("test_plot_1.dat", "w")

def getReading(channel):
    # pull raw data from chip
    rawData = spi.xfer([1, (8 + channel) << 4, 0])
    # process raw data to something we understand
    processedData = ((rawData[1]&3) << 8) + rawData[2]
    return processedData
"""
def getReading1(channel):
    # pull raw data from chip
    rawData = spi1.xfer([1, (8 + channel) << 4, 0])
    # process raw data to something we understand
    processedData = ((rawData[1]&3) << 8) + rawData[2]
    return processedData
"""
while True:
    if (((time.time() * 1000) - now) < 10001):
        data.write("%d\n" % getReading(ch0))
        #data1.write("%d\n" % getReading1(ch1))
    else:
        break

# http://www.takaitra.com/posts/492
| improve this question | |
  • It depends on what your code is doing. I don't think anyone can answer this question as it's written. If you're going for performance though, consider dumping Python completely. – Brick Jul 25 '17 at 19:42
  • I have uploaded my code – Pi3-user-here Jul 25 '17 at 19:46
  • 3
    If this is meant to just read from SPI a bunch of times you are effectively limited to the speed of the SPI operation, which is relatively slow. GetReading will block the process until its complete so 99.99999% of this script is just waiting for IO. – tobyd Jul 25 '17 at 21:02
  • Is there a way to get rid of that 99.999% of the time? Because I tried print, it was slower, I tried sys.stdout, that was slower too. Writing to the file was the fastest that I could find. However, I haven't tried cStringIO, can that help? – Always Learning Forever Jul 25 '17 at 21:30
3

You have two bottlenecks here that are going to slow you down. First, there's the SPI, which was mentioned in a comment. That takes a few clock cycles in each direction and will limit you some. The second is writing to the SD card. I expect that the second is more important for your speed. Disk writes are generally much slower than computations, and the SD card is even slower than disk.

If you just want to use the whole number of CPU cycles, try taking those out and see what happens. Of course that means you won't be accomplishing your task, so then you'll need to put them back in some other way. A possible option would be to buffer your write-to-disk operations and have them trickle out in their own thread while the rest of the code runs. This may or may not work - For example, you might over-run the buffer and crash. You'll have to do some design and experimentation.

| improve this answer | |
  • Is there any way of reducing those clock cycles? MCP3008's datasheet mentions that it takes about 24 clock cycles per sample. I suspect that even if I can reduce it to about 20 cycles per sample, I should be able to increase the speed substantially. So, how exactly do I write to buffer and then to disk? Any tutorials or something? Will creating a list of all the input and then writing it to a file once I am done with my data collection do the trick? – Always Learning Forever Jul 26 '17 at 1:21
  • @Akshat Tripathi, what is your real objective? You seem focused on increasing CPU usage but I suspect that's the wrong metric for what you really want to do. Are you really trying to increase your sample rate? – Daddy the Runner Jul 26 '17 at 10:06
  • Yes, that's exactly what I am trying to do. I want to get as many sample points per second as possible. Ideally, 200k samples/second with this ADC – Pi3-user-here Jul 26 '17 at 15:12
  • @AkshatTripathi I don't believe that you can reduce the cycles for the samples via SPI. The protocol works by sending a series of rising and falling edges over the line, and that necessarily takes clock cycles. The only option that I see is to use multiple SPI buses in parallel on different threads / processes, but there's a limit on the number of buses that you can have on the Pi. Moreover, you'll need to understand how to design and write the parallel program, which may or may not be an issue for you. Python does have "multiprocessing" but it does NOT have multi-threading, just FYI. – Brick Jul 26 '17 at 16:21
  • 1
    Profiling before optimizing existing code is always good advice. For new design, you'll have to relay on understanding of principals as a guide. Regarding language, you should match what you use to the job. Personally I avoid Python unless there's a specific open source module that I want to use and cannot find in a better language. For low level data connects, I'd go to C/C++. Java's sometimes handy due to its relatively user-friendly threading model. I have tools in place that let me easily use all of the above in a single program. If you want a simple yes/no, however, I'd say yes to C here. – Brick Jul 28 '17 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.