0

I am trying to sample from two analog sensors at highest frequency possible (using and Arduino Uno, MPC3002 ADC), then send the data to raspberry pi to save them as binary file.

Arduino and MCP3002 communicates at 8MHz through SPI and it all works fine (double checked with oscilloscope). Here is my Arduino Code:

#include <SPI.h>
#include "MCP3002.h"

MCP3002 adc(10);

uint16_t ISR_BUF_FULL_COUNT = 256;
ISR(TIMER2_COMPA_vect) {
  byte buff[2];


  adc.analogRead0(buff);
  Serial.write(buff,2);
  adc.analogRead1(buff);
  Serial.write(buff,2);

}

void setup() {

  pinMode(7, OUTPUT);
  //set timer2 interrupt at 64kHz
  TCCR2A = 0;// set entire TCCR2A register to 0
  TCCR2B = 0;// same for TCCR2B
  TCNT2  = 0;//initialize counter value to 0
  // set compare match register for 15.625khz increments
  OCR2A = 127;// = (16*10^6) / (50000??*8) - 1 (must be <256)
  // turn on CTC mode
  TCCR2A |= (1 << WGM21);
  // Set CS21 bit for 8 prescaler
  TCCR2B |= (1 << CS21);
  //TCCR2B |= (1 << CS20);  
  // enable timer compare interrupt
  TIMSK2 |= (1 << OCIE2A);

  adc.begin();
  Serial.begin(1000000);

}

void loop() {

}

and Here is my python code in Pi side:

import serial
ser = serial.Serial('/dev/ttyUSB0',baudrate=1000000,timeout=None, bytesize=8, parity='N', stopbits=1)
ser.flushInput()
ser.flushOutput()
newFile = open("data.bin", "wb")
while True:
    newFile.write(ser.read(ser.inWaiting()))
    ##print ser.inWaiting()

Pi save the data correctly at the beginning and then suddenly (like after 200 samples) the data start be irrelevant values (my strong guess is that it is related to the buffer of pi) I intend to transfer 4 bytes (2 bytes for each sensor) at 1 MPS serial. I checked with oscilloscope the tx and the whole procedure takes less than 40 us.

ISR timer of Arduino is called every 64 us (checked with osilloscope) I checked the TX pin of Arduino and sending each 2 bytes will take exactly less than 40us, including reading time from MCP3002 which all looks fine.

Can any body suggest any solution why Pi behave like this? Can it be related to the buffer? Is Python on Pi fast enough to read at 1MBPS rates?

UPDATE: So here is the C code I used (using read/write functions) and It still doesn't work properly.

/*====================================================================================================*/
/* www.xanthium.in                                            */
/* Copyright (C) 2014 Rahul.S                                                                         */
/*====================================================================================================*/

/*====================================================================================================*/
/* Running the executable                                                                             */
/* ---------------------------------------------------------------------------------------------------*/ 
/* 1) Compile the  serialport_read.c  file using gcc on the terminal (without quotes)                 */
    /*                                                                                                    */
/*  " gcc -o serialport_read serialport_read.c "                                                  */
/*                                                                                                    */
    /* 2) Linux will not allow you to access the serial port from user space,you have to be root.So use   */
    /*    "sudo" command to execute the compiled binary as super user.                                    */
    /*                                                                                                    */
    /*       "sudo ./serialport_read"                                                                     */
/*                                                                                                    */
/*====================================================================================================*/

/*====================================================================================================*/
/* Sellecting the Serial port Number on Linux                                                         */
/* ---------------------------------------------------------------------------------------------------*/ 
/* /dev/ttyUSBx - when using USB to Serial Converter, where x can be 0,1,2...etc                      */
/* /dev/ttySx   - for PC hardware based Serial ports, where x can be 0,1,2...etc                      */
    /*====================================================================================================*/

/*-------------------------------------------------------------*/
    /* termios structure -  /usr/include/asm-generic/termbits.h    */ 
/* use "man termios" to get more info about  termios structure */
/*-------------------------------------------------------------*/

    #include <stdio.h>
    #include <fcntl.h>   /* File Control Definitions           */
    #include <termios.h> /* POSIX Terminal Control Definitions */
    #include <unistd.h>  /* UNIX Standard Definitions      */ 
    #include <errno.h>   /* ERROR Number Definitions           */

void main(void)
    {
        int fd;/*File Descriptor*/

    printf("\n +----------------------------------+");
    printf("\n |        Serial Port Read          |");
    printf("\n +----------------------------------+");

    /*------------------------------- Opening the Serial Port -------------------------------*/

    /* Change /dev/ttyUSB0 to the one corresponding to your system */

        fd = open("/dev/ttyUSB0",O_RDWR | O_NOCTTY);    /* ttyUSB0 is the FT232 based USB2SERIAL Converter   */
                            /* O_RDWR   - Read/Write access to serial port       */
                            /* O_NOCTTY - No terminal will control the process   */
                            /* Open in blocking mode,read will wait              */



        if(fd == -1)                        /* Error Checking */
               printf("\n  Error! in Opening ttyUSB0  ");
        else
               printf("\n  ttyUSB0 Opened Successfully ");


    /*---------- Setting the Attributes of the serial port using termios structure --------- */

    struct termios SerialPortSettings;  /* Create the structure                          */

    tcgetattr(fd, &SerialPortSettings); /* Get the current attributes of the Serial port */

    /* Setting the Baud rate */
    cfsetispeed(&SerialPortSettings,B1000000); /* Set Read  Speed as 9600                       */
    cfsetospeed(&SerialPortSettings,B1000000); /* Set Write Speed as 9600                       */

    /* 8N1 Mode */
    SerialPortSettings.c_cflag &= ~PARENB;   /* Disables the Parity Enable bit(PARENB),So No Parity   */
    SerialPortSettings.c_cflag &= ~CSTOPB;   /* CSTOPB = 2 Stop bits,here it is cleared so 1 Stop bit */
    SerialPortSettings.c_cflag &= ~CSIZE;    /* Clears the mask for setting the data size             */
    SerialPortSettings.c_cflag |=  CS8;      /* Set the data bits = 8                                 */

    SerialPortSettings.c_cflag &= ~CRTSCTS;       /* No Hardware flow Control                         */
    SerialPortSettings.c_cflag |= CREAD | CLOCAL; /* Enable receiver,Ignore Modem Control lines       */ 


    SerialPortSettings.c_iflag &= ~(IXON | IXOFF | IXANY);          /* Disable XON/XOFF flow control both i/p and o/p */
    SerialPortSettings.c_iflag &= ~(ICANON | ECHO | ECHOE | ISIG);  /* Non Cannonical mode                            */

    SerialPortSettings.c_oflag &= ~OPOST;/*No Output Processing*/

    /* Setting Time outs */
    SerialPortSettings.c_cc[VMIN] = 10; /* Read at least 10 characters */
    SerialPortSettings.c_cc[VTIME] = 0; /* Wait indefinetly   */


    if((tcsetattr(fd,TCSANOW,&SerialPortSettings)) != 0) /* Set the attributes to the termios structure*/
        printf("\n  ERROR ! in Setting attributes");
    else
                printf("\n  BaudRate = 1000000 \n  StopBits = 1 \n  Parity   = none");

        /*------------------------------- Read data from serial port -----------------------------*/

    tcflush(fd, TCIFLUSH);   /* Discards old data in the rx buffer            */

    char read_buffer[32];   /* Buffer to store the data received              */
    int  bytes_read = 0;    /* Number of bytes read by the read() system call */
    FILE *write_ptr;
    write_ptr = fopen("testc.bin","wb");  // w for write, b for binary
    while (1){
        bytes_read = read(fd,&read_buffer,32); /* Read the data                   */
        fwrite(&read_buffer,1,bytes_read,write_ptr); 
        printf("%d",bytes_read);


    }
    close(fd); /* Close the serial port */

    }
  • This has nothing to do with the Pi. Linux is NOT a real time OS. Using Python just adds additional delay. It is possible to increase serial buffer size, but expecting continuous logging at 1Mbps is probably unrealistic. You need to implement flow control and/or accept lost data. – Milliways Jun 6 '18 at 9:26
  • Thank you, How can I increase serial buffer size? It seems that it is already 4095, if i can double it, I think it would be solved. – Mojtaba Jun 6 '18 at 9:42
  • You picked only one component of my comment. If I wanted to do this I would use 'C' and you can specify your buffer size. Python probably uses the buffer size specified by the kernel - maybe this could be modified with termios. The real message was continuous logging at 1Mbps is probably unrealistic - it could possibly be logged with a large buffer, but presumably you want to do something with the data. – Milliways Jun 6 '18 at 11:24
  • You were right, I tried it with C and again the same problem. However when I leave enough time gap between the data packages (i.e. I increased the timer of Arduino from 64 to to 240 us) it works perfect. Isn't it weird? – Mojtaba Jun 6 '18 at 12:52
  • Do you think I should give up or I can still try increasing the buffer size? Can you please help me a bit more how I can increase it? – Mojtaba Jun 6 '18 at 13:14
2

Taking a step back, Arduino is real-time and Pi is not. Instead of trying to make the Pi what it is not, rethink the challenge as "how can Pi periodically poll for information from the Arduino over serial?". It's better to buffer things more on the Arduino side than to try to make the Pi real-time. For example, the Pi can ask for last 100 samples and the Arduino can buffer and provide that with some potential loss. If temporal data loss is unacceptable, use Arduino to summarize your data at a consistent timerate and have the Pi request the summary. I used JSON, you can use binary if it helps.

Your Arduino ISR code is sending data immediately on each analog sensor read. That is bad. Arduino ISR code should JUST read the data and store it in an Arduino ring buffer. Make your ring buffer as large as needed to accommodate your data sample. Then use your main loop() for responding to serial requests from the Pi for the contents of the ring buffer. This will separate the two real-time activities cleanly. Right now your sensor ISR is fighting the serial transfer to Pi. This is an Arduino issue, not a Pi issue.

Some other points: 1) serial connections degrade with cable length. With long cables between Arduino/Pi, lower your baud rate. I had 20'cables once and had to almost send morse code. 2) If you are sending high precision data from Arduino to Pi, a differential value protocol (e.g., +1 instead of 674313, 674314) may reduce your bandwidth requirements. 3) plan for serial data errors and compute checksum for Arduino data. Pi should verify and repoll as required.

  • I dont think adding any extra codes will help. I am already on the margin and any extra comments will costs me few microseconds which is not desired. – Mojtaba Jun 6 '18 at 15:40
  • Mojtaba, i just realized you are sending serial data on your sensor ISR. This is bad. See my edit above. – OyaMist Jun 6 '18 at 17:59
  • It is not a problem of Arduino. I checked it with PC connected to Arduino and it works without any problem. The problem is clearly from the buffer size of Pi which losses data. – Mojtaba Jun 7 '18 at 8:38
0

I doubt this has anything to do with the buffer or Linux being non-real time.

I believe the serial buffer to be about 16k bytes in size. USB transfers are handled at a low level with high priority.

You are sending 15625 messages per second (1000000/64). There is no way that Python will be able to keep up.

Write your receiver in C.

  • Thank you, I was exactly thinking the same that python is slow, it will not read from the buffer fast enough and then the buffer gets full and data are lost. I am a bit slow in Linux and takes me quite some time to find a proper library for receiver in C. do you have any library or kind of ready solution? or at least were should i start from? – Mojtaba Jun 6 '18 at 10:29
  • Have a look at elinux.org/RPi_GPIO_Code_Samples – joan Jun 6 '18 at 10:38
  • Thanks, I did it and have the same problem. please read my above comment. – Mojtaba Jun 6 '18 at 12:53
  • My guess is you did it wrong. Can you share the code you used? – joan Jun 6 '18 at 13:17
  • This is the C code I compiled, I compare the readings with python as well: – Mojtaba Jun 6 '18 at 13:19
0

This is the C code I compiled, I compare the readings with python as well: #include #include #include

#include <wiringPi.h>
#include <wiringSerial.h>




int main ()
{
  int fd ;
  int val;
  char c;
  FILE *write_ptr;

  if ((fd = serialOpen ("/dev/ttyUSB0", 1000000)) < 0)
  {
    fprintf (stderr, "Unable to open serial device: %s\n", strerror (errno)) ;
    return 1 ;
  }

  if (wiringPiSetup () == -1)
  {
    fprintf (stdout, "Unable to start wiringPi: %s\n", strerror (errno)) ;
    return 1 ;
  }


  write_ptr = fopen("testc.bin","wb");  // w for write, b for binary
  while (1)
  {
    c = serialGetchar (fd);
    //fwrite((const void*) & c,sizeof(int),1,write_ptr); 
    fwrite(&c,1,1,write_ptr); 

  }


  return 0 ;
}
  • Please note that when I increase the timer of Arduino to 200us every thing is fine. I am kind of sure that the buffer is the problem not the baudrate. – Mojtaba Jun 6 '18 at 13:22
  • 1
    You should edit the code into your original question rather than adding it as an answer. I would not use those calls. Reading/writing a single byte at a time is rarely a good idea if you are after any level of performance. Use the C read/write functions. – joan Jun 6 '18 at 13:33
  • Can you please send my some links? – Mojtaba Jun 6 '18 at 14:05
  • read and write are standard C functions. There are millions of usage examples on the www. – joan Jun 6 '18 at 14:11
  • Please check my update, still no result – Mojtaba Jun 6 '18 at 15:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.