Arduino-Raspberry Pi fast data communication through serial

Question

I am trying to sample from two analog sensors at highest frequency possible (using and Arduino Uno, MPC3002 ADC), then send the data to raspberry pi to save them as binary file.

Arduino and MCP3002 communicates at 8MHz through SPI and it all works fine (double checked with oscilloscope). Here is my Arduino Code:

#include <SPI.h>
#include "MCP3002.h"

MCP3002 adc(10);

uint16_t ISR_BUF_FULL_COUNT = 256;
ISR(TIMER2_COMPA_vect) {
  byte buff[2];


  adc.analogRead0(buff);
  Serial.write(buff,2);
  adc.analogRead1(buff);
  Serial.write(buff,2);

}

void setup() {

  pinMode(7, OUTPUT);
  //set timer2 interrupt at 64kHz
  TCCR2A = 0;// set entire TCCR2A register to 0
  TCCR2B = 0;// same for TCCR2B
  TCNT2  = 0;//initialize counter value to 0
  // set compare match register for 15.625khz increments
  OCR2A = 127;// = (16*10^6) / (50000??*8) - 1 (must be <256)
  // turn on CTC mode
  TCCR2A |= (1 << WGM21);
  // Set CS21 bit for 8 prescaler
  TCCR2B |= (1 << CS21);
  //TCCR2B |= (1 << CS20);  
  // enable timer compare interrupt
  TIMSK2 |= (1 << OCIE2A);

  adc.begin();
  Serial.begin(1000000);

}

void loop() {

}

and Here is my python code in Pi side:

import serial
ser = serial.Serial('/dev/ttyUSB0',baudrate=1000000,timeout=None, bytesize=8, parity='N', stopbits=1)
ser.flushInput()
ser.flushOutput()
newFile = open("data.bin", "wb")
while True:
    newFile.write(ser.read(ser.inWaiting()))
    ##print ser.inWaiting()

Pi save the data correctly at the beginning and then suddenly (like after 200 samples) the data start be irrelevant values (my strong guess is that it is related to the buffer of pi) I intend to transfer 4 bytes (2 bytes for each sensor) at 1 MPS serial. I checked with oscilloscope the tx and the whole procedure takes less than 40 us.

ISR timer of Arduino is called every 64 us (checked with osilloscope) I checked the TX pin of Arduino and sending each 2 bytes will take exactly less than 40us, including reading time from MCP3002 which all looks fine.

Can any body suggest any solution why Pi behave like this? Can it be related to the buffer? Is Python on Pi fast enough to read at 1MBPS rates?

UPDATE: So here is the C code I used (using read/write functions) and It still doesn't work properly.

/*====================================================================================================*/
/* www.xanthium.in                                            */
/* Copyright (C) 2014 Rahul.S                                                                         */
/*====================================================================================================*/

/*====================================================================================================*/
/* Running the executable                                                                             */
/* ---------------------------------------------------------------------------------------------------*/ 
/* 1) Compile the  serialport_read.c  file using gcc on the terminal (without quotes)                 */
    /*                                                                                                    */
/*  " gcc -o serialport_read serialport_read.c "                                                  */
/*                                                                                                    */
    /* 2) Linux will not allow you to access the serial port from user space,you have to be root.So use   */
    /*    "sudo" command to execute the compiled binary as super user.                                    */
    /*                                                                                                    */
    /*       "sudo ./serialport_read"                                                                     */
/*                                                                                                    */
/*====================================================================================================*/

/*====================================================================================================*/
/* Sellecting the Serial port Number on Linux                                                         */
/* ---------------------------------------------------------------------------------------------------*/ 
/* /dev/ttyUSBx - when using USB to Serial Converter, where x can be 0,1,2...etc                      */
/* /dev/ttySx   - for PC hardware based Serial ports, where x can be 0,1,2...etc                      */
    /*====================================================================================================*/

/*-------------------------------------------------------------*/
    /* termios structure -  /usr/include/asm-generic/termbits.h    */ 
/* use "man termios" to get more info about  termios structure */
/*-------------------------------------------------------------*/

    #include <stdio.h>
    #include <fcntl.h>   /* File Control Definitions           */
    #include <termios.h> /* POSIX Terminal Control Definitions */
    #include <unistd.h>  /* UNIX Standard Definitions      */ 
    #include <errno.h>   /* ERROR Number Definitions           */

void main(void)
    {
        int fd;/*File Descriptor*/

    printf("\n +----------------------------------+");
    printf("\n |        Serial Port Read          |");
    printf("\n +----------------------------------+");

    /*------------------------------- Opening the Serial Port -------------------------------*/

    /* Change /dev/ttyUSB0 to the one corresponding to your system */

        fd = open("/dev/ttyUSB0",O_RDWR | O_NOCTTY);    /* ttyUSB0 is the FT232 based USB2SERIAL Converter   */
                            /* O_RDWR   - Read/Write access to serial port       */
                            /* O_NOCTTY - No terminal will control the process   */
                            /* Open in blocking mode,read will wait              */



        if(fd == -1)                        /* Error Checking */
               printf("\n  Error! in Opening ttyUSB0  ");
        else
               printf("\n  ttyUSB0 Opened Successfully ");


    /*---------- Setting the Attributes of the serial port using termios structure --------- */

    struct termios SerialPortSettings;  /* Create the structure                          */

    tcgetattr(fd, &SerialPortSettings); /* Get the current attributes of the Serial port */

    /* Setting the Baud rate */
    cfsetispeed(&SerialPortSettings,B1000000); /* Set Read  Speed as 9600                       */
    cfsetospeed(&SerialPortSettings,B1000000); /* Set Write Speed as 9600                       */

    /* 8N1 Mode */
    SerialPortSettings.c_cflag &= ~PARENB;   /* Disables the Parity Enable bit(PARENB),So No Parity   */
    SerialPortSettings.c_cflag &= ~CSTOPB;   /* CSTOPB = 2 Stop bits,here it is cleared so 1 Stop bit */
    SerialPortSettings.c_cflag &= ~CSIZE;    /* Clears the mask for setting the data size             */
    SerialPortSettings.c_cflag |=  CS8;      /* Set the data bits = 8                                 */

    SerialPortSettings.c_cflag &= ~CRTSCTS;       /* No Hardware flow Control                         */
    SerialPortSettings.c_cflag |= CREAD | CLOCAL; /* Enable receiver,Ignore Modem Control lines       */ 


    SerialPortSettings.c_iflag &= ~(IXON | IXOFF | IXANY);          /* Disable XON/XOFF flow control both i/p and o/p */
    SerialPortSettings.c_iflag &= ~(ICANON | ECHO | ECHOE | ISIG);  /* Non Cannonical mode                            */

    SerialPortSettings.c_oflag &= ~OPOST;/*No Output Processing*/

    /* Setting Time outs */
    SerialPortSettings.c_cc[VMIN] = 10; /* Read at least 10 characters */
    SerialPortSettings.c_cc[VTIME] = 0; /* Wait indefinetly   */


    if((tcsetattr(fd,TCSANOW,&SerialPortSettings)) != 0) /* Set the attributes to the termios structure*/
        printf("\n  ERROR ! in Setting attributes");
    else
                printf("\n  BaudRate = 1000000 \n  StopBits = 1 \n  Parity   = none");

        /*------------------------------- Read data from serial port -----------------------------*/

    tcflush(fd, TCIFLUSH);   /* Discards old data in the rx buffer            */

    char read_buffer[32];   /* Buffer to store the data received              */
    int  bytes_read = 0;    /* Number of bytes read by the read() system call */
    FILE *write_ptr;
    write_ptr = fopen("testc.bin","wb");  // w for write, b for binary
    while (1){
        bytes_read = read(fd,&read_buffer,32); /* Read the data                   */
        fwrite(&read_buffer,1,bytes_read,write_ptr); 
        printf("%d",bytes_read);


    }
    close(fd); /* Close the serial port */

    }

Answer 1

Taking a step back, Arduino is real-time and Pi is not. Instead of trying to make the Pi what it is not, rethink the challenge as "how can Pi periodically poll for information from the Arduino over serial?". It's better to buffer things more on the Arduino side than to try to make the Pi real-time. For example, the Pi can ask for last 100 samples and the Arduino can buffer and provide that with some potential loss. If temporal data loss is unacceptable, use Arduino to summarize your data at a consistent timerate and have the Pi request the summary. I used JSON, you can use binary if it helps.

Your Arduino ISR code is sending data immediately on each analog sensor read. That is bad. Arduino ISR code should JUST read the data and store it in an Arduino ring buffer. Make your ring buffer as large as needed to accommodate your data sample. Then use your main loop() for responding to serial requests from the Pi for the contents of the ring buffer. This will separate the two real-time activities cleanly. Right now your sensor ISR is fighting the serial transfer to Pi. This is an Arduino issue, not a Pi issue.

Some other points: 1) serial connections degrade with cable length. With long cables between Arduino/Pi, lower your baud rate. I had 20'cables once and had to almost send morse code. 2) If you are sending high precision data from Arduino to Pi, a differential value protocol (e.g., +1 instead of 674313, 674314) may reduce your bandwidth requirements. 3) plan for serial data errors and compute checksum for Arduino data. Pi should verify and repoll as required.

Answer 2

I doubt this has anything to do with the buffer or Linux being non-real time.

I believe the serial buffer to be about 16k bytes in size. USB transfers are handled at a low level with high priority.

You are sending 15625 messages per second (1000000/64). There is no way that Python will be able to keep up.

Write your receiver in C.

Answer 3

This is the C code I compiled, I compare the readings with python as well: #include #include #include

#include <wiringPi.h>
#include <wiringSerial.h>




int main ()
{
  int fd ;
  int val;
  char c;
  FILE *write_ptr;

  if ((fd = serialOpen ("/dev/ttyUSB0", 1000000)) < 0)
  {
    fprintf (stderr, "Unable to open serial device: %s\n", strerror (errno)) ;
    return 1 ;
  }

  if (wiringPiSetup () == -1)
  {
    fprintf (stdout, "Unable to start wiringPi: %s\n", strerror (errno)) ;
    return 1 ;
  }


  write_ptr = fopen("testc.bin","wb");  // w for write, b for binary
  while (1)
  {
    c = serialGetchar (fd);
    //fwrite((const void*) & c,sizeof(int),1,write_ptr); 
    fwrite(&c,1,1,write_ptr); 

  }


  return 0 ;
}