Skip to main content
added 368 characters in body
Source Link

Guys I'm having a problem when I write/save many files in Raspberry Pi.

I'm using the RaspiCam to record videos in HD and at the same time I'm collecting alot of data and writing them in XML files every 1 second. The data is collected at 10 Hz.

I don't know why, but sometimes when I'm saving the XML file, it takes 3-4 seconds to save the file, that's too much. It happens once within 30-40 seconds.

It occurs very often and I'm losing data because of that. I tried to debug and I noticed that when it occurs is where I call the function fopen.

Does somebody know, what is happening?

I'm using the tinyxml2 library to write and save the XML files.

"Edited"

Here is a log file that I did to see what times it's freezing.

[INFO]: [OBD Buffer]: 0 [GPS Buffer]: 26    [IMU Buffer]: 3  [XML Time]: 0.988022(s)

[INFO]: [OBD Buffer]: 2 [GPS Buffer]: 45    [IMU Buffer]: 22 [XML Time]: 2.89335(s)

As you can see, the XML files were saving normally, but suddenly it took 3 seconds to save. Because of that, as my other threads are still running and collecting data, my OBD/GPS/IMU buffers gets full and I start to lose data because the XML is freezing when saving.

Solution: I had to increase the time of saving the XML's files. RaspbianOS isn't a good choice for perfomance in real time. I'm saving the XML's files every minute now and I'm not losing information anymore. So, to sum up I cannot save the XML's files every second. I think the nsilent22' answer can describe it better.

Thank you all for your support.

Guys I'm having a problem when I write/save many files in Raspberry Pi.

I'm using the RaspiCam to record videos in HD and at the same time I'm collecting alot of data and writing them in XML files every 1 second. The data is collected at 10 Hz.

I don't know why, but sometimes when I'm saving the XML file, it takes 3-4 seconds to save the file, that's too much. It happens once within 30-40 seconds.

It occurs very often and I'm losing data because of that. I tried to debug and I noticed that when it occurs is where I call the function fopen.

Does somebody know, what is happening?

I'm using the tinyxml2 library to write and save the XML files.

"Edited"

Here is a log file that I did to see what times it's freezing.

[INFO]: [OBD Buffer]: 0 [GPS Buffer]: 26    [IMU Buffer]: 3  [XML Time]: 0.988022(s)

[INFO]: [OBD Buffer]: 2 [GPS Buffer]: 45    [IMU Buffer]: 22 [XML Time]: 2.89335(s)

As you can see, the XML files were saving normally, but suddenly it took 3 seconds to save. Because of that, as my other threads are still running and collecting data, my OBD/GPS/IMU buffers gets full and I start to lose data because the XML is freezing when saving.

Guys I'm having a problem when I write/save many files in Raspberry Pi.

I'm using the RaspiCam to record videos in HD and at the same time I'm collecting alot of data and writing them in XML files every 1 second. The data is collected at 10 Hz.

I don't know why, but sometimes when I'm saving the XML file, it takes 3-4 seconds to save the file, that's too much. It happens once within 30-40 seconds.

It occurs very often and I'm losing data because of that. I tried to debug and I noticed that when it occurs is where I call the function fopen.

Does somebody know, what is happening?

I'm using the tinyxml2 library to write and save the XML files.

"Edited"

Here is a log file that I did to see what times it's freezing.

[INFO]: [OBD Buffer]: 0 [GPS Buffer]: 26    [IMU Buffer]: 3  [XML Time]: 0.988022(s)

[INFO]: [OBD Buffer]: 2 [GPS Buffer]: 45    [IMU Buffer]: 22 [XML Time]: 2.89335(s)

As you can see, the XML files were saving normally, but suddenly it took 3 seconds to save. Because of that, as my other threads are still running and collecting data, my OBD/GPS/IMU buffers gets full and I start to lose data because the XML is freezing when saving.

Solution: I had to increase the time of saving the XML's files. RaspbianOS isn't a good choice for perfomance in real time. I'm saving the XML's files every minute now and I'm not losing information anymore. So, to sum up I cannot save the XML's files every second. I think the nsilent22' answer can describe it better.

Thank you all for your support.

grammar
Source Link
Jacobm001
  • 11.9k
  • 7
  • 47
  • 57

Guys I'm having a problem when I write/save many files in Raspberry Pi.

I'm using the RaspiCam to record videos in HD and at the same time I'm collecting alot of data and writing them in XML files every 1 second. The data areis collected at 10 Hz.

I don't know why, but sometimes when I'm saving the XML file, it takes 3-4 seconds to save the file, that's too much. It happens once within 30-40 seconds.

It occurs very often and I'm losing data because of that. I tried to debug and I noticed that when it occurs is where I call the function fopen.

Does somebody know, what is happening?

I'm using the tinyxml2 library to write and save the XML files.

"Edited"

Here is a log file that I did to see what times it's freezing.

[INFO]: [OBD Buffer]: 0 [GPS Buffer]: 26    [IMU Buffer]: 3  [XML Time]: 0.988022(s)

[INFO]: [OBD Buffer]: 2 [GPS Buffer]: 45    [IMU Buffer]: 22 [XML Time]: 2.89335(s)

As you can see, the XML files waswere saving normally, but suddenly heit took 3 seconds to save. And becauseBecause of that, as my other threads are still running and collecting data, after some time my OBD/GPS/IMU buffers gets full and I start to lose data because the XML is freezing when saving.

Guys I'm having a problem when I write/save many files in Raspberry Pi.

I'm using the RaspiCam to record videos in HD and at the same time I'm collecting alot of data and writing them in XML files every 1 second. The data are collected at 10 Hz.

I don't know why, but sometimes when I'm saving the XML file, it takes 3-4 seconds to save the file, that's too much. It happens once within 30-40 seconds.

It occurs very often and I'm losing data because of that. I tried to debug and I noticed that when it occurs is where I call the function fopen.

Does somebody know, what is happening?

I'm using the tinyxml2 library to write and save the XML files.

"Edited"

Here is a log file that I did to see what times it's freezing.

[INFO]: [OBD Buffer]: 0 [GPS Buffer]: 26    [IMU Buffer]: 3  [XML Time]: 0.988022(s)

[INFO]: [OBD Buffer]: 2 [GPS Buffer]: 45    [IMU Buffer]: 22 [XML Time]: 2.89335(s)

As you can see, the XML files was saving normally, but suddenly he took 3 seconds to save. And because of that, as my other threads still running and collecting data, after some time my OBD/GPS/IMU buffers gets full and I start to lose data because the XML is freezing when saving.

Guys I'm having a problem when I write/save many files in Raspberry Pi.

I'm using the RaspiCam to record videos in HD and at the same time I'm collecting alot of data and writing them in XML files every 1 second. The data is collected at 10 Hz.

I don't know why, but sometimes when I'm saving the XML file, it takes 3-4 seconds to save the file, that's too much. It happens once within 30-40 seconds.

It occurs very often and I'm losing data because of that. I tried to debug and I noticed that when it occurs is where I call the function fopen.

Does somebody know, what is happening?

I'm using the tinyxml2 library to write and save the XML files.

"Edited"

Here is a log file that I did to see what times it's freezing.

[INFO]: [OBD Buffer]: 0 [GPS Buffer]: 26    [IMU Buffer]: 3  [XML Time]: 0.988022(s)

[INFO]: [OBD Buffer]: 2 [GPS Buffer]: 45    [IMU Buffer]: 22 [XML Time]: 2.89335(s)

As you can see, the XML files were saving normally, but suddenly it took 3 seconds to save. Because of that, as my other threads are still running and collecting data, my OBD/GPS/IMU buffers gets full and I start to lose data because the XML is freezing when saving.

deleted 5 characters in body
Source Link

Guys I'm having a problem when I write/save many files in Raspberry Pi.

I'm using the RaspiCam to record videos in HD and at the same time I'm collecting alot of data and writing them in XML files every 1 second. The data are collected at 10 Hz.

I don't know why, but sometimes when I'm saving the XML file, it takes 3-4 seconds to save the file, that's too much. It happens once within 30-40 seconds.

It occurs very often and I'm losing data because of that. I tried to debug and I noticed that when it occurs is where I call the function fopen.

Does somebody know, what is happening?

I'm using the tinyxml2 library to write and save the XML files.

"Edited"

Here is a log file that I did to see what times it's freezing.

**[INFO]**: [OBD Buffer]: 0   [GPS Buffer]: 26    [IMU Buffer]: 3 [XML Time]: 0.988022(s)

**[INFO]:** [OBD Buffer]: 2   [GPS Buffer]: 45    [IMU Buffer]: 22    [XML Time]: 2.89335(s)

As you can see, the XML files was saving normally, but suddenly he took 3 seconds to save. And because of that, as my other threads still running and collecting data, after some time my buffer gets full and I start to lose data because the XML is freezing.

Here is a log file that I did to see what times it's freezing.

[INFO]: [OBD Buffer]: 0 [GPS Buffer]: 26    [IMU Buffer]: 3  [XML Time]: 0.988022(s)

[INFO]: [OBD Buffer]: 2 [GPS Buffer]: 45    [IMU Buffer]: 22 [XML Time]: 2.89335(s)

As you can see, the XML files was saving normally, but suddenly he took 3 seconds to save. And because of that, as my other threads still running and collecting data, after some time my OBD/GPS/IMU buffers gets full and I start to lose data because the XML is freezing when saving.

Guys I'm having a problem when I write/save many files in Raspberry Pi.

I'm using the RaspiCam to record videos in HD and at the same time I'm collecting alot of data and writing them in XML files every 1 second. The data are collected at 10 Hz.

I don't know why, but sometimes when I'm saving the XML file, it takes 3-4 seconds to save the file, that's too much. It happens once within 30-40 seconds.

It occurs very often and I'm losing data because of that. I tried to debug and I noticed that when it occurs is where I call the function fopen.

Does somebody know, what is happening?

I'm using the tinyxml2 library to write and save the XML files.

"Edited"

Here is a log file that I did to see what times it's freezing.

**[INFO]**: [OBD Buffer]: 0   [GPS Buffer]: 26    [IMU Buffer]: 3 [XML Time]: 0.988022(s)

**[INFO]:** [OBD Buffer]: 2   [GPS Buffer]: 45    [IMU Buffer]: 22    [XML Time]: 2.89335(s)

As you can see, the XML files was saving normally, but suddenly he took 3 seconds to save. And because of that, as my other threads still running and collecting data, after some time my buffer gets full and I start to lose data because the XML is freezing.

Guys I'm having a problem when I write/save many files in Raspberry Pi.

I'm using the RaspiCam to record videos in HD and at the same time I'm collecting alot of data and writing them in XML files every 1 second. The data are collected at 10 Hz.

I don't know why, but sometimes when I'm saving the XML file, it takes 3-4 seconds to save the file, that's too much. It happens once within 30-40 seconds.

It occurs very often and I'm losing data because of that. I tried to debug and I noticed that when it occurs is where I call the function fopen.

Does somebody know, what is happening?

I'm using the tinyxml2 library to write and save the XML files.

"Edited"

Here is a log file that I did to see what times it's freezing.

[INFO]: [OBD Buffer]: 0 [GPS Buffer]: 26    [IMU Buffer]: 3  [XML Time]: 0.988022(s)

[INFO]: [OBD Buffer]: 2 [GPS Buffer]: 45    [IMU Buffer]: 22 [XML Time]: 2.89335(s)

As you can see, the XML files was saving normally, but suddenly he took 3 seconds to save. And because of that, as my other threads still running and collecting data, after some time my OBD/GPS/IMU buffers gets full and I start to lose data because the XML is freezing when saving.

added 554 characters in body
Source Link
Loading
Tweeted twitter.com/StackRaspi/status/689429495418191872
removed tag raspberry-pi-foundation
Link
Loading
Source Link
Loading