1

I'm looking at portability of some code that others want to use on the Raspberry Pi using gcc (Raspbian 6.3.0-18+rpi1+deb9u1) 6.3.0 20170516 ("Raspbian GNU/Linux 9 (stretch)").

In particular I'm looking at addressing files that are >4Gb, so a pure 32bit data model fails.

It's not clear for Raspbian if it is a 32bit or 64bit or mixed data model? E.g see http://nickdesaulniers.github.io/blog/2016/05/30/data-models-and-word-size/.

The code should work on Windows and on the various Linuxes so there can be confusion as to the various implementation defined compatibility requirements.

What is the data model inuse?

0

Sounds like someone is going for a hail mary pass ;) You are probably in luck though.

#include <stdio.h>

int main (int argc, const char *argv[]) 
{
        printf("%d\n", sizeof(char*));
        return 0;
}

Compiled and run on current Raspbian, the output is: 4. Using long gives the same thing -- but keep reading.

What is the data model inuse?

The basic math model: 32 / 8 = 4 just as 4 * 8 = 32. To be fair it's what that article refers to as ILP32 but wrongly states it is used on linux x86-64 -- it isn't, but then Raspbian isn't linux x86-64. It's linux armhf.

In particular I'm looking at addressing files that are >4Gb, so a pure 32bit data model fails.

In that context, this is only really a problem if you intend to load the entire file into RAM, which you can't do anyway. A pointer points to memory addresses, not file locations. There is a correspondence with regard to binary objects loaded as code, but I am sure this is not what you are up to. You can index a file by byte number, and while there are more than 4 billion bytes in a file > 4GB in size, you don't have to use a pointer to store that number -- you can use, eg., a uint64_t, which holds values from 0 to 18446744073709551615. Don't forget:

#include <inttypes.h>
// Or for C++ tidiness <cinttypes>. 

Using that stuff is more portable (it's part of the C99 standard) than implementation dependent things such as the size of longs and pointers. There is no ambiguity about what uint64_t refers to.

  • It's more that parts of the application may want to stream blocks of a large external file in smaller chunks so the external file offset is >4Gb, even thought the internal black us small <2Gb. At the moment size_t is the go to big pointer but there is so much inconsistency... – Philip Oakley Sep 12 '19 at 12:59
  • That's an XY problem; nothing you describe sounds particularly complicated (the "Y" part of the problem is your fixation on using a pointer). You use fixed chunk sizes and index them plus use an offset, or you use variable chunk sizes and keep track of their order and size, plus use an offset. But this is all stuff more appropriate to Stack Overflow proper. Good luck! – goldilocks Sep 12 '19 at 13:25
  • Hi @goldilocks. The use of the (pointer,length) method is part of the mandated library, so it's more an XYZ problem - between a rock and a hard place. Part of the XY problem is a curved ball that threw in the Pi as a possible compilation target, leaving me scrambling to get a handle on it's data model. Undefined behaviour is easy compared to implementation defined portability ! – Philip Oakley Sep 12 '19 at 19:54
0

If you want to read long files, what you need is the size of the type used for file offsets, off_t.

Every somewhat recent (at least 10 years) Linux system supports 64 Bit off_t, so you should have no problem with large files.

You may need -D_FILE_OFFSET_BITS=64 or #define _FILE_OFFSET_BITS 64 to force 64 Bit file offsets. This works on Linux and Windows.

  • 1
    WRT off_t: 1) May not be portable to Windows as it is POSIX, not standard C. 2) Does not have a standard specified size and so may differ from platform to platform. See also: stackoverflow.com/a/9073762/1151724 – goldilocks Sep 10 '19 at 19:56
  • 1
    ...And whoops, in fact on Raspbian, sizeof(off_t) is 32 bits... – goldilocks Sep 10 '19 at 20:05
  • @goldilocks The question is not about standard C, it is about Windows and Linux. I added the necessary define to force 64 Bit off_t on Windows and Linux. – RalfFriedl Sep 10 '19 at 20:36
  • @RalfFriedl Part of my issue is that the application code should be portable to other compilers as well, so it gets fun for MSVC etc. What is size_t on Rasp. I understood it to be 64bit capable (think zlib streams) – Philip Oakley Sep 12 '19 at 12:56
  • There is a difference between 64 Bit for file size and 64 Bit for address space. off_t is for file size, size_t is for address space. – RalfFriedl Sep 12 '19 at 17:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.