0

I increased my cluster from 16 to 20 machines. One issue which has come up is that I cannot connect more that 16 machines as NFS clients to the raspberry pi nfs server.

The nfs server machine runs the latest raspbian (stretch), with no exotic cofiguration.

I have looked through /lib/systemd/system to try find where and how the nfs daemon is started, with little success so far. The file nfs-kernel-sever.service has a shell variable $RPCNFSDARGS which I think is key, but which I cannot determine where it is assigned.

Any help to understand systemd and increase number of clients is appreciated.

PS. I want to understand a solution to this particular nfs problem, I am not looking for alternative data sharing techniques.

PPS: I have build a brand new server machine and I still have the same issue. I can't get past 15 machines

UPDATE: This issue seems to appear only in using Version 4 protocol. With version 3 there is no problem.

Improve this question
3
  • there is no concurrent connection limit in NFS
    – Jaromanda X
    Commented Jun 13, 2018 at 4:28
  • Do you get any specific error message when trying to connect the 16th raspi? You are on a cluster so I may ask if each client has its own ip address? Does the limit occurs for connections to one export or is it also if you try to connect to 16 different exports?
    – Ingo
    Commented Jun 24, 2018 at 10:42
  • "how the nfs daemon is started, with little success so far" -> The daemon is actually a set of kernel threads/processes, see man 8 nfsd (and apropos nfs). You can find them with ps -C nfsd, but other than their PPID (which will be that of kthreadd) they have no characteristics. You could try changing your configuration to use more threads (the default is 8).
    – goldilocks ♦
    Commented Jun 25, 2018 at 20:06

2 Answers 2

Reset to default
1

Do you use NFSv3 or NFSv4? The only limitation I know is that a user can only have 16 group ids for access control. In /etc/default/nfs-kernel-server I have set RPCMOUNTDOPTS="--manage-gids --no-nfs-version 3". The only way I know to understand how systemd starts nfs-kernel-server is to go step by step through the unit what it Wants and Requires Before and After starting. systemctl cat <unit-name> may help a little. For analyzing dependencies you can use one of these commands:

rpi ~$ systemctl list-dependencies
rpi ~$ systemctl list-dependencies nfs-kernel-server.service
rpi ~$ systemd-analyze dot nfs-kernel-server.service
rpi ~$ systemd-analyze plot


references:
[1] man systemd.unit
[2] systemd bootup
[3] How To Use Systemctl to Manage Systemd Services and Units

Improve this answer
1
  • It looks like the issue takes place in NFSv4 only. Version three has no problems.
    – John Smith
    Commented Jun 24, 2018 at 3:57
0

The NFS daemon is actually a kernel entity; if you look at ps -C nfsd you'll see eight instances running. If you dig a little more (e.g. watch one in top; note that to see them in htop you'll have to toggle shift-K to show kernel threads) you'll notice they have no memory stats, etc. This is because they are kernel threads.

By default there are eight, and no more will be added. I don't know how many clients each can handle. I would think more than 2, but this seems like a place to start since the NFS documentation doesn't mention any limits that I could find. Edit /etc/nfs.conf; the default version has everything commented out, scroll down and uncomment #[nfsd] then #threads=8 under that and try 16.

Re-start nfsd and check the number of threads with ps -C nfsd. If there isn't 16, try changing the rpc.nfsd command line argument (explained below); the man pages don't say which takes precedence.

how the nfs daemon is started

As per /lib/systemd/system/nfs-server.service (which you looked at), and man nfsd (which is actually a synonym for man 8 rpc.nfsd) they're started by rpc.nfsd.

a shell variable $RPCNFSDARGS

These come from the EnvironmentFile referenced in the systemd service file, /run/sysconfig/nfs-utils (beware I'm actually looking at a Fedora machine running NFS; Debian/Raspbian may use slightly different paths but you should be able to follow the same breadcrumb trail). RPCNFSDARGS is set to " 8". That's being appended to the rpc.nfsd invocation. If you look at the man page, it takes various options but a number alone/after other options stipulates the number of NFS server threads.

So you could edit this instead or as well as the conf file.

Improve this answer
3
  • Technically man nfsd means look in any section of the manual pages - for some commands they appear in more than one place and by default you only pull up the lowest sectioned entry (IIRC) - all that the extra 8 does in the man invocation is to limit the search to section 8 (which is system administrator - or for users in the wheel group if my memory is not too messed up) stuff... 8-P
    – SlySven
    Commented Jun 29, 2018 at 15:41
  • Yes. And I don't think there is more than one for either of them in the normal distribution, so the 8 is superfluous -- except to some of the more savvy of us it is a tidbit of information ;P Although if there is more than one of them, and that other one is in section 1, if you don't use 8, and you are ignorant of the issue, you will get the wrong page and wonder what the heck that fairy tale character is gibbering on about. @SlySven
    – goldilocks ♦
    Commented Jun 30, 2018 at 10:37
  • Yeah, I should have mentioned the -a option to show all the sections' entry for a particular topic...
    – SlySven
    Commented Jul 4, 2018 at 3:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.