Using a RPi Zero to build a WiFi USB Stick with `modprobe g_mass_storage`

Question

I am trying to build a WiFi USB Stick using a Raspberry Pi Zero W. I want to be able to plug in the RPI to a computer or whatever device able to output data to a USB port and then the specific mounted partition of the RPI will be recognized as a normal USB drive. Then, when a file is saved on this "special" USB drive the file gets automatically uploaded to a server.

I started following the tutorial to build the pi zero w into a smart usb flash drive (link) and then try to inverse the process where an external device saves some information onto the RPi.

Until here I managed but the problem is that the files I saved on the RPi don't appear in the RPi's filesystem unless I reboot the RPi. It seems to be coming from the IO cache however, the sync function is not helping at all.

This is really problematic as I want to use something like watchdog that would trigger some actions once a new file is detected (basically, upload the new file and then delete it from the device).

In my mind, this is very close to what an Eye-Fi WiFi SD Card does but I am not sure anymore the Raspberry Pi Zero W has the hardware necessary to build such a thing.

Thanks a lot for your time,

EDIT 1:

Here are the steps I did in details to try to emulate a USB mass storage where I could then trigger specific actions when new files are save to it:

1. Use Etcher with a 64Gb SD Card to flash RASPBIAN STRETCH LITE November 2018

2. Setup the headless RPi by adding on the boot mount an empty ssh file and the needed information for the wifi connection in wpa_supplicant.conf

3. Plug the RPi with a USB cable to a Windows computer (not the PWR_IN port on the RPi)

4. Connect to the RPi using ssh and updating the system: sudo apt-get update and sudo apt-get upgrade.

5. Enable the USB driver: add dtoverlay=dwc2 to the bottom of /boot/config.txt and dwc2 the bottom of /etc/modules (In my case i2c-dev was not in /etc/modules as mentioned by the instructions.

6. Reboot the RPi with sudo reboot

7. Create a container file with sudo dd bs=1M if=/dev/zero of=/piusb.bin count=2048 and sudo mkdosfs /piusb.bin -F 32 -I

8. Mount the container file with sudo mkdir /mnt/usb_share, adding /piusb.bin /mnt/usb_share vfat users,umask=000 0 2 at the end of /etc/fstab and mounting it with sudo mount -a.

9. Use the mass storage device mode: sudo modprobe g_mass_storage file=/piusb.bin stall=0 removable=y. (In comparison to the initial instructions I removed ro=1 and added removable=y to allow Windows to write on the USB)

10. At this point I could see the USB Mass Storage detected on Windows but I couldn't write to it (and Windows displays a warning that the USB Mass Storage may have a problem), then I use the Windows dialog to "Fix" the problem.

11. At this stage I can read and write on the emulated USB mass storage from my Windows machine. The problem is that when I do ls -lha /mnt/usb_share I don't see any modification that I made on the Windows computer. Even unmounting with sudo modprobe -r g_mass_storage and then remounting it with sudo modprobe g_mass_storage file=/piusb.bin stall=0 removable=y doesn't change it. The only way to see the changes in the folder from the terminal (The RPi side) is to reboot the RPi with sudo reboot.

12. I also tried to do some sync or to flush the cache and the swap using sync; echo 3 > /proc/sys/vm/drop_caches and swapoff -a && swapon -a but it doesn't make a difference.

EDIT 2:

A quick fix I found is to unmount and remount the partition to force it to write it to the disk. Therefore, I do something like this:

sudo modprobe -r g_mass_storage; sudo umount /mnt/usb_share; sudo mount -a; sudo modprobe g_mass_storage file=/piusb.bin stall=0 removable=y

However, I would definitely like something less extreme to "refresh" the filesystem periodically and then be able to monitor the changes on it. I am open to any suggestions 😛

Answer 1

NOT POSSIBLE WITH THIS METHOD

As put by @jogco, using g_mass_storage you can't have the RPi being notified of the changes on the partition as the other device thinks it is its own and does not handle it as shared.

Therefore the only workaround found at the moment (which is not rebooting the whole system) is to periodically unmount and remount the partition as stated in the EDIT 2 which is not the optimal option.

As certain type of hardware similar to the Eye-Fi WiFi SD Card have a similar behavior as what I was trying to get it might be possible using a different method or a different hardware. I will update this answer in case I found such solution in the future.

Answer 2

g_mass_storage provides low-level (cluster/block) access to a partition, which can only be exclusive. Having the same container mounted locally and accessed via mass storage at the same time will not only miss any filesystem changes, but can actually lead to filesystem corruption.

What you should use is one of the protocols which work on filesystem level, such as MTP. That's how modern smartphones let you browse photos on your computer via USB, while still using the same storage internally. AFAIK in Linux you will need g_ffs driver and mtpd service to get this working.

Answer 3

While this should only be used with a readonly view from the raspberry pi, based on these questions for dropping the filesystem cache you can "reset" the listings without remounting the drive. This also keeps everything working on the remote side as well. This isn't super reliable since you have to wait for the host to flush the data (sync on linux), the command is

echo 3 | sudo tee /proc/sys/vm/drop_caches

Mounting the drive with the "sync" option or calling sync from the raspberry pi probably won't hurt, but doesn't seem to help either.

Answer 4

Depending on how often you expect changes, you could setup Watchdog to monitor the /piusb.bin and initiate a bash script to (re-) mount it only, when changes have occurred.

The modified date changes, when you write something to it:

pi@raspberrypi:~ $ stat /piusb.bin 
  File: /piusb.bin
  Size: 4294967296  Blocks: 8388616    IO Block: 4096   regular file
Device: b302h/45826d    Inode: 24135       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-11-02 21:11:48.831225005 +0000
Modify: 2021-11-03 08:23:56.869005088 +0000
Change: 2021-11-03 08:23:56.869005088 +0000
 Birth: -

I typically have the image unmounted. When I scan, I have the scanner write the scanned file to the piusb.bin as above. The modified date changes, Watchdog notices it and the bashscript mounts the image and copies the files to a remote location and then umount it again.

#!/bin/sh

sudo mount -a
sleep 1
if [ -z "$(ls -A /mnt/usb_share/SCANFILE)" ];
then
  echo "Folder is empty"
else
  rsync /mnt/usb_share/SCANFILE/* [email protected]:/volume1/Scanner/
  rm /mnt/usb_share/SCANFILE/*
fi
sudo umount -f -l /mnt/usb_share