Skip to main content
Raspberry Pi

Return to Question

edited title
Link
Longwelwind
Longwelwind
  • 83
  • 1
  • 6

Programmatically and remotely flash the SD card of Netbooting multiple "workers" RPi from a Raspberry Pi 3B+"master" RPi

There's only one master in the system
Source Link
Longwelwind
Longwelwind
  • 83
  • 1
  • 6

My question is, I believe, a little bit complex, and I think it would be better to fully describe what I have and what I want to achieve, with what I am able to do.

What I have

I have a group of RPi 3B+, with one RPi serving as a Master of the cluster and the others serving as Workers. The goal is to create a cluster of RPi manageable from the master RPi to do distributed computations. They are all connected via Ethernet through a switch. I have setup a network with a DHCP server on the Master RPi (with dnsmasq) and I am able to SSH to any Worker RPi without any problem.

I have also set up a TFTP server on the Master RPi, for reasons I will explain in the next section.

What I want to achieve

I want to be able to flash the SD cards of the Workers remotely and without having to do any physical interactions with the Workers RPi, all from the mastersmaster. I can manually flash a special system to the SD cards of the Workers once, but I'd like to then be able to update the OS of the Workers without having to move anything, as stated above.

The "plan" that I have now is the following:

  • Flash the Workers SD cards with a special system composed of 2 systems:
  • A piCore (TinyCore) system that, on boot, would launch a script that checks if a new image is available on the Master RPi, and if yes, downloads it with TFTP, flashes it and reboot.
  • A second partition that would contain a "normal" system, set up by the piCore system. Ideally, piCore could set up any kind of OS (Raspbian, an other piCore system or whatever).

What I am able to do

I am able to flash an SD card with piCore, use fdisk to create a new partition intended for the "normal" system. I can set up a script in piCore that will check if a new image is present on the Master RPi, and download it.

What I don't know how to do

How can I, once I have the image on the piCore partition, flash it in the second partition ? I only know the unix command dd that allows me to flash an entire SD card from an .img file, and I obviously don't want to flash the entire SD card of the Worker but only the special partition dedicated to the "normal" system.

How can I configure the bootloader of the SD card to make sure it always boot the piCore partition, and how can I, from the special script in the piCore partition, make sure that when I reboot, the newly flashed "normal" system is booted up and not again the piCore system.

I think I have a correct plan, at least "conceptually", to solve the problem, but I lack the technical knowledge on which tool I should use and how to configure them. I don't have a lot of experiences on this kind of low-level things, unfortunately.

I also heard of the process of "netbooting" the Worker to a filesystem located on the Master, but I'm not sure it would fit the requirements of my problem. Mainly, I don't know if it possible for multiple to netboot to the same (and more specifically, use the same remote filesystem at the same time) and if it possible to set up the Worker such that it reboots on the local filesystem on the SD card after it has checked if a new image is available on the Master RPi.

Thanks in advance!

My question is, I believe, a little bit complex, and I think it would be better to fully describe what I have and what I want to achieve, with what I am able to do.

What I have

I have a group of RPi 3B+, with one RPi serving as a Master of the cluster and the others serving as Workers. The goal is to create a cluster of RPi manageable from the master RPi to do distributed computations. They are all connected via Ethernet through a switch. I have setup a network with a DHCP server on the Master RPi (with dnsmasq) and I am able to SSH to any Worker RPi without any problem.

I have also set up a TFTP server on the Master RPi, for reasons I will explain in the next section.

What I want to achieve

I want to be able to flash the SD cards of the Workers remotely and without having to do any physical interactions with the Workers RPi, all from the masters. I can manually flash a special system to the SD cards of the Workers once, but I'd like to then be able to update the OS of the Workers without having to move anything, as stated above.

The "plan" that I have now is the following:

  • Flash the Workers SD cards with a special system composed of 2 systems:
  • A piCore (TinyCore) system that, on boot, would launch a script that checks if a new image is available on the Master RPi, and if yes, downloads it with TFTP, flashes it and reboot.
  • A second partition that would contain a "normal" system, set up by the piCore system. Ideally, piCore could set up any kind of OS (Raspbian, an other piCore system or whatever).

What I am able to do

I am able to flash an SD card with piCore, use fdisk to create a new partition intended for the "normal" system. I can set up a script in piCore that will check if a new image is present on the Master RPi, and download it.

What I don't know how to do

How can I, once I have the image on the piCore partition, flash it in the second partition ? I only know the unix command dd that allows me to flash an entire SD card from an .img file, and I obviously don't want to flash the entire SD card of the Worker but only the special partition dedicated to the "normal" system.

How can I configure the bootloader of the SD card to make sure it always boot the piCore partition, and how can I, from the special script in the piCore partition, make sure that when I reboot, the newly flashed "normal" system is booted up and not again the piCore system.

I think I have a correct plan, at least "conceptually", to solve the problem, but I lack the technical knowledge on which tool I should use and how to configure them. I don't have a lot of experiences on this kind of low-level things, unfortunately.

I also heard of the process of "netbooting" the Worker to a filesystem located on the Master, but I'm not sure it would fit the requirements of my problem. Mainly, I don't know if it possible for multiple to netboot to the same (and more specifically, use the same remote filesystem at the same time) and if it possible to set up the Worker such that it reboots on the local filesystem on the SD card after it has checked if a new image is available on the Master RPi.

Thanks in advance!

My question is, I believe, a little bit complex, and I think it would be better to fully describe what I have and what I want to achieve, with what I am able to do.

What I have

I have a group of RPi 3B+, with one RPi serving as a Master of the cluster and the others serving as Workers. The goal is to create a cluster of RPi manageable from the master RPi to do distributed computations. They are all connected via Ethernet through a switch. I have setup a network with a DHCP server on the Master RPi (with dnsmasq) and I am able to SSH to any Worker RPi without any problem.

I have also set up a TFTP server on the Master RPi, for reasons I will explain in the next section.

What I want to achieve

I want to be able to flash the SD cards of the Workers remotely and without having to do any physical interactions with the Workers RPi, all from the master. I can manually flash a special system to the SD cards of the Workers once, but I'd like to then be able to update the OS of the Workers without having to move anything, as stated above.

The "plan" that I have now is the following:

  • Flash the Workers SD cards with a special system composed of 2 systems:
  • A piCore (TinyCore) system that, on boot, would launch a script that checks if a new image is available on the Master RPi, and if yes, downloads it with TFTP, flashes it and reboot.
  • A second partition that would contain a "normal" system, set up by the piCore system. Ideally, piCore could set up any kind of OS (Raspbian, an other piCore system or whatever).

What I am able to do

I am able to flash an SD card with piCore, use fdisk to create a new partition intended for the "normal" system. I can set up a script in piCore that will check if a new image is present on the Master RPi, and download it.

What I don't know how to do

How can I, once I have the image on the piCore partition, flash it in the second partition ? I only know the unix command dd that allows me to flash an entire SD card from an .img file, and I obviously don't want to flash the entire SD card of the Worker but only the special partition dedicated to the "normal" system.

How can I configure the bootloader of the SD card to make sure it always boot the piCore partition, and how can I, from the special script in the piCore partition, make sure that when I reboot, the newly flashed "normal" system is booted up and not again the piCore system.

I think I have a correct plan, at least "conceptually", to solve the problem, but I lack the technical knowledge on which tool I should use and how to configure them. I don't have a lot of experiences on this kind of low-level things, unfortunately.

I also heard of the process of "netbooting" the Worker to a filesystem located on the Master, but I'm not sure it would fit the requirements of my problem. Mainly, I don't know if it possible for multiple to netboot to the same (and more specifically, use the same remote filesystem at the same time) and if it possible to set up the Worker such that it reboots on the local filesystem on the SD card after it has checked if a new image is available on the Master RPi.

Thanks in advance!

Tweeted twitter.com/StackRaspi/status/1021018691432042496
Source Link
Longwelwind
Longwelwind
  • 83
  • 1
  • 6

Programmatically and remotely flash the SD card of a Raspberry Pi 3B+

My question is, I believe, a little bit complex, and I think it would be better to fully describe what I have and what I want to achieve, with what I am able to do.

What I have

I have a group of RPi 3B+, with one RPi serving as a Master of the cluster and the others serving as Workers. The goal is to create a cluster of RPi manageable from the master RPi to do distributed computations. They are all connected via Ethernet through a switch. I have setup a network with a DHCP server on the Master RPi (with dnsmasq) and I am able to SSH to any Worker RPi without any problem.

I have also set up a TFTP server on the Master RPi, for reasons I will explain in the next section.

What I want to achieve

I want to be able to flash the SD cards of the Workers remotely and without having to do any physical interactions with the Workers RPi, all from the masters. I can manually flash a special system to the SD cards of the Workers once, but I'd like to then be able to update the OS of the Workers without having to move anything, as stated above.

The "plan" that I have now is the following:

  • Flash the Workers SD cards with a special system composed of 2 systems:
  • A piCore (TinyCore) system that, on boot, would launch a script that checks if a new image is available on the Master RPi, and if yes, downloads it with TFTP, flashes it and reboot.
  • A second partition that would contain a "normal" system, set up by the piCore system. Ideally, piCore could set up any kind of OS (Raspbian, an other piCore system or whatever).

What I am able to do

I am able to flash an SD card with piCore, use fdisk to create a new partition intended for the "normal" system. I can set up a script in piCore that will check if a new image is present on the Master RPi, and download it.

What I don't know how to do

How can I, once I have the image on the piCore partition, flash it in the second partition ? I only know the unix command dd that allows me to flash an entire SD card from an .img file, and I obviously don't want to flash the entire SD card of the Worker but only the special partition dedicated to the "normal" system.

How can I configure the bootloader of the SD card to make sure it always boot the piCore partition, and how can I, from the special script in the piCore partition, make sure that when I reboot, the newly flashed "normal" system is booted up and not again the piCore system.

I think I have a correct plan, at least "conceptually", to solve the problem, but I lack the technical knowledge on which tool I should use and how to configure them. I don't have a lot of experiences on this kind of low-level things, unfortunately.

I also heard of the process of "netbooting" the Worker to a filesystem located on the Master, but I'm not sure it would fit the requirements of my problem. Mainly, I don't know if it possible for multiple to netboot to the same (and more specifically, use the same remote filesystem at the same time) and if it possible to set up the Worker such that it reboots on the local filesystem on the SD card after it has checked if a new image is available on the Master RPi.

Thanks in advance!