pi-cluster

Home · Getting Started · Troubleshooting · About

Cluster Guide

Follow these steps in order, from an empty desk to 311 trained models. Each page starts with its goal and where to run it, and ends with a check that the step worked before you move on.

Step Page At the end you have Run on
1 Hardware List Everything bought and assembled, with enough power and cooling –
2 Install OS Five cards flashed with Raspberry Pi OS Lite, each node reachable over SSH Your computer
3 The Pi Network Internet access, fixed addresses, nodes reachable by name, master able to log in everywhere Your computer, master
4 Cluster Commands One command to run or copy anything on every node master
5 Cluster Security Key-only logins, fail2ban and a firewall master
6 Single Node Setup Java on every node; HDFS working on master alone master
7 Cluster Setup HDFS across four workers, with two copies of every block master
8 Spark Spark with one 4-core executor per worker, reading from HDFS master
9 Workload 311 models trained, one per worker at a time, and a benchmark master

Stuck? See Troubleshooting.

Learn more

The steps link to these when you need them. They’re also handy once the cluster is running.

Page Covers
Software Raspberry Pi OS, which edition to use, and the Java, Hadoop and Spark versions the guide pins
Maintenance Updating packages and firmware on every node; backing up, restoring and cloning cards