I had a need to run a utility that requires Linux (specifically, iprutils to configure a RAID card) onto a Power S814 that I had installed at a datacentre, and no remote hands to boot a CD with. However, there’s not much information documented on how to boot Linux on these systems from a network. This article aims to synthesize what I’ve found online, since information about IBM Power systems tends to be scant outside of IBM documentation..
Assumptions
I’m using an x86 Ubuntu 22.04 system to provide the server side, because it was what was provisioned for me already. While you could use any system in theory, the steps to use GRUB for a different architecture are annoyingly Debian/Ubuntu-specific due to issues with their packaging. If you could use a PowerPC system to netboot from, that would likely save a ton of the effort I had to go through. If you have suggestions on how to make that step easier, please do share.
I’m also going to be booting Alpine 3.19, mostly because I just need to run a utility from RAM, and Alpine makes netbooting resources available easily. Not all distros do, unfortunately. Alpine presents some challenges with things like modloop. If you’re netbooting something like i.e. Debian’s netboot kernel and initrd pair, then you don’t need things like modloop and can skip things involving that. If you’re knowledgeable enough with your distro of choice, you should know what to omit.
The server I’m using is a Power S814, which can boot 64-bit little endian OSes. If you’re running older hardware, you’re going to have to use a different distro, and possibly run into firmware bugs with large initrd files. It’s also a PowerVM system that boots via Open Firmware; this guide likely doesn’t apply to PowerNV systems (i.e. the Talos II) or Power Macs.
Setting up TFTP
Pretty much every old netboot mechanism involves TFTP, since it’s very simple. Let’s log into the netboot server and run this command to install a TFTP daemon. You could use a different one, but HPA’s works fine in my experience.
sudo apt-get install tftpd-hpa
You’ll want to verify the directory that it uses. The config file is in /etc/default/tftpd-hpa. At least on Ubuntu, it uses /srv/tftp as the root for TFTP by default. That’s fine, and it’s what I’ll use in this article. Mentally replace all instances of this path if yours will differ.
Building a PowerPC GRUB image
We need a bootloader to load Linux. Other platforms have pxelinux or petitboot, but Open Firmware on PPC isn’t as lucky. However, GRUB is a swiss army knife of booting, and it can slurp up kernels over TFTP and boot on Open Firmware. So, let’s just use GRUB.
But, we have an x86 server. Cross-architecture netbooting isn’t hard; surely, we can just install the grub-ieee1275-bin package, and get the Open Firmware version of GRUB, right? Unfortunately, things aren’t that easy in the Debian/Ubuntu world. The package on x86 doesn’t contain a PowerPC version, but rather an x86 Open Firmware version of GRUB. (The OLPC still haunts in 2024.)
If you are doing this from a PowerPC system, congratulations! Just skip ahead to actually installing the modules.
It’s possible there’s a better way (i.e. maybe building from source is easier?), or that other distros package GRUB in a way that makes this hassle-free. I’ve discounted using a debootstrap chroot as complex and unnecessary, at least.
Setting up multiarch
Thankfully, we don’t need to run any code to set up i.e. an emulator, just install the package so we have the files. Even though there’s no code for Linux to run, the package is marked as for PowerPC only, so we need to use packages across multiple architectures. Let’s set up multiarch for 64-bit little endian PowerPC, which Debian and Ubuntu both have:
sudo dpkg --add-architecture ppc64el
This isn’t needed on Debian, so skip these steps if you’re using that. If you’re on Ubuntu, their mirror layout is a little silly, where PowerPC packages are elsewhere, and doing an update now would confuse apt. Let’s tell it the repos in /etc/apt/sources.list are x86 only by adding [arch=amd64] (or [arch=amd64,i386] if you have 32-bit too) after deb on each line beginning with deb. Then add a file called /etc/apt/sources.list.d/ppc.list (or something else, just a .list file in that directory), with contents like (changing jammy for a different release if needed):
deb [arch=ppc64el] http://ports.ubuntu.com/ jammy main restricted deb [arch=ppc64el] http://ports.ubuntu.com/ jammy-updates main restricted deb [arch=ppc64el] http://ports.ubuntu.com/ jammy universe deb [arch=ppc64el] http://ports.ubuntu.com/ jammy-updates universe deb [arch=ppc64el] http://ports.ubuntu.com/ jammy multiverse deb [arch=ppc64el] http://ports.ubuntu.com/ jammy-updates multiverse deb [arch=ppc64el] http://ports.ubuntu.com/ jammy-backports main restricted universe multiverse
Then we can finally update the repository:
sudo apt-get update
Installing the GRUB OF modules
We can now install the PowerPC modules. (If you’re on a PowerPC system, just drop the :ppc64el part, since we don’t need to specify architecture.) If you tried installing the x86 Open Firmware modules, it’ll uninstall those since they conflict.
sudo apt-get install grub-ieee1275-bin:ppc64el
Actually building the image
Finally, we have the modules. Let’s make a GRUB executable that bundles a bunch of modules we need together.
grub-mkimage --output=core.elf --format=powerpc-ieee1275 boot configfile echo elf http ieee1275_fb linux loadenv ls net normal ofnet reboot regexp serial sleep tftp time true date gzio xzio zstd -p /
This makes a single executable containing the listed modules. In this case, we need some network and compression modules. The -p flag tells GRUB the prefix to look for things in. In this case, that puts it in the TFTP root, so it’ll look for files like grub.cfg later there.
Copy it into into the TFTP root now.
cp core.elf /srv/tftp/
Setting up BOOTP
While not necessary, it helps the BOOTP server not complain if you have the hostname you’ll use for it in DNS. If it’s not there, just add it to the hosts file, like so (obviously, change based on IP and name):
echo "10.1.2.59 vios814" >> /etc/hosts
Now, we need to make a bootptab entry. The syntax is ugly; it’s a set of colon separated two-letter tags. The breakdown of those is as follows:
vios814:– the hostname matching in /etc/hostsbf=– the file in in tftp to downloadip=– the netbooting system’s IP addresssm=– the netmaskgw=– default gatewayns=– default nameserversa=– this server’s IP address
Let’s append such a line:
echo "vios814:bf=core.elf:ip=10.1.2.59:sm=255.255.0.0:gw=10.1.0.1:ns=10.1.0.1:sa=10.1.2.60" >> /etc/bootptab
Now, we need to start bootpd. (If you want extra credit on your book report, it’d be a good idea to make i.e. a systemd unit for it. For some reason, none is provided. We’ll just run it manually for now, killing the old one if it’s still there. We’ll do it from a root shell too, since inherited file descriptors is always confusing with sudo.)
sudo -i pkill bootpd /usr/sbin/bootpd -d4 /etc/bootptab 2> /var/log/bootpd.log & exit
Setting up the Alpine image
Alpine provides a netboot tarball on the downloads page, but doesn’t provide much guidance on to use them except for on x86 PXE environments. However, we can easily piece together what’s needed, especially since many parts are the same.
Download the file, then extract the tarball. Copy the kernel and ramdisk to the TFTP root.
wget https://dl-cdn.alpinelinux.org/alpine/v3.19/releases/ppc64le/alpine-netboot-3.19.0-ppc64le.tar.gz tar xvf alpine-netboot-3.19.0-ppc64le.tar.gz cp boot/vmlinuz-lts boot/initramfs-lts /srv/tftp/
Modloop over HTTP
The modloop file contains things that get loaded after the kernel and initrd. While in theory you don’t need the modloop file, you really do. It contains drivers and firmware for devices you actually want on a Power system, like the ethernet and SAS controllers.
Alpine can fetch the modloop over HTTP or FTP (source). Annoyingly, it doesn’t seem to do TFTP, which means we have to set up a web server now too. Pick your poison; I’m used to Apache like so:
sudo apt-get install apache24
Then you can just copy the modloop to the document root, and we’ll just use that later.
sudo cp boot/modloop-lts /var/www/html
Setting up a GRUB config
Make a grub.cfg in /srv/tftp, adjacent to the GRUB file, with contents like (obviously, adapt the modloop URL and ip= for your own network):
menuentry 'Alpine' --class os --class linux {
linux (tftp)/vmlinuz-lts modules=loop,squashfs alpine_repo=https://dl-cdn.alpinelinux.org/alpine/v3.19/main modloop=http://10.1.2.60/modloop-lts ip=:::::eth11:dhcp
initrd (tftp)/initramfs-lts
}
Most of these are typical options for booting Alpine over the network (i.e. passing the repo and modloop parameters), but note the strange ip= instance. Usually, most set this to just ip=dhcp. However, Power systems tend to have a ton of NICs in them, so you might need to specify which device to use DHCP from. Unfortunately, you might need to just try booting and see what interfaces are available, and what it tries by default and what it enumerates as. For me, it tried the ethernet port in P1-C10-T1 by default, which wasn’t plugged in. (It makes me wish Alpine had systemd’s predictable device names.)
Note that while it’s tempting to use a static IP in ip=, it won’t cover DNS, and that breaks the initial loading of packages from the repo. Unless you want to get into mirroring them, but that’s another set of pain.
Making Open Firmware use BOOTP
We’re finally done on the netboot server; now we can finally configure the Power system to boot from it. Attach to a terminal for the LPAR you want to configure. Break into SMS by pressing 1 when the system is booting; it might do so if it can’t find anything to boot from. You should see this screen:
PowerPC Firmware
Version FW860.12 (SV860_070)
SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved.
-------------------------------------------------------------------------------
Main Menu
1. Select Language
2. Setup Remote IPL (Initial Program Load)
3. I/O Device Information
4. Select Console
5. Select Boot Options
-------------------------------------------------------------------------------
Navigation Keys:
X = eXit System Management Services
-------------------------------------------------------------------------------
Type menu item number and press Enter or select Navigation key:
Select option 2 to configure netbooting.
PowerPC Firmware
Version FW860.12 (SV860_070)
SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved.
-------------------------------------------------------------------------------
NIC Adapters
Device Location Code Hardware
Address
1. PCIe2 4-port 1GbE Adapter U78C9.001.WZS0ZZ4-P1-C10-T1 98be9407a438
2. PCIe2 4-port 1GbE Adapter U78C9.001.WZS0ZZ4-P1-C10-T2 98be9407a439
3. PCIe2 4-port 1GbE Adapter U78C9.001.WZS0ZZ4-P1-C10-T3 98be9407a43a
4. PCIe2 4-port 1GbE Adapter U78C9.001.WZS0ZZ4-P1-C10-T4 98be9407a43b
-------------------------------------------------------------------------------
Navigation keys:
M = return to Main Menu
ESC key = return to previous screen X = eXit System Management Services
-------------------------------------------------------------------------------
Type menu item number and press Enter or select Navigation key:
Select the interface and port that’s plugged into the same network. In this case, I need #4, or P1-C10-T4. (Look at the IBM documentation for what ports and slots are mapped where on your system, it helps a lot when figuring out what network interface you’re talking about.)
PowerPC Firmware Version FW860.12 (SV860_070) SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved. ------------------------------------------------------------------------------- Select Internet Protocol Version. 1. IPv4 - Address Format 123.231.111.222 2. IPv6 - Address Format 1234:5678:90ab:cdef:1234:5678:90ab:cdef ------------------------------------------------------------------------------- Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key:
`
We’re going to be using IPv4. Select BOOTP if it gives you a choice between BOOTP and iSCSI.
PowerPC Firmware Version FW860.12 (SV860_070) SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved. ------------------------------------------------------------------------------- Network Parameters PCIe2 4-port 1GbE Adapter: U78C9.001.WZS0ZZ4-P1-C10-T4 1. IP Parameters 2. Adapter Configuration 3. Ping Test 4. Advanced Setup: BOOTP ------------------------------------------------------------------------------- Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key:
You need to change the IP parameters here.
PowerPC Firmware Version FW860.12 (SV860_070) SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved. ------------------------------------------------------------------------------- IP Parameters PCIe2 4-port 1GbE Adapter: U78C9.001.WZS0ZZ4-P1-C10-T4 1. Client IP Address [10.1.2.59] 2. Server IP Address [10.1.2.60] 3. Gateway IP Address [10.1.0.1] 4. Subnet Mask [255.255.0.0] ------------------------------------------------------------------------------- Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key:
Change these to match your network. When done, hit escape to go back to the previous screen and run the ping test. Then you can press M to go back to the main menu.
Booting from the network with Open Firmware
Back at the SMS main menu, hit option 5 for selecting boot options. You’ll be dropped into this screen:
PowerPC Firmware Version FW860.12 (SV860_070) SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved. ------------------------------------------------------------------------------- Multiboot 1. Select Install/Boot Device 2. Configure Boot Device Order 3. Multiboot Startup <OFF> 4. SAN Zoning Support ------------------------------------------------------------------------------- Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key:
Select install/boot device.
PowerPC Firmware Version FW860.12 (SV860_070) SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved. ------------------------------------------------------------------------------- Select Device Type 1. Tape 2. CD/DVD 3. Hard Drive 4. Network 5. List all Devices ------------------------------------------------------------------------------- Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key:
You want a network device. Select BOOTP if it asks you to.
PowerPC Firmware
Version FW860.12 (SV860_070)
SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved.
-------------------------------------------------------------------------------
Select Device
Device Current Device
Number Position Name
1. 2 PCIe2 4-port 1GbE Adapter
( loc=U78C9.001.WZS0ZZ4-P1-C10-T1 )
2. - PCIe2 4-port 1GbE Adapter
( loc=U78C9.001.WZS0ZZ4-P1-C10-T2 )
3. - PCIe2 4-port 1GbE Adapter
( loc=U78C9.001.WZS0ZZ4-P1-C10-T3 )
4. - PCIe2 4-port 1GbE Adapter
( loc=U78C9.001.WZS0ZZ4-P1-C10-T4 )
-------------------------------------------------------------------------------
Navigation keys:
M = return to Main Menu
ESC key = return to previous screen X = eXit System Management Services
-------------------------------------------------------------------------------
Type menu item number and press Enter or select Navigation key:
Pick the network adapter you previously configured. In my case, it’s #4, or P1-C10-T4.
PowerPC Firmware
Version FW860.12 (SV860_070)
SMS (c) Copyright IBM Corp. 2000,2016 All rights reserved.
-------------------------------------------------------------------------------
Select Task
PCIe2 4-port 1GbE Adapter
( loc=U78C9.001.WZS0ZZ4-P1-C10-T4 )
1. Information
2. Normal Mode Boot
3. Service Mode Boot
-------------------------------------------------------------------------------
Navigation keys:
M = return to Main Menu
ESC key = return to previous screen X = eXit System Management Services
-------------------------------------------------------------------------------
Type menu item number and press Enter or select Navigation key:
Select a normal mode boot here. Confirm that you want to boot.
After it slurps the files over TFTP, it should boot GRUB. If it fails, it’ll likely throw a BA017021 SRC code if it couldn’t find the file, or a BA01B015 if it timed out trying to use BOOTP.
Finishing up
It should come up into Alpine, load the modloop over the network, and download APKs at the provided repository. It should look like this, and you’ll know something went wrong if it i.e. gets stuck at DHCP, can’t download the APKs, etc. In my case, I was able to add the community repository and install iprutils.
Mounting boot media: ok.
ok.
Obtaining IP via DHCP (eth11)...
* Obtaining IP via DHCP (eth11): udhcpc: started, v1.36.1
udhcpc: broadcasting discover
udhcpc: broadcasting discover
tg3 001e:80:00.3 eth11: Link is up at 1000 Mbps, full duplex
tg3 001e:80:00.3 eth11: Flow control is off for TX and off for RX
tg3 001e:80:00.3 eth11: EEE is disabled
udhcpc: broadcasting discover
udhcpc: broadcasting discover
udhcpc: broadcasting select for 10.1.1.195, server 10.1.0.1
udhcpc: lease of 10.1.1.195 obtained from 10.1.0.1, lease time 7200
Obtaining IP via DHCP (eth11): ok.
ok.
Installing packages to root filesystem...
* Installing packages to root filesystem: fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/ppc64le/APKINDEX.tar.gz
NET: Registered PF_INET6 protocol family
Segment Routing with IPv6
In-situ OAM (IOAM) with IPv6
(1/26) Installing alpine-baselayout-data (3.4.3-r2)
(2/26) Installing musl (1.2.4_git20230717-r4)
(3/26) Installing busybox (1.36.1-r15)
Executing busybox-1.36.1-r15.post-install
(4/26) Installing busybox-binsh (1.36.1-r15)
(5/26) Installing alpine-baselayout (3.4.3-r2)
Executing alpine-baselayout-3.4.3-r2.pre-install
Executing alpine-baselayout-3.4.3-r2.post-install
(6/26) Installing ifupdown-ng (0.12.1-r4)
(7/26) Installing libcap2 (2.69-r1)
(8/26) Installing openrc (0.52.1-r2)
Executing openrc-0.52.1-r2.post-install
(9/26) Installing mdev-conf (4.6-r0)
(10/26) Installing busybox-mdev-openrc (1.36.1-r15)
(11/26) Installing alpine-conf (3.17.0-r0)
(12/26) Installing alpine-keys (2.4-r1)
(13/26) Installing alpine-release (3.19.0-r0)
(14/26) Installing ca-certificates-bundle (20230506-r0)
(15/26) Installing libcrypto3 (3.1.4-r4)
(16/26) Installing libssl3 (3.1.4-r4)
(17/26) Installing ssl_client (1.36.1-r15)
(18/26) Installing zlib (1.3-r2)
(19/26) Installing apk-tools (2.14.0-r5)
(20/26) Installing busybox-openrc (1.36.1-r15)
(21/26) Installing busybox-suid (1.36.1-r15)
(22/26) Installing scanelf (1.3.7-r2)
(23/26) Installing musl-utils (1.2.4_git20230717-r4)
(24/26) Installing libc-utils (0.7.2-r5)
(25/26) Installing alpine-base (3.19.0-r0)
(26/26) Installing openssl (3.1.4-r4)
Executing busybox-1.36.1-r15.trigger
OK: 15 MiB in 26 packages
Installing packages to root filesystem: ok.
ok.
OpenRC 0.52.1 is starting up Linux 6.6.4-1-lts (ppc64le)
* /proc is already mounted
* Mounting /run ... [ ok ]
* /run/openrc: creating directory
* /run/lock: creating directory
* /run/lock: correcting owner
* Caching service dependencies ... [ ok ]
* Remounting devtmpfs on /dev ... [ ok ]
* Mounting /dev/mqueue ... [ ok ]
Connecting to 10.1.2.60 (10.1.2.60:80)
saving to '/lib/modloop-lts'
modloop-lts 100% |********************************| 43.4M 0:00:00 ETA
'/lib/modloop-lts' saved
* Mounting modloop /lib/modloop-lts ... * Verifying modloop
[ ok ]
* Mounting security filesystem ... [ ok ]
* Mounting debug filesystem ... [ ok ]
* Mounting persistent storage (pstore) filesystem ... [ ok ]
* Starting busybox mdev ... [ ok ]
* Scanning hardware for mdev ... [ ok ]
* Loading hardware drivers ... [ ok ]
* Setting the local clock based on last shutdown time ... [ ok ]
* Loading modules ... [ ok ]
* Checking local filesystems ... [ ok ]
* Remounting filesystems ... [ ok ]
* Mounting local filesystems ... [ ok ]
* Configuring kernel parameters ... [ ok ]
* Migrating /var/lock to /run/lock ... [ ok ]
* Creating user login records ... [ ok ]
* Cleaning /tmp directory ... [ ok ]
* Setting hostname ... [ ok ]
* Starting busybox syslog ... [ ok ]
/lib/rc/sh/openrc-run.sh: eval: line 7: syntax error: unexpected "("
* ERROR: firstboot failed to start
Welcome to Alpine Linux 3.19
Kernel 6.6.4-1-lts on an ppc64le (/dev/hvc0)
localhost login:
The error in the init script is concerning, but seems like the system works when I log into it.
Bibliography
- https://wiki.alpinelinux.org/wiki/PXE_boot
- https://www.ibm.com/docs/en/power10?topic=novalink-setting-up-network-installation-powervm
- https://askubuntu.com/questions/430705/how-to-use-apt-get-to-download-multi-arch-library
One thought on “Netbooting Alpine Linux on a PowerVM system”