Porting Apache NuttX RTOS to Avaota-A1 SBC (Allwinner A527 SoC)

📝 16 Mar 2025

Avaota-A1 SBC with SDWire MicroSD Multiplexer and Smart Power Plug

This article explains how we ported NuttX from QEMU Arm64 Kernel Build to PINE64 Yuzuki Avaota-A1 SBC based on Allwinner A527 SoC … Completed within 24 Hours!

Why are we doing this?

Anyone porting NuttX from QEMU to Real SBC? This walkthrough shall be mighty helpful!
Avaota-A1 SBC is Open Source Hardware (CERN OHL Licensed). PINE64 sells it today, maybe we’ll see more manufacturers.
This could be the First Port of Arm64 in NuttX Kernel Build. (NXP i.MX93 might be another?)
We’ll run it as PR Test Bot for validating Arm64 Pull Requests on Real Hardware. PR Test Bot will be fully automated thanks to the MicroSD Multiplexer.

We’re ready for volunteers to build NuttX Drivers for Avaota-A1 / Allwinner A527 (GPIO, SPI, I2C, MIPI CSI / DSI, Ethernet, WiFi, …) Please lemme know, maybe we can draw something on the Onboard LCD! 🙏

Sunxi Docs on Allwinner A527
Allwinner A527 Datasheet
Allwinner A523 User Manual (A527 is similar to A523)
Avaota-A1 Schematic
Avaota-A1 Device Tree

(BTW I bought all the hardware covered in this article. Nope, nothing was sponsored: Avaota-A1, SDWire, IKEA TRETAKT)

Avaota-A1 SBC connected to USB UART

§1 Boot Linux on our SBC

Nifty Trick for Booting NuttX on Any Arm64 SBC (RISC-V too)

Arm64 Bootloader (U-Boot / SyterKit) will boot Linux by loading the Image file

(Containing the Linux Kernel)
Thus we “Hijack” the Image file, replace it by NuttX Kernel
Which means NuttX Kernel shall look and feel like a Linux Kernel
That’s why we have a Linux Kernel Header at the top of NuttX

To begin, we observe our SBC and its Natural Behaviour… How does it Boot Linux?

Connect a USB UART Dongle (CH340 or CP2102) to the UART0 Port (pic above)

Avaota-A1 USB UART Colour

GND (Pin 6) GND Yellow

TX (Pin 8) RX Orange

RX (Pin 10) TX Red

Avaota-A1	USB UART	Colour
GND (Pin 6)	GND	Yellow
TX (Pin 8)	RX	Orange
RX (Pin 10)	TX	Red

Boot Log will appear at /dev/ttyUSB0…

## Allow the user to access the USB UART port
## Logout and login to refresh the permissions
sudo usermod -a -G dialout $USER
logout

## Connect to USB UART Console
screen /dev/ttyUSB0 115200

Download the Latest AvaotaOS Release (Ubuntu Noble GNOME) and uncompress it…

wget https://github.com/AvaotaSBC/AvaotaOS/releases/download/0.3.0.4/AvaotaOS-0.3.0.4-noble-gnome-arm64-avaota-a1.img.xz
xz -d AvaotaOS-0.3.0.4-noble-gnome-arm64-avaota-a1.img.xz

Write the .img file to a MicroSD with Balena Etcher.

Insert the MicroSD into our SBC and Boot AvaotaOS. We’ll see the Boot Log…

read /Image addr=40800000
Kernel addr: 0x40800000
BL31: v2.5(debug):9241004a9
sunxi-arisc driver is starting
ERROR: Error initializing runtime service opteed_fast

Aha! Kernel Boot Address 0x4080_0000 is super important, we’ll use it in a while

§2 NuttX Kernel Build for Arm64 QEMU

Follow these steps to Build and Run NuttX for Arm64 QEMU (Kernel Build)

## Build NuttX Kernel (NuttX Kernel Build)
git clone https://github.com/apache/nuttx
git clone https://github.com/apache/nuttx-apps apps
cd nuttx
tools/configure.sh qemu-armv8a:knsh
make -j

## Build NuttX Apps (NuttX Kernel Build)
make -j export
pushd ../apps
./tools/mkimport.sh -z -x ../nuttx/nuttx-export-*.tar.gz
make -j import
popd

## Boot NuttX on QEMU
qemu-system-aarch64 \
  -semihosting \
  -cpu cortex-a53 \
  -nographic \
  -machine virt,virtualization=on,gic-version=3 \
  -net none \
  -chardev stdio,id=con,mux=on \
  -serial chardev:con \
  -mon chardev=con,mode=readline \
  -kernel ./nuttx

Check that it works…

- Ready to Boot Primary CPU
- Boot from EL2
- Boot from EL1
- Boot to C runtime for OS Initialize

NuttShell (NSH) NuttX-12.8.0
nsh> uname -a
nxposix_spawn_exec: ERROR: exec failed: 2
NuttX 12.8.0 96eb5e7819 Mar 13 2025 15:45:11 arm64 qemu-armv8a

nsh> hello
Hello, World!!

## No worries about `nxposix_spawn_exec`
## To Quit: Press Ctrl-a then x

We’re ready to boot nuttx.bin on our SBC.

What’s this semihosting business in QEMU?

## Boot NuttX on QEMU, needs Semihosting
qemu-system-aarch64 \
  -semihosting ...

NuttX Kernel Build will call out to HostFS Semihosting (pic above) to access NSH Shell and NuttX Apps. We’ll change this for our SBC.

Why start with NuttX Kernel Build? Not NuttX Flat Build?

Our SBC is a mighty monster with Eight Arm64 Cores and plenty of RAM (2 GB). It makes more sense to boot NuttX Kernel Build and run lots of cool powerful NuttX App, thanks to Virtual Memory.

(NuttX Flat Build was created for Simpler Microcontrollers with Limited RAM)

Yuzuki Avaota-A1 SBC with PinePhone MicroSD Extender

§3 Boot NuttX on our SBC

Remember the MicroSD we downloaded? Inside the MicroSD is a 28 MB Linux Kernel, named “Image”…

$ ls -l /media/$USER/boot
   78769  bl31.bin
  180233  config-5.15.154-ga464bc4feaff
     512  dtb
     512  extlinux
27783176  Image
  180228  scp.bin
   12960  splash.bin
 5193581  System.map-5.15.154-ga464bc4feaff
 6497300  uInitrd

We replace it with NuttX…

Take the NuttX Kernel nuttx.bin from the previous section

(Yes the QEMU one)

Overwrite the Image file by nuttx.bin…

## Backup and overwrite `Image` on MicroSD
mv \
  /media/$USER/boot/Image \
  /media/$USER/boot/Image.old
cp \
  nuttx.bin \
  /media/$USER/boot/Image

## `Image` should be a lot smaller now
## Remember to Unmount and prevent filesystem corruption
ls -l /media/$USER/boot/Image
umount /media/$USER/boot

Insert the MicroSD into our SBC. Boot it…

read /Image addr=40800000
Kernel addr: 0x40800000
BL31: v2.5(debug):9241004a9
sunxi-arisc driver is starting
ERROR: Error initializing runtime service opteed_fast

Nothing happens. We tweak this iteratively, in tiny steps…

§4 Print to UART in Arm64 Assembly

Is NuttX actually booting on our SBC?

Let’s print something. UART0 Base Address is here…

A523 User Manual	Page 1839
Module	Base Address
UART0	0x0250_0000

16550 Transmit Register is at Offset 0…

A523 User Manual	Page 1839
Offset	Register
0x0000	UART_THR (Transmit Holding Register)
0x0004	UART_DLH (Divisor Latch High Register)
0x0008	UART_IIR (Interrupt Identity Register)
0x000C	UART_LCR (Line Control)

This says we can Print to UART like so…

// Print `123` to UART0
*(volatile uint8_t *) 0x02500000 = '1';
*(volatile uint8_t *) 0x02500000 = '2';
*(volatile uint8_t *) 0x02500000 = '3';

But we’ll do it in Arm64 Assembly: arm64_head.S

/* Bootloader starts NuttX here */
__start:
  add x13, x18, #0x16 /* "MZ": Magic Number for Linux Kernel Header */
  b   real_start      /* Jump to Executable Code      */
  ...                 /* Omitted: Linux Kernel Header */

/* Executable Code begins here */
/* We print `123` to UART0     */
real_start:

  /* Load UART0 Base Address into Register X15 */
  mov  x15, #0x02500000

  /* Load character `1` into Register W16 */
  mov  w16, #0x31

  /* Store the lower byte from Register W16 (`1`) to UART0 Base Address */
  strb w16, [x15]

  /* Load and Store the lower byte from Register W16 (`2`) to UART0 Base Address */
  mov  w16, #0x32
  strb w16, [x15]

  /* Load and Store the lower byte from Register W16 (`3`) to UART0 Base Address */
  mov  w16, #0x33
  strb w16, [x15]

(RISC-V? Same same)

Rebuild NuttX and recopy nuttx.bin to MicroSD, overwriting the Image file. NuttX will boot and print 123! 🎉

read /Image addr=40800000
Kernel addr: 0x40800000
BL31: v2.5(debug):9241004a9
sunxi-arisc driver is starting
ERROR: Error initializing runtime service opteed_fast
123

Indeed NuttX is booting on our SBC, then crashing later. (Ignore the error: opteed_fast)

Why print in Arm64 Assembly? Why not C?

Arm64 Assembly is the very first thing that boots when Bootloader starts NuttX
This happens before anything complicated and crash-prone begins: UART Driver, Memory Management, Task Scheduler, …
The Arm64 Assembly above is Address-Independent Code: It will execute at Any Arm64 Address

Next we move our code and make it Address-Dependent…

§5 Set the Start Address

NuttX boots a tiny bit on our SBC. Where’s the rest?

Our SBC boots NuttX at a different address from QEMU. We set the Start Address inside NuttX…

read /Image addr=40800000
Kernel addr: 0x40800000
123

Remember the Boot Log from earlier? It says that the SyterKit Bootloader starts NuttX at Address 0x4080_0000. We set it here: ld-kernel.script

MEMORY {
  /* Previously: QEMU boots at 0x4028_0000 */
  dram (rwx)  : ORIGIN = 0x40800000, LENGTH = 2M

  /* Previously: QEMU Paged Memory is at 0x4028_0000 */
  pgram (rwx) : ORIGIN = 0x40A00000, LENGTH = 4M   /* w/ cache */

  /* Why? Because 0x4080_0000 + 2 MB = 0x40A0_0000 */

(Note that Paged Memory Pool shifts down)

Since we changed the Paged Memory Pool (pgram), we update ARCH_PGPOOL_PBASE and VBASE: configs/knsh/defconfig

## Physical Address of Paged Memory Pool
## Previously: QEMU Paged Memory is at 0x4028_0000
CONFIG_ARCH_PGPOOL_PBASE=0x40A00000

## Virtual Address of Paged Memory Pool
## Previously: QEMU Paged Memory is at 0x4028_0000
CONFIG_ARCH_PGPOOL_VBASE=0x40A00000

(Paged Memory Pool shall be dished out as Virtual Memory to NuttX Apps)

NuttX QEMU declares the RAM Size as 128 MB in RAMBANK1_SIZE. We set RAM_SIZE accordingly: configs/knsh/defconfig
```
## RAM Size is a paltry 128 MB
CONFIG_RAM_SIZE=134217728
```
(Kinda tiny, but sufficient)

Linux Kernel Header has an incorrect Image Load Offset. Arm64 Bootloaders don’t care, we’ll let it be…

/* Bootloader starts NuttX here, followed by Linux Kernel Header */
__start:
  ...
  /* Image Load Offset from Start of RAM         */
  /* Boot Address - CONFIG_RAM_START = 0x800000  */
  /* But we won't change this, since it's unused */
  .quad 0x800000

With these mods, our C Code in NuttX shall boot correctly. FYI: Boot Address also appears on the Onboard LCD…

Avaota-A1 SBC with Onboard LCD

§6 UART Driver for 16550

Our C Code can print to UART now?

To watch the Boot Progress (Sesame Street-style), we can print primitively to UART like this: qemu_boot.c

// 0x0250_0000 is the UART0 Base Address
void arm64_boot_primary_c_routine(void) {
  *(volatile uint8_t *) 0x02500000 = 'A';
  arm64_chip_boot();
  ...

void arm64_chip_boot(void) {
  *(volatile uint8_t *) 0x02500000 = 'B';
  arm64_mmu_init(true);  // Init the Memory Mgmt Unit

  *(volatile uint8_t *) 0x02500000 = 'C';
  arm64_enable_mte();    // Init the Memory Tagging Extension

  *(volatile uint8_t *) 0x02500000 = 'D';
  qemu_board_initialize();  // Init the Board

  *(volatile uint8_t *) 0x02500000 = 'E';
  arm64_earlyserialinit();  // Init the Serial Driver

  *(volatile uint8_t *) 0x02500000 = 'F';
  syslog_rpmsg_init_early(...);  // Init the System Logger

  *(volatile uint8_t *) 0x02500000 = 'G';
  up_perf_init(..);  // Init the Performance Counters

Beyond Big Bird: We need the 16550 UART Driver. Based on the A527 UART Doc…

NuttX Boot Code (Arm64 Assembly) will print to UART. We patch it: qemu_lowputc.S

// Base Address and Baud Rate for 16550 UART
#define UART1_BASE_ADDRESS          0x02500000
#define EARLY_UART_PL011_BAUD_RATE  115200

NuttX Boot Code will drop UART Output, unless we wait for UART Ready: qemu_lowputc.S

/* Wait for 16550 UART to be ready to transmit
* xb: Register that contains the UART Base Address
* wt: Scratch register number */
.macro early_uart_ready xb, wt
1:
  ldrh  \wt, [\xb, #0x14] /* UART_LSR (Line Status Register) */
  tst   \wt, #0x20        /* Check THRE (TX Holding Register Empty) */
  b.eq  1b                /* Wait for the UART to be ready (THRE=1) */
.endm

(Thanks to PinePhone)

QEMU uses PL011 UART. We switch to 16550 UART: qemu_serial.c

// Switch from PL011 UART (QEMU) to 16550 UART
#include <nuttx/serial/uart_16550.h>

// Enable the 16550 Console UART at Startup
void arm64_earlyserialinit(void) {
  // Previously for QEMU: pl011_earlyserialinit
  u16550_earlyserialinit();
}

// Ditto but not so early
void arm64_serialinit(void) {
  // Previous for QEMU: pl011_serialinit
  u16550_serialinit();
}

16550 UART shall be configured: configs/knsh/defconfig

CONFIG_16550_ADDRWIDTH=0
CONFIG_16550_REGINCR=4
CONFIG_16550_UART0=y
CONFIG_16550_UART0_BASE=0x02500000
CONFIG_16550_UART0_CLOCK=198144000
CONFIG_16550_UART0_IRQ=125
CONFIG_16550_UART0_SERIAL_CONSOLE=y
CONFIG_16550_UART=y
CONFIG_16550_WAIT_LCR=y
CONFIG_SERIAL_UART_ARCH_MMIO=y

PL011 UART shall be removed: configs/knsh/defconfig

## Remove PL011 UART from NuttX Config:
## CONFIG_UART1_BASE=0x9000000
## CONFIG_UART1_IRQ=33
## CONFIG_UART1_PL011=y
## CONFIG_UART1_SERIAL_CONSOLE=y
## CONFIG_UART_PL011=y

16550_UART0_CLOCK isn’t quite correct, we’ll settle later. Meanwhile we disable the UART Clock Configuration: uart_16550.c

// We disable the UART Clock Configuration...
static int u16550_setup(FAR struct uart_dev_s *dev) { ...
#ifdef FIX_LATER  // We'll fix it later
  // Enter DLAB=1
  u16550_serialout(priv, UART_LCR_OFFSET, (lcr | UART_LCR_DLAB));

  // Omitted: Set the UART Baud Divisor
  // ...

  // Clear DLAB
  u16550_serialout(priv, UART_LCR_OFFSET, lcr);
#endif

Same old drill: Rebuild, recopy and reboot NuttX. We see plenty more debug output…

123
- Ready to Boot Primary CPU
- Boot from EL2
- Boot from EL1
- Boot to C runtime for OS Initialize
AB

OK the repeated rebuilding, recopying and rebooting of NuttX is getting really tiresome. We automate…

Avaota-A1 SBC with SDWire MicroSD Multiplexer and Smart Power Plug

§7 MicroSD Multiplexer + Smart Power Plug

What if we could rebuild-recopy-reboot NuttX… In One Single Script?

Thankfully our Avaota-A1 SBC is connected to SDWire MicroSD Multiplexer and Smart Power Plug (pic above). Our Build Script shall do everything for us…

Copy NuttX to MicroSD
Swap the MicroSD from our Test PC to SBC
Power up SBC and boot NuttX
Run OSTest and verify the result
How it looks? Watch the Demo

Avaota-A1 SBC with SDWire MicroSD Multiplexer and Smart Power Plug

All this happens in our nifty Build Script: run.sh

## Build NuttX and Apps (NuttX Kernel Build)
make -j
make -j export
pushd ../apps
./tools/mkimport.sh -z -x ../nuttx/nuttx-export-*.tar.gz
make -j import
popd

## Generate the Initial RAM Disk
## Prepare a Padding with 64 KB of zeroes
## Append Padding and Initial RAM Disk to the NuttX Kernel
genromfs -f initrd -d ../apps/bin -V "NuttXBootVol"
head -c 65536 /dev/zero >/tmp/nuttx.pad
cat nuttx.bin /tmp/nuttx.pad initrd \
  >Image

## Get the Home Assistant Token
## That we copied from http://localhost:8123/profile/security
## export token=xxxx
. $HOME/home-assistant-token.sh

## Power Off the SBC
curl \
  -X POST \
  -H "Authorization: Bearer $token" \
  -H "Content-Type: application/json" \
  -d '{"entity_id": "automation.avaota_power_off"}' \
  http://localhost:8123/api/services/automation/trigger

## Copy NuttX Image to MicroSD
## No password needed for sudo, see below
## Change `thinkcentre` to your Test PC
## https://github.com/lupyuen/nuttx-avaota-a1/blob/main/copy-image.sh
scp Image thinkcentre:/tmp/Image
ssh thinkcentre ls -l /tmp/Image
ssh thinkcentre sudo /home/user/copy-image.sh

## Power On the SBC
curl \
  -X POST \
  -H "Authorization: Bearer $token" \
  -H "Content-Type: application/json" \
  -d '{"entity_id": "automation.avaota_power_on"}' \
  http://localhost:8123/api/services/automation/trigger

## Wait for SBC to finish booting
sleep 30

## Power Off the SBC
curl \
  -X POST \
  -H "Authorization: Bearer $token" \
  -H "Content-Type: application/json" \
  -d '{"entity_id": "automation.avaota_power_off"}' \
  http://localhost:8123/api/services/automation/trigger

(See the Build Log)

(Watch the Demo on YouTube)

Smart Power Plug in IKEA App and Google Home

This script assumes that we have…

Installed a Home Assistant Server

(Works fine with Docker)
Added the Smart Power Plug to Google Assistant

“Avaota Power” (pic above)
Installed the Google Assistant SDK for Home Assistant

(So we don’t need Zigbee programming)
Created the Power Automation in Home Assistant

“Avaota Power On” and “Avaota Power Off” (pic below)

Smart Power Plug in Home Assistant

What’s copy_image.sh?

This is the script that copies our NuttX Image to MicroSD, via the SDWire MicroSD Multiplexer, explained here…

“SDWire MicroSD Multiplexer”

Can we automate the testing? Including OSTest?

Yep we have an Expect Script that will execute and verify OSTest: run.sh

## Copy NuttX Image to MicroSD
## No password needed for sudo, see below
## Change `thinkcentre` to your Test PC
## https://github.com/lupyuen/nuttx-avaota-a1/blob/main/copy-image.sh
scp Image thinkcentre:/tmp/Image
ssh thinkcentre ls -l /tmp/Image
ssh thinkcentre sudo /home/user/copy-image.sh

## Boot and Test NuttX with OSTest
## https://github.com/lupyuen/nuttx-build-farm/blob/main/avaota.exp
export AVAOTA_SERVER=thinkcentre
pushd $HOME/nuttx-build-farm
expect ./avaota.exp
popd

§8 Arm64 Memory Management Unit

It’s getting late. Can we get back to NuttX now?

24 Hours is all we need no worries! Earlier we saw NuttX stuck at “AB”…

123
- Ready to Boot Primary CPU
- Boot from EL2
- Boot from EL1
- Boot to C runtime for OS Initialize
AB

Which says that NuttX is stranded inside arm64_mmu_init: qemu_boot.c

// 0x0250_0000 is the UART0 Base Address
void arm64_boot_primary_c_routine(void) {
  *(volatile uint8_t *) 0x02500000 = 'A';
  arm64_chip_boot();
  ...

// `AB` means that NuttX is stuck inside arm64_mmu_init()
void arm64_chip_boot(void) {
  *(volatile uint8_t *) 0x02500000 = 'B';
  arm64_mmu_init(true);  // Init the Memory Mgmt Unit

  // Stuck above, never came here
  *(volatile uint8_t *) 0x02500000 = 'C';
  arm64_enable_mte();    // Init the Memory Tagging Extension

What’s arm64_mmu_init?

NuttX calls arm64_mmu_init to start the Arm64 Memory Management Unit (MMU). We add some logs inside: arm64_mmu.c

// Enable Debugging for MMU
#define CONFIG_MMU_ASSERT 1
#define CONFIG_MMU_DEBUG  1
#define trace_printf _info

// We fix the Debug Output, changing `%lux` to `%p`
static void init_xlat_tables(const struct arm_mmu_region *region) {
  ...
  sinfo("mmap: virt %p phys %p size %p\n", virt, phys, size);

// To enable the MMU at Exception Level 1...
static void enable_mmu_el1(unsigned int flags) {
  ...
  // Flush the Cached Data before Enabling MMU
  _info("UP_MB");
  UP_MB();

  // Enable the MMU and Data Cache
  _info("Enable the MMU and data cache");
  write_sysreg(value | SCTLR_M_BIT | SCTLR_C_BIT, sctlr_el1);

  // Ensure that MMU Enable takes effect immediately
  _info("UP_ISB");
  UP_ISB();

And we enable the logs for Scheduler and Memory Manager: configs/knsh/defconfig

## Enable Logging for Memory Manager
CONFIG_DEBUG_MM=y
CONFIG_DEBUG_MM_ERROR=y
CONFIG_DEBUG_MM_INFO=y
CONFIG_DEBUG_MM_WARN=y

## Enable Logging for Scheduler
CONFIG_DEBUG_SCHED=y
CONFIG_DEBUG_SCHED_ERROR=y
CONFIG_DEBUG_SCHED_INFO=y
CONFIG_DEBUG_SCHED_WARN=y

Ah OK we’re stuck just before Enabling the MMU…

## Init the MMU Page Translation Tables
init_xlat_tables: mmap: virt 0x7000000    phys 0x7000000    size 0x20000000
init_xlat_tables: mmap: virt 0x40000000   phys 0x40000000   size 0x8000000
init_xlat_tables: mmap: virt 0x4010000000 phys 0x4010000000 size 0x10000000
init_xlat_tables: mmap: virt 0x8000000000 phys 0x8000000000 size 0x8000000000
init_xlat_tables: mmap: virt 0x3eff0000 phys 0x3eff0000 size 0x10000
init_xlat_tables: mmap: virt 0x40800000 phys 0x40800000 size 0x2a000
init_xlat_tables: mmap: virt 0x4082a000 phys 0x4082a000 size 0x6000
init_xlat_tables: mmap: virt 0x40830000 phys 0x40830000 size 0x13000
init_xlat_tables: mmap: virt 0x40a00000 phys 0x40a00000 size 0x400000

## Enable the MMU at Exception Level 1
enable_mmu_el1: UP_MB
enable_mmu_el1: Enable the MMU and data cache

(Exception Level explained)

Something sus about the above Mystery Addresses, what are they?

Virtual	Physical	Size
0x0700_0000	0x0700_0000	0x2000_0000
0x4000_0000	0x4000_0000	0x0800_0000
0x40_1000_0000	0x40_1000_0000	0x1000_0000
0x80_0000_0000	0x80_0000_0000	0x80_0000_0000
0x3EFF_0000	0x3EFF_0000	0x0001_0000
0x4080_0000	0x4080_0000	0x0002_A000
0x4082_A000	0x4082_A000	0x0000_6000
0x4083_0000	0x4083_0000	0x0001_3000
0x40A0_0000	0x40A0_0000	0x0040_0000

A527 Memory Map

§9 Fix the Memory Map

Why do we need Arm64 MMU? (Memory Management Unit)

We require MMU for…

Memory Protection: Prevent Applications (and Kernel) from meddling with things (in System Memory) that they’re not supposed to
Virtual Memory: Allow Applications to access chunks of “Imaginary Memory” at Exotic Addresses (0x8000_0000!)

But in reality: They’re System RAM recycled from boring old addresses (like 0x40A0_4000)

If we don’t configure MMU with the correct Memory Map…

NuttX Kernel won’t boot: “Help! I can’t access my Kernel Code and Data!”
NuttX Apps won’t run: “Whoops where’s the App Code and Data that Kernel promised?”

Arm64 MMU won’t turn on. Maybe our Memory Map is incorrect?

We verify our A527 Memory Map (pic above)

A523 User Manual	Page 40
Module	Address
Boot ROM & SRAM	0x0000_0000 to …
PCIE	0x2000_0000 to 0x2FFF_FFFF
DRAM	0x4000_0000 to …

How does this compare with NuttX? We do extra MMU Logging: arm64_mmu.c

// Print the Names of the MMU Memory Regions
static void init_xlat_tables(const struct arm_mmu_region *region) { ...
  _info("name=%s\n", region->name);
  sinfo("mmap: virt %p phys %p size %p\n", virt, phys, size);

Ah much clearer! Now we see the Names of Memory Regions for the MMU…

Name	Physical	Size
DEVICE_REGION	0x0700_0000	0x2000_0000
DRAM0_S0	0x4000_0000	0x0800_0000
PCI_CFG	0x40_1000_0000	0x1000_0000
PCI_MEM	0x80_0000_0000	0x80_0000_0000
PCI_IO	0x3EFF_0000	0x0001_0000
nx_code	0x4080_0000	0x0002_A000
nx_rodata	0x4082_A000	0x0000_6000
nx_data	0x4083_0000	0x0001_3000
nx_pgpool	0x40A0_0000	0x0040_0000

Two Tweaks…

DEVICE_REGION: This says I/O Memory Space ends at 0x2700_0000. Based on the earlier A527 Memory Map, we extend this to 0x4000_0000 (1 GB): qemu/chip.h

// Fix the I/O Memory Space: Base Address and Size
#define CONFIG_DEVICEIO_BASEADDR 0x00000000
#define CONFIG_DEVICEIO_SIZE     MB(1024)

// We don't need PCI, for now
// #define CONFIG_PCI_CFG_BASEADDR 0x4010000000
// #define CONFIG_PCI_CFG_SIZE     MB(256)
// #define CONFIG_PCI_MEM_BASEADDR 0x8000000000
// #define CONFIG_PCI_MEM_SIZE     GB(512)
// #define CONFIG_PCI_IO_BASEADDR  0x3eff0000
// #define CONFIG_PCI_IO_SIZE      KB(64)

PCI: We remove these for now: qemu_boot.c

static const struct arm_mmu_region g_mmu_regions[] = {
  ...
  // We don't need PCI, for now
  // MMU_REGION_FLAT_ENTRY("PCI_CFG", ...
  // MMU_REGION_FLAT_ENTRY("PCI_MEM", ...
  // MMU_REGION_FLAT_ENTRY("PCI_IO", ...

The rest are hunky dory…

DRAM0_S0 says that RAM Address Space ends at 0x4800_0000 (128 MB)

(Kinda small, but sufficient for now)
nx_code (0x4080_0000): Kernel Code begins here
nx_rodata (0x4082_A000): Read-Only Data for Kernel
nx_data (0x4083_0000): Read-Write Data for Kernel
nx_pgpool (0x40A0_0000): Remember the Paged Memory Pool? This shall be dished out as Virtual Memory to NuttX Apps

We rebuild, recopy, reboot NuttX. Our Memory Map looks much better now…

Name	Physical	Size
DEVICE_REGION	0x0000_0000	0x4000_0000
DRAM0_S0	0x4000_0000	0x0800_0000
nx_code	0x4080_0000	0x0002_A000
nx_rodata	0x4082_A000	0x0000_6000
nx_data	0x4083_0000	0x0001_3000
nx_pgpool	0x40A0_0000	0x0040_0000

Though it crashes elsewhere…

§10 Arm64 Generic Interrupt Controller

Why is NuttX failing with an Undefined Instruction?

gic_validate_dist_version: No GIC version detect
arm64_gic_initialize: no distributor detected, giving up ret=-19
...
nx_start_application: Starting init task: /system/bin/init
arm64_el1_undef: Undefined instruction at 0x408276e4, dump:
Assertion failed panic: at file: common/arm64_fatal.c:572

Yeah this failure is totally misleading. Real Culprit: NuttX couldn’t Init the GIC…

No GIC version detect
No distributor detected, giving up

What’s this GIC?

It’s the Arm64 Generic Interrupt Controller (GIC), version 3. GIC shall…

Receive Input / Output Interrupts

(Like keypresses)
Forward them to a CPU Core for processing

(Works like RISC-V PLIC)

GIC is here…

A523 User Manual	Page 263
Module	Base Address
GIC	0x0340_0000

Which has these GIC Registers inside, handling 8 Arm64 Cores…

A523 User Manual	Page 264
Offset	Register
0x00_0000	GICD_CTLR (Distributor Control Register)
0x06_0000	GICR_CTLR_C0 (Redistributor Control Register, Core 0)
0x08_0000	GICR_CTLR_C1 (Ditto, Core 1)
0x0A_0000	GICR_CTLR_C2 (Ditto, Core 2)
0x0C_0000	GICR_CTLR_C3 (Ditto, Core 3)
0x0E_0000	GICR_CTLR_C4 (Ditto, Core 4)
0x10_0000	GICR_CTLR_C5 (Ditto, Core 5)
0x12_0000	GICR_CTLR_C6 (Ditto, Core 6)
0x14_0000	GICR_CTLR_C7 (Ditto, Core 7)
0x16_0000	GICDA_CTLR (Distributor Control Register A)

Based on the above, we set the Addresses of GICD and GICR (Distributor / Redistributor): qemu/chip.h

// Base Address of GIC Distributor and Redistributor
#define CONFIG_GICD_BASE   0x3400000
#define CONFIG_GICR_BASE   0x3460000

// Spaced 0x20000 bytes per Arm64 Core
#define CONFIG_GICR_OFFSET 0x20000

Remember to Disable Memory Manager Logging. NuttX GIC Driver starts correctly and complains no more!

## SPI = Physical Interrupt Signal (not the typical SPI)
gic_validate_dist_version:
  GICv3 version detect
  GICD_TYPER = 0x7b0408
  256 SPIs implemented

We’ll call GIC to handle UART Interrupts. Before that: We need NSH Shell…

NuttX Apps Filesystem on ROMFS

§11 NuttX Apps Filesystem

Are we done yet?

For a Simple NuttX Port (Flat Build): Congrats, just fix the UART Interrupt and we’re done!

However we’re doing NuttX Kernel Build. Which needs more work…

nx_start_application:
  Starting init task: /system/bin/init
arm64_el1_undef:
  Undefined instruction at 0x408274a4
Assertion failed panic:
  common/arm64_fatal.c:572

What’s /system/bin/init? Why is it failing?

/system/bin/init is NSH Shell. NuttX Kernel Build will load NuttX Apps from a Local Filesystem, and it’s missing right now. (NuttX Flat Build will bind binary Apps directly into Kernel)

To solve this: We bundle the NuttX Apps together into a ROMFS Filesystem…

## Generate the Initial RAM Disk
genromfs \
  -f initrd \
  -d ../apps/bin \
  -V "NuttXBootVol"

Then we package NuttX Kernel + NuttX Apps into a NuttX Image…

## Prepare a Padding with 64 KB of zeroes
## Append Padding and Initial RAM Disk to the NuttX Kernel
head -c 65536 /dev/zero >/tmp/nuttx.pad
cat nuttx.bin /tmp/nuttx.pad initrd \
  >Image

(See the Build Script)

When NuttX Boots: It will…

Locate the ROMFS Filesystem in memory
Copy it to the designated Memory Region
Mount it as a RAM Disk
Allowing NuttX Kernel to start NSH Shell

(And other NuttX Apps)

How? See this…

“NuttX Apps Filesystem”

NSH Prompt still missing? It won’t appear until we handle the UART Interrupt…

NuttX on Avaota-A1

§12 Fix the UART Interrupt

One Last Thing: Settle the UART Interrupt and we’re done!

A523 User Manual	Page 256
Interrupt Number	Interrupt Source
34	UART0

This is how we set the UART0 Interrupt and watch for keypresses: configs/knsh/defconfig

## Set the UART0 Interrupt to 34
CONFIG_16550_UART0_IRQ=34

To Wrap Up: We Disable Logging for Memory Manager and Scheduler. And Disable MMU Debugging.

NSH Prompt finally appears and OSTest completes successfully. Our NuttX Porting is complete yay!

NuttShell (NSH) NuttX-12.4.0
nsh> uname -a
NuttX 12.4.0 6c5c1a5f9f-dirty Mar  8 2025 21:57:02 arm64 qemu-armv8a

nsh> ostest
...
user_main: Exiting
ostest_main: Exiting with status 0

(See the Complete Log)

(See the Final Code)

(Ready for NuttX Upstreaming)

NSH Prompt won’t appear if UART Interrupt is disabled?

That’s because NSH runs as a NuttX App in User Space. When NSH Shell prints this…

NuttShell (NSH) NuttX-12.4.0
nsh>

It calls the Serial Driver. Which will wait for a UART Interrupt to signal that the Transmit Buffer is empty and available.

Thus if UART Interrupt is disabled, nothing gets printed in NuttX Apps. (Explained here)

NuttX might run OK on Radxa Cubie A5E (Allwinner T527)

NuttX might run OK on Radxa Cubie A5E (Allwinner T527)

§13 What’s Next

Right now we’re upstreaming Avatoa-A1 SBC to NuttX Mainline…

We’re seeking volunteers to build NuttX Drivers for Avaota-A1 (GPIO, SPI, I2C, MIPI CSI / DSI, Ethernet, WiFi, …) Please lemme know, maybe we can draw something on the Onboard LCD!

Special Thanks to My Sponsors for supporting my writing. Your support means so much to me 🙏

Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…

lupyuen.org/src/avaota.md

Yuzuki Avaota-A1 SBC with PinePhone MicroSD Extender

§14 Appendix: Build NuttX for Avaota-A1

To boot NuttX on Avatoa-A1: We may download Image from here…

NuttX Release for Avatoa-A1 SBC

Or follow these steps to compile our (Work-In-Progress) NuttX for Avaota-A1: run.sh

## Download Source Code for NuttX and Apps
git clone https://github.com/lupyuen2/wip-nuttx nuttx --branch avaota2
git clone https://github.com/lupyuen2/wip-nuttx-apps apps --branch avaota2
cd nuttx

## Build NuttX and Apps (NuttX Kernel Build)
tools/configure.sh avaota-a1:nsh
make -j
make -j export
pushd ../apps
./tools/mkimport.sh -z -x ../nuttx/nuttx-export-*.tar.gz
make -j import
popd

## Generate the Initial RAM Disk
## Prepare a Padding with 64 KB of zeroes
## Append Padding and Initial RAM Disk to the NuttX Kernel
genromfs -f initrd -d ../apps/bin -V "NuttXBootVol"
head -c 65536 /dev/zero >/tmp/nuttx.pad
cat nuttx.bin /tmp/nuttx.pad initrd \
  >Image

Read on to boot the NuttX Image on our SBC…

(See the Build Log)

NuttX on Avaota-A1

§15 Appendix: Boot NuttX on Avaota-A1

Earlier we built NuttX for Avaota-A1 and created the Image file, containing the NuttX Kernel + NuttX Apps. Let’s boot it on MicroSD…

Prepare the AvaotaOS MicroSD…

“Boot NuttX on our SBC”

Copy the NuttX Image to MicroSD…

## Copy NuttX Image to AvaotaOS MicroSD
## Overwrite the `Image` file
mv \
  /media/$USER/boot/Image \
  /media/$USER/boot/Image.old
cp \
  Image \
  /media/$USER/boot/Image

## Unmount and boot it on Avaota-A1
ls -l /media/$USER/boot/Image
umount /media/$USER/boot

Boot the MicroSD on our SBC

We can automate the last two steps with a MicroSD Multiplexer and Smart Power Plug: run.sh

## Get the Home Assistant Token
## That we copied from http://localhost:8123/profile/security
## export token=xxxx
. $HOME/home-assistant-token.sh

## Power Off the SBC
curl \
  -X POST \
  -H "Authorization: Bearer $token" \
  -H "Content-Type: application/json" \
  -d '{"entity_id": "automation.avaota_power_off"}' \
  http://localhost:8123/api/services/automation/trigger

## Copy NuttX Image to MicroSD
## No password needed for sudo, see below
## Change `thinkcentre` to your Test PC
## https://github.com/lupyuen/nuttx-avaota-a1/blob/main/copy-image.sh
scp Image thinkcentre:/tmp/Image
ssh thinkcentre ls -l /tmp/Image
ssh thinkcentre sudo /home/user/copy-image.sh

## Power On the SBC
curl \
  -X POST \
  -H "Authorization: Bearer $token" \
  -H "Content-Type: application/json" \
  -d '{"entity_id": "automation.avaota_power_on"}' \
  http://localhost:8123/api/services/automation/trigger

## Wait for SBC to finish testing
echo Press Enter to Power Off
read

## Power Off the SBC
curl \
  -X POST \
  -H "Authorization: Bearer $token" \
  -H "Content-Type: application/json" \
  -d '{"entity_id": "automation.avaota_power_off"}' \
  http://localhost:8123/api/services/automation/trigger

Or to Automate Everything including OSTest: run.sh

## Copy NuttX Image to MicroSD
## No password needed for sudo, see below
## Change `thinkcentre` to your Test PC
## https://github.com/lupyuen/nuttx-avaota-a1/blob/main/copy-image.sh
scp Image thinkcentre:/tmp/Image
ssh thinkcentre ls -l /tmp/Image
ssh thinkcentre sudo /home/user/copy-image.sh

## Boot and Test NuttX with OSTest
## https://github.com/lupyuen/nuttx-build-farm/blob/main/avaota.exp
export AVAOTA_SERVER=thinkcentre
pushd $HOME/nuttx-build-farm
expect ./avaota.exp
popd

(Watch the Demo on YouTube)

(copy-image.sh is explained here)

(Smart Power Plug also)

NuttX boots to NSH Shell. And passes OSTest yay!

NOTICE:  BL31: v2.5(debug):9241004a9
NOTICE:  BL31: Built : 13:37:46, Nov 16 2023
NOTICE:  BL31: No DTB found.
NOTICE:  [SCP] :wait arisc ready....
NOTICE:  [SCP] :arisc version: []
NOTICE:  [SCP] :arisc startup ready
NOTICE:  [SCP] :arisc startup notify message feedback
NOTICE:  [SCP] :sunxi-arisc driver is starting
ERROR:   Error initializing runtime service opteed_fast
- Ready to Boot Primary CPU
- Boot from EL2
- Boot from EL1
- Boot to C runtime for OS Initialize

NuttShell (NSH) NuttX-12.4.0
nsh> uname -a
NuttX 12.4.0 42c0ed4a89 Mar 13 2025 09:13:56 arm64 avaota-a1

nsh> ls -l /dev
/dev:
 crw-rw-rw-           0 console
 crw-rw-rw-           0 null
 brw-rw-rw-    16777216 ram0
 crw-rw-rw-           0 ttyS0
 crw-rw-rw-           0 zero

nsh> ps
  PID GROUP PRI POLICY   TYPE    NPX STATE    EVENT     SIGMASK            STACK    USED FILLED COMMAND
    0     0   0 FIFO     Kthread   - Ready              0000000000000000 0008176 0000928  11.3%  Idle_Task
    1     0 192 RR       Kthread   - Waiting  Semaphore 0000000000000000 0008112 0000992  12.2%  hpwork 0x40833568 0x408335b8
    2     0 100 RR       Kthread   - Waiting  Semaphore 0000000000000000 0008112 0000992  12.2%  lpwork 0x408334e8 0x40833538
    4     4 100 RR       Task      - Running            0000000000000000 0008128 0002192  26.9%  /system/bin/init

nsh> free
      total       used       free    maxused    maxfree  nused  nfree name
  125542400      33880  125508520      53032  125484976     58      5 Kmem
    4194304     245760    3948544               3948544               Page

nsh> hello
Hello, World!!

nsh> getprime
Set thread priority to 10
Set thread policy to SCHED_RR
Start thread #0
thread #0 started, looking for primes < 10000, doing 10 run(s)
thread #0 finished, found 1230 primes, last one was 9973
Done
getprime took 162 msec

nsh> ostest
...
Final memory usage:
VARIABLE  BEFORE   AFTER
======== ======== ========
arena        a000    26000
ordblks         2        4
mxordblk     6ff8    1aff8
uordblks     27e8     6700
fordblks     7818    1f900
user_main: Exiting
ostest_main: Exiting with status 0
nsh>

(See the NuttX Log)

Upstreaming NuttX for Avaota-A1

§16 Appendix: Upstream NuttX for Avaota-A1

In this article we ported NuttX QEMU Arm64 (Kernel Build) iteratively to Avaota-A1. Up Next: Upstreaming our code to NuttX Mainline!

Here’s how we copy-n-pasted our Modified Files into a proper NuttX Arch (Allwinner A527) and NuttX Board (Avaota-A1)

Upstreaming becomes lotsa copypasta…

We create a Staging PR in our own repo…

github.com/lupyuen2/wip-nuttx/pull/99

Dump the list of Modified Files…

## Change this to our Staging PR
$ pr=https://github.com/lupyuen2/wip-pinephone-nuttx/pull/99
$ curl -L $pr.diff \
  | grep "diff --git" \
  | sort \
  | cut -d" " -f3 \
  | cut -c3-

## Here are the Modified Files for our PR
Documentation/platforms/arm64/a527/boards/avaota-a1/avaota-a1.jpg
Documentation/platforms/arm64/a527/boards/avaota-a1/index.rst
Documentation/platforms/arm64/a527/index.rst
arch/arm64/Kconfig
arch/arm64/include/a527/chip.h
arch/arm64/include/a527/irq.h
arch/arm64/src/a527/CMakeLists.txt
arch/arm64/src/a527/Kconfig
arch/arm64/src/a527/Make.defs
arch/arm64/src/a527/a527_boot.c
arch/arm64/src/a527/a527_boot.h
arch/arm64/src/a527/a527_initialize.c
arch/arm64/src/a527/a527_lowputc.S
arch/arm64/src/a527/a527_serial.c
arch/arm64/src/a527/a527_textheap.c
arch/arm64/src/a527/a527_timer.c
arch/arm64/src/a527/chip.h
boards/Kconfig
boards/arm64/a527/avaota-a1/CMakeLists.txt
boards/arm64/a527/avaota-a1/Kconfig
boards/arm64/a527/avaota-a1/configs/nsh/defconfig
boards/arm64/a527/avaota-a1/include/board.h
boards/arm64/a527/avaota-a1/include/board_memorymap.h
boards/arm64/a527/avaota-a1/scripts/Make.defs
boards/arm64/a527/avaota-a1/scripts/gnu-elf.ld
boards/arm64/a527/avaota-a1/scripts/ld.script
boards/arm64/a527/avaota-a1/src/CMakeLists.txt
boards/arm64/a527/avaota-a1/src/Makefile
boards/arm64/a527/avaota-a1/src/a527_appinit.c
boards/arm64/a527/avaota-a1/src/a527_boardinit.c
boards/arm64/a527/avaota-a1/src/a527_bringup.c
boards/arm64/a527/avaota-a1/src/a527_power.c
boards/arm64/a527/avaota-a1/src/avaota-a1.h

Check nxstyle on the Modified Files…

## Run nxstyle on the Modified Files
nxstyle Documentation/platforms/arm64/a527/boards/avaota-a1/avaota-a1.jpg
nxstyle Documentation/platforms/arm64/a527/boards/avaota-a1/index.rst
nxstyle Documentation/platforms/arm64/a527/index.rst
nxstyle arch/arm64/Kconfig
nxstyle arch/arm64/include/a527/chip.h
nxstyle arch/arm64/include/a527/irq.h
nxstyle arch/arm64/src/a527/CMakeLists.txt
nxstyle arch/arm64/src/a527/Kconfig
nxstyle arch/arm64/src/a527/Make.defs
nxstyle arch/arm64/src/a527/a527_boot.c
nxstyle arch/arm64/src/a527/a527_boot.h
nxstyle arch/arm64/src/a527/a527_initialize.c
nxstyle arch/arm64/src/a527/a527_lowputc.S
nxstyle arch/arm64/src/a527/a527_serial.c
nxstyle arch/arm64/src/a527/a527_textheap.c
nxstyle arch/arm64/src/a527/a527_timer.c
nxstyle arch/arm64/src/a527/chip.h
nxstyle boards/Kconfig
nxstyle boards/arm64/a527/avaota-a1/CMakeLists.txt
nxstyle boards/arm64/a527/avaota-a1/Kconfig
nxstyle boards/arm64/a527/avaota-a1/configs/nsh/defconfig
nxstyle boards/arm64/a527/avaota-a1/include/board.h
nxstyle boards/arm64/a527/avaota-a1/include/board_memorymap.h
nxstyle boards/arm64/a527/avaota-a1/scripts/Make.defs
nxstyle boards/arm64/a527/avaota-a1/scripts/gnu-elf.ld
nxstyle boards/arm64/a527/avaota-a1/scripts/ld.script
nxstyle boards/arm64/a527/avaota-a1/src/CMakeLists.txt
nxstyle boards/arm64/a527/avaota-a1/src/Makefile
nxstyle boards/arm64/a527/avaota-a1/src/a527_appinit.c
nxstyle boards/arm64/a527/avaota-a1/src/a527_boardinit.c
nxstyle boards/arm64/a527/avaota-a1/src/a527_bringup.c
nxstyle boards/arm64/a527/avaota-a1/src/a527_power.c
nxstyle boards/arm64/a527/avaota-a1/src/avaota-a1.h

Copy the Arch Files into the Arch Pull Request

“arch/arm64/a527: Add support for Allwinner A527 SoC”

## Download the Branch for Avaota Arch (initially empty)
pushd /tmp 
git clone https://github.com/lupyuen2/wip-nuttx avaota-arch --branch avaota-arch
popd

## Copy the Arch Files from src to dest
function copy_files() {
  src=.
  dest=/tmp/avaota-arch
  for file in \
    Documentation/platforms/arm64/a527/index.rst \
    arch/arm64/Kconfig \
    arch/arm64/include/a527/chip.h \
    arch/arm64/include/a527/irq.h \
    arch/arm64/src/a527/CMakeLists.txt \
    arch/arm64/src/a527/Kconfig \
    arch/arm64/src/a527/Make.defs \
    arch/arm64/src/a527/a527_boot.c \
    arch/arm64/src/a527/a527_boot.h \
    arch/arm64/src/a527/a527_initialize.c \
    arch/arm64/src/a527/a527_lowputc.S \
    arch/arm64/src/a527/a527_serial.c \
    arch/arm64/src/a527/a527_textheap.c \
    arch/arm64/src/a527/a527_timer.c \
    arch/arm64/src/a527/chip.h \

  do
    src_file=$src/$file
    dest_file=$dest/$file
    dest_dir=$(dirname -- "$dest_file")
    set -x
    mkdir -p $dest_dir
    cp $src_file $dest_file
    set +x
  done
}

## Copy and commit /tmp/avaota-arch
## Remove the "Supported Boards" (toctree) from Arch Doc
copy_files
code /tmp/avaota-arch

Copy the Board Files into the Board Pull Request

## Download the Branch for Avaota Board (initially empty)
pushd /tmp 
git clone https://github.com/lupyuen2/wip-nuttx avaota-board --branch avaota-board
popd

## Copy the Board Files from src to dest
## Copy the Arch Doc again because we restored the "Supported Boards" 
function copy_files() {
  src=.
  dest=/tmp/avaota-board
  for file in \
    Documentation/platforms/arm64/a527/index.rst \
    Documentation/platforms/arm64/a527/boards/avaota-a1/avaota-a1.jpg \
    Documentation/platforms/arm64/a527/boards/avaota-a1/index.rst \
    boards/Kconfig \
    boards/arm64/a527/avaota-a1/CMakeLists.txt \
    boards/arm64/a527/avaota-a1/Kconfig \
    boards/arm64/a527/avaota-a1/configs/nsh/defconfig \
    boards/arm64/a527/avaota-a1/include/board.h \
    boards/arm64/a527/avaota-a1/include/board_memorymap.h \
    boards/arm64/a527/avaota-a1/scripts/Make.defs \
    boards/arm64/a527/avaota-a1/scripts/gnu-elf.ld \
    boards/arm64/a527/avaota-a1/scripts/ld.script \
    boards/arm64/a527/avaota-a1/src/CMakeLists.txt \
    boards/arm64/a527/avaota-a1/src/Makefile \
    boards/arm64/a527/avaota-a1/src/a527_appinit.c \
    boards/arm64/a527/avaota-a1/src/a527_boardinit.c \
    boards/arm64/a527/avaota-a1/src/a527_bringup.c \
    boards/arm64/a527/avaota-a1/src/a527_power.c \
    boards/arm64/a527/avaota-a1/src/avaota-a1.h \

  do
    src_file=$src/$file
    dest_file=$dest/$file
    dest_dir=$(dirname -- "$dest_file")
    set -x
    mkdir -p $dest_dir
    cp $src_file $dest_file
    set +x
  done
}

## Copy and commit /tmp/avaota-board
copy_files
code /tmp/avaota-board

Remember to create Two Commits Per PR: One Commit for Code, Another Commit for Docs
Need to Squash the Commits (or amend them), but another Code or Doc Commit is stuck in between?

Try Reordering the Commits to the top, before squashing or amending.
We’re finally ready to Submit our Pull Requests!

“arch/arm64/a527: Add support for Allwinner A527 SoC”

SDWire MicroSD Multiplexer

§17 Appendix: SDWire MicroSD Multiplexer

Let’s make our Tweak-Build-Test Cycle quicker for NuttX. We use SDWire MicroSD Multiplexer (pic above) to flip our MicroSD between Test PC and SBC…

“SDWire MicroSD Multiplexer”

SDWire needs Plenty of Sudo Passwords to flip the multiplexer, mount the filesystem, copy to MicroSD. We make it Sudo Password-Less with visudo…

Wrap all the Sudo Commands into a script: copy-image.sh

## Create a Bash Script: copy-image.sh
## Containing these commands...

set -e  ## Exit when any command fails
set -x  ## Echo commands
whoami  ## I am root!

## Copy /tmp/Image to MicroSD
sd-mux-ctrl --device-serial=sd-wire_02-09 --ts
sleep 5
mkdir -p /tmp/sda1
mount /dev/sda1 /tmp/sda1
cp /tmp/Image /tmp/sda1/
ls -l /tmp/sda1

## Unmount MicroSD and flip it to the Test Device (Avaota-A1)
umount /tmp/sda1
sd-mux-ctrl --device-serial=sd-wire_02-09 --dut

Configure visudo so that our script will run as Sudo Without Password…

## Make our script executable
## Start the Sudoers Editor
chmod +x /home/user/copy-image.sh
sudo visudo

## Add this line:
user ALL=(ALL) NOPASSWD: /home/user/copy-image.sh

Then we can trigger our script remotely via SSH, Without Sudo Password: run.sh

## Copy NuttX Image to MicroSD
## No password needed for sudo yay!
scp Image thinkcentre:/tmp/Image
ssh thinkcentre \
  ls -l /tmp/Image
ssh thinkcentre \
  sudo /home/user/copy-image.sh

Everything goes into our Build Script for NuttX

How it looks? Watch the Demo

(Actually we could allow anyone in the world to Remotely Build and Test NuttX on our Avaota-A1 SBC hmmm…)

NuttX Apps Filesystem in ROMFS

§18 Appendix: NuttX Apps Filesystem

Earlier we talked about the ROMFS Filesystem for NuttX Apps (Initial RAM Disk, pic above)

“NuttX Apps Filesystem”

This section explains how we implemented the NuttX Apps Filesystem…

Modified Files for NuttX Apps Filesystem

After this implementation, /system/bin/init (NSH Shell) shall start successfully…

qemu_bringup:
mount_ramdisk:
nx_start_application: ret=0
nx_start_application: Starting init task: /system/bin/init
nxtask_activate: /system/bin/init pid=4,TCB=0x408469f0
nxtask_exit: AppBringUp pid=3,TCB=0x40846190
board_app_initialize:
nx_start: CPU0: Beginning Idle Loop

§18.1 HostFS becomes ROMFS

QEMU uses Semihosting and HostFS to access the NuttX Apps Filesystem. We change to ROMFS: configs/knsh/defconfig

## We added ROMFS...
CONFIG_BOARDCTL_ROMDISK=y
CONFIG_BOARD_LATE_INITIALIZE=y
CONFIG_INIT_MOUNT_TARGET="/system/bin"

## And removed Semihosting HostFS...
## CONFIG_FS_HOSTFS=y
## CONFIG_ARM64_SEMIHOSTING_HOSTFS=y
## CONFIG_ARM64_SEMIHOSTING_HOSTFS_CACHE_COHERENCE=y
## CONFIG_INIT_MOUNT_DATA="fs=../apps"
## CONFIG_INIT_MOUNT_FSTYPE="hostfs"
## CONFIG_INIT_MOUNT_SOURCE=""
## CONFIG_INIT_MOUNT_TARGET="/system"

BOARD_LATE_INITIALIZE is needed because we’ll Mount the ROMFS Filesystem inside qemu_bringup. (See below)

§18.2 Linker Script

We reserve 16 MB of RAM for the ROMFS Filesystem that will host the NuttX Apps: ld-kernel.script

/* Linker Script: We added the RAM Disk (16 MB) */
MEMORY {
  dram (rwx)    : ORIGIN = 0x40800000, LENGTH = 2M
  pgram (rwx)   : ORIGIN = 0x40A00000, LENGTH = 4M    /* w/ cache */
  ramdisk (rwx) : ORIGIN = 0x40E00000, LENGTH = 16M   /* w/ cache */
}

/* We'll reference these in our code */
__ramdisk_start = ORIGIN(ramdisk);
__ramdisk_size  = LENGTH(ramdisk);
__ramdisk_end   = ORIGIN(ramdisk) + LENGTH(ramdisk);

§18.3 Mount the ROMFS

At Startup: We mount the ROMFS Filesystem (inside RAM) as /dev/ram0: qemu_bringup.c

// At NuttX Startup...
int qemu_bringup(void) {
  // We Mount the RAM Disk
  mount_ramdisk();
  ...
}

// Mount a RAM Disk defined in ld.script to /dev/ramX.  The RAM Disk
// contains a ROMFS filesystem with applications that can be spawned at
// runtime.
static int mount_ramdisk(void) {
  struct boardioc_romdisk_s desc;
  desc.minor    = RAMDISK_DEVICE_MINOR;
  desc.nsectors = NSECTORS((ssize_t)__ramdisk_size);
  desc.sectsize = SECTORSIZE;
  desc.image    = __ramdisk_start;

  int ret = boardctl(BOARDIOC_ROMDISK, (uintptr_t)&desc);
  if (ret < 0) {
    syslog(LOG_ERR, "Ramdisk register failed: %s\n", strerror(errno));
    syslog(LOG_ERR, "Ramdisk mountpoint /dev/ram%d\n",RAMDISK_DEVICE_MINOR);
    syslog(LOG_ERR, "Ramdisk length %lu, origin %lx\n", (ssize_t)__ramdisk_size, (uintptr_t)__ramdisk_start);
  }
  return ret;
}

// RAM Disk Definition
#define SECTORSIZE   512
#define NSECTORS(b)  (((b) + SECTORSIZE - 1) / SECTORSIZE)
#define RAMDISK_DEVICE_MINOR 0

§18.4 Copy the ROMFS

But Before That: We safely copy the ROMFS Filesystem (Initial RAM Disk) from the NuttX Image into the ramdisk Memory Region…

Mounting the ROMFS Filesystem

This happens just after Bootloader starts NuttX: qemu_boot.c

// Needed for the `aligned_data` macro
#include <nuttx/compiler.h>

// Just after Bootloader has started NuttX...
void arm64_chip_boot(void) {

  // We copy the RAM Disk
  qemu_copy_ramdisk();

  // Omitted: Other initialisation (MMU, ...)
  arm64_mmu_init(true);
  ...
}

// Copy the RAM Disk from NuttX Image to RAM Disk Region.
static void qemu_copy_ramdisk(void) {
  const uint8_t aligned_data(8) header[8] = "-rom1fs-";
  const uint8_t *limit = (uint8_t *)g_idle_topstack + (256 * 1024);
  uint8_t *ramdisk_addr = NULL;
  uint8_t *addr;
  uint32_t size;

  // After Idle Stack Top, search for "-rom1fs-". This is the RAM Disk Address.
  // Limit search to 256 KB after Idle Stack Top.
  for (addr = g_idle_topstack; addr < limit; addr += 8) {
      if (memcmp(addr, header, sizeof(header)) == 0) {
        ramdisk_addr = addr;
        break;
      }
  }

  // Stop if RAM Disk is missing
  if (ramdisk_addr == NULL) {
    _err("Missing RAM Disk. Check the initrd padding.");
    PANIC();
  }

  // Read the Filesystem Size from the next 4 bytes (Big Endian)
  size = (ramdisk_addr[8] << 24) + (ramdisk_addr[9] << 16) +
         (ramdisk_addr[10] << 8) + ramdisk_addr[11] + 0x1f0;

  // Filesystem Size must be less than RAM Disk Memory Region
  if (size > (size_t)__ramdisk_size) {
    _err("RAM Disk Region too small. Increase by %lu bytes.\n", size - (size_t)__ramdisk_size);
    PANIC();
  }

  // Copy the RAM Disk from NuttX Image to RAM Disk Region.
  // __ramdisk_start overlaps with ramdisk_addr + size.
  qemu_copy_overlap(__ramdisk_start, ramdisk_addr, size);
}

// Copy an overlapping memory region.  dest overlaps with src + count.
static void qemu_copy_overlap(uint8_t *dest, const uint8_t *src, size_t count) {
  uint8_t *d = dest + count - 1;
  const uint8_t *s = src + count - 1;
  if (dest <= src) { _err("dest and src should overlap"); PANIC(); }
  while (count--) {
    volatile uint8_t c = *s;  // Prevent compiler optimization
    *d = c;
    d--;
    s--;
  }
} 

// RAM Disk Region is defined in Linker Script
extern uint8_t __ramdisk_start[];
extern uint8_t __ramdisk_size[];

(Moved here)

Why the aligned addresses?

// Header is aligned to 8 bytes
const uint8_t
  aligned_data(8) header[8] =
  "-rom1fs-";

// Address is also aligned to 8 bytes
for (
  addr = g_idle_topstack;
  addr < limit;
  addr += 8
) {
  // Otherwise this will hit Alignment Fault
  memcmp(addr, header, sizeof(header));
  ...
}

We align our Memory Accesses to 8 Bytes. Otherwise we’ll hit an Alignment Fault…

## Alignment Fault at `memcmp(addr, header, sizeof(header))`
default_fatal_handler:
  (IFSC/DFSC) for Data/Instruction aborts:
  alignment fault

(Strangely: This Alignment isn’t needed for RISC-V)

Porting NuttX to Avaota-A1

§19 Appendix: Port NuttX to Avaota-A1

In this article, we took NuttX for Arm64 QEMU Kernel Build (qemu-armv8a:knsh) and changed it slightly for Avaota-A1 SBC. To help our PR Reviewers: This section explains the Modified Code in our Pull Request…

Modified Files for Avaota-A1

Only Seven Files were modified from QEMU NuttX. All other files were simply copied and renamed, from QEMU NuttX to Avaota-A1. (Pic above)

§19.1 Memory Map

arch/arm64/include/a527/chip.h

We define the I/O Memory Space…

// I/O Memory Space
#define CONFIG_DEVICEIO_BASEADDR   0x00000000
#define CONFIG_DEVICEIO_SIZE       MB(1024)

// Kernel Boot Address from SBC Bootloader
#define CONFIG_LOAD_BASE           0x40800000

Based on the A527 Memory Map…

A523 User Manual	Page 40
Module	Address
Boot ROM & SRAM	0x0000_0000 to …
PCIE	0x2000_0000 to 0x2FFF_FFFF
DRAM	0x4000_0000 to …

(Explained here)

§19.2 GIC Interrupt Controller

arch/arm64/include/a527/chip.h

We set the GIC Base Addresses…

// GICD and GICD Base Addresses
#define CONFIG_GICD_BASE           0x3400000
#define CONFIG_GICR_BASE           0x3460000

Based on the GIC Doc…

A523 User Manual	Page 263
Module	Base Address
GIC	0x0340_0000

And GIC Registers…

A523 User Manual	Page 264
Offset	Register
0x00_0000	GICD_CTLR (Distributor Control Register)
0x06_0000	GICR_CTLR_C0 (Redistributor Control Register, Core 0)
0x08_0000	GICR_CTLR_C1 (Ditto, Core 1)
0x0A_0000	GICR_CTLR_C2 (Ditto, Core 2)
0x0C_0000	GICR_CTLR_C3 (Ditto, Core 3)
0x0E_0000	GICR_CTLR_C4 (Ditto, Core 4)
0x10_0000	GICR_CTLR_C5 (Ditto, Core 5)
0x12_0000	GICR_CTLR_C6 (Ditto, Core 6)
0x14_0000	GICR_CTLR_C7 (Ditto, Core 7)
0x16_0000	GICDA_CTLR (Distributor Control Register A)

(Explained here)

boards/arm64/a527/avaota-a1/configs/nsh/defconfig

We set the UART0 Interrupt…

## Set the UART0 Interrupt to 34
CONFIG_16550_UART0_IRQ=34

Based on the A527 Doc…

A523 User Manual	Page 256
Interrupt Number	Interrupt Source
34	UART0

(Explained here)

§19.3 Arm64 Boot Code

arch/arm64/src/a527/a527_lowputc.S

We updated the Arm64 Boot Code for 16550 UART Driver…

We modified the UART Base Address…

// Base Address for 16550 UART
#define UART0_BASE_ADDRESS 0x02500000

QEMU was using PL011 UART. We fixed this for 16550 UART, to Wait for UART Ready (derived from NuttX A64)

/* Wait for 16550 UART to be ready to transmit
* xb: Register that contains the UART Base Address
* wt: Scratch register number */
.macro early_uart_ready xb, wt
1:
  ldrh  \wt, [\xb, #0x14] /* UART_LSR (Line Status Register) */
  tst   \wt, #0x20        /* Check THRE (TX Holding Register Empty) */
  b.eq  1b                /* Wait for the UART to be ready (THRE=1) */
.endm

UART Base Address came from the A527 Doc…

A523 User Manual	Page 1839
Module	Base Address
UART0	0x0250_0000

With these UART Registers…

A523 User Manual	Page 1839
Offset	Register
0x0000	UART_THR (Transmit Holding Register)
0x0004	UART_DLH (Divisor Latch High Register)
0x0008	UART_IIR (Interrupt Identity Register)
0x000C	UART_LCR (Line Control)

(Explained here)

Mounting the ROMFS Filesystem containing the NuttX Apps

§19.4 NuttX Start Code

arch/arm64/src/a527/a527_boot.c

At NuttX Startup: We mount the ROMFS Filesystem (Initial RAM Disk, pic above) containing the NuttX Apps…

“NuttX Apps Filesystem”

How? We safely copy the ROMFS Filesystem from the NuttX Image into the ramdisk Memory Region. This code comes from NuttX EIC7700X…

// Needed for the `aligned_data` macro
#include <nuttx/compiler.h>

// Just after Bootloader has started NuttX...
void arm64_chip_boot(void) {

  // We copy the RAM Disk
  qemu_copy_ramdisk();

  // Omitted: Other initialisation (MMU, ...)
  arm64_mmu_init(true);
  ...
}

// Copy the RAM Disk from NuttX Image to RAM Disk Region.
static void a527_copy_ramdisk(void) {
  const uint8_t aligned_data(8) header[8] = "-rom1fs-";
  const uint8_t *limit = (uint8_t *)g_idle_topstack + (256 * 1024);
  uint8_t *ramdisk_addr = NULL;
  uint8_t *addr;
  uint32_t size;

  // After Idle Stack Top, search for "-rom1fs-". This is the RAM Disk Address.
  // Limit search to 256 KB after Idle Stack Top.
  for (addr = g_idle_topstack; addr < limit; addr += 8) {
      if (memcmp(addr, header, sizeof(header)) == 0) {
        ramdisk_addr = addr;
        break;
      }
  }

  // Stop if RAM Disk is missing
  if (ramdisk_addr == NULL) {
    _err("Missing RAM Disk. Check the initrd padding.");
    PANIC();
  }

  // Read the Filesystem Size from the next 4 bytes (Big Endian)
  size = (ramdisk_addr[8] << 24) + (ramdisk_addr[9] << 16) +
         (ramdisk_addr[10] << 8) + ramdisk_addr[11] + 0x1f0;

  // Filesystem Size must be less than RAM Disk Memory Region
  if (size > (size_t)__ramdisk_size) {
    _err("RAM Disk Region too small. Increase by %lu bytes.\n", size - (size_t)__ramdisk_size);
    PANIC();
  }

  // Copy the RAM Disk from NuttX Image to RAM Disk Region.
  // __ramdisk_start overlaps with ramdisk_addr + size.
  a527_copy_overlap(__ramdisk_start, ramdisk_addr, size);
}

// Copy an overlapping memory region.  dest overlaps with src + count.
static void a527_copy_overlap(uint8_t *dest, const uint8_t *src, size_t count) {
  uint8_t *d = dest + count - 1;
  const uint8_t *s = src + count - 1;
  if (dest <= src) { _err("dest and src should overlap"); PANIC(); }
  while (count--) {
    volatile uint8_t c = *s;  // Prevent compiler optimization
    *d = c;
    d--;
    s--;
  }
} 

// RAM Disk Region is defined in Linker Script
extern uint8_t __ramdisk_start[];
extern uint8_t __ramdisk_size[];

(Previously here)

Why the aligned addresses?

// Header is aligned to 8 bytes
const uint8_t
  aligned_data(8) header[8] =
  "-rom1fs-";

// Address is also aligned to 8 bytes
for (
  addr = g_idle_topstack;
  addr < limit;
  addr += 8
) {
  // Otherwise this will hit Alignment Fault
  memcmp(addr, header, sizeof(header));
  ...
}

We align our Memory Accesses to 8 Bytes. Otherwise we’ll hit an Alignment Fault…

## Alignment Fault at `memcmp(addr, header, sizeof(header))`
default_fatal_handler:
  (IFSC/DFSC) for Data/Instruction aborts:
  alignment fault

(Explained here)

§19.5 Board Bringup Code

boards/arm64/a527/avaota-a1/src/a527_bringup.c

At Board Startup: We mount the ROMFS Filesystem (inside RAM) as /dev/ram0…

// At NuttX Startup...
int a527_bringup(void) {
  // We Mount the RAM Disk
  mount_ramdisk();
  ...
}

// Mount a RAM Disk defined in ld.script to /dev/ramX.  The RAM Disk
// contains a ROMFS filesystem with applications that can be spawned at
// runtime.
static int mount_ramdisk(void) {
  struct boardioc_romdisk_s desc;
  desc.minor    = RAMDISK_DEVICE_MINOR;
  desc.nsectors = NSECTORS((ssize_t)__ramdisk_size);
  desc.sectsize = SECTORSIZE;
  desc.image    = __ramdisk_start;

  int ret = boardctl(BOARDIOC_ROMDISK, (uintptr_t)&desc);
  if (ret < 0) {
    syslog(LOG_ERR, "Ramdisk register failed: %s\n", strerror(errno));
    syslog(LOG_ERR, "Ramdisk mountpoint /dev/ram%d\n",RAMDISK_DEVICE_MINOR);
    syslog(LOG_ERR, "Ramdisk length %lu, origin %lx\n", (ssize_t)__ramdisk_size, (uintptr_t)__ramdisk_start);
  }
  return ret;
}

// RAM Disk Definition
#define SECTORSIZE   512
#define NSECTORS(b)  (((b) + SECTORSIZE - 1) / SECTORSIZE)
#define RAMDISK_DEVICE_MINOR 0

(Explained here)

§19.6 Linker Script

boards/arm64/a527/avaota-a1/scripts/ld.script

We reserve 16 MB of RAM for the ROMFS Filesystem that will host the NuttX Apps…

/* Linker Script: We moved the Paged Pool and added the RAM Disk (16 MB) */
MEMORY {
  /* Previously: QEMU boots at 0x4028_0000 */
  dram (rwx)  : ORIGIN = 0x40800000, LENGTH = 2M

  /* Previously: QEMU Paged Memory is at 0x4028_0000 */
  /* Why? Because 0x4080_0000 + 2 MB = 0x40A0_0000   */
  pgram (rwx) : ORIGIN = 0x40A00000, LENGTH = 4M   /* w/ cache */

  /* Added the RAM Disk */
  ramdisk (rwx) : ORIGIN = 0x40E00000, LENGTH = 16M   /* w/ cache */
}

/* We'll reference these in our code */
__ramdisk_start = ORIGIN(ramdisk);
__ramdisk_size  = LENGTH(ramdisk);
__ramdisk_end   = ORIGIN(ramdisk) + LENGTH(ramdisk);

Also we moved the Paged Pool because the Boot Address has changed to 0x4080_0000.

(Explained here)

§19.7 NuttX Config

boards/arm64/a527/avaota-a1/configs/nsh/defconfig

Since we changed the Paged Memory Pool (pgram), we update ARCH_PGPOOL_PBASE and VBASE: configs/knsh/defconfig

## Physical Address of Paged Memory Pool
## Previously: QEMU Paged Memory is at 0x4028_0000
CONFIG_ARCH_PGPOOL_PBASE=0x40A00000

## Virtual Address of Paged Memory Pool
## Previously: QEMU Paged Memory is at 0x4028_0000
CONFIG_ARCH_PGPOOL_VBASE=0x40A00000

(Explained here)

NuttX QEMU declares the RAM Size as 128 MB in RAMBANK1_SIZE. We set RAM_SIZE accordingly…

## RAM Size is 128 MB
CONFIG_RAM_SIZE=134217728

(Explained here)

Based on the 16550 UART Registers above: We configured the 16550 UART and removed PL011 UART…

CONFIG_16550_ADDRWIDTH=0
CONFIG_16550_REGINCR=4
CONFIG_16550_UART0=y
CONFIG_16550_UART0_BASE=0x02500000
CONFIG_16550_UART0_CLOCK=23040000
CONFIG_16550_UART0_IRQ=125
CONFIG_16550_UART0_SERIAL_CONSOLE=y
CONFIG_16550_UART=y
CONFIG_16550_WAIT_LCR=y
CONFIG_SERIAL_UART_ARCH_MMIO=y

(Explained here)

16550_UART0_CLOCK was computed according to these instructions…

NuttX UART Debug Log shows:
  dlm = 0x00
  dll = 0x0D

We know that:
  dlm = 0x00 = (div >> 8)
  dll = 0x0D = (div & 0xFF)

Which means:
  div = 0x0D

We know that:
  baud = 115200
  div  = (uartclk + (baud << 3)) / (baud << 4)

Therefore:
  0x0D    = (uartclk + 921600) / 1843200
  uartclk = (0x0D * 1843200) - 921600
          = 23040000

arch/arm64/src/a527/a527_serial.c

QEMU was using PL011 UART. We switched the Serial Driver to 16550 UART…

// Switch from PL011 UART (QEMU) to 16550 UART
#include <nuttx/serial/uart_16550.h>

// Enable the 16550 Console UART at Startup
void arm64_earlyserialinit(void) {
  // Previously for QEMU: pl011_earlyserialinit
  u16550_earlyserialinit();
}

// Ditto but not so early
void arm64_serialinit(void) {
  // Previous for QEMU: pl011_serialinit
  u16550_serialinit();
}

(Explained here)