Assembly and Low-Level Language Fundamentals Questions
Programming and debugging at the instruction level. Covers reading and writing assembly for x86-64 and ARM (A32, Thumb, AArch64), including small hand-written routines, vector (SIMD) code and exclusive load/store atomics, registers, the stack and frame layout, prologues and epilogues, calling conventions and ABIs (System V, Microsoft x64, AAPCS), variadic calls, inline assembly and its constraints and clobbers, Cortex-M exception entry and context-switch code written in assembly, and how compilers translate and optimize source into machine code (optimization flags, inlining, tail calls, LTO, aliasing, strength reduction, virtual dispatch, memcpy lowering, stack spills). Also covers object files and linking as they affect generated code (ELF, PE/COFF and Mach-O, relocations, GOT and PLT, position-independent code, static linking, stack unwinding), the compiler backend ideas behind it (SSA, register allocation, instruction selection, peephole passes) and emitting machine code from a minimal JIT. On the debugging side: reading disassembly, using gdb and lldb for registers, frames, breakpoints and watchpoints, analyzing core dumps and stripped binaries with addr2line and build IDs, and diagnosing crashes from instruction-level state such as corrupted returns, stack smashing, ABI mismatches and optimizer-induced bugs. Debugging method in general, hardware probe tooling, malware analysis and exploit-mitigation design are covered elsewhere.
On a Cortex-M with tasks running on the process stack pointer and exceptions on the main stack pointer, walk through what the hardware stacks on exception entry, then write the PendSV handler in assembly that saves the outgoing task and restores the next one. How do you return to thread mode on the right stack?
Sample Answer
Direct answer
On exception entry the Cortex-M hardware pushes eight words onto the stack the interrupted code was using (the process stack pointer, PSP, for a task): R0, R1, R2, R3, R12, LR (the link register, the return address of the current function), PC (the program counter) and xPSR (the combined program status register: the condition flags, the Thumb-state bit and the number of the exception being handled). It does not push R4 to R11. A PendSV handler (PendSV is the pendable service call exception, one that software can trigger by setting a bit, and it is given the lowest priority so it runs only after every real interrupt has finished) therefore does the other half of the context switch by hand: it pushes R4 to R11 onto the outgoing task's stack, stores the new stack pointer in that task's control block (TCB, the small struct that records a task's saved stack pointer), loads the incoming task's saved stack pointer, pops R4 to R11, writes PSP, and returns with the special value 0xFFFFFFFD in LR. That value is not an address. It is an EXC_RETURN code that tells the hardware to return to thread mode (ordinary, non-exception code) and to unstack the remaining eight words from PSP. Bit 2 of the code selects the stack, so 0xFFFFFFFD is "thread mode, PSP" and 0xFFFFFFF9 would be "thread mode, MSP", which is how you return "on the right stack".
Said aloud in under a minute: the hardware stacks eight words (R0 to R3, R12, LR, PC, xPSR) on PSP; PendSV, at the lowest priority, pushes R4 to R11 under them and saves the resulting stack pointer in the outgoing task's control block; it loads the incoming task's stack pointer, pops R4 to R11, writes PSP, and executes bx lr with 0xFFFFFFFD; the hardware then pops the other eight words and resumes the new task. A task that has never run gets a fabricated frame so that this same path starts it. The sections after the handler walk-through test that story on a Cortex-M3 model and cover the stack-alignment padding and the floating-point variant.
What the hardware stacks, in order
The eight words are stored at ascending addresses starting from the new, lower stack pointer (the 8-byte alignment rule below can add one padding word above them):
| Offset from the stacked SP | Word |
|---|---|
| +0x00 | R0 |
| +0x04 | R1 |
| +0x08 | R2 |
| +0x0C | R3 |
| +0x10 | R12 |
| +0x14 | LR (the interrupted code's link register) |
| +0x18 | PC (the address to resume at) |
| +0x1C | xPSR (flags, Thumb bit, exception number) |
These are the registers the AAPCS (the Arm Architecture Procedure Call Standard, the calling convention that says which registers a function may overwrite) lets any called function overwrite, which is why the hardware can safely stack just those: an ordinary C function used as a handler may clobber them. R4 to R11 are callee-saved (a function that uses them must put the original values back before returning), so a C handler would restore them itself, but a context switch changes whose registers are live, so the switch code must save and restore them explicitly. The order above is the frame order documented in the ARMv7-M Architecture Reference Manual (Figure B1-3 in section B1.5.7, with the PushStack() pseudocode in B1.5.6 that stores R0 at offset 0x00 up to xPSR at 0x1C), and the run below checks it on a Cortex-M3.
The handler then runs in handler mode (the processor state used while an exception handler executes; tasks run in thread mode) on the main stack pointer (MSP), with LR holding the EXC_RETURN value, so the first thing it must do is read PSP to find the task's stack.
The context switch, run on a Cortex-M3
The complete program below (Cortex-M3, bare metal, two tasks, a SysTick timer tick that pends PendSV every 20,000 core clocks) was built with arm-none-eabi-gcc 14.2.1 and run on QEMU 10.0.13 (-M lm3s6965evb, a Cortex-M3 board model) inside a gcc:14 container on aarch64. QEMU's Cortex-M3 model implements the exception entry, stacking, EXC_RETURN and PendSV behaviour being tested, but it does not model cycle timing, so only behaviour is shown, never speed.
Reading notes for the assembly. ldr r3, =current_tcb is not a plain load: the assembler stores the address of current_tcb as a constant in a literal pool (a small table of constants placed in the code, near the instruction) and emits a load relative to the program counter; .ltorg tells the assembler to put the pool at that spot. tst lr, #4 ANDs LR with 4 only to set the flags. ite eq (if-then-else) makes the next two instructions conditional: the first runs if the flags say equal (bit 2 was 0), the second if not equal. A @ starts a comment in this assembler syntax.
switch.S, the assembly (the handlers, plus test helpers described after the output):
.syntax unified
.cpu cortex-m3
.thumb
.global PendSV_Handler
.type PendSV_Handler, %function
PendSV_Handler:
mrs r0, psp @ outgoing task's stack pointer (hardware frame is just above it)
isb
ldr r3, =current_tcb
ldr r2, [r3] @ r2 = outgoing TCB
#ifndef BROKEN_NO_R4_R11
stmdb r0!, {r4-r11} @ push the registers the hardware did not stack
#else
sub r0, r0, #32
#endif
str r0, [r2] @ save outgoing sp in its TCB
ldr r1, =next_tcb
ldr r1, [r1] @ r1 = incoming TCB
str r1, [r3] @ current_tcb = next_tcb
ldr r0, [r1] @ incoming saved sp
#ifndef BROKEN_NO_R4_R11
ldmia r0!, {r4-r11} @ pop r4-r11; r0 now points at the hardware frame
#else
add r0, r0, #32
#endif
msr psp, r0
isb
ldr lr, =0xFFFFFFFD @ EXC_RETURN: thread mode, process stack
bx lr
.ltorg
.size PendSV_Handler, .-PendSV_Handler
.global SysTick_Handler
.type SysTick_Handler, %function
SysTick_Handler:
tst lr, #4 @ EXC_RETURN bit 2: which stack was stacked onto
ite eq
mrseq r0, msp
mrsne r0, psp
mov r1, lr
b tick_c
.size SysTick_Handler, .-SysTick_Handler
.global SVC_Handler
.type SVC_Handler, %function
SVC_Handler:
tst lr, #4
ite eq
mrseq r0, msp
mrsne r0, psp
b svc_c
.size SVC_Handler, .-SVC_Handler
@ uint32_t spin_check(uint32_t base, uint32_t iters)
@ Loads r4-r11 with base+0..base+7, spins, then counts how many changed.
.macro setreg reg, n
add \reg, r0, #\n
.endm
.macro chkreg reg, n
add r2, r0, #\n
cmp \reg, r2
it ne
addne r3, r3, #1
.endm
.global spin_check
.type spin_check, %function
spin_check:
push {r4-r11, lr}
setreg r4, 0
setreg r5, 1
setreg r6, 2
setreg r7, 3
setreg r8, 4
setreg r9, 5
setreg r10, 6
setreg r11, 7
1: subs r1, r1, #1
bne 1b
movs r3, #0
chkreg r4, 0
chkreg r5, 1
chkreg r6, 2
chkreg r7, 3
chkreg r8, 4
chkreg r9, 5
chkreg r10, 6
chkreg r11, 7
mov r0, r3
pop {r4-r11, pc}
.size spin_check, .-spin_check
@ void frame_probe(void): load known values into r0-r3 and r12, then wait
@ until probe_done becomes non-zero. A tick arriving here stacks those values.
.global frame_probe
.type frame_probe, %function
frame_probe:
push {r4-r6, lr}
ldr r5, =probe_done
ldr r0, =0x10101010
ldr r1, =0x11111111
ldr r2, =0x12121212
ldr r3, =0x13131313
ldr r12, =0x1C1C1C1C
.global probe_loop
probe_loop:
ldr r6, [r5]
cmp r6, #0
beq probe_loop
.global probe_loop_end
probe_loop_end:
pop {r4-r6, pc}
.ltorg
.size frame_probe, .-frame_probe
@ void align_probe(uint32_t psp_value): run an SVC from thread mode with PSP = psp_value
.global align_probe
.type align_probe, %function
align_probe:
mov r3, lr
msr psp, r0
mrs r1, control
orr r2, r1, #2
msr control, r2
isb
svc #0
msr control, r1
isb
bx r3
.size align_probe, .-align_probe
main.c, the C side (task setup, the scheduler decision, and the checks):
#include <stdint.h>
#define REG(a) (*(volatile uint32_t *)(a))
#define UART0_DR REG(0x4000C000u)
#define SYST_CSR REG(0xE000E010u)
#define SYST_RVR REG(0xE000E014u)
#define SYST_CVR REG(0xE000E018u)
#define ICSR REG(0xE000ED04u)
#define CCR REG(0xE000ED14u)
#define SHPR3 REG(0xE000ED20u)
#define ICSR_PENDSVSET (1u << 28)
#define CCR_STKALIGN (1u << 9)
typedef struct { uint32_t *sp; } tcb_t;
tcb_t *volatile current_tcb;
tcb_t *volatile next_tcb;
volatile uint32_t probe_done;
static volatile uint32_t ticks, a_done, b_done, a_bad, b_bad;
extern uint32_t spin_check(uint32_t base, uint32_t iters);
extern void frame_probe(void);
extern void align_probe(uint32_t psp_value);
extern char probe_loop[], probe_loop_end[];
static void putc_(char c) { UART0_DR = (uint32_t)c; }
static void puts_(const char *s) { while (*s) putc_(*s++); }
static void hex(uint32_t v)
{
puts_("0x");
for (int i = 28; i >= 0; i -= 4) putc_("0123456789abcdef"[(v >> i) & 0xF]);
}
static void line(const char *name, uint32_t v) { puts_(name); hex(v); puts_("\n"); }
static void semihost_exit(int ok)
{
/* SYS_EXIT (0x18); r1 holds the reason code on a 32-bit target */
register uint32_t op __asm__("r0") = 0x18;
register uint32_t arg __asm__("r1") = ok ? 0x20026u : 0x20023u;
__asm__ volatile("bkpt 0xAB" : : "r"(op), "r"(arg) : "memory");
for (;;) {}
}
#define STACK_WORDS 128
static uint32_t stack_a[STACK_WORDS] __attribute__((aligned(8)));
static uint32_t stack_b[STACK_WORDS] __attribute__((aligned(8)));
static uint32_t boot_stack[STACK_WORDS] __attribute__((aligned(8)));
static tcb_t tcb_boot, tcb_a, tcb_b;
static void task_exit(void) { for (;;) {} }
static void task_a(void);
static void task_b(void);
static void init_task(tcb_t *t, uint32_t *stack, void (*entry)(void))
{
uint32_t *sp = stack + STACK_WORDS; /* top of stack, 8-byte aligned */
*--sp = 0x01000000u; /* xPSR: Thumb bit set */
*--sp = (uint32_t)entry & ~1u; /* PC */
*--sp = (uint32_t)task_exit; /* LR */
*--sp = 0; *--sp = 0; *--sp = 0; *--sp = 0; *--sp = 0; /* R12, R3, R2, R1, R0 */
for (int i = 0; i < 8; i++) *--sp = 0; /* R11..R4 (software-saved block) */
t->sp = sp;
}
void tick_c(uint32_t *frame, uint32_t exc_return)
{
ticks++;
if (!probe_done && frame[6] >= (uint32_t)probe_loop && frame[6] < (uint32_t)probe_loop_end) {
puts_("-- hardware frame at first tick inside frame_probe --\n");
line("EXC_RETURN = ", exc_return);
line("frame mod 8 = ", (uint32_t)frame & 7u);
line("in task_a stack = ", (uint32_t)((uintptr_t)frame >= (uintptr_t)stack_a &&
(uintptr_t)frame < (uintptr_t)(stack_a + STACK_WORDS)));
line("frame[0] R0 = ", frame[0]);
line("frame[1] R1 = ", frame[1]);
line("frame[2] R2 = ", frame[2]);
line("frame[3] R3 = ", frame[3]);
line("frame[4] R12 = ", frame[4]);
line("frame[5] LR in task_a = ", (frame[5] & ~1u) >= (uint32_t)task_a &&
(frame[5] & ~1u) < (uint32_t)task_a + 0x100);
line("frame[6] PC in probe loop = ", frame[6] >= (uint32_t)probe_loop &&
frame[6] < (uint32_t)probe_loop_end);
line("frame[7] xPSR (T bit, IPSR) = ", frame[7] & 0x010001FFu);
probe_done = 1;
}
next_tcb = (current_tcb == &tcb_a) ? &tcb_b : &tcb_a;
ICSR = ICSR_PENDSVSET; /* ask for the context switch */
}
void svc_c(uint32_t *frame)
{
uint32_t xpsr = frame[7];
line(" frame address mod 8 = ", (uint32_t)frame & 7u);
line(" xPSR bit 9 = ", (xpsr >> 9) & 1u);
}
#define ROUNDS 6
#define ITERS 4000000u
static void task_a(void)
{
frame_probe();
for (int r = 0; r < ROUNDS; r++) a_bad += spin_check(0xA0000000u, ITERS);
a_done = 1;
while (!b_done) {}
puts_("-- result --\n");
line("ticks > 4 : ", ticks > 4);
line("task_a changed registers = ", a_bad);
line("task_b changed registers = ", b_bad);
semihost_exit(a_bad == 0 && b_bad == 0 && ticks > 4);
}
static void task_b(void)
{
for (int r = 0; r < ROUNDS; r++) b_bad += spin_check(0xB0000000u, ITERS);
b_done = 1;
for (;;) {}
}
int main(void)
{
puts_("-- STKALIGN = 1, PSP 8-byte aligned --\n");
CCR |= CCR_STKALIGN;
align_probe((uint32_t)(boot_stack + STACK_WORDS));
puts_("-- STKALIGN = 1, PSP = 4 mod 8 --\n");
align_probe((uint32_t)(boot_stack + STACK_WORDS) - 4);
puts_("-- STKALIGN = 0, PSP = 4 mod 8 --\n");
CCR &= ~CCR_STKALIGN;
align_probe((uint32_t)(boot_stack + STACK_WORDS) - 4);
CCR |= CCR_STKALIGN;
init_task(&tcb_a, stack_a, task_a);
init_task(&tcb_b, stack_b, task_b);
tcb_boot.sp = boot_stack + STACK_WORDS; /* scratch area the first PendSV saves into */
current_tcb = &tcb_boot;
next_tcb = &tcb_a;
SHPR3 = (SHPR3 & 0x0000FFFFu) | (0xFFu << 16) | (0x80u << 24); /* PendSV lowest, SysTick above it */
SYST_RVR = 20000u - 1u;
SYST_CVR = 0;
SYST_CSR = 7u; /* enable, tick interrupt, core clock */
ICSR = ICSR_PENDSVSET; /* first switch: boot context -> task_a */
for (;;) {}
}
vectors.c:
#include <stdint.h>
extern int main(void);
extern uint32_t _estack, _sbss, _ebss;
extern void PendSV_Handler(void), SysTick_Handler(void), SVC_Handler(void);
void Reset_Handler(void)
{
for (uint32_t *p = &_sbss; p < &_ebss; p++) *p = 0;
main();
for (;;) {}
}
static void Default_Handler(void) { for (;;) {} }
__attribute__((section(".isr_vector"), used))
void (*const vector_table[16])(void) = {
(void (*)(void))&_estack, Reset_Handler, Default_Handler, Default_Handler,
Default_Handler, Default_Handler, Default_Handler, 0,
0, 0, 0, SVC_Handler, Default_Handler, 0, PendSV_Handler, SysTick_Handler,
};
link.ld:
MEMORY { FLASH (rx) : ORIGIN = 0x00000000, LENGTH = 256K
RAM (rwx) : ORIGIN = 0x20000000, LENGTH = 64K }
SECTIONS {
.text : { KEEP(*(.isr_vector)) *(.text*) *(.rodata*) . = ALIGN(4); } > FLASH
.bss (NOLOAD) : { . = ALIGN(4); _sbss = .; *(.bss*) *(COMMON) . = ALIGN(4); _ebss = .; } > RAM
_estack = ORIGIN(RAM) + LENGTH(RAM);
}
Build and run from the directory that holds the four files above (inside the container, after installing gcc-arm-none-eabi and qemu-system-arm with apt):
arm-none-eabi-gcc -mcpu=cortex-m3 -mthumb -O2 -Wall -Wextra -ffreestanding -nostdlib -nostartfiles -T link.ld main.c vectors.c switch.S -o ctxswitch.elf
timeout 100 qemu-system-arm -M lm3s6965evb -nographic -semihosting-config enable=on,target=native -kernel ctxswitch.elf; echo "qemu exit=$?"
Run this way, it prints the following. The output is the same on every run because the program only prints facts that do not depend on timing (the last line is the echo of the QEMU exit status, 0 meaning the program called semihosting exit with success):
-- STKALIGN = 1, PSP 8-byte aligned --
frame address mod 8 = 0x00000000
xPSR bit 9 = 0x00000000
-- STKALIGN = 1, PSP = 4 mod 8 --
frame address mod 8 = 0x00000000
xPSR bit 9 = 0x00000001
-- STKALIGN = 0, PSP = 4 mod 8 --
frame address mod 8 = 0x00000004
xPSR bit 9 = 0x00000000
-- hardware frame at first tick inside frame_probe --
EXC_RETURN = 0xfffffffd
frame mod 8 = 0x00000000
in task_a stack = 0x00000001
frame[0] R0 = 0x10101010
frame[1] R1 = 0x11111111
frame[2] R2 = 0x12121212
frame[3] R3 = 0x13131313
frame[4] R12 = 0x1c1c1c1c
frame[5] LR in task_a = 0x00000001
frame[6] PC in probe loop = 0x00000001
frame[7] xPSR (T bit, IPSR) = 0x01000000
-- result --
ticks > 4 : 0x00000001
task_a changed registers = 0x00000000
task_b changed registers = 0x00000000
qemu exit=0
Reading the handler, line by line
The handler is the block in switch.S from PendSV_Handler: to bx lr, repeated here without the test-only #ifndef branch:
PendSV_Handler:
mrs r0, psp @ outgoing task's stack pointer (hardware frame is just above it)
isb
ldr r3, =current_tcb
ldr r2, [r3] @ r2 = outgoing TCB
stmdb r0!, {r4-r11} @ push the registers the hardware did not stack
str r0, [r2] @ save outgoing sp in its TCB
ldr r1, =next_tcb
ldr r1, [r1] @ r1 = incoming TCB
str r1, [r3] @ current_tcb = next_tcb
ldr r0, [r1] @ incoming saved sp
ldmia r0!, {r4-r11} @ pop r4-r11; r0 now points at the hardware frame
msr psp, r0
isb
ldr lr, =0xFFFFFFFD @ EXC_RETURN: thread mode, process stack
bx lr
mrs r0, pspreads the outgoing task's stack pointer. The handler is on MSP, so PSP is still the task's, pointing at the eight hardware-stacked words.stmdb r0!, {r4-r11}(store multiple, decrement before, write the new address back) pushes R4 to R11 below the hardware frame. R4 ends at the lowest address. Each task's saved context is now 16 words: R4 to R11, then the hardware frame.str r0, [r2]stores that lowest address in the outgoing TCB. Saving it is the whole point: it is the only thing that identifies the task's context.current_tcb = next_tcb, thenldr r0, [r1]loads the incoming task's saved stack pointer. The scheduler decision lives in C (tick_csetsnext_tcb), so the assembly never calls C and LR stays intact. If the handler did call C it would have to push LR first and pop it into PC at the end.ldmia r0!, {r4-r11}pops the incoming task's software-saved registers, andr0now points at its hardware frame.msr psp, r0makes that the process stack pointer.ldr lr, =0xFFFFFFFDandbx lrbegin the exception return: the hardware reads the value (bit 2 picks the stack, bit 3 the mode), unstacks R0 to R3, R12, LR, PC and xPSR from PSP, and resumes the incoming task at its stacked PC.
The isb instructions (instruction synchronisation barrier, which flushes the pipeline so later instructions see the effect of earlier register writes) copy the pattern in FreeRTOS's Cortex-M3 xPortPendSVHandler, which has one after mrs r0, psp and one after msr psp, r0. They are defensive rather than load-bearing: Memfault's context-switching write-up says of the first one: "Technically it doesn't really serve any purpose here and shouldn't be required at all", whereas the same write-up says of the isb that FreeRTOS places after its msr basepri write (this demo does not use BASEPRI): "When lowering the effective execution level, an isb instruction is required for the new priority to be visible for future instructions." The demo also places one after writing the CONTROL register in align_probe, because a CONTROL change that selects PSP should be followed by an isb before code relies on the new stack. A dsb is the stronger sibling: it waits until earlier memory writes, such as the ICSR write that pends PendSV, have completed.
Starting the first task, and a task that yields
A task that has never run has no hardware frame, so init_task builds a fake one: xPSR with only the Thumb bit set (0x01000000), PC set to the entry function with bit 0 cleared, LR set to a function that loops (what a task that returns would fall into), and zeros for the rest, with eight more zero words below it for R4 to R11. The handler cannot tell that frame from a real one. For the very first switch the demo pends PendSV from main (running on MSP) with current_tcb pointing at a scratch "boot" TCB, so the handler has somewhere harmless to save into. The frame the hardware stacked onto MSP for that first entry is simply abandoned, which costs 32 bytes of MSP and is fine because main never returns. The handler's first mrs r0, psp and stmdb also need PSP to point at valid RAM before the first switch: in the demo the last align_probe call happens to leave PSP inside boot_stack, but a real port should write PSP explicitly (for example msr psp with the top of a scratch stack) before pending the first PendSV, because the architecture leaves the reset value of PSP UNKNOWN, so a PSP that was never written would send the stmdb to wherever it happens to point. On the QEMU model used here PSP starts at 0, the stmdb writes at 0xFFFFFFE0 and above, and nothing stops it (observed: the program still passes), which hides the mistake in an emulator run; that tolerance is a property of the emulator, not evidence the code is safe on a real part. A task that wants to give up the CPU voluntarily sets next_tcb and writes ICSR = 1 << 28 (PENDSVSET); code that must not run another line before the switch happens follows that write with dsb and isb (a data synchronisation barrier waits for the write to complete).
The PendSV priority matters. It is set to the lowest value in the demo (SHPR3, the system handler priority register 3, bits 23:16 = 0xFF; bigger number means lower priority) and SysTick (0x80, bits 31:24) above it. If PendSV could preempt another interrupt handler, the exception return would land in that handler, not in a task, and the 0xFFFFFFFD forced into LR would be an illegal return to thread mode while the other handler is still active, which the core reports as a UsageFault (the INVPC flag) unless CCR.NONBASETHRDENA is set. Lowest priority guarantees that PendSV only ever interrupts thread-mode task code, which is what makes the hard-coded value valid.
Reading the test helpers
The helpers are what let the program check itself.
SysTick_Handleris the timer interrupt. LR holds the EXC_RETURN value, and its bit 2 says which stack the hardware used for the frame:tst lr, #4followed byite eqpicksmrseq r0, mspwhen the bit is 0 (the interrupted code was on MSP) andmrsne r0, pspwhen it is 1 (a task on PSP).r0then points at the stacked frame,mov r1, lrpasses EXC_RETURN along, andb tick_cjumps to the C function with those as its two argumentsframeandexc_return.SVC_Handlerdoes the same pick for thesvcinstruction used byalign_probe.spin_check(base, iters)is the register-preservation test.setregloads R4 to R11 withbase + 0tobase + 7(the macro expands toadd r4, r0, #0and so on, sincer0isbase). The1: subs r1, r1, #1; bne 1bloop burnsitersiterations, which is long enough for several SysTick ticks, and therefore several task switches, to land in the middle. Thenchkregrecomputes each expected value intor2, compares it with the register, and adds 1 to the counter inr3if they differ; the function returns that count. With a correct handler no switch changes a task's R4 to R11, so the count is 0.frame_probeloads known values into R0 to R3 and R12 and loops untilprobe_doneis set. The first tick that lands inside the loop stacks exactly those values, sotick_ccan read them back out of the frame and print them, which is how the table of frame positions is checked against hardware.align_probe(psp_value)sets PSP to a chosen value, sets CONTROL bit 1 (orr r2, r1, #2thenmsr control, r2) so thread code uses PSP, executessvc #0so the hardware stacks a frame on that PSP, and then restores CONTROL.svc_cprints the frame address modulo 8 and xPSR bit 9, which is how the three alignment cases below are observed.
Proof that the hardware frame is what the table says, and that the save is required
The first block of output comes from the SysTick handler catching the first tick inside frame_probe, a loop that has put known values into R0 to R3 and R12. SysTick_Handler tests bit 2 of LR to pick MSP or PSP, then passes the frame pointer to C. The values read back at frame[0] to frame[4] are exactly the planted ones, frame[6] is an address inside the probe loop, and frame[7] has the Thumb bit set. The EXC_RETURN seen by SysTick is 0xfffffffd, so the interrupted code was a task on PSP. The result block shows both tasks, which constantly hold distinct patterns in R4 to R11 while being preempted by the tick, finishing with no register changed.
Removing the R4 to R11 save and restore (building with -DBROKEN_NO_R4_R11, which keeps the stack pointer arithmetic but drops the register copies) makes the same harness report a nonzero count of changed registers for both tasks, and the program no longer exits cleanly. The exact count depends on where each tick lands, so it differs from run to run: it is nonzero in almost every run, task_b changed registers is usually one higher than task_a, other values are possible, and QEMU exits with status 1 (the checks failed). Do not rely on any particular number. One sample run printed (the result block only):
-- result --
ticks > 4 : 0x00000001
task_a changed registers = 0x00000020
task_b changed registers = 0x00000021
Each task is then reading the other task's R4 to R11.
8-byte stack alignment
The AAPCS requires the stack to be 8-byte aligned at public interfaces (calls between separately compiled functions), and the core can enforce this on exception entry. STKALIGN is the name of the setting: when the CCR.STKALIGN bit (bit 9 of the configuration control register at 0xE000ED14) is set, hardware inserts one padding word if the stack pointer is only 4-byte aligned, and records that in bit 9 of the stacked xPSR so the exception return knows to skip the pad. The first output block shows all three cases on the M3 model: with STKALIGN set and an aligned PSP the frame is aligned and xPSR bit 9 is 0; with STKALIGN set and PSP at 4 mod 8 the frame address is still aligned and xPSR bit 9 is 1; with STKALIGN clear and PSP at 4 mod 8 the frame is left at 4 mod 8 and bit 9 is 0. Consequences for the handler: never assume the hardware frame sits exactly 32 bytes above the saved stack pointer unless you know STKALIGN and the stack alignment; take the address from PSP as the handler does. Keeping every task stack 8-byte aligned and the initial sp a multiple of 8, as init_task does, avoids the pad entirely. The architecture leaves the reset value of STKALIGN implementation defined (Arm recommends 1); the QEMU Cortex-M3 model reads back 0x00000200 at start (STKALIGN set), so the demo's explicit CCR |= CCR_STKALIGN is a safeguard, and it is the clearing in the third case that makes the output differ.
Floating-point cores (Cortex-M4F and M7)
The demo core has no FPU, so the FP case is a variant (the floating-point registers are named s0 to s31, 32-bit each), assembled with arm-none-eabi-gcc -c -mcpu=cortex-m4 -mthumb -mfloat-abi=hard -mfpu=fpv4-sp-d16 and disassembled, but not executed (QEMU's M3 board has no FPU):
.syntax unified
.cpu cortex-m4
.fpu fpv4-sp-d16
.thumb
.global PendSV_Handler
.type PendSV_Handler, %function
PendSV_Handler:
mrs r0, psp
isb
ldr r3, =current_tcb
ldr r2, [r3]
tst lr, #0x10 @ EXC_RETURN bit 4 = 0 means an FP frame was stacked
it eq
vstmdbeq r0!, {s16-s31} @ save the FP registers the hardware did not stack
stmdb r0!, {r4-r11, lr} @ keep EXC_RETURN with the task so the FP flag travels with it
str r0, [r2]
ldr r1, =next_tcb
ldr r1, [r1]
str r1, [r3]
ldr r0, [r1]
ldmia r0!, {r4-r11, lr}
tst lr, #0x10
it eq
vldmiaeq r0!, {s16-s31}
msr psp, r0
isb
bx lr
.ltorg
When a task has used the FPU, the exception frame is extended: after the eight basic words the hardware also stacks the caller-saved FP registers (the first sixteen single-precision registers and FPSCR). Lazy stacking is controlled by the LSPEN bit of FPCCR (the floating-point context control register, an FPU setting in the system control space). When it is on, the hardware reserves space on the stack for s0 to s15 and FPSCR (the floating-point status and control register) in the frame, so the extended frame has the same size with or without lazy stacking (the eight basic words, s0 to s15, FPSCR and one reserved word, 26 words or 104 bytes in all), but it does not copy the values. LSPEN, and ASPEN which makes the first FP instruction set CONTROL.FPCA, both reset to 1 (ARMv7-M Architecture Reference Manual, FPCCR register description), as Memfault's write-up also states. Only if a floating-point instruction executes while inside the exception does the hardware then store them into the reserved space. The saving is that interrupts that never touch the FPU never pay for 17 words of FP state. In the handler above, the first FP instruction, vstmdbeq, is what triggers that deferred copy, so by the time it runs the extended frame is complete. The callee-saved FP registers (16 to 31) are never stacked by hardware, so the handler must save them, but only when the task has an FP frame. EXC_RETURN says which: 0xFFFFFFFD (bit 4 set) means a basic frame, 0xFFFFFFED (bit 4 clear) means an extended frame on PSP, and the handler tests bit 4 with tst lr, #0x10. The full set of EXC_RETURN values is 0xFFFFFFF1 handler/MSP, 0xFFFFFFF9 thread/MSP, 0xFFFFFFFD thread/PSP, and the FP forms 0xFFFFFFE1, 0xFFFFFFE9, 0xFFFFFFED. Because the FP flag is per switch, the FP handler stores LR together with R4 to R11 so each task's own EXC_RETURN is restored with it (init_task would therefore also put 0xFFFFFFFD in the saved-LR slot of a task that has never run), and it restores the 16 FP registers (if any) after R4 to R11 and before writing PSP.
Common mistakes
- Forgetting that the handler runs on MSP, then using
spwhere it meant PSP. - Saving R4 to R11 but not storing the new stack pointer back into the outgoing TCB, or loading the incoming one before storing the outgoing one.
- Giving PendSV a higher priority than an interrupt that can pend it or that a task depends on, so it preempts a handler.
- Building the initial frame with a PC whose bit 0 is set (the architecture makes an exception return to a PC that is not halfword aligned UNPREDICTABLE, which is why
init_taskclears bit 0), or with an xPSR that lacks the Thumb bit; the second makes the exception return raise a UsageFault, because the core would run in ARM state, which Cortex-M cannot. - Misaligned task stacks, or hard-coding the 32-byte frame size.
- Using the hardware-saved values to infer what a task "was doing" without allowing for the STKALIGN pad.
Write a GCC/Clang inline assembly function uint64_t read_tsc(void) that reads the x86 time stamp counter. Explain what the compiler needs to be told for the statement to be correct, and what can go wrong if you read the counter without any ordering around it.
Sample Answer
Direct answer
rdtsc (read time-stamp counter) puts the 64-bit counter into two registers: low half in eax, high half in edx. The inline asm statement therefore needs two outputs bound to exactly those registers ("=a" and "=d": a constraint is the letter in quotes that tells the compiler where an operand must live, a meaning eax/rax, d meaning edx/rdx, and = meaning the asm writes it), must be volatile so the compiler does not delete or merge it, and needs a "memory" clobber (a declaration that the asm touches things the operand list does not mention) if it is used to bracket other code, so the compiler does not move loads and stores across it. Separately, the processor executes instructions out of order (it may start later instructions before earlier ones finish) and can therefore run rdtsc earlier or later than neighbouring instructions: Intel documents RDTSC as not a serializing instruction (one that waits for all earlier instructions to finish and blocks later ones from starting), so a measurement needs lfence around the read (or rdtscp, which waits for earlier instructions but still lets later ones start early).
The function and its ordered variants
Compiled with gcc -O2 -Wall -Wextra (GCC 14.4, x86-64, linux/amd64 container run under emulation on an arm64 host):
#include <stdint.h>
#include <stdio.h>
/* Correct minimal form: two outputs, volatile. */
static inline uint64_t read_tsc(void) {
uint32_t lo, hi;
__asm__ volatile("rdtsc" : "=a"(lo), "=d"(hi));
return ((uint64_t)hi << 32) | lo;
}
/* Ordered form: lfence keeps earlier instructions from finishing after the
read, and the second lfence keeps later ones from starting before it.
The "memory" clobber stops the compiler moving loads and stores across it. */
static inline uint64_t read_tsc_ordered(void) {
uint32_t lo, hi;
__asm__ volatile("lfence\n\trdtsc\n\tlfence" : "=a"(lo), "=d"(hi) : : "memory");
return ((uint64_t)hi << 32) | lo;
}
/* rdtscp also returns IA32_TSC_AUX in ecx, so ecx must be declared as an output. */
static inline uint64_t read_tscp_ordered(uint32_t *aux) {
uint32_t lo, hi, c;
__asm__ volatile("rdtscp\n\tlfence" : "=a"(lo), "=d"(hi), "=c"(c) : : "memory");
*aux = c;
return ((uint64_t)hi << 32) | lo;
}
int main(void) {
uint32_t aux;
uint64_t prev = read_tsc_ordered();
int backwards = 0;
for (int i = 0; i < 100000; i++) {
uint64_t now = (i & 1) ? read_tsc_ordered() : read_tscp_ordered(&aux);
if (now < prev) backwards++;
prev = now;
}
printf("reads that went backwards on one thread: %d\n", backwards);
return 0;
}
Output of that program under emulation (the check is that one thread never sees the counter go backwards across 100000 ordered reads, so no timing value is shown):
reads that went backwards on one thread: 0
An emulated counter does not reproduce the processor's out-of-order execution, so this run shows only that the program works and that a decreasing read would be counted (changing the test to force one prints 1); it is not evidence about hardware ordering.
What each piece tells the compiler:
"=a"(lo), "=d"(hi): the results appear ineaxandedx, so the compiler knows those two registers are overwritten and where to find the values.volatile: the statement has an effect the compiler cannot see (reading a changing counter), so it may not be removed or treated as a pure function."memory": the statement may read or write memory in ways the compiler cannot see, so it must finish earlier memory accesses first and reload cached values afterwards. It is a compiler-only barrier and emits no instruction."=c"(c)on therdtscpform: that instruction also writesecx(with the low 32 bits of IA32_TSC_AUX, a model-specific register whose value the operating system sets, commonly to identify the CPU the code is running on), so leaving it out would silently corrupt whatever the compiler kept inecx.
What goes wrong, executed
These functions from one file, compiled with gcc -O2 -S -masm=intel pitfalls.c on the same platform (the listing below is that output, trimmed as described under it):
#include <stdint.h>
static inline void discard_plain(void) { uint32_t lo, hi; __asm__("rdtsc" : "=a"(lo), "=d"(hi)); }
static inline void discard_volatile(void) { uint32_t lo, hi; __asm__ volatile("rdtsc" : "=a"(lo), "=d"(hi)); }
void flush_plain(void) { discard_plain(); }
void flush_volatile(void) { discard_volatile(); }
/* "=A" on x86-64 means rax OR rdx, not the pair edx:eax. */
uint64_t read_bad_constraint(void) {
uint64_t t;
__asm__ volatile("rdtsc" : "=A"(t));
return t;
}
static inline uint64_t tsc_no_clobber(void) {
uint32_t lo, hi;
__asm__ volatile("rdtsc" : "=a"(lo), "=d"(hi));
return ((uint64_t)hi << 32) | lo;
}
static inline uint64_t tsc_clobber(void) {
uint32_t lo, hi;
__asm__ volatile("rdtsc" : "=a"(lo), "=d"(hi) : : "memory");
return ((uint64_t)hi << 32) | lo;
}
/* The work between the two reads is *p + *p * 3. */
int work_no_clobber(const int *p, uint64_t *t0, uint64_t *t1) {
int v = *p * 3;
*t0 = tsc_no_clobber();
int w = *p + v;
*t1 = tsc_no_clobber();
return w;
}
int work_clobber(const int *p, uint64_t *t0, uint64_t *t1) {
int v = *p * 3;
*t0 = tsc_clobber();
int w = *p + v;
*t1 = tsc_clobber();
return w;
}
Listing, in Intel syntax (gcc -O2 -S -masm=intel; comments, .cfi lines, the local .LFB/.LFE labels and other directives removed, otherwise as emitted). In Intel syntax the destination comes first: mov rax, rdi copies rdi into rax, and DWORD PTR [rax] is a 32-bit memory operand. GCC's default output is AT&T syntax, which reads the other way round (source first, % before register names, $ before constants, a size suffix on the mnemonic): the same line is movq %rdi, %rax, and lea eax, [0+rcx*4] is leal 0(,%rcx,4), %eax. Every rdtsc shows in the listing as written, with the compiler's own #APP/#NO_APP markers removed.
flush_plain:
ret
flush_volatile:
rdtsc
ret
read_bad_constraint:
rdtsc
ret
work_no_clobber:
mov rax, rdi
mov rdi, rdx
mov ecx, DWORD PTR [rax]
rdtsc
sal rdx, 32
mov eax, eax
or rdx, rax
mov QWORD PTR [rsi], rdx
rdtsc
sal rdx, 32
mov eax, eax
or rdx, rax
lea eax, [0+rcx*4]
mov QWORD PTR [rdi], rdx
ret
work_clobber:
mov eax, DWORD PTR [rdi]
mov r8, rdx
lea ecx, [rax+rax*2]
rdtsc
sal rdx, 32
mov eax, eax
add ecx, DWORD PTR [rdi]
or rdx, rax
mov QWORD PTR [rsi], rdx
rdtsc
sal rdx, 32
mov eax, eax
or rdx, rax
mov eax, ecx
mov QWORD PTR [r8], rdx
ret
Reading the listing:
flush_plaincompiles to a bareret: withoutvolatile, an asm whose outputs are unused is deleted.flush_volatilekeeps therdtsc.read_bad_constraint: on x86-64 the"=A"constraint meansraxorrdx, not the pair. The compiler emitsrdtscand returnsrax, so the upper 32 bits come back as zero and the value is silently truncated.work_no_clobberis the counter-intuitive one. Read it in order. The pointer arguments are moved out of the way (mov rax, rdi,mov rdi, rdx), then the only load of*p(mov ecx, DWORD PTR [rax]) executes before the firstrdtsc. After the firstrdtsccome three instructions that build the 64-bit result (sal rdx, 32shifts the high half up,mov eax, eaxclears the upper 32 bits ofrax,or rdx, raxmerges the halves), then the store oft0. The secondrdtscfollows immediately. The arithmetic that depends on*p,lea eax, [0+rcx*4](*p + *p * 3simplified to4 * *p), appears after the secondrdtsc. So the "timed" region contains no work at all, only the bookkeeping for the first result: the compiler was free to move the arithmetic out because nothing told it the asm statements depend on memory or on the work.- In
work_clobberthe"memory"clobber forces the second load of*pto be done between the reads:add ecx, DWORD PTR [rdi]sits between the twordtscinstructions. The multiply*p * 3(lea ecx, [rax+rax*2]) is still computed before the first read: pure register arithmetic is not constrained by a memory clobber. To time a computation reliably, make its result feed something the asm depends on, or keep the code under test in a separate non-inlined function.
The hardware ordering problem
Even with the compiler held in order, the out-of-order processor may run rdtsc before earlier instructions finish or after later ones start. The Intel manual text for RDTSC says it is not a serializing instruction and does not necessarily wait until all previous instructions have executed, and recommends lfence immediately before it to wait for earlier instructions and loads, with mfence; lfence when earlier stores must also be globally visible, and lfence immediately after it to stop later instructions beginning early. RDTSCP waits for all previous instructions and loads, but not for previous stores, and later instructions may start before the read. So the lfence; rdtsc; lfence in read_tsc_ordered follows that guidance. Without any ordering, a short measured sequence can appear shorter or longer than it really was, or even negative when the two reads execute out of order relative to the work.
Two further limits matter for the interpretation: the counter counts at a rate that is not guaranteed to match the core's current clock speed, so convert with a calibrated frequency rather than treating it as core cycles, and on a machine with several cores or a migrating thread, compare reads only from the same core (the IA32_TSC_AUX value returned by rdtscp lets you detect a migration).
The Cortex-M version
The same measurement problem exists on a Cortex-M microcontroller (Arm, Thumb instruction set), where the answer to "what must the asm say" is different. The cycle counter is the DWT (data watchpoint and trace) unit's CYCCNT register. It is memory-mapped, so no asm is needed to read it: a volatile 32-bit load is the read. CMSIS (Arm's core header) puts DWT at 0xE0001000 with CTRL at offset 0 and CYCCNT at offset 4, uses bit 0 of DWT CTRL (CYCCNTENA) to start counting, and bit 24 (TRCENA) of the CoreDebug DEMCR register to switch the DWT block on. Asm only adds control over ordering: an isb (instruction synchronisation barrier) makes sure earlier register writes (such as enabling the counter) have taken effect and the read is not performed early, and the "memory" clobber stops the compiler from moving the work around it.
Compiled with arm-none-eabi-gcc 14.2.1 -mcpu=cortex-m3 -mthumb -O2 -c:
#include <stdint.h>
#define DEMCR (*(volatile uint32_t *)0xE000EDFCu) /* CoreDebug DEMCR, bit 24 = TRCENA */
#define DWT_CTRL (*(volatile uint32_t *)0xE0001000u) /* bit 0 = CYCCNTENA */
#define DWT_CYCCNT (*(volatile uint32_t *)0xE0001004u)
static inline void cycle_counter_init(void) {
DEMCR |= (1u << 24); /* enable the DWT/ITM block */
DWT_CYCCNT = 0;
DWT_CTRL |= 1u; /* start counting core clock cycles */
}
static inline uint32_t cycles_now(void) {
return DWT_CYCCNT; /* one volatile 32-bit load is the whole read */
}
/* Only needed to stop the CPU's own pipeline reordering the read relative to the work. */
static inline uint32_t cycles_now_ordered(void) {
uint32_t c;
__asm__ volatile("isb" : : : "memory");
c = DWT_CYCCNT;
__asm__ volatile("isb" : : : "memory");
return c;
}
uint32_t measure(void (*fn)(void)) {
uint32_t t0 = cycles_now_ordered();
fn();
uint32_t t1 = cycles_now_ordered();
return t1 - t0; /* unsigned subtraction is correct across one wrap */
}
void init(void) { cycle_counter_init(); }
objdump -d of measure shows the two isb pairs around a single ldr r5, [r4, #4] (CYCCNT is at offset 4 from 0xe0001000) before and after the call through blx r0:
2: f3bf 8f6f isb sy
6: 4c06 ldr r4, [pc, #24] @ (20 <measure+0x20>)
8: 6865 ldr r5, [r4, #4]
a: f3bf 8f6f isb sy
e: 4780 blx r0
10: f3bf 8f6f isb sy
14: 6860 ldr r0, [r4, #4]
16: f3bf 8f6f isb sy
1a: 1b40 subs r0, r0, r5
The isb sy lines are the instruction synchronisation barriers, the ldr r5, [r4, #4] and ldr r0, [r4, #4] are the two reads of CYCCNT (r4 holds 0xe0001000), and blx r0 calls fn. This is the measure function from objdump -d of the object file, trimmed: the file-format and section header lines, the 00000000 <measure>: function label, the push {r3, r4, r5, lr} line before the first isb, the pop {r3, r4, r5, pc} line after the subs, the padding nop and the literal-pool word .word 0xe0001000 at the end of the function, and the whole second function init are omitted; the bytes, addresses and the @ (20 <measure+0x20>) comment on the ldr r4, [pc, #24] line are objdump's own (the comment notes the literal-pool word that line loads). CYCCNT is 32 bits wide, so it wraps after 2^32 cycles: 42.9 s at 100 MHz and 59.7 s at 72 MHz. Unsigned subtraction handles one wrap, as in measure, but not two. Check that your core implements the DWT cycle counter before relying on it; it is optional and not present on every Cortex-M.
When to use each
Prefer rdtsc with the ordering fences for short x86 measurements, a plain volatile load of CYCCNT on Cortex-M, and a library timer (clock_gettime) when you need wall-clock seconds, since it handles frequency and migration for you.
A bug disappears when the code is compiled without optimization, but it crashes with -O2 or -O3. How would you investigate whether the root cause is undefined behavior, instruction reordering, or an optimizer transformation, and what evidence would you collect at the assembly level to support your conclusion?
Sample Answer
Direct answer
Work through it in a fixed order, cheapest evidence first: (1) confirm the bug is the optimizer's doing by bisecting flags and files, (2) run the sanitizers (UBSan for undefined behaviour, TSan for data races) on the failing build, (3) diff the assembly of the one suspect function between the working and failing builds, and (4) prove the cause by making the suspected construct well-defined and watching the failure disappear at -O2. "Works at -O0, breaks at -O2" almost always means the source contains undefined behaviour (UB: code the language standard gives no meaning, so the compiler may assume it never happens) or a missing synchronization, not a compiler bug. Compiler bugs exist but are the last hypothesis.
Step 1: narrow it
- Build each optimization level (
-O0,-O1,-O2,-O3) and note the first level that fails. The optimizer passes that turn on at that level are the suspects. - Put the file-by-file bisection first: compile all files at
-O0except one, and move the-O2file until the failure appears. Then do it per function with__attribute__((optimize("O0")))or__attribute__((noinline))on the suspect. - Try the targeted switches, each of which turns off one assumption the optimizer makes. Let the symptom pick which to try first:
| Symptom | Switch to try | What it turns off |
|---|---|---|
| Arithmetic gives a wrong result after a value passes the integer limit, or an overflow check vanishes | -fwrapv | the assumption that signed overflow never happens (it wraps instead) |
| A value read through a pointer is stale after a write through a pointer of another type | -fno-strict-aliasing | the assumption that pointers to different types never point at the same memory (strict aliasing) |
| A null check disappears after the pointer was already dereferenced | -fno-delete-null-pointer-checks | the assumption that a dereferenced pointer is not null |
| A loop never sees a change made by another thread | none: go to ThreadSanitizer (step 2) | n/a: this is a data race, not a switch |
If one cures it, the bug is a source-level assumption in that category. The switches are for diagnosis; the repair is in the source (step 4).
Step 2: sanitizers
Run the failing build under -fsanitize=undefined and -fsanitize=thread (and address for memory errors). A sanitizer is a compiler-inserted runtime checker: it reports the source line directly, which is better evidence than assembly. Run natively on AArch64 with GCC 14.4, this program (next_slot.c) intends to return -1 when the increment wraps:
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
/* Intended: return -1 when the increment wraps. Signed overflow is undefined
behaviour, so the optimizer may assume (i + 1) is greater than i and drop
the test. */
__attribute__((noinline)) int next_slot(int i)
{
int next = i + 1;
if (next < i)
return -1;
return next;
}
int main(int argc, char **argv)
{
int start = argc > 1 ? atoi(argv[1]) : INT_MAX;
printf("next_slot(%d) = %d\n", start, next_slot(start));
return 0;
}
Built and run with the default input INT_MAX (the largest int, 2147483647). Each line below is a build command followed by what the run printed; the ... stands for -o ns next_slot.c and the -> marks the printed output (the UBSan message goes to standard error, then the program's own line follows it):
gcc -O0 -o ns next_slot.c -> next_slot(2147483647) = -1
gcc -O1 -o ns next_slot.c -> next_slot(2147483647) = -2147483648
gcc -O2 -o ns next_slot.c -> next_slot(2147483647) = -2147483648
gcc -O2 -fsanitize=undefined ... -> next_slot.c:10:9: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
next_slot(2147483647) = -1
gcc -O2 -fwrapv ... -> next_slot(2147483647) = -1
The sanitizer names the line; -fwrapv makes the failure vanish at -O2. That pair (the sanitizer points at the line, the flag that defines the behaviour cures it) is the proof of cause. Note that the UBSan build also prints -1: instrumentation changes the generated code and can hide the bug, so it diagnoses but does not reproduce.
Step 3: assembly evidence
Compare the suspect function in the two builds. The listing is AArch64 (w0 is the 32-bit argument and return register, sp the stack pointer; str and ldr store and load; the destination register comes first). gcc -O0 -S (only next_slot is shown, main and the format-string data are omitted; assembler directives and the .LFB / .LFE labels are removed; the # comment on the bge line was added by hand):
next_slot:
sub sp, sp, #32
str w0, [sp, 12]
ldr w0, [sp, 12]
add w0, w0, 1
str w0, [sp, 28]
ldr w1, [sp, 28]
ldr w0, [sp, 12]
cmp w1, w0
bge .L2 # the test next < i is present
mov w0, -1
b .L3
.L2:
ldr w0, [sp, 28]
.L3:
add sp, sp, 32
ret
gcc -O2 -S (same omissions: only next_slot, directives and .LFB / .LFE labels removed):
next_slot:
add w0, w0, 1
ret
The comparison, the -1 and the branch are gone. That is the evidence an optimizer-level cause leaves: a check that exists in the source has no instruction in the optimized build. The same pair on x86-64 (Intel syntax, destination first, gcc -O0 then gcc -O2, GCC 14.4 in a linux/amd64 container) shows the same disappearance. At -O0 it has add eax, 1, cmp eax, DWORD PTR [rbp-20], jge .L2 and a mov eax, -1 for the overflow path; at -O2 the whole function is lea eax, [rdi+1] then ret, so next_slot(2147483647) prints -2147483648 there as well. The reason is that next < i can only be true if i + 1 overflowed, which is UB, so the compiler assumes it is false and deletes the branch. The fix is to test before the addition, for example if (i == INT_MAX) return -1;.
A different cause: reordering and hoisting
The same symptom has a second, unrelated source: an unsynchronized shared variable. The optimizer's transformation here is hoisting (moving a computation or load out of a loop because it expects the value not to change) and, in general, reordering of memory accesses. The three suspects in the question are told apart by evidence: undefined behaviour that lets the compiler delete a test leaves a missing instruction (the next_slot case, and the sanitizer names the line); a data race that lets it hoist a load leaves one load before a loop that never reloads (this case, and ThreadSanitizer names the race); and an optimizer bug leaves neither, with the sanitizers silent and the source clean. This program (spin.c) waits for a flag set by another thread:
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
int done; /* plain int: no volatile, no atomic */
static void *worker(void *arg)
{
(void)arg;
usleep(100000);
done = 1;
return NULL;
}
int main(void)
{
pthread_t t;
pthread_create(&t, NULL, worker, NULL);
while (!done)
;
pthread_join(t, NULL);
puts("worker finished");
return 0;
}
Built with gcc -O0 -o spin0 spin.c -lpthread, it prints worker finished. Built with -O2, it never finishes (killed by timeout 5, exit status 124). The -O2 listing of main shows the reason (the whole function from gcc -O2 -S, directives and the .LFB13 / .LFE13 labels removed; AArch64; adrp and :lo12:.LANCHOR0 together form the address of the global done, .LANCHOR0 being a compiler-generated label for the block holding it, and cbz w0, .L6 means branch if w0 is zero):
main:
stp x29, x30, [sp, -32]!
adrp x2, worker
mov x3, 0
mov x29, sp
add x2, x2, :lo12:worker
add x0, sp, 24
mov x1, 0
bl pthread_create
adrp x0, .LANCHOR0
ldr w0, [x0, #:lo12:.LANCHOR0]
cbz w0, .L6
ldr x0, [sp, 24]
mov x1, 0
bl pthread_join
adrp x0, .LC0
add x0, x0, :lo12:.LC0
bl puts
mov w0, 0
ldp x29, x30, [sp], 32
ret
.L6:
b .L6
Read it in order: the program starts the thread, loads done once into w0 (the ldr right after adrp x0, .LANCHOR0), and if the value is zero (cbz) jumps to .L6; if it is not zero it falls through to pthread_join, puts and the return. .L6 contains one instruction, b .L6, a branch to itself: nothing in that loop reads memory again, so it cannot see the worker's write. On x86-64 (Intel syntax) the same shape is mov eax, DWORD PTR done[rip], test eax, eax, je .L6 and .L6: jmp .L6; compiled and run in a linux/amd64 container it also hangs at -O2.
The compiler is allowed to read done once because a plain int read by one thread while another writes it is a data race (two threads access the same memory, at least one writes, with no synchronization), which is UB, and a single-threaded reading of the loop shows nothing changes done. Built with -O1 -g -fsanitize=thread (ThreadSanitizer, TSan, the compiler's data-race detector), the program prints the report below, abridged: the ================== delimiter lines, the blank lines, the stack frames inside the ThreadSanitizer runtime, the thread-creation stack, the 'As if synchronized via sleep' block, the (pid=...) suffix, source-file paths, (binary+offset) suffixes and the trailing address on the Location line are removed, and the addresses are those of this build:
WARNING: ThreadSanitizer: data race
Write of size 4 at 0x00000042007c by thread T1:
#0 worker spin.c:11
Previous read of size 4 at 0x00000042007c by main thread:
#0 main spin.c:19
Location is global 'done' of size 4
SUMMARY: ThreadSanitizer: data race spin.c:11 in worker
The report reads: thread T1 (the worker) wrote the 4-byte global done at line 11 while the main thread had read it at line 19, with no synchronization between them. Those two lines are the race. The fix is atomic_int (an integer type from <stdatomic.h> whose reads and writes are indivisible and synchronize between threads) with an atomic load in the loop, or a condition variable, not volatile: volatile forces the load to be repeated, but the C language does not treat it as synchronization, so the race remains and it orders no other data. With atomic_int done; and while (!atomic_load(&done)), the -O2 build finishes and prints worker finished, and its listing has the load inside the loop: .L5: ldar w0, [x1] then cbz w0, .L5 (ldar is an AArch64 load-acquire).
Step 4: prove the fix
- Apply the source-level fix, not a flag. The flags (
-fwrapv) are diagnostics here, not the repair. - Rebuild at
-O2and-O3and re-run the failing case; add the case as a regression test that runs in the optimized configuration. - Keep UBSan and TSan builds in CI so the same class cannot return.
If the sanitizers are silent and the bisection points at one function, the remaining suspects are strict-aliasing violations, uninitialized variables (their values differ by build) and, last, a compiler bug: reduce the case, check a different compiler version, and only then report it.
An example of the strict-aliasing case (the rule that an object may be accessed only through a pointer of its own type, so the compiler may assume a float * store cannot change an int):
#include <stdio.h>
__attribute__((noinline)) int store_both(int *i, float *f)
{
*i = 1;
*f = 2.0f; /* a float store: the compiler assumes it cannot change *i */
return *i;
}
int main(void)
{
int x = 0;
printf("%d\n", store_both(&x, (float *)&x));
return 0;
}
Run natively on AArch64 with GCC 14.4, -O0 prints 1073741824 (the bit pattern of 2.0f, 0x40000000, which is what is really in x), -O2 prints 1, and -O2 -fno-strict-aliasing prints 1073741824 again. The -O2 listing shows why: mov w0, 1, str w0, [x2] and str s31, [x1] store the two values, and the function returns w0, which still holds 1; *i is never reloaded after the float store. The same evidence pattern as before applies: a load that exists in the source has no instruction in the optimized build, and the one switch that restores the assumption cures it. An uninitialized local behaves differently by build because the compiler may put it in any register or stack slot; -Wall -Wextra and -fsanitize=memory (Clang) or valgrind find it.
Why can compiler optimizations make local variables appear missing, reordered, or 'optimized out' in a debugger, and how would you explain that to a developer who expects source-level stepping to match assembly exactly?
Sample Answer
Direct answer
The debugger does not read your source: it reads the machine code and a separate table of debug information that says "at this instruction address, variable x lives in this register or stack slot, and these instructions came from this line". At -O0 that table is simple, because every variable has its own stack slot for the whole function and the instructions follow the source order. When the optimizer is on, it keeps a value in a register only while it is needed, reuses that register afterwards, merges or deletes computations, and interleaves instructions from different lines. The table then has to say "valid only between these two addresses", or nothing at all. So variables seem to disappear (<optimized out>), lines seem to run out of order, and a function can seem to have no frame of its own. The program still behaves the same; the picture the debugger can draw of it does not.
An analogy for a developer who expects one line, one step
Think of a recipe written line by line, and a cook who reads the whole recipe first, chops all the onions for steps 2, 5 and 7 together, and throws away a step that produces something nobody uses. Someone watching the kitchen with the recipe in hand sees step 5 happen before step 3 and no sign of step 4. The recipe (source) describes what the dish must be; it does not promise the order of the work. Optimizers are allowed to do exactly that, because only the observable result must stay the same.
A concrete session
#include <stdio.h>
#include <stdlib.h>
__attribute__((noinline)) unsigned rescale(const unsigned *v, int n)
{
unsigned tag = (unsigned)rand(); /* result never read again */
unsigned sum = 0;
for (int i = 0; i < n; i++)
sum += v[i] * 3;
return sum;
}
int main(void)
{
unsigned v[4] = {1, 2, 3, 4};
printf("%u\n", rescale(v, 4));
return 0;
}
tag receives a result that is never read again. Built twice with gcc -g (GCC 14.4, AArch64, run natively), and driven by gdb -batch -ex 'break rescale' -ex 'run' -ex 'info locals' -ex 'next' -ex 'next' -ex 'next' -ex 'next' -ex 'info locals' -ex 'print tag' -ex 'info address tag' ./rs (GDB 16.3; each run's Breakpoint 1 at ... line, the library-loading lines, and a closing line holding a single . after the info address output are removed, and the values shown for v and the uninitialized slots depend on the stack contents of that run):
-O0 -g:
Breakpoint 1, rescale (v=0xfffffffff980, n=4) at rescale.c:6
6 unsigned tag = (unsigned)rand(); /* result never read again */
tag = 65535
sum = 65535
7 unsigned sum = 0;
8 for (int i = 0; i < n; i++)
9 sum += v[i] * 3;
8 for (int i = 0; i < n; i++)
i = 0
tag = 1804289383
sum = 3
$1 = 1804289383
Symbol "tag" is a complex DWARF expression:
0: DW_OP_fbreg -12
-O2 -g:
Breakpoint 1, rescale (v=v@entry=0xfffffffff980, n=n@entry=4) at rescale.c:6
6 unsigned tag = (unsigned)rand(); /* result never read again */
tag = <optimized out>
sum = <optimized out>
8 for (int i = 0; i < n; i++)
9 sum += v[i] * 3;
8 for (int i = 0; i < n; i++)
9 sum += v[i] * 3;
i = 1
tag = <optimized out>
sum = 3
$1 = <optimized out>
Symbol "tag" is optimized out.
What each difference means:
- Before its line runs, a variable holds garbage (
tag = 65535,sum = 65535at-O0): its stack slot has not been written yet. The debugger shows the slot, not an "unset" marker. tag = <optimized out>at-O2:rand()must still be called (it has side effects), but its result is not needed, so no register or slot keeps it.info address tagsaysSymbol "tag" is optimized out.There is nothing to show. At-O0, the same variable is a stack slot at frame offset -12 (DW_OP_fbreg -12, a DWARF location expression: "frame base minus 12").sum = <optimized out>at the start, thensum = 3later: the variable has a location only in a range of addresses, after the instruction that creates it.- Line 7 never appears in the
-O2stepping:unsigned sum = 0;generated no instruction of its own, so there is no place to stop. The loop's lines interleave (8, 9, 8, 9) because the compiler places the test and the body together. v=v@entry: GDB is saying "this is the value the argument had on entry", which it can recover even after the register was reused.
What to tell the developer, and what to do
- The debugger is reading a map drawn for the unoptimized layout; the optimized code is correct, just not shaped like the source.
- For source-level stepping, build with
-Ogor-O0(or-O0for only the file under investigation). - If the bug only exists in the optimized build, debug that build:
disassemble /sshows source lines interleaved with instructions,info address varshows where the variable lives over which ranges, andstepisteps one instruction at a time. - Adding
volatileto make a variable visible changes the code being debugged, so it can hide the bug it was added to find.
You read the compiler-generated assembly for a function that processes sensor data and see large stack spills that blow your RAM budget on a small microcontroller. How do you find which constructs in the source cause them, and what can you change in the code or the build to reduce stack use?
Sample Answer
Direct answer
Do not start from the word "spill". Measure the function's stack frame, then split it into causes, because a large frame on a small microcontroller usually comes from something other than register spills (spills are values the compiler cannot keep in registers and parks in stack slots). The order I would follow: (1) get the exact number per function with the compiler (-fstack-usage, which writes a .su file listing each function's frame size, -Wframe-larger-than, a call-graph file, meaning a listing of which function calls which) and confirm it at run time by painting the stack (filling unused stack with a known pattern, running the code, and seeing how much of the pattern was overwritten); (2) read the disassembly with source lines (objdump -dl) and classify what the frame holds: large local objects and struct copies (calls to memcpy), true spills (loads and stores relative to sp inside loops), inlined callee frames, and registers saved in the prologue (the function's first instructions, which save registers and reserve the frame); (3) fix the largest cause first and re-measure after each change; (4) lock the budget in with a build-time warning. In the example below, a "spill" problem turned out to be struct copies: 568 bytes of stack became 160 by changing code, while build flags alone saved 16 to 24 bytes.
The code under investigation
The function reads a window of 64 sensor samples, computes a spread (variance-like sum) and runs an eight-tap filter. The source has three switches so each fix can be applied and measured on its own (BYPTR: pass the window by pointer; NARROW: store samples as 16-bit; TAPS: loop over a constant table of taps). With none of them defined it is the original code.
#include <stdint.h>
#define N 64
#ifdef NARROW
typedef int16_t sample_t; /* samples already arrive as 16-bit values */
#else
typedef int32_t sample_t;
#endif
typedef struct {
sample_t v[N];
int32_t sum;
int32_t peak;
} window_t;
#ifdef BYPTR
static void load_window(window_t *w, const int16_t *raw, uint32_t n)
{
w->sum = 0;
w->peak = 0;
for (uint32_t i = 0; i < N; i++) {
int32_t x = (i < n) ? raw[i] : 0;
w->v[i] = (sample_t)x;
w->sum += x;
if (x > w->peak)
w->peak = x;
}
}
static int32_t spread(const window_t *w)
{
int32_t mean = w->sum / N, acc = 0;
for (uint32_t i = 0; i < N; i++) {
int32_t d = w->v[i] - mean;
acc += d * d;
}
return acc;
}
#else
static window_t load_window(const int16_t *raw, uint32_t n)
{
window_t w;
w.sum = 0;
w.peak = 0;
for (uint32_t i = 0; i < N; i++) {
int32_t x = (i < n) ? raw[i] : 0;
w.v[i] = (sample_t)x;
w.sum += x;
if (x > w.peak)
w.peak = x;
}
return w;
}
static int32_t spread(window_t w)
{
int32_t mean = w.sum / N, acc = 0;
for (uint32_t i = 0; i < N; i++) {
int32_t d = w.v[i] - mean;
acc += d * d;
}
return acc;
}
#endif
#ifdef TAPS
static const int8_t taps[8] = { 1, 2, 3, 4, 4, 3, 2, 1 };
static int32_t fir8(const sample_t *p)
{
int32_t acc = 0;
for (uint32_t k = 0; k < 8; k++)
acc += p[k] * taps[k];
return acc >> 4;
}
#else
static int32_t fir8(const sample_t *p, int32_t c0, int32_t c1, int32_t c2, int32_t c3,
int32_t c4, int32_t c5, int32_t c6, int32_t c7)
{
int32_t a0 = p[0] * c0, a1 = p[1] * c1, a2 = p[2] * c2, a3 = p[3] * c3;
int32_t a4 = p[4] * c4, a5 = p[5] * c5, a6 = p[6] * c6, a7 = p[7] * c7;
return (a0 + a1 + a2 + a3 + a4 + a5 + a6 + a7) >> 4;
}
#endif
int32_t process(const int16_t *raw, uint32_t n, int32_t *out)
{
#ifdef BYPTR
window_t w;
load_window(&w, raw, n);
int32_t var = spread(&w);
#else
window_t w = load_window(raw, n);
int32_t var = spread(w);
#endif
for (uint32_t i = 0; i + 8 <= N; i++) {
#ifdef TAPS
out[i] = fir8(&w.v[i]);
#else
out[i] = fir8(&w.v[i], 1, 2, 3, 4, 4, 3, 2, 1);
#endif
}
return var + w.peak;
}
Step 1: measure per function
Built with arm-none-eabi-gcc 14.2.1 (-mcpu=cortex-m3 -mthumb -O2 -fstack-usage), the compiler writes sensor.su, one line per function, with the frame size in bytes and whether it is static (known at compile time) or dynamic (VLAs, variable-length arrays whose size is set at run time, and alloca, a call that grows the frame at run time):
sensor.c:87:9:process 568 static
-Wframe-larger-than=256 turns the same fact into a warning at build time (the In function 'process': line and the source and caret lines that GCC prints around it are left out):
sensor.c:105:1: warning: the frame size of 528 bytes is larger than 256 bytes [-Wframe-larger-than=]
The warning's 528 and the .su file's 568 differ by 40 bytes, and the prologue shows where they go. With the same compiler and flags, process begins stmdb sp!, {r4, r5, r6, r7, r8, r9, sl, fp, lr} (nine registers pushed, 9 x 4 = 36 bytes) followed by sub.w sp, sp, #532. The warning counts only the locals, 528 bytes, which is the two 264-byte windows; the sub is 532 because GCC rounds the local area up by 4 bytes to keep the stack 8-byte aligned; and the .su figure is the whole frame: 532 + 36 = 568. So the extra 40 bytes are 36 bytes of saved registers plus 4 bytes of alignment padding, and the 568 figure is the one that matched the run-time measurement below. -fcallgraph-info=su writes a .ci graph file with the same per-function bytes plus the call edges, which is what you need to add frames along the deepest call path, since worst-case stack is the sum along that path plus the largest nested interrupt. Static numbers cannot see recursion, VLAs or interrupts, so the run-time check follows.
The run-time check paints 2 KiB below the stack pointer with a pattern, calls the function, and finds the first word that is no longer the pattern. Because the stack grows toward lower addresses, the lowest overwritten word is the deepest point reached, and the distance from the starting stack pointer to it is the stack used. Walking the harness: the freestanding build has no C library, and the compiler emits memcpy calls for struct copies, so the harness supplies its own memcpy (marked with optimize("no-tree-loop-distribute-patterns") as a precaution, because a copy loop can in principle be recognised and replaced by a call to memcpy, which would recurse; arm-none-eabi-gcc 14.2.1 at -O2 compiled this particular loop to a plain byte loop even without the attribute, so here it is harmless rather than required); put, put_u and semihost_exit print over the serial port and ask the emulator to exit (on a 32-bit Arm core the semihosting SYS_EXIT call takes the reason code 0x20026 directly in r1; the pointer-to-a-block form is the 64-bit Arm convention); probe reads sp, sets top 64 bytes below it (clear of probe's own frame) and bot 512 words (2 KiB) below top, fills bot to top with 0xA5A5A5A5, calls process, then scans up from bot while the words still hold the pattern and returns sp_now - p; main fills a test input, prints the result and a checksum of the outputs; the last lines are the vector table and reset handler. Run on QEMU's LM3S6965 board (Cortex-M3), with the harness below linked with sensor.c:
#include <stdint.h>
int32_t process(const int16_t *raw, uint32_t n, int32_t *out);
#define UART0_DR (*(volatile uint32_t *)0x4000C000u)
#define PAINT 0xA5A5A5A5u
__attribute__((used, optimize("no-tree-loop-distribute-patterns")))
void *memcpy(void *d, const void *s, unsigned long n)
{
unsigned char *dp = d; const unsigned char *sp = s;
while (n--) *dp++ = *sp++;
return d;
}
static void put(const char *s) { while (*s) UART0_DR = (uint8_t)*s++; }
static void put_u(uint32_t v)
{
char b[12]; int i = 0;
do { b[i++] = (char)('0' + v % 10); v /= 10; } while (v);
while (i) UART0_DR = (uint8_t)b[--i];
}
static void semihost_exit(void)
{
register uint32_t r0 __asm__("r0") = 0x18u; /* SYS_EXIT */
register uint32_t r1 __asm__("r1") = 0x20026u; /* ADP_Stopped_ApplicationExit: the reason code itself on 32-bit Arm */
__asm__ volatile("bkpt 0xAB" : : "r"(r0), "r"(r1) : "memory");
}
static int16_t raw[64];
static int32_t out[64];
__attribute__((noinline)) static uint32_t probe(int32_t *result)
{
uint32_t sp_now;
__asm__ volatile("mov %0, sp" : "=r"(sp_now));
volatile uint32_t *top = (volatile uint32_t *)(sp_now - 64); /* clear of this frame */
volatile uint32_t *bot = top - 512; /* 2 KiB of paint */
for (volatile uint32_t *p = bot; p < top; p++) *p = PAINT;
*result = process(raw, 64, out);
volatile uint32_t *p = bot;
while (p < top && *p == PAINT) p++; /* first word the call touched */
return sp_now - (uint32_t)p;
}
int main(void)
{
for (uint32_t i = 0; i < 64; i++) raw[i] = (int16_t)((i * 37u) % 200u) - 60;
int32_t ret;
uint32_t used = probe(&ret);
int32_t check = ret;
for (uint32_t i = 0; i < 57; i++) check += out[i] * (int32_t)(i + 1);
put("stack bytes used by process(): "); put_u(used);
put(" checksum: "); put_u((uint32_t)check); put("\n");
semihost_exit();
for (;;) { }
}
extern uint32_t _estack;
void reset_handler(void) { main(); for (;;) { } }
__attribute__((section(".vectors"), used))
void (*const vectors[2])(void) = { (void (*)(void))&_estack, reset_handler };
arm-none-eabi-gcc -mcpu=cortex-m3 -mthumb -O2 -Wall -Wextra -ffreestanding -nostdlib -T link.ld stack_probe.c sensor.c -o probe.elf
qemu-system-arm -M lm3s6965evb -nographic -semihosting-config enable=on,target=native -kernel probe.elf
stack bytes used by process(): 568 checksum: 302574
568 measured against 568 from -fstack-usage. The harness also prints a checksum of the outputs, so that every later change can be checked to compute the same results. The input values are offset so that the window sum is far from zero: with a near-zero sum a wrong mean would not change the checksum, and with this input replacing mean by 0 does. The link.ld is a minimal flash-and-RAM script for the same board (vectors first in flash, stack at the top of a 64 KiB RAM):
ENTRY(reset_handler)
MEMORY {
FLASH (rx) : ORIGIN = 0x00000000, LENGTH = 256K
RAM (rwx) : ORIGIN = 0x20000000, LENGTH = 64K
}
SECTIONS {
.text : { KEEP(*(.vectors)) *(.text*) *(.rodata*) } > FLASH
.data : { *(.data*) } > RAM AT > FLASH
.bss : { *(.bss*) *(COMMON) } > RAM
_estack = ORIGIN(RAM) + LENGTH(RAM);
}
Step 2: classify what the frame holds
Read the disassembly with source line numbers: arm-none-eabi-gcc -g -c then arm-none-eabi-objdump -dl. Three observations from the original build:
- Struct copies. Two
bl memcpycalls, mapped by-dltosensor.c:53(return w;inload_window, a struct returned by value) andsensor.c:95(spread(w), a struct passed by value). Awindow_tis 64 x 4 + 8 = 264 bytes, and the original keeps two of them on the stack: 2 x 264 = 528, exactly the frame size in the warning above. This is what dominates. - True spills. For
-mcpu=cortex-m0(which runs almost only the compact 16-bit Thumb instructions (a few instructions such asbl,mrsandmsrare 32-bit) and uses mostly the eight low registers, unlike Cortex-M3's mix of 16- and 32-bit Thumb-2 instructions and wider register access) the same source has 10ldr/strinstructions with an[sp, #n]operand inprocess: 8 listed undersensor.c:97, the loop that inlinesfir8with its eight coefficients and eight products live at once, and 2 undersensor.c:104, thereturn var + w.peak;line; on Cortex-M3 the count is 0 to 1. Those are a few words, not hundreds of bytes. - Prologue. A frame this large cannot be added with a small immediate constant in the instruction: the M0 build loads the size from a literal (
ldr r4, [pc, #232], thenadd sp, r4), which is the visible sign of a big frame.
So the answer to "which constructs cause them" is: by-value structs first, then eight simultaneously live values, then a 264-byte window of 32-bit samples that is bigger than it needs to be (136 bytes once the samples are stored as the 16-bit values they already are).
Step 3: fix one cause at a time and re-measure
Each row adds one change to the previous row; sizes come from -fstack-usage at -O2, and the last column was measured on QEMU's Cortex-M3 with the harness above (all four rows print the same checksum, 302574, so the results are identical):
| Build flags | M0 static | M3 static | M3 measured | sp-relative ld/st (M0, M3) |
|---|---|---|---|---|
| none (original) | 576 | 568 | 568 | 10, 0 |
-DBYPTR | 312 | 304 | 304 | 10, 1 |
-DBYPTR -DNARROW | 192 | 176 | 176 | 7, 0 |
-DBYPTR -DNARROW -DTAPS | 168 | 160 | 160 | 0, 0 |
The last column counts word ldr and str instructions with an sp base address inside process. Halfword and double-word forms are not counted: the original Cortex-M3 build has one strd that initialises the window's sum and peak, and the -DBYPTR -DNARROW Cortex-M3 build has six ldrsh.w loads with an sp base, which read window samples held in the frame and are data reads, not spills.
Pass the window by pointer (and fill it through an output pointer instead of returning it): -264 bytes, one of the two window objects gone and both memcpy calls with it (0 calls remain in the object file; the warning's frame size drops from 528 to 264). Store samples as 16-bit, which is what the data already is: another -128 on M3 (-120 on M0). Replace the eight coefficient arguments with a loop over a static const table in flash: the spills vanish on M0 and 16 more bytes go on M3 (24 on M0). The M0 column is static only, because the simulator in use models a Cortex-M3. After -DTAPS the frame is 160 bytes on M3 and 168 on M0, of which the window itself is 136 (64 x 2 + 8).
Build-flag changes on the original source (-fstack-usage, process):
| Flags | M0 | M3 |
|---|---|---|
-O2 | 576 | 568 |
-Os | 560 | 552 |
-Os -fconserve-stack | 552 | 544 |
-Os saves 16 bytes and adding -fconserve-stack (GCC's option to reduce stack use at some cost in speed) 8 more, so flags help a little but do not touch the 264-byte copies.
Other code-level options when the window must stay large: make it a static or pool-allocated buffer (this makes the function non-reentrant, so it needs a lock or a single caller, and the RAM is permanently used), process in chunks so the window is smaller, mark rarely-used large helpers noinline so their frames are not merged into a hot caller, and avoid VLAs and alloca.
Step 4: keep it fixed
Add -Wframe-larger-than=<budget> (or -Wstack-usage=<budget>) to the build so a regression fails at compile time, record the .su numbers in CI, and keep the painted-stack check in a test that runs the deepest path. Re-run the checksum after every change: it caught nothing here because each change preserved the result, but it is what makes "narrow the samples" a safe change rather than an assumption.
Unlock Full Question Bank
Get access to all 26 Assembly and Low-Level Language Fundamentals interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.