Validation and code generation

Relevant source files

• src/aarch64.rs 
• jit.md 
• tests/aarch64_jit.rs 

All of src/aarch64.rs runs without allocation. It never formats strings, and it never uses a panic as a capacity check. All offset, length and displacement arithmetic is checked.

Two passes over one Sink

Sink has an optional output buffer, a position, and a byte limit. In the sizing pass output is None: words are counted but not written, and branch range checks are skipped because targets are not known yet. The emit pass uses the offsets recorded during sizing, so forward branches resolve directly. No relocation list is needed.

The limit is min(options.max_code_size, MAX_CODE_SIZE), where MAX_CODE_SIZE = 2^27 - 4. Keeping the whole image under +128 MiB guarantees that every unconditional B (±128 MiB range) can reach any point in the image. Going over the limit returns CodeTooLarge.

Validation rules (validate)

JitError.pc holds the original BPF slot index whenever the error belongs to an instruction.

Instruction lowering

Conditional branches use a fixed layout: an inverted conditional branch skips over an unconditional B. The conditional encoding therefore never needs range relaxation, and the reach of B is guaranteed by the 128 MiB image cap.

Condition codes per BPF op: JEQ EQ, JNE/JSET NE, JGT HI, JGE HS, JLT LO, JLE LS, JSGT GT, JSGE GE, JSLT LT, JSLE LE.

Checked failure blocks (require)

require(valid_cond, status, pc, epilogue) reserves one word for a B.<valid_cond> and emits the failure block: load Invocation*, write fault_pc, set W0 = status, B epilogue. It then patches the reserved word to branch past that block. The conditional displacement (±1 MiB) is range-checked. The prologue’s invocation checks, the divide-by-zero checks, and callback status checks all use this helper.

Immediates (Sink::imm)

• A 64-bit value whose upper 48 bits are all ones becomes a single MOVN (fork commit 208a940). Small negative constants therefore take 1 instruction instead of 4.
• Otherwise: MOVZ for halfword 0, plus MOVK only for the nonzero higher halfwords (fork commit 785f1d7). MOVZ clears the other bits, so skipping zero halfwords is safe.
• 32-bit forms use W registers (2 halfwords at most).
• ALU and JMP immediates are sign-extended from i32 to the operation width, so JMP64 compares against the sign-extended 64-bit value.

compact_immediates_use_fewer_halfword_writes (host) and compact_immediates_clear_old_register_bits (native) test this.

Prologue and epilogue

SUB  SP, SP, #160
STP  X19..X30 pairs at [SP, #0..#80]
ADD  X29, SP, #80
STR  X0, [SP, #104]                 ; Invocation*
STR  XZR, [X0, #40]                 ; result = 0
MOV  TMP, #NO_FAULT ; STR TMP, [X0, #48]
check callbacks != 0, stack != 0, stack_len >= stack_size,
      ADDS X28, stack, stack_size (no carry)   -> else STATUS_INVALID_INVOCATION
MOV  X9, X19..X27 <- XZR             ; BPF R0..R9 = 0
LDR  X19, [X0, #0]                   ; R1 = context
...
epilogue:
LDP  X19..X30 pairs
ADD  SP, SP, #160
RET

Encoding tests

The host-side tests in tests/aarch64_jit.rs run on every architecture:

• rejects_invalid_streams_and_targets: every rejection category, with the expected pc.
• bounded_buffers_and_canaries: short output and scratch are rejected, and canary bytes around the output stay intact.
• compact_immediates_use_fewer_halfword_writes: checks the MOVN/MOVK savings.
• rejects_invalid_options_and_checks_callback_branch_encoding: checks the options validation and the exact encoding of the callback status branch.

The native suite runs only under all(target_arch = "aarch64", target_os = "linux"). See Testing and CI.