LATX, AOT: statically reduce L1I set conflicts

[LaurenIsACoder]

Background

This work started from an unexpected result in the EFLAGS optimization effort: the translator emitted fewer LoongArch instructions, but several SPECint workloads did not become faster and some became slower. Since the translated instruction count had already decreased, we treated the remaining regression as a separate code-fetch and layout problem instead of concluding that EFLAGS elimination had no value.

PMU measurements showed that performance could change significantly when the generated LoongArch code moved in the AOT code buffer, even when the instruction sequence and count remained effectively unchanged. This drew attention to L1 instruction-cache set conflicts.

The validation host is a Loongson 3A6000 with a 64 KiB, 4-way L1I, 64-byte lines, and 256 sets. Its set-index cycle is 16 KiB and a line maps approximately as:

set = (host_address / 64) % 256;

Analysis of the existing layout

We compared the x86 ELF layout with AOT=2 LoongArch addresses for all 12 SPEC CPU2000 integer workloads:

ELF functions:                       28,758
Functions represented in AOT:        5,602
Guest TU order inversions:                0
ELF cold functions/fragments:         1,167
Cold functions represented in AOT:        1

The existing AOT implementation already preserves guest TU order and naturally excludes nearly all compiler-marked cold code. The main lost property was larger address alignment: 16/32/64-byte alignment was preserved at 99.1%/98.6%/97.7%, while 512-byte alignment was preserved at 10.5% and 2/4 KiB alignment at 0%.

An early experiment restored 512-byte to 4 KiB boundaries for direct-call targets and improved SPECint. Causal ablations showed that the boundary itself was not beneficial: the original functions had at most 15 bytes of preceding padding, exact-512-only and 4-KiB-only variants regressed, and delaying padding to the following TU kept most of the gain even though the call target was no longer aligned. Sampling showed that downstream ordinary TUs, not the aligned entries, accounted for most saved misses. Sparse padding was changing the relative L1I-set placement of hot code groups.

Summary

• build a static TU control-flow graph while storing an AOT cache
• identify loop working sets with strongly connected components and include their direct callees
• model the host L1I geometry from sysfs and minimize predicted set overflow
• add at most ways * cacheline bytes before safe direct-CALL entries, under a 1% per-segment padding budget
• persist the chosen padding and cache geometry in existing AOT metadata space
• apply the stored fixed plan in O(1) per movable entry during AOT=2 loading
• reuse stored component IDs during default AOT=1 lazy loading and score only five bounded placements against the pages already loaded by that thread
• keep page, TU, and TB order unchanged while supporting both AOT=2 and the default AOT=1

The feature is experimental, defaults off, and has one LATX_AOT_STATIC_LAYOUT=0/1 switch.

Rationale

The previous guest-address-alignment experiment improved SPECint, but causal ablations showed that 512-byte or 4KB guest alignment was not itself beneficial. Its padding happened to change downstream LoongArch code placement across L1I sets. This version replaces the guest-address proxy with an explicit static set-pressure model.

Only direct-CALL targets may receive padding. Allowing padding before arbitrary TUs caused a real vortex wait-path failure because the current AOT recovery path has implicit layout constraints at ordinary TU boundaries.

Results

Loongson 3A6000, AOT=2, SPEC CPU2000 integer train, three-run medians, latest master 3de0d400f51:

Benchmark	Runtime change
gzip	+0.482%
vpr	-0.141%
gcc	+0.362%
mcf	-0.006%
crafty	-0.522%
parser	-0.383%
eon	-1.788%
perlbmk	-0.436%
gap	-0.995%
vortex	-0.726%
bzip2	-0.072%
twolf	-0.142%

Geometric mean runtime: -0.366%. All 12 benchmarks passed.

Default AOT=1, same candidate binary and byte-identical AOT caches, switch off/on:

• geometric mean runtime: -0.130%
• summed runtime: approximately neutral (+0.049%)
• gap -2.54%, vortex -1.64%, twolf -1.01%
• eon +1.47%, crafty +0.85%, gcc +0.49%
• all 12 benchmarks passed

The AOT=1 path keeps per-thread, per-segment, per-CF_PARALLEL set pressure and clears it when the global TB flush count changes. Padding is restricted to direct-CALL entries, the same 1% stored budget, and 0-ways cache lines. Its score includes both set overflow and the number of added cache lines.

Default AOT=1 held-out SQLite, same binary and byte-identical caches, seven alternating off/on runs:

• median cycles about -1.15% in the final three-pair confirmation (an earlier seven-pair run was -1.43%)
• L1I misses about -9.8%
• host instruction overhead about +0.08%
• outputs matched exactly

Held-out official SQLite 3.53.4 x86_64 workloads, not used to tune the algorithm:

• 300,000-row in-memory SQL workload: median cycles about -1.21%
• 20 repeated sqlite3_analyzer runs: median cycles about -0.95%
• outputs matched master exactly

A deterministic random-padding control used the same safe boundaries, maximum padding, and global budget. In seven same-period alternating SQLite runs, the scored layout was about 2.17% faster than the random layout with essentially identical host instruction counts.

The hardware evidence is mixed across programs: vortex reduced L1I misses, while SQLite improved cycles even when aggregate L1I misses did not. The PR therefore remains draft and does not claim that L1I set conflicts are the only layout-sensitive effect; branch-predictor and iTLB interactions remain follow-up work.

Validation

• full incremental ninja build on LoongArch
• external static-layout validation harness: 8/8 pass
• latx-aot-cache-reader: pass, including disk write/mmap header round trip
• latx-aot-file-publish: pass
• latx-aot-pe-load-address: pass
• final-format AOT caches recorded 64-byte lines, 256 sets, 4 ways and the layout magic
• LATX_AOT_STATIC_LAYOUT=2 exits with an error
• AOT=1 off/on use byte-identical caches and produce identical output
• git diff --check: pass
• checkpatch: 0 errors; only generic MAINTAINERS warnings for new files

Known limitations

• existing AOT caches must be regenerated to contain a layout plan; normal committed builds already invalidate old caches through AOT_VERSION
• missing or mismatched host cache geometry disables the layout plan
• if highwater prevents one planned padding, the rest of that segment's plan is disabled to avoid applying a shifted partial plan
• the static model currently covers loop SCCs and their direct callees; it does not model indirect-call targets or branch-predictor indexing
• AOT=1 is positive in geometric mean but has benchmark-level regressions and remains experimental/default-off
• the feature remains default-off

Configuration

LATX_AOT_STATIC_LAYOUT=0  # disabled, default
LATX_AOT_STATIC_LAYOUT=1  # enabled

---

背景

这项工作起源于EFLAGS优化中的一个反常结果：翻译器生成了更少的LoongArch指令，但部分SPECint程序没有变快，个别程序反而变慢。既然翻译指令数已经下降，我们没有据此否定EFLAGS消除，而是把剩余退化作为独立的取指和代码布局问题继续分析。

PMU测量显示，即使LoongArch指令序列和数量基本不变，只要代码在AOT code buffer中的位置改变，性能就可能明显变化，因此我们开始关注L1指令缓存set冲突。

验证机器Loongson 3A6000的L1I为64 KiB、4路、64字节cacheline、256个set，set索引周期为16 KiB：

set = (host_address / 64) % 256;

对现有AOT布局的分析

我们比较了12个SPEC CPU2000 integer程序的x86 ELF布局和AOT=2恢复后的LoongArch地址：

ELF函数总数：                    28,758
进入AOT的函数：                  5,602
Guest TU顺序逆转：                    0
ELF cold函数或片段：              1,167
进入AOT的cold函数或片段：             1

现有AOT已经保持guest TU顺序，并通过只缓存实际翻译代码自然过滤掉绝大多数cold代码。主要丢失的是较大的地址边界：16/32/64字节对齐分别保留99.1%/98.6%/97.7%，512字节只保留10.5%，2/4 KiB为0%。

早期实验恢复直接CALL目标的512字节到4 KiB边界后SPECint出现正收益，但因果消融证明边界本身不是原因：原函数前最多只有15字节填充，只处理exact-512或4 KiB都会退化，把padding延迟到下一个TU仍能保留大部分收益。采样显示减少miss的主要是后续普通TU，而不是被对齐的入口。真正的作用是稀疏padding改变了多个热点代码组之间的L1I set映射。

方案

AOT存储阶段利用已有TB元数据构造TU控制流图，通过强连通分量识别静态循环，并把循环及其直接callee作为静态工作集合。算法不依赖ELF符号，因此stripped程序也可以使用。

存储时已知最终LoongArch tu_size。每个组件维护256项set压力，代价为：

overflow = max(0, resident_lines - associativity);
cost += overflow * overflow;

只有直接CALL目标允许padding。任意TU边界padding曾使vortex进入错误等待路径，说明普通TU边界仍有隐含布局约束。候选固定为0到ways条cacheline，在3A6000上是0、64、128、192、256字节；每个segment总预算为LoongArch AOT代码尺寸的1%。AOT=1评分还加入新增cacheline数量，避免为很小的冲突收益扩大代码跨度。

AOT文件保存L1I结构、布局magic、每段预算、每个TU的组件ID、安全CALL入口标志和AOT=2固定padding。

AOT=2

AOT=2按固定顺序整段恢复，加载时只检查L1I结构、读取padding、检查highwater并应用，每个入口O(1)。某次计划padding无法应用时，停用该segment剩余计划，避免部分计划错位。

AOT=1

AOT=1是默认模式，每次按需加载连续四页，不能复用AOT=2固定序列。它读取存储的静态组件ID，按线程、segment和CF_PARALLEL维护已加载代码的set压力。普通TU只登记footprint，安全CALL入口只评估五个有界位置；状态查询复用于整个四页批次，TB flush计数变化时清空，同一TU不会重复登记。

这属于运行时布局而不是运行时profile：只记录静态选中的代码已放在什么位置，不记录执行次数。

性能

以下负数表示更快。

AOT=2 SPEC CPU2000 integer train

Benchmark	运行时间变化
gzip	+0.482%
vpr	-0.141%
gcc	+0.362%
mcf	-0.006%
crafty	-0.522%
parser	-0.383%
eon	-1.788%
perlbmk	-0.436%
gap	-0.995%
vortex	-0.726%
bzip2	-0.072%
twolf	-0.142%

几何平均： -0.366%
总时间：   -0.260%
正确性：   12/12通过

AOT=1 SPEC CPU2000 integer train

相同候选二进制、字节完全一致的AOT文件，只切换布局开关：

Benchmark	运行时间变化
gzip	-0.121%
vpr	+0.321%
gcc	+0.492%
mcf	+0.142%
crafty	+0.853%
parser	+0.330%
eon	+1.472%
perlbmk	+0.483%
gap	-2.539%
vortex	-1.637%
bzip2	-0.282%
twolf	-1.008%

几何平均： -0.130%
总时间：   +0.049%，基本持平
正确性：   12/12通过

Held-out SQLite

官方SQLite 3.53.4 Linux x86_64工具未参与算法设计或参数选择，A/B输出哈希一致：

AOT=2 30万行内存数据库：      cycles约-1.21%
AOT=2 20次sqlite3_analyzer：  cycles约-0.95%
AOT=1 30万行内存数据库：      cycles约-1.15%（之前7组为-1.43%）
AOT=1 L1I miss：              约-9.8%
AOT=1 host指令开销：          约+0.08%

随机padding否证

固定种子随机对照使用相同安全入口、最大padding和1%预算，但不使用set评分。同一时段7组SQLite交替中，评分方案cycles中位数约3.355B，随机方案约3.429B，评分方案快约2.17%，host指令数一致。这说明收益不能只用任意padding解释。

验证

• LoongArch完整增量ninja构建；
• 现有AOT文件发布、cache reader和PE load-address测试；
• 外部harness覆盖静态图、SCC、set压力、预算、highwater、持久化元数据、lazy placement、重复placement和页面乱序加载，8/8通过；
• AOT元数据写入和mmap回读；
• 新格式缓存执行AOT=1和AOT=2；
• 两种模式均完成off/on和SPECint正确性；
• SQLite输出哈希和随机padding否证；
• git diff --check及产品提交checkpatch。

配置

LATX_AOT_STATIC_LAYOUT=0  # 关闭，默认
LATX_AOT_STATIC_LAYOUT=1  # 开启

已知限制

• 静态模型覆盖循环SCC和直接callee，不预测间接CALL目标。
• L1I miss不能解释全部运行时间变化，布局还可能影响BTB、分支历史索引、iTLB和前端时序。
• AOT=1几何平均略有正收益，但存在单项退化，暂不适合默认开启。
• Lazy状态按线程维护，其他翻译线程放置的代码不会进入当前线程组件压力。
• SMC reload复用原TB地址；TB flush会清空状态。长期线程中已卸载segment对应TLS状态仍是后续清理方向。
• 无法读取host cache结构或结构不匹配时忽略计划。
• 只有直接CALL目标允许padding，任意TU边界目前不是安全位置。