AC28 and ME1a: all risk models compared

These are fits to provisional masks on the training runs, with unvalidated scan registration. Output resolutions, CAD mapping coverage, loss functions and epoch counts differ. The table can compare saved diagnostics; it cannot identify the best production predictor. Other model families have no held-out run scores.

AC28 highest AP
Risk GNN: 0.764

AC28 highest IoU
Residual Geo GNN (joint): 0.465

ME1a highest AP
Residual Geo GNN (individual): 0.629

ME1a highest IoU
Residual Geo GNN (individual): 0.476

Risk GNN: AC28 AP 0.764, IoU 0.309; ME1a AP 0.253, IoU 0.036. Its AC28 AP lead over Mesh GNN is 0.007. It does not lead AC28 IoU or either ME1a metric. The optimized residual GNN predicts risk only; Risk GNN also emits a deformation field.

All saved model results on common CAD nodes

Model	AC28 AP	AC28 IoU	ME1a AP	ME1a IoU	AC28 coverage	ME1a coverage	AC28 epochs	ME1a epochs
Residual Geo GNN (individual)	0.446	0.136	0.629	0.476	1.000	1.000	180	180
Residual Geo GNN (joint)	0.671	0.465	0.476	0.053	1.000	1.000	180	180
FNO2d	0.620	0.296	0.444	0.143	0.987	0.922	60	60
2D CNN/U-Net	0.622	0.333	0.298	0.143	0.987	0.922	80	80
Risk GNN	0.764	0.309	0.253	0.036	1.000	1.000	400	400
Sparse 3D U-Net	0.359	0.047	0.112	0.019	1.000	1.000	100	100
Dense 3D U-Net	0.358	0.048	0.097	0.019	1.000	1.000	100	100
PointNet	0.093	0.114	0.076	0.009	1.000	0.997	120	120
Mesh GNN	0.758	0.147	0.045	0.021	1.000	1.000	100	100
DGCNN	0.002	0.000	0.038	0.065	0.736	0.630	100	100
UV-Net	0.076	0.043	0.011	0.011	0.979	0.966	100	100
Tabular MLP	0.002	0.000	0.001	0.000	1.000	1.000	120	120

AP and IoU of all models on AC28 and ME1a

Training scope

The Risk GNN used both AC28 runs for its AC28 entry. Other original models used AC28 18h01. The residual GNN individual entry uses one run of each part; its joint entry uses both AC28 runs and ME1a. Every displayed score evaluates a run included in that model's training. Only the residual GNN has a separate leave-one-run-out experiment, and its held-out IoU was zero for all three runs. The five conditional diffusion variants generate outputs on different native domains; their diagnostic risk scores are in the main report and are not ranked here.

Model	AC28 training	ME1a training
2D CNN/U-Net	AC28 18h01	ME1a 14h06
FNO2d	AC28 18h01	ME1a 14h06
Dense 3D U-Net	AC28 18h01	ME1a 14h06
Sparse 3D U-Net	AC28 18h01	ME1a 14h06
PointNet	AC28 18h01	ME1a 14h06
DGCNN	AC28 18h01	ME1a 14h06
Mesh GNN	AC28 18h01	ME1a 14h06
Risk GNN	AC28 16h + 18h01	ME1a 14h06
UV-Net	AC28 18h01	ME1a 14h06
Tabular MLP	AC28 18h01	ME1a 14h06
Residual Geo GNN (individual)	AC28_18h01	ME1a_14h06
Residual Geo GNN (joint)	AC28_16h+AC28_18h01+ME1a_14h06	AC28_16h+AC28_18h01+ME1a_14h06

Residual GNN held-out experiment · AC28 model gallery · ME1a model gallery

Is Risk GNN the best risk model?

All saved model results on common CAD nodes

Training scope