Observed issues › afar-c-lastprivate-distribute-parallel-for-small-type
Appears to behave incorrectlyWrong answer · C · OpenMP · Open
A lastprivate item smaller than 8 bytes on target teams distribute parallel for comes back as 0 or a garbage value
Legality: independently confirmed. 2 independent readings find the program legal and the result required. How this is checked
Affected variants
| Compiler | MI300A gfx942 unified memory (XNACK 1) | MI300A gfx942 copy (XNACK 0) | MI250X gfx90a copy (XNACK 0) | MI300X gfx942 copy (XNACK 0) | MI350X gfx950 copy (XNACK 0) | A100 sm_80 separate memory | A100 sm_80 managed memory |
|---|---|---|---|---|---|---|---|
| AFAR 24.3.0 | behaves incorrectly | behaves incorrectly | unknown not run | behaves incorrectly | behaves incorrectly | n/a | n/a |
| CCE 19.0.0 | behaves incorrectly | behaves incorrectly | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 20.0.2 | behaves incorrectly | behaves incorrectly | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.3 | behaves incorrectly | behaves incorrectly | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.4 | behaves incorrectly | behaves incorrectly | unknown not run | unknown not run | unknown not run | n/a | n/a |
| NVHPC 25.11 | n/a | n/a | n/a | n/a | n/a | behaves incorrectly | behaves incorrectly |
| NVHPC 26.9 | n/a | n/a | n/a | n/a | n/a | behaves incorrectly | behaves incorrectly |
| GCC 16.2.0 | correct | correct | correct | correct | correct | correct | n/a |
"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.
Messages
| Result | Compiler and GPU | Message |
|---|---|---|
| behaves incorrectly | AFAR 24.3.0, MI300A unified AFAR 24.3.0, MI300A copy AFAR 24.3.0, MI300X AFAR 24.3.0, MI350X | showlp= 0 dev= 1 |
| behaves incorrectly | CCE 19.0.0, MI300A unified CCE 19.0.0, MI300A copy CCE 20.0.2, MI300A unified CCE 20.0.2, MI300A copy CCE 21.0.3, MI300A unified CCE 21.0.3, MI300A copy CCE 21.0.4, MI300A unified CCE 21.0.4, MI300A copy NVHPC 25.11, A100 separate NVHPC 25.11, A100 managed NVHPC 26.9, A100 separate NVHPC 26.9, A100 managed | showlp= -1 dev= 1 |
Reproduce
AFAR 24.3.0, MI300A unified
$AFAR/bin/amdclang -fopenmp --offload-arch=gfx942 -O2 repro.c -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
CCE 19.0.0, MI300A unified
module load cce/19.0.0 craype-accel-amd-gfx942 rocm/6.4.3 module unload cray-mpich craycc -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx942 -O2 repro.c -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
CCE 20.0.2, MI300A unified
module load cce/20.0.2 craype-accel-amd-gfx942 rocm/6.4.3 module unload cray-mpich craycc -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx942 -O2 repro.c -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
CCE 21.0.3, MI300A unified
module load cce/21.0.3 craype-accel-amd-gfx942 rocm/7.2.0 module unload cray-mpich craycc -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx942 -O2 repro.c -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
CCE 21.0.4, MI300A unified
module load cce/21.0.4 craype-accel-amd-gfx942 rocm/7.2.0 module unload cray-mpich craycc -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx942 -O2 repro.c -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
NVHPC 25.11, A100 separate
export LD_LIBRARY_PATH=$NVHPC_25_11/compilers/lib nvc -mp=gpu -gpu=cc80 -gpu=mem:separate -O2 repro.c -o repro OMP_TARGET_OFFLOAD=mandatory ./repro
NVHPC 26.9, A100 separate
export LD_LIBRARY_PATH=$NVHPC_26_9/compilers/lib nvc -mp=gpu -gpu=cc80 -gpu=mem:separate -O2 repro.c -o repro OMP_TARGET_OFFLOAD=mandatory ./repro
GCC 16.2.0, MI300A unified (runs correctly)
export LD_LIBRARY_PATH=$GCC_16_2/lib64:/opt/rocm-7.2.1/lib $GCC_16_2/bin/gcc -fopenmp -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx942 -O2 repro.c -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
Program (53 lines)
/* # Finding (lead 2): AFAR amdclang returns 0 or a garbage value for a lastprivate item
# of fewer than 8 bytes on target teams distribute parallel for (C)
- compiler: AFAR amdclang 24.0.0git (drop 24.3.0), -fopenmp --offload-arch=gfx942 (MI300A,
HSA_XNACK 1 and 0) and gfx90a (MI250X, Tioga, XNACK 0), -fopenmp-version=52, -O2 and -O0
- expected: lp= 192 dev= 1. lastprivate gives the original lp the value of the sequentially
last iteration (i = 64: 3 * 64): "the value of each new list item from the sequentially last
iteration of the associated loops ... is assigned to the original list item" (5.2 §5.4.5
p.115 l.12-16); on the combined construct it applies to the distribute and for leaves (5.2
§17.2 p.340 l.21-22) and map(tofrom: lp) on the target leaf brings the value to the host.
- actual: MI300A (XNACK 1 and 0, -O2 and -O0): lp= 0 or a value that varies from run to run
(-1191462675, 948123843, 16246, ...; the trial saw 104571, 82972, 165175, 151912, 17038;
the 12 later runs all printed 0);
MI250X: lp= 0 in every run. The same with short (14674, -18864, ...) and float (-1.1e-05, ...);
with target followed by teams distribute parallel for lastprivate(lp) inside it.
- same-compiler controls (each one change): long long lp or double lp gives 192 in every run
on both GPUs; target teams distribute lastprivate(lp) map(tofrom: lp) with the int gives 192;
target parallel for lastprivate(lp) map(tofrom: lp) with the int gives 192.
- no other family can witness: CCE craycc 19/20/21 give lp= -1 (the original unchanged) with
int and long long alike, their lastprivate defect on a for inside a target region
(cce-c-parallel-for-in-target.c); NVIDIA's guide documents lastprivate on distribute as
unsupported (HPC Compilers User's Guide 25.11, §7.4, 2.9.4), so nvc is no witness; AFAR
amdflang stops at "not yet implemented: lastprivate clause in constructs other than
simd/worksharing-loop/taskloop" on the Fortran form. The expected value rests on the spec
text and AFAR's own 8-byte and single-leaf controls.
- relation: stress/afar-c-lastprivate-combined-target-not-mapped.c is the missing implicit map
(no map clause: lp stays -1 with int and long long alike); this is a second defect, which is
why that finding's explicit-map control passes only with its long long item .
- legality: offload.engine.rules accepts it (with the race rule and the restriction checks:
offload.engine.prerun.check, rerun, data/evidence/lead_probe/logs/legality_new.jsonl), and nvc
26.9 and 25.11 (NVHPC, not under test for an AFAR finding) compile it with -mp=gpu
(data/evidence/lead_probe/logs/wingtip_witness.log); the check's other witness, the amdclang front
end, is AFAR's own and is not counted; no gfortran or gcc is a legality witness.
Race-free: lp is lastprivate; every iteration writes only its own copy.
- evidence: data/evidence/lead_probe/leads/ (lp_*.c), runs in data/evidence/lead_probe/logs/leads_*.jsonl;
reproduction (the gate's four environments by hand, as for
afar-c-teams-distribute-reduction-first-team.c): MI300A, XNACK 1 and 0, wrong in every run of
(a), (b), (c) and (d), but the value differs between runs (0, 16246, 337455926, -1176291223,
...: the gate reads such a row as nondeterministic); MI250X (<host>), (a), (b), (d): lp= 0 in
every run (confirmed). The long long control gave 192 in every environment. */
#include <stdio.h>
#include <omp.h>
int main(void) {
int lp = -1;
int dev = -1;
#pragma omp target teams distribute parallel for lastprivate(lp) map(tofrom: lp)
for (int i = 1; i <= 64; i++)
lp = 3 * i;
#pragma omp target map(tofrom: dev)
dev = omp_is_initial_device() ? 0 : 1;
printf("lp= %d dev= %d\n", lp, dev);
return 0;
}
Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page
Expected and observed
Expected
lp= 192 dev= 1
Observed where it behaves incorrectly
lp= 0 dev= 1
What the specification requires
The specification text this program relies on and the compiler is expected to support.
OpenMP 5.2 §5.4.5 p.115 l.12-16
the value of each new list item from the sequentially last iteration of the associated loops ... is assigned to the original list item
OpenMP 5.2 §17.2 p.340 l.21-22
Evidence
Other compilers run it correctly
GCC 16.2.0