Observed issues › catalog/afar-flang-critical-target-parallel-wrong
Appears to behave incorrectlyWrong answer · Fortran · OpenMP · Open
amdflang: an unnamed critical in a target parallel do does not exclude other threads (lost updates)
Legality: independently confirmed. 2 independent readings find the program legal and the result required. How this is checked
Affected variants
| Compiler | MI300A gfx942 unified memory (XNACK 1) | MI300A gfx942 copy (XNACK 0) | MI250X gfx90a copy (XNACK 0) | MI300X gfx942 copy (XNACK 0) | MI350X gfx950 copy (XNACK 0) | A100 sm_80 separate memory | A100 sm_80 managed memory |
|---|---|---|---|---|---|---|---|
| AFAR 24.3.0 | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | n/a | n/a |
| CCE 19.0.0 | fails another way | fails another way | fails another way | unknown not run | unknown not run | n/a | n/a |
| CCE 20.0.2 | fails another way | fails another way | fails another way | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.3 | fails another way | fails another way | fails another way | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.4 | fails another way | fails another way | fails another way | unknown not run | unknown not run | n/a | n/a |
| NVHPC 25.11 | n/a | n/a | n/a | n/a | n/a | fails another way | fails another way |
| NVHPC 26.9 | n/a | n/a | n/a | n/a | n/a | fails another way | fails another way |
| GCC 16.2.0 | correct | correct | correct | correct | correct | correct | n/a |
"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.
Messages
| Result | Compiler and GPU | Message |
|---|---|---|
| behaves incorrectly | AFAR 24.3.0, MI300A unified AFAR 24.3.0, MI300A copy AFAR 24.3.0, MI250X AFAR 24.3.0, MI300X AFAR 24.3.0, MI350X | showRESULT ran_on_host=0 check=fail |
| fails another way | CCE 19.0.0, MI300A unified CCE 19.0.0, MI300A copy CCE 19.0.0, MI250X | showWarning: Headers and libraries from cray-libsci/25.03.0 will be ignored because depends on PE_MPICH which is unavailable.. MT_CONTEXT_CRITICAL_SECTION Error message :: unhandled construct type Error detected :: File '/workspace/crayftn/pdgcs/v_openmp.c', line 7617 Initiated from :: Line 660 (v_main.c) Optimizer built :: 2025-02-07 (production) File :: afar-flang-critical-target-parallel-wrong.f90 |
| fails another way | CCE 20.0.2, MI300A unified CCE 20.0.2, MI300A copy CCE 20.0.2, MI250X | showWarning: Headers and libraries from cray-libsci/25.09.0 will be ignored because depends on PE_MPICH which is unavailable.. MT_CONTEXT_CRITICAL_SECTION Error message :: unhandled construct type Error detected :: File '/workspace/crayftn/pdgcs/v_openmp.c', line 7814 Initiated from :: Line 660 (v_main.c) Optimizer built :: 2025-11-17 (production) File :: afar-flang-critical-target-parallel-wrong.f90 |
| fails another way | CCE 21.0.3, MI300A unified CCE 21.0.3, MI300A copy CCE 21.0.3, MI250X | showWarning: Headers and libraries from cray-libsci/26.03.0 will be ignored because depends on PE_MPICH which is unavailable.. MT_CONTEXT_CRITICAL_SECTION Error message :: unhandled construct type Error detected :: File '/workspace/crayftn/pdgcs/v_openmp.c', line 8197 Initiated from :: Line 732 (v_main.c) Optimizer built :: 2026-07-14 (production) File :: afar-flang-critical-target-parallel-wrong.f90 |
| fails another way | CCE 21.0.4, MI300A unified CCE 21.0.4, MI300A copy CCE 21.0.4, MI250X | showWarning: Headers and libraries from cray-libsci/26.03.0 will be ignored because depends on PE_MPICH which is unavailable.. MT_CONTEXT_CRITICAL_SECTION Error message :: unhandled construct type Error detected :: File '/workspace/crayftn/pdgcs/v_openmp.c', line 8197 Initiated from :: Line 732 (v_main.c) Optimizer built :: 2026-09-24 (production) File :: afar-flang-critical-target-parallel-wrong.f90 |
| fails another way | NVHPC 25.11, A100 separate NVHPC 25.11, A100 managed | showNVFORTRAN-F-1196-OpenMP - omp critical is not supported in GPU region (afar-flang-critical-target-parallel-wrong.f90: 19) NVFORTRAN/x86-64 Linux 25.11-0: compilation aborted |
| fails another way | NVHPC 26.9, A100 separate NVHPC 26.9, A100 managed | showNVFORTRAN-F-1196-OpenMP - omp critical is not supported in GPU region (afar-flang-critical-target-parallel-wrong.f90: 19) NVFORTRAN/x86-64 Linux 26.9-0: compilation aborted |
Reproduce
AFAR 24.3.0, MI300A unified
$AFAR/bin/amdflang -fopenmp --offload-arch=gfx942 -O2 repro.f90 -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
GCC 16.2.0, MI300A unified (runs correctly)
export LD_LIBRARY_PATH=$GCC_16_2/lib64:/opt/rocm-7.2.1/lib $GCC_16_2/bin/gfortran -fopenmp -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx942 -O2 repro.f90 -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
Program (68 lines)
! # Draft finding: omp52-w-critical on afar
!
! - compiler: afar
! - toolchain: afar; arch: gfx942; machine: tuolumne; memory mode: unified; opt: -O2
! - program: catalog omp52-w-critical (f90, version 52, sha256 7f0f6d194e31f04f)
! - expected: check=pass (trusted; elision + model)
! - actual: wrong_answer check=fail (check_fail: ran_on_host=0 check=fail)
! - oracle audit: run on AFAR, CCE 19, 20, 21 (copy mode) and NVHPC: no toolchain passes the original (CCE ICEs, NVHPC
! rejects), so the mutant cannot be credited (unaudited, not weak); the C port's identical check is valid (its mutant killed on
! AFAR), and the mutant fails only by a race (a device effect)
! - verdict: compiler_bug by rule R20-runtime-bug: a legal program fails at run time, deterministically, against an expected value
! from two or more independent sources with an executed witness, and another toolchain, mode, arch or the elision gives the
! expected value
! - reason: expected trusted check=pass (elision + model): stored main verdict: elision + model agree; witnesses: afar/gfx942/copy
! (port:omp52-c-w-critical), afar/gfx942/unified (port:omp52-c-w-critical), cce19/gfx942/copy (port:omp52-c-w-critical), cc ...
! - citations: RUL-DERIVED-NEEDS-EXECUTION (rulings.md); RUL-WITNESS-PER-MODE (rulings.md);
! RUL-SAME-COMPILER-RUNS (rulings.md); RUL-PRESENCE-WITNESS (rulings.md); RUL-METAMORPHIC-OTHER-FAMILY
! (rulings.md)
! - legality: legal: offload.engine.rules accepts it (offload.engine.prerun.check, rerun: with the
! race rule and the restriction checks); NVHPC is no witness: it does not implement a feature the
! program uses (NVFORTRAN-F-1196-OpenMP - omp critical is not supported in GPU
! region); the check's other witness, the AFAR flang front end, is the compiler under test and is
! not counted; no gfortran or gcc is a legality witness
! - witnesses: afar/gfx942/copy (port:omp52-c-w-critical), afar/gfx942/unified (port:omp52-c-w-critical), cce19/gfx942/copy
! (port:omp52-c-w-critical), cce19/gfx942/unified (port:omp52-c-w-critical), cce20/gfx942/copy (port:omp52-c-w-critical),
! cce20/gfx94 ...
! - withdrawn: none
!
! Generated by compliance.widen (OpenMP 5.2, free-form Fortran; catalog widening). Do not edit.
! id: omp52-w-critical
! primary atom: omp.directive.critical
! atoms: omp.directive.critical
! legal: yes
! observable: yes
! ran_on_host: inside the target region of interest
! evidence: construct
! needs: none
! std: f2008
! feature: an unnamed critical construct in a target parallel loop
! spec: OpenMP 5.2 §15.2, p.299 (critical construct); its binding thread set is the contention group, so a target parallel region
! (one team) is used
program wide_prog
use omp_lib, only: omp_is_initial_device
implicit none
integer, parameter :: n = 64
integer :: a(n), b(n), ref(n)
integer :: s, t
integer :: i, host_flag
logical :: ok
do i = 1, n
a(i) = i
b(i) = 0
ref(i) = 2*i + 1
end do
host_flag = 1
s = 0
!$omp target parallel do map(to: a) map(tofrom: s, host_flag) private(t)
do i = 1, n
!$omp critical
t = s
s = t + a(i)
!$omp end critical
if (i == 1) host_flag = merge(1, 0, omp_is_initial_device())
end do
!$omp end target parallel do
ok = s == n*(n + 1)/2
print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', ok)
end program wide_prog
Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page
Expected and observed
Expected
check=pass
Observed where it behaves incorrectly
RESULT ran_on_host=0 check=fail
What the specification requires
The specification text this program relies on and the compiler is expected to support.
OpenMP 5.2 §15.2 p.299
Evidence
Same compiler, one change
- an atomic update instead of the critical construct: AFAR 24.3.0: correct; CCE 19.0.0: correct; CCE 20.0.2: correct; CCE 21.0.3: correct; CCE 21.0.4: correct
Control program
program wide_prog use omp_lib, only: omp_is_initial_device implicit none integer, parameter :: n = 64 integer :: a(n), b(n), ref(n) integer :: s, t integer :: i, host_flag logical :: ok do i = 1, n a(i) = i b(i) = 0 ref(i) = 2*i + 1 end do host_flag = 1 s = 0 !$omp target parallel do map(to: a) map(tofrom: s, host_flag) private(t) do i = 1, n !$omp atomic s = s + a(i) if (i == 1) host_flag = merge(1, 0, omp_is_initial_device()) end do !$omp end target parallel do ok = s == n*(n + 1)/2 print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', ok) end program wide_prog
Other compilers run it correctly
GCC 16.2.0