Observed issues › catalog/afar-flang-ordered-wrong
Appears to behave incorrectlyWrong answer · Fortran · OpenMP · Open
amdflang: ordered regions in a parallel do in a target region do not run in iteration order
Legality: independently confirmed. 2 independent readings find the program legal and the result required. How this is checked
Affected variants
| Compiler | MI300A gfx942 unified memory (XNACK 1) | MI300A gfx942 copy (XNACK 0) | MI250X gfx90a copy (XNACK 0) | MI300X gfx942 copy (XNACK 0) | MI350X gfx950 copy (XNACK 0) | A100 sm_80 separate memory | A100 sm_80 managed memory |
|---|---|---|---|---|---|---|---|
| AFAR 24.3.0 | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | n/a | n/a |
| CCE 19.0.0 | fails another way | fails another way | fails another way | unknown not run | unknown not run | n/a | n/a |
| CCE 20.0.2 | fails another way | fails another way | fails another way | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.3 | fails another way | fails another way | fails another way | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.4 | fails another way | fails another way | fails another way | unknown not run | unknown not run | n/a | n/a |
| NVHPC 25.11 | n/a | n/a | n/a | n/a | n/a | fails another way | fails another way |
| NVHPC 26.9 | n/a | n/a | n/a | n/a | n/a | fails another way | fails another way |
| GCC 16.2.0 | correct | correct | correct | correct | correct | correct | n/a |
"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.
Messages
| Result | Compiler and GPU | Message |
|---|---|---|
| behaves incorrectly | AFAR 24.3.0, MI300A unified AFAR 24.3.0, MI300A copy AFAR 24.3.0, MI250X AFAR 24.3.0, MI300X AFAR 24.3.0, MI350X | showRESULT ran_on_host=0 check=fail |
| fails another way | CCE 19.0.0, MI300A unified CCE 19.0.0, MI300A copy CCE 19.0.0, MI250X | showWarning: Headers and libraries from cray-libsci/25.03.0 will be ignored because depends on PE_MPICH which is unavailable.. MT_CONTEXT_ORDERED Error message :: unhandled construct type Error detected :: File '/workspace/crayftn/pdgcs/v_openmp.c', line 7617 Initiated from :: Line 660 (v_main.c) Optimizer built :: 2025-02-07 (production) File :: afar-flang-ordered-wrong.f90 |
| fails another way | CCE 20.0.2, MI300A unified CCE 20.0.2, MI300A copy CCE 20.0.2, MI250X | showWarning: Headers and libraries from cray-libsci/25.09.0 will be ignored because depends on PE_MPICH which is unavailable.. MT_CONTEXT_ORDERED Error message :: unhandled construct type Error detected :: File '/workspace/crayftn/pdgcs/v_openmp.c', line 7814 Initiated from :: Line 660 (v_main.c) Optimizer built :: 2025-11-17 (production) File :: afar-flang-ordered-wrong.f90 |
| fails another way | CCE 21.0.3, MI300A unified CCE 21.0.3, MI300A copy CCE 21.0.3, MI250X | showWarning: Headers and libraries from cray-libsci/26.03.0 will be ignored because depends on PE_MPICH which is unavailable.. MT_CONTEXT_ORDERED Error message :: unhandled construct type Error detected :: File '/workspace/crayftn/pdgcs/v_openmp.c', line 8197 Initiated from :: Line 732 (v_main.c) Optimizer built :: 2026-07-14 (production) File :: afar-flang-ordered-wrong.f90 |
| fails another way | CCE 21.0.4, MI300A unified CCE 21.0.4, MI300A copy CCE 21.0.4, MI250X | showWarning: Headers and libraries from cray-libsci/26.03.0 will be ignored because depends on PE_MPICH which is unavailable.. MT_CONTEXT_ORDERED Error message :: unhandled construct type Error detected :: File '/workspace/crayftn/pdgcs/v_openmp.c', line 8197 Initiated from :: Line 732 (v_main.c) Optimizer built :: 2026-09-24 (production) File :: afar-flang-ordered-wrong.f90 |
| fails another way | NVHPC 25.11, A100 separate NVHPC 25.11, A100 managed | showNVFORTRAN-F-1196-OpenMP - ordered is not supported in GPU region (afar-flang-ordered-wrong.f90: 24) NVFORTRAN/x86-64 Linux 25.11-0: compilation aborted |
| fails another way | NVHPC 26.9, A100 separate NVHPC 26.9, A100 managed | showNVFORTRAN-F-1196-OpenMP - ordered is not supported in GPU region (afar-flang-ordered-wrong.f90: 24) NVFORTRAN/x86-64 Linux 26.9-0: compilation aborted |
Reproduce
AFAR 24.3.0, MI300A unified
$AFAR/bin/amdflang -fopenmp --offload-arch=gfx942 -O2 repro.f90 -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
GCC 16.2.0, MI300A unified (runs correctly)
export LD_LIBRARY_PATH=$GCC_16_2/lib64:/opt/rocm-7.2.1/lib $GCC_16_2/bin/gfortran -fopenmp -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx942 -O2 repro.f90 -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
Program (72 lines)
! # Draft finding: omp52-clause-ordered on afar
!
! - compiler: afar
! - toolchain: afar; arch: gfx942; machine: tuolumne; memory mode: unified; opt: -O2
! - program: catalog omp52-clause-ordered (f90, version 52, sha256 50b2a05bbea64844)
! - expected: check=pass (trusted; elision + model)
! - actual: wrong_answer check=fail (check_fail: ran_on_host=0 check=fail)
! - verdict: compiler_bug by rule R20-runtime-bug: a legal program fails at run time, deterministically, against an expected value
! from two or more independent sources with an executed witness, and another toolchain, mode, arch or the elision gives the
! expected value
! - reason: expected trusted check=pass (elision + model): stored main verdict: elision + model agree; witnesses: afar/gfx942/copy
! (port:omp52-c-clause-ordered), afar/gfx942/unified (port:omp52-c-clause-ordered), elision
! - citations: RUL-DERIVED-NEEDS-EXECUTION (rulings.md); RUL-WITNESS-PER-MODE (rulings.md);
! RUL-SAME-COMPILER-RUNS (rulings.md)
! - legality: legal (legal by construction: offload.engine.gen / c_gen / acc_gen check every program against the catalog rules
! and the
! offload scope as it is built (no compiler))
! - witnesses: afar/gfx942/copy (port:omp52-c-clause-ordered), afar/gfx942/unified (port:omp52-c-clause-ordered), elision
! - withdrawn: none
!
! Generated by compliance.gen (OpenMP 5.2, free-form Fortran). Do not edit.
! id: omp52-clause-ordered
! primary atom: omp.clause.ordered
! legal: yes
! observable: yes
! ran_on_host: inside the target region of interest
! evidence: construct
! needs: none
! std: f2008
program gen_prog
use omp_lib, only: omp_is_initial_device
implicit none
integer, parameter :: n = 64
integer, allocatable, target :: a(:), b(:)
integer :: ref(n)
integer :: i, host_flag
logical :: ok
integer :: cnt, seq(n)
allocate(a(n), b(n))
do i = 1, n
a(i) = i
b(i) = 0
ref(i) = 2*i + 1
end do
host_flag = 1
cnt = 0
seq = -1
!$omp target map(to: a) map(from: b) map(tofrom: cnt) map(from: seq) map(tofrom: host_flag)
host_flag = merge(1, 0, omp_is_initial_device())
!$omp parallel do default(none) shared(a, b, cnt, seq) ordered
do i = 1, n
b(i) = 2*a(i) + 1
!$omp ordered
cnt = cnt + 1
seq(i) = cnt
!$omp end ordered
end do
!$omp end parallel do
!$omp end target
ok = all(b == ref) .and. all(2*seq + 1 == ref)
print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', ok)
deallocate(a, b)
end program gen_prog
! spec (requirement): Ordered regions of a worksharing loop with an ordered clause execute sequentially in the order
! of the loop iterations.
! "the threads in the team that is executing the worksharing-loop region execute ordered regions sequentially in the
! order of the loop iterations." (5.2 §15.10.2 p.330 l.21-22)
! spec (requirement): An ordered construct without clauses has the effect of the threads clause.
! "If no clauses are specified, the effect is as if the threads parallelization-level clause was specified." (5.2
! §15.10.2 p.330 l.20-21)
! spec (this program): The ordered region increments cnt and records seq(i), so the regions run in iteration order and
! seq(i) = i for every i; on the device amdflang runs them out of order and check=fail results.
Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page
Expected and observed
Expected
check=pass
Observed where it behaves incorrectly
RESULT ran_on_host=0 check=fail
What the specification requires
The specification text this program relies on and the compiler is expected to support.
OpenMP 5.2 §15.10.2 p.330 l.21-22
the threads in the team that is executing the worksharing-loop region execute ordered regions sequentially in the order of the loop iterations.
OpenMP 5.2 §15.10.2 p.330 l.20-21
If no clauses are specified, the effect is as if the threads parallelization-level clause was specified.
Evidence
Same compiler, one change
- the same parallel do on the host (the target directive removed): AFAR 24.3.0: correct; CCE 19.0.0: correct; CCE 20.0.2: correct; CCE 21.0.3: correct; CCE 21.0.4: correct
Control program
program gen_prog use omp_lib, only: omp_is_initial_device implicit none integer, parameter :: n = 64 integer, allocatable, target :: a(:), b(:) integer :: ref(n) integer :: i, host_flag logical :: ok integer :: cnt, seq(n) allocate(a(n), b(n)) do i = 1, n a(i) = i b(i) = 0 ref(i) = 2*i + 1 end do host_flag = 1 cnt = 0 seq = -1 host_flag = merge(1, 0, omp_is_initial_device()) !$omp parallel do default(none) shared(a, b, cnt, seq) ordered do i = 1, n b(i) = 2*a(i) + 1 !$omp ordered cnt = cnt + 1 seq(i) = cnt !$omp end ordered end do !$omp end parallel do ok = all(b == ref) .and. all(2*seq + 1 == ref) print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', ok) deallocate(a, b) end program gen_prog
Other compilers run it correctly
GCC 16.2.0