Observed issues › oop/cce-copy-second-member-section-lost
the second of two member sections of a structure is not copied back
Legality: independently confirmed. 2 independent readings find the program legal and the result required. How this is checked
map(tofrom: f%b(3:6), f%d(3:6)) on one target construct: the device's writes to f%d do not come back with CCE 19, 20 and 21 when memory is separate (f%b does). One member section works; unified memory works; AFAR and GCC are correct.
Affected variants
| Compiler | MI300A gfx942 unified memory (XNACK 1) | MI300A gfx942 copy (XNACK 0) | MI250X gfx90a copy (XNACK 0) | MI300X gfx942 copy (XNACK 0) | MI350X gfx950 copy (XNACK 0) | A100 sm_80 separate memory | A100 sm_80 managed memory |
|---|---|---|---|---|---|---|---|
| AFAR 24.3.0 | correct | correct | unknown not run | correct | correct | n/a | n/a |
| CCE 19.0.0 | correct | behaves incorrectly | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 20.0.2 | correct | behaves incorrectly | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.3 | correct | behaves incorrectly | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.4 | correct | behaves incorrectly | unknown not run | unknown not run | unknown not run | n/a | n/a |
| NVHPC 25.11 | n/a | n/a | n/a | n/a | n/a | correct | correct |
| NVHPC 26.9 | n/a | n/a | n/a | n/a | n/a | correct | correct |
| GCC 16.2.0 | correct | correct | correct | correct | correct | correct | n/a |
"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.
Messages
| Result | Compiler and GPU | Message |
|---|---|---|
| behaves incorrectly | CCE 19.0.0, MI300A copy CCE 20.0.2, MI300A copy CCE 21.0.3, MI300A copy CCE 21.0.4, MI300A copy | showr1= 18 r2= 0 RESULT ran_on_host=0 check=fail |
Reproduce
CCE 19.0.0, MI300A copy
module load cce/19.0.0 craype-accel-amd-gfx942 rocm/6.4.3 module unload cray-mpich ftn -fopenmp -O2 repro.f90 -o repro HSA_XNACK=0 OMP_TARGET_OFFLOAD=mandatory ./repro
CCE 20.0.2, MI300A copy
module load cce/20.0.2 craype-accel-amd-gfx942 rocm/6.4.3 module unload cray-mpich ftn -fopenmp -O2 repro.f90 -o repro HSA_XNACK=0 OMP_TARGET_OFFLOAD=mandatory ./repro
CCE 21.0.3, MI300A copy
module load cce/21.0.3 craype-accel-amd-gfx942 rocm/7.2.0 module unload cray-mpich ftn -fopenmp -O2 repro.f90 -o repro HSA_XNACK=0 OMP_TARGET_OFFLOAD=mandatory ./repro
CCE 21.0.4, MI300A copy
module load cce/21.0.4 craype-accel-amd-gfx942 rocm/7.2.0 module unload cray-mpich ftn -fopenmp -O2 repro.f90 -o repro HSA_XNACK=0 OMP_TARGET_OFFLOAD=mandatory ./repro
AFAR 24.3.0, MI300A unified (runs correctly)
$AFAR/bin/amdflang -fopenmp --offload-arch=gfx942 -O2 repro.f90 -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
Program (59 lines)
! # Finding (fortran-oop): two member sections of one structure on one map clause: the second is not copied back (copy mode)
!
! - toolchains and modes that fail: cce19 (CCE 19.0.0: copy), cce20 (CCE 20.0.2: copy), cce21 (CCE 21.0.3: copy), cce21_4 (CCE
! 21.0.4: copy); -O2
! - build: ftn -fopenmp -O2 (module load cce/<v> craype-accel-amd-gfx942 rocm/<r>; OpenACC: ftn -h acc)
! - failure: wrong_answer (check=fail); OMP_TARGET_OFFLOAD=mandatory
! - observed: `r1= 18 r2= 0` at HSA_XNACK=0 on every CCE release: f%b(3:6) comes back, f%d(3:6) does not; with two structures the
! second member of each is lost (`r1= 18 r2= 54`, expected 54 126); HSA_XNACK=1 correct
! - expected: `r1= 18 r2= 36`
! - rule: 5.2 §5.8.3 p.152 l.28-31 (list items with the same containing structure form one mappable storage block); §5.8.3 p.154
! exit step 1a (each tofrom list item is copied back); §13.8 p.286 l.27-28 (every referenced element is mapped)
! - same-compiler control: one member section (controls/section-fixed.f90): correct on every CCE in both modes
! (results/oop/main/rows.part*.jsonl)
! - other families: AFAR and GCC 16.2 correct; the AOMP leads derived-type-member-array-map-3 and
! double-derived-type-double-member-array-map also fail on CCE
! - gate and confirmer: 8 cells, 8 confirmed in the gate's four environments (a-d; results/oop/main/gate_reruns*.jsonl); confirmer
! (results/oop/main/confirm.jsonl): needs_human (R22-expected-weak) 8; R22-expected-weak means the expected value has one
! source, the generator's Python model; the values follow from the program's arithmetic (checked by hand), and AFAR, GCC 16.2
! (gfx90a), GCC 16.2 (sm_80), NVHPC 25.11, NVHPC 26.9 print them on this program
! - programs (2): oop-elemsec-two-members, oop-elemsec-two-structs (offload/engine/oop_gen.py; this file is the first one)
! - legality: for the cce cells: the rules (offload.engine.rules, with the fortran-oop restrictions) and AFAR's flang front end
! (-fsyntax-only) and NVHPC 26.9 nvfortran (compile only) (tools/campaigns/oop.py legality, results/oop/main/legality.json)
module oop_m
implicit none
integer, parameter :: n = 64
type da
integer :: b(10) = 0
integer :: d(10) = 0
end type da
end module oop_m
program oop
use omp_lib
use oop_m
implicit none
integer :: host_flag
logical :: ok
integer :: r1, r2
type(da) :: f
integer :: l
host_flag = 1
!$omp target map(from: host_flag)
host_flag = merge(1, 0, omp_is_initial_device())
!$omp end target
!$omp target map(tofrom: f%b(3:6), f%d(3:6))
do l = 3, 6
f%b(l) = l
f%d(l) = 2 * l
end do
!$omp end target
r1 = sum(f%b)
r2 = sum(f%d)
ok = .true.
ok = ok .and. (r1 == 18)
ok = ok .and. (r2 == 36)
print '(a,i0)', 'r1= ', r1
print '(a,i0)', 'r2= ', r2
print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', ok)
end program oop
Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page
Expected and observed
Expected
r2= 36
Observed where it behaves incorrectly
r1= 18 r2= 0 RESULT ran_on_host=0 check=fail
What the specification requires
The specification text this program relies on and the compiler is expected to support.
OpenMP 5.2 §5.8.3 p.152 l.28-31
OpenMP 5.2 §5.8.3 p.154
OpenMP 5.2 §13.8 p.286 l.27-28
Evidence
Same compiler, one change
- one member section instead of two: not run yet
Control program
! control (fortran-oop family program oop-section-fixed, correct on every CCE in both modes) ! fortran-oop section/fixed (OpenMP 5.2, free-form Fortran): ! map(tofrom: s%b(17:48)): a section of a fixed-size array component ! spec: OpenMP 5.2 §5.8.3 p.151 l.13 (structure elements and array sections are list items); §5.8.3 ! p.157 l.21-22; OpenMP 5.2 §13.8 p.286 l.27-28 (other referenced elements of a mapped structure ! element are mapped too) ! legality: defined ! expected (computed by compliance/oop_gen.py): r1= 3120 module oop_m implicit none integer, parameter :: n = 64 type fs integer :: b(n) = 0 integer :: k = 0 end type fs end module oop_m program oop use omp_lib use oop_m implicit none integer :: host_flag logical :: ok integer :: r1 type(fs) :: s integer :: i host_flag = 1 !$omp target map(from: host_flag) host_flag = merge(1, 0, omp_is_initial_device()) !$omp end target do i = 1, n s%b(i) = i end do !$omp target teams distribute parallel do map(tofrom: s%b(17:48)) do i = 17, 48 s%b(i) = 2 * s%b(i) end do r1 = sum(s%b) ok = .true. ok = ok .and. (r1 == 3120) print '(a,i0)', 'r1= ', r1 print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', ok) end program oop
Other compilers run it correctly
AFAR 24.3.0, NVHPC 25.11, NVHPC 26.9, GCC 16.2.0