Observed issues › fuzz/b3-278-cce19
ftn: target teams with reduction(max) reading an allocatable component of an implicitly mapped structure ends in an illegal memory access
Legality: independently confirmed. One independent reading was uncertain; 2 adjudicating readings that weighed both find the program legal and the result required: dr, ib and ic are non-scalars implicitly mapped tofrom; the predefined default mapper maps each structure element of dr, including the allocatable component q, and 5.2 added support for allocatable components. j is a Fortran loop variable inside a teams construct and so private; i is the distribute loop variable. r1b is a max reduction and the ic index stays in 1..5, so the program is race free. A device fault is a compiler or runtime error. How this is checked
Affected variants
| Compiler | MI300A gfx942 unified memory (XNACK 1) | MI300A gfx942 copy (XNACK 0) | MI250X gfx90a copy (XNACK 0) | MI300X gfx942 copy (XNACK 0) | MI350X gfx950 copy (XNACK 0) | A100 sm_80 separate memory | A100 sm_80 managed memory |
|---|---|---|---|---|---|---|---|
| AFAR 24.3.0 | correct | correct | unknown not run | correct | correct | n/a | n/a |
| CCE 19.0.0 | behaves incorrectly | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 20.0.2 | behaves incorrectly | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.3 | fails another way | fails another way | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.4 | fails another way | fails another way | unknown not run | unknown not run | unknown not run | n/a | n/a |
| NVHPC 25.11 | n/a | n/a | n/a | n/a | n/a | fails another way | correct |
| NVHPC 26.9 | n/a | n/a | n/a | n/a | n/a | fails another way | correct |
| GCC 16.2.0 | correct | correct | correct | correct | correct | correct | n/a |
"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.
Messages
| Result | Compiler and GPU | Message |
|---|---|---|
| behaves incorrectly | CCE 19.0.0, MI300A unified | showMemory access fault by GPU node-7 (Agent handle: 0x4c9cb0) on address 0x3ff000. Reason: Unknown. |
| behaves incorrectly | CCE 20.0.2, MI300A unified | showMemory access fault by GPU node-5 (Agent handle: 0x4c9f90) on address 0x3fd000. Reason: Unknown. |
| fails another way | CCE 21.0.3, MI300A unified CCE 21.0.4, MI300A unified | showACC: libcrayacc/acc_hw_amd.c:1927 CRAY_ACC_ERROR - hipStreamSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' from fuzz-b3-278-cce19.f90:26 ACC: libcrayacc/acc_hw_amd.c:1476 CRAY_ACC_ERROR - hipDeviceSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' |
| fails another way | CCE 21.0.3, MI300A copy CCE 21.0.4, MI300A copy | showACC: libcrayacc/acc_hw_amd.c:1496 CRAY_ACC_ERROR - hipStreamSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' from fuzz-b3-278-cce19.f90:34 ACC: libcrayacc/acc_hw_amd.c:1476 CRAY_ACC_ERROR - hipDeviceSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' |
| fails another way | NVHPC 25.11, A100 separate NVHPC 26.9, A100 separate | showFailing in Thread:1 Accelerator Fatal Error: call to cuMemcpyDtoHAsync returned error 700 (CUDA_ERROR_ILLEGAL_ADDRESS): Illegal address during kernel execution File: <dir>/fuzz-b3-278-cce19.f90 Function: fuzz_prog:1 Line: 34 |
Reproduce
CCE 19.0.0, MI300A unified
module load cce/19.0.0 craype-accel-amd-gfx942 rocm/6.4.3 module unload cray-mpich ftn -fopenmp -O2 repro.f90 -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
CCE 20.0.2, MI300A unified
module load cce/20.0.2 craype-accel-amd-gfx942 rocm/6.4.3 module unload cray-mpich ftn -fopenmp -O2 repro.f90 -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
AFAR 24.3.0, MI300A unified (runs correctly)
$AFAR/bin/amdflang -fopenmp --offload-arch=gfx942 -O2 repro.f90 -o repro HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
Program (63 lines)
! Minimized by compliance.fuzz_min from fuzz seed 278 (fuzz-3).
! toolchain: cce19 (-O2, HSA_XNACK=1, OMP_TARGET_OFFLOAD=mandatory)
! failure: run_error (crash: SIGABRT)
! expected: runs to completion (the host run does)
! observed: device: fail crash: SIGABRT; host reference (afar) checksum=144
! Legal OpenMP 5.2: offload.engine.prerun.check accepts it (rerun: offload.engine.rules with the
! race rule and the restriction checks, and the AFAR flang front end, not under test here); no
! compiler under test, gfortran or gcc is a legality witness.
! also failed on (before refinement): cce20
! : CCE 21.0.3 and 21.0.4 fail it too (hipErrorIllegalAddress, HSA_XNACK 0
! and 1; reruns); it was recorded as correct on CCE 21.
program fuzz_prog
use omp_lib, only: omp_is_initial_device
implicit none
integer, parameter :: n = 64, m = 5
type :: rec_t
integer, allocatable :: q(:)
end type rec_t
type(rec_t) :: dr
integer, allocatable, target :: ib(:)
integer :: ic(n, m)
integer :: i, j, r1b
integer(8) :: cs
allocate(ib(n))
allocate(dr%q(n))
do i = 1, n
ib(i) = mod(i * 42 + 15, 101) - 28
end do
do j = 1, m
do i = 1, n
ic(i, j) = mod((i + 6 * j) * 19 + 21, 97) - 20
end do
end do
do i = 1, n
dr%q(i) = mod(i * 17 + 13, 1009) - 449
end do
r1b = -13
! fuzz: kernels
!$omp target teams reduction(max: r1b)
!$omp distribute
do i = n, 1, -1
do j = 1, m
r1b = max(r1b, max((dr%q(i) / 8), (ic(8, (6 - j)) + ib(i))))
end do
end do
!$omp end distribute
!$omp end target teams
! fuzz: checksum
cs = 0
cs = modulo(cs * 31_8 + int(r1b, 8), 2147483629_8)
print '(a,i0,a,i0)', 'RESULT checksum=', cs, ' ran_on_host=', 0
end program fuzz_prog
! spec (requirement): a structure variable that a target region references, with no clause naming it, is mapped tofrom with each
! structure element (an allocatable component included), and a reduction on a combined target teams construct is combined into
! the original list item.
! "A structure type T has a predefined default mapper that is defined as if by a declare mapper directive that specifies v in a
! map clause with the alloc map-type and each structure element of v in a map clause with the tofrom map-type." (5.2 §5.8.2
! p.150 l.2-4)
! "If a variable is not a scalar then it is treated as if it had appeared in a map clause with a map-type of tofrom." (5.2
! §5.8.1 p.149 l.4-5)
! spec (this program): target teams reduction(max: r1b) reads the allocatable component dr%q(i) of the implicitly mapped
! structure dr in the distribute loop; the component is mapped with the structure, but CCE 19, 20 and 21 end in an illegal
! memory access, and the program is correct when the loop reads ib(i) instead.
Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page
Expected and observed
Observed where it behaves incorrectly
Memory access fault by GPU node-7 (Agent handle: 0x4c9cb0) on address 0x3ff000. Reason: Unknown.
What the specification requires
The specification text this program relies on and the compiler is expected to support.
OpenMP 5.2 §5.8.2 p.150 l.2-4
A structure type T has a predefined default mapper that is defined as if by a declare mapper directive that specifies v in a map clause with the alloc map-type and each structure element of v in a map clause with the tofrom map-type.
OpenMP 5.2 §5.8.1 p.149 l.4-5
If a variable is not a scalar then it is treated as if it had appeared in a map clause with a map-type of tofrom.
Evidence
Same compiler, one change
- the device loop reads ib(i) instead of the allocatable component dr%q(i) (dr is not referenced in the region): AFAR 24.3.0: correct; CCE 19.0.0: correct; CCE 20.0.2: correct; CCE 21.0.3: correct; CCE 21.0.4: correct
Control program
program fuzz_prog implicit none integer, parameter :: n = 64, m = 5 type :: rec_t integer, allocatable :: q(:) end type rec_t type(rec_t) :: dr integer, allocatable, target :: ib(:) integer :: ic(n, m) integer :: i, j, r1b integer(8) :: cs allocate(ib(n)) allocate(dr%q(n)) do i = 1, n ib(i) = mod(i * 42 + 15, 101) - 28 end do do j = 1, m do i = 1, n ic(i, j) = mod((i + 6 * j) * 19 + 21, 97) - 20 end do end do do i = 1, n dr%q(i) = mod(i * 17 + 13, 1009) - 449 end do r1b = -13 !$omp target teams reduction(max: r1b) !$omp distribute do i = n, 1, -1 do j = 1, m r1b = max(r1b, max((ib(i) / 8), (ic(8, (6 - j)) + ib(i)))) end do end do !$omp end distribute !$omp end target teams cs = 0 cs = modulo(cs * 31_8 + int(r1b, 8), 2147483629_8) print '(a,i0)', 'RESULT checksum=', cs end program fuzz_prog
Other compilers run it correctly
AFAR 24.3.0, NVHPC 25.11, NVHPC 26.9, GCC 16.2.0