OffTarget

Observed issues › fuzz/b3-278-cce19

Appears to behave incorrectlyCrashes or aborts at run time · Fortran · OpenMP · Changed in 21.0.3

ftn: target teams with reduction(max) reading an allocatable component of an implicitly mapped structure ends in an illegal memory access

Legality: independently confirmed. One independent reading was uncertain; 2 adjudicating readings that weighed both find the program legal and the result required: dr, ib and ic are non-scalars implicitly mapped tofrom; the predefined default mapper maps each structure element of dr, including the allocatable component q, and 5.2 added support for allocatable components. j is a Fortran loop variable inside a teams construct and so private; i is the distribute loop variable. r1b is a max reduction and the ic index stays in 1..5, so the program is race free. A device fault is a compiler or runtime error. How this is checked

Affected variants

CompilerMI300A gfx942
unified memory (XNACK 1)
MI300A gfx942
copy (XNACK 0)
MI250X gfx90a
copy (XNACK 0)
MI300X gfx942
copy (XNACK 0)
MI350X gfx950
copy (XNACK 0)
A100 sm_80
separate memory
A100 sm_80
managed memory
AFAR 24.3.0correctcorrectunknown
not run
correctcorrectn/an/a
CCE 19.0.0behaves incorrectlycorrectunknown
not run
unknown
not run
unknown
not run
n/an/a
CCE 20.0.2behaves incorrectlycorrectunknown
not run
unknown
not run
unknown
not run
n/an/a
CCE 21.0.3fails another wayfails another wayunknown
not run
unknown
not run
unknown
not run
n/an/a
CCE 21.0.4fails another wayfails another wayunknown
not run
unknown
not run
unknown
not run
n/an/a
NVHPC 25.11n/an/an/an/an/afails another waycorrect
NVHPC 26.9n/an/an/an/an/afails another waycorrect
GCC 16.2.0correctcorrectcorrectcorrectcorrectcorrectn/a

"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.

Messages

ResultCompiler and GPUMessage
behaves incorrectlyCCE 19.0.0, MI300A unified
show
Memory access fault by GPU node-7 (Agent handle: 0x4c9cb0) on address 0x3ff000. Reason: Unknown.
behaves incorrectlyCCE 20.0.2, MI300A unified
show
Memory access fault by GPU node-5 (Agent handle: 0x4c9f90) on address 0x3fd000. Reason: Unknown.
fails another wayCCE 21.0.3, MI300A unified
CCE 21.0.4, MI300A unified
show
ACC: libcrayacc/acc_hw_amd.c:1927 CRAY_ACC_ERROR -  hipStreamSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' from fuzz-b3-278-cce19.f90:26
ACC: libcrayacc/acc_hw_amd.c:1476 CRAY_ACC_ERROR -  hipDeviceSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered'
fails another wayCCE 21.0.3, MI300A copy
CCE 21.0.4, MI300A copy
show
ACC: libcrayacc/acc_hw_amd.c:1496 CRAY_ACC_ERROR -  hipStreamSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' from fuzz-b3-278-cce19.f90:34
ACC: libcrayacc/acc_hw_amd.c:1476 CRAY_ACC_ERROR -  hipDeviceSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered'
fails another wayNVHPC 25.11, A100 separate
NVHPC 26.9, A100 separate
show
Failing in Thread:1
Accelerator Fatal Error: call to cuMemcpyDtoHAsync returned error 700 (CUDA_ERROR_ILLEGAL_ADDRESS): Illegal address during kernel execution
 File: <dir>/fuzz-b3-278-cce19.f90
 Function: fuzz_prog:1
 Line: 34

Reproduce

CCE 19.0.0, MI300A unified

module load cce/19.0.0 craype-accel-amd-gfx942 rocm/6.4.3
module unload cray-mpich
ftn -fopenmp -O2 repro.f90 -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro

CCE 20.0.2, MI300A unified

module load cce/20.0.2 craype-accel-amd-gfx942 rocm/6.4.3
module unload cray-mpich
ftn -fopenmp -O2 repro.f90 -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro

AFAR 24.3.0, MI300A unified (runs correctly)

$AFAR/bin/amdflang -fopenmp --offload-arch=gfx942 -O2 repro.f90 -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
Program (63 lines)
! Minimized by compliance.fuzz_min from fuzz seed 278 (fuzz-3).
! toolchain: cce19 (-O2, HSA_XNACK=1, OMP_TARGET_OFFLOAD=mandatory)
! failure: run_error (crash: SIGABRT)
! expected: runs to completion (the host run does)
! observed: device: fail crash: SIGABRT; host reference (afar) checksum=144
! Legal OpenMP 5.2: offload.engine.prerun.check accepts it (rerun: offload.engine.rules with the
!   race rule and the restriction checks, and the AFAR flang front end, not under test here); no
!   compiler under test, gfortran or gcc is a legality witness.
! also failed on (before refinement): cce20
! : CCE 21.0.3 and 21.0.4 fail it too (hipErrorIllegalAddress, HSA_XNACK 0
!   and 1; reruns); it was recorded as correct on CCE 21.
program fuzz_prog
  use omp_lib, only: omp_is_initial_device
  implicit none
  integer, parameter :: n = 64, m = 5
  type :: rec_t
    integer, allocatable :: q(:)
  end type rec_t
  type(rec_t) :: dr
  integer, allocatable, target :: ib(:)
  integer :: ic(n, m)
  integer :: i, j, r1b
  integer(8) :: cs
  allocate(ib(n))
  allocate(dr%q(n))
  do i = 1, n
    ib(i) = mod(i * 42 + 15, 101) - 28
  end do
  do j = 1, m
    do i = 1, n
      ic(i, j) = mod((i + 6 * j) * 19 + 21, 97) - 20
    end do
  end do
  do i = 1, n
    dr%q(i) = mod(i * 17 + 13, 1009) - 449
  end do
  r1b = -13
  ! fuzz: kernels
    !$omp target teams reduction(max: r1b)
      !$omp distribute
      do i = n, 1, -1
        do j = 1, m
          r1b = max(r1b, max((dr%q(i) / 8), (ic(8, (6 - j)) + ib(i))))
        end do
      end do
      !$omp end distribute
    !$omp end target teams
  ! fuzz: checksum
  cs = 0
  cs = modulo(cs * 31_8 + int(r1b, 8), 2147483629_8)
  print '(a,i0,a,i0)', 'RESULT checksum=', cs, ' ran_on_host=', 0
end program fuzz_prog
! spec (requirement): a structure variable that a target region references, with no clause naming it, is mapped tofrom with each
!   structure element (an allocatable component included), and a reduction on a combined target teams construct is combined into
!   the original list item.
! "A structure type T has a predefined default mapper that is defined as if by a declare mapper directive that specifies v in a
!   map clause with the alloc map-type and each structure element of v in a map clause with the tofrom map-type." (5.2 §5.8.2
!   p.150 l.2-4)
! "If a variable is not a scalar then it is treated as if it had appeared in a map clause with a map-type of tofrom." (5.2
!   §5.8.1 p.149 l.4-5)
! spec (this program): target teams reduction(max: r1b) reads the allocatable component dr%q(i) of the implicitly mapped
!   structure dr in the distribute loop; the component is mapped with the structure, but CCE 19, 20 and 21 end in an illegal
!   memory access, and the program is correct when the loop reads ib(i) instead.

Source in the repository

Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page

Expected and observed

Observed where it behaves incorrectly

Memory access fault by GPU node-7 (Agent handle: 0x4c9cb0) on address 0x3ff000. Reason: Unknown.

What the specification requires

The specification text this program relies on and the compiler is expected to support.

OpenMP 5.2 §5.8.2 p.150 l.2-4

A structure type T has a predefined default mapper that is defined as if by a declare mapper directive that specifies v in a map clause with the alloc map-type and each structure element of v in a map clause with the tofrom map-type.

OpenMP 5.2 §5.8.1 p.149 l.4-5

If a variable is not a scalar then it is treated as if it had appeared in a map clause with a map-type of tofrom.

Evidence

Same compiler, one change

  • the device loop reads ib(i) instead of the allocatable component dr%q(i) (dr is not referenced in the region): AFAR 24.3.0: correct; CCE 19.0.0: correct; CCE 20.0.2: correct; CCE 21.0.3: correct; CCE 21.0.4: correct
    Control program
    program fuzz_prog
      implicit none
      integer, parameter :: n = 64, m = 5
      type :: rec_t
        integer, allocatable :: q(:)
      end type rec_t
      type(rec_t) :: dr
      integer, allocatable, target :: ib(:)
      integer :: ic(n, m)
      integer :: i, j, r1b
      integer(8) :: cs
      allocate(ib(n))
      allocate(dr%q(n))
      do i = 1, n
        ib(i) = mod(i * 42 + 15, 101) - 28
      end do
      do j = 1, m
        do i = 1, n
          ic(i, j) = mod((i + 6 * j) * 19 + 21, 97) - 20
        end do
      end do
      do i = 1, n
        dr%q(i) = mod(i * 17 + 13, 1009) - 449
      end do
      r1b = -13
        !$omp target teams reduction(max: r1b)
          !$omp distribute
          do i = n, 1, -1
            do j = 1, m
              r1b = max(r1b, max((ib(i) / 8), (ic(8, (6 - j)) + ib(i))))
            end do
          end do
          !$omp end distribute
        !$omp end target teams
      cs = 0
      cs = modulo(cs * 31_8 + int(r1b, 8), 2147483629_8)
      print '(a,i0)', 'RESULT checksum=', cs
    end program fuzz_prog
    

Other compilers run it correctly

AFAR 24.3.0, NVHPC 25.11, NVHPC 26.9, GCC 16.2.0