OffTarget

Observed issues › harvest/gcc-acc-declare-create-allocatable-not-mapped

Appears to behave incorrectlyRun-time error · Fortran · OpenACC · Open

OpenACC: declare create of a Fortran allocatable gives no device copy at the allocate statement

Legality: not independently confirmed. Independent readings are uncertain: Section 2.13.2 says a global-context declare sets the structured reference counter to one, and also says that for an allocatable the allocate statement raises a runtime error if that counter is not zero. Read literally, the program's allocate would be an error. How this is checked

OpenACC 2.6 and later make an allocate statement for an allocatable in a declare create directive allocate the device copy too. GCC 16.2 does not: the first compute construct that names the array stops with "is not mapped" (exit 1), on the MI250X and the A100. With an explicit enter data create after the allocate statement the same program runs correctly. CCE and NVHPC run the harvested programs correctly.

Affected variants

CompilerMI300A gfx942
unified memory (XNACK 1)
MI300A gfx942
copy (XNACK 0)
MI250X gfx90a
copy (XNACK 0)
MI300X gfx942
copy (XNACK 0)
MI350X gfx950
copy (XNACK 0)
A100 sm_80
separate memory
A100 sm_80
managed memory
AFAR 24.3.0n/an/an/an/an/an/an/a
CCE 19.0.0fails another waycorrectunknown
not run
unknown
not run
unknown
not run
n/an/a
CCE 20.0.2fails another waycorrectunknown
not run
unknown
not run
unknown
not run
n/an/a
CCE 21.0.3behaves incorrectlycorrectunknown
not run
unknown
not run
unknown
not run
n/an/a
CCE 21.0.4behaves incorrectlycorrectunknown
not run
unknown
not run
unknown
not run
n/an/a
NVHPC 25.11n/an/an/an/an/acorrectcorrect
NVHPC 26.9n/an/an/an/an/acorrectcorrect
GCC 16.2.0behaves incorrectlybehaves incorrectlybehaves incorrectlybehaves incorrectlybehaves incorrectlybehaves incorrectlyn/a

"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.

Messages

ResultCompiler and GPUMessage
behaves incorrectlyCCE 21.0.3, MI300A unified
CCE 21.0.4, MI300A unified
show
ACC: libcrayacc/acc_hw_amd.c:1927 CRAY_ACC_ERROR -  hipStreamSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' from site-harvest--gcc-acc-declare-create-allocatab
ACC: libcrayacc/acc_hw_amd.c:1476 CRAY_ACC_ERROR -  hipDeviceSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered'
behaves incorrectlyGCC 16.2.0, MI300A unified
GCC 16.2.0, MI300A copy
show
libgomp: [0x8d02e0,4000] is not mapped
behaves incorrectlyGCC 16.2.0, MI250X
show
libgomp: [0xa74c80,4000] is not mapped
behaves incorrectlyGCC 16.2.0, MI300X
show
libgomp: [0x1a283d0,4000] is not mapped
behaves incorrectlyGCC 16.2.0, MI350X
show
libgomp: [0x101a040,4000] is not mapped
behaves incorrectlyGCC 16.2.0, A100 separate
show
libgomp: [0x139b1840,4000] is not mapped
fails another wayCCE 19.0.0, MI300A unified
show
Memory access fault by GPU node-7 (Agent handle: 0x4c6a30) on address (nil). Reason: Write access to a read-only page.
fails another wayCCE 20.0.2, MI300A unified
show
Memory access fault by GPU node-5 (Agent handle: 0x4c7f10) on address (nil). Reason: Write access to a read-only page.

Reproduce

CCE 21.0.3, MI300A unified

module load cce/21.0.3 craype-accel-amd-gfx942 rocm/7.2.0
module unload cray-mpich
ftn -h acc -O2 repro.f90 -o repro
HSA_XNACK=1 ./repro

CCE 21.0.4, MI300A unified

module load cce/21.0.4 craype-accel-amd-gfx942 rocm/7.2.0
module unload cray-mpich
ftn -h acc -O2 repro.f90 -o repro
HSA_XNACK=1 ./repro

GCC 16.2.0, MI300A unified

export LD_LIBRARY_PATH=$GCC_16_2/lib64:/opt/rocm-7.2.1/lib
$GCC_16_2/bin/gfortran -fopenacc -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx942 -O2 repro.f90 -o repro
HSA_XNACK=1 ./repro

GCC 16.2.0, A100 separate

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=$GCC_16_2/lib64
$GCC_16_2/bin/gfortran -fopenacc -foffload=nvptx-none -foffload-options=nvptx-none=-march=sm_80 -foffload-options=nvptx-none=-latomic -O2 repro.f90 -o repro
./repro

NVHPC 25.11, A100 separate (runs correctly)

export LD_LIBRARY_PATH=$NVHPC_25_11/compilers/lib
nvfortran -acc=gpu -gpu=cc80 -gpu=mem:separate -O2 repro.f90 -o repro
./repro
Program (58 lines)
! # Finding (GCC 16.2, known-issues harvest): OpenACC declare create of a module allocatable: the allocate statement
!   creates no device copy ("is not mapped")
!
! - toolchains: gcc_g90 (GCC 16.2.0, AMD Instinct MI250X gfx90a, Tioga) and gcc_sm80 (GCC 16.2.0, NVIDIA A100 sm_80, wingtip);
!   separate memory only; -O2
! - build: gfortran -fopenacc -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx90a -O2 (gcc_g90);
!   -foffload=nvptx-none -foffload-options=nvptx-none=-march=sm_80 -foffload-options=nvptx-none=-latomic -O2 (gcc_sm80)
! - failure: run_error: is not mapped
! - observed: `libgomp: [0x...,4000] is not mapped` and exit 1 at the first compute construct that names the array (present(a)), in
!   every run (gcc_sm80: mandatory exit 5 of 5; unset exit 5 of 5; gcc_g90: mandatory exit 5 of 5; unset exit 5 of 5
!   (results/parity/<machine>/<tc>/probes.jsonl))
! - expected: `bad= 0`, then `RESULT ran_on_host=0 check=pass`
! - rule: in Fortran, for an allocatable in a declare create directive and a non-shared memory device, at an allocate statement
!   "memory will be allocated in both local memory as well as in the current device memory and the dynamic reference counter will be
!   set to one" (OpenACC 3.4 §2.13.2 l.3056-3057). The behaviour is OpenACC 2.6's (the changelog of OpenACC 3.4, "Changes from
!   Version 2.5 to 2.6", margin line 588: "The declare create directive with a Fortran allocatable has new behavior"), the version
!   GCC documents (libgomp.texi, "Enabling OpenACC": OpenACC 2.6).
! - same-compiler control (correct, gcc_sm80: mandatory pass 5 of 5; unset pass 5 of 5; gcc_g90: mandatory pass 5 of 5; unset pass 5
!   of 5 (results/parity/<machine>/<tc>/probes.jsonl)): the same program with `!$acc enter data create(a)` after the allocate
!   statement (controls/gcc-acc-declare-create-allocatable-control.f90).
! - other families: CCE 19/20/21 (copy mode, MI300A and MI250X) and NVHPC 25.11/26.9 (separate and managed memory) run the harvested
!   programs correctly (hk-007, hk-013, hk-087: results/harvest-known/main*). The site's finding matrix runs this file on every
!   compiler.
! - harvested programs (6 cells (3 gcc_g90, 3 gcc_sm80); gate: confirmed 6; confirmer: needs_human (R22-expected-weak) 6): hk-007,
!   hk-013, hk-087
! - legality: the pre-run check (offload.engine.prerun.check: the rules and AFAR's front end, two witnesses) accepts the harvested
!   programs; this file is read against the rule above. GCC is a compiler under test only and never a legality witness.
module declare_create_m
  implicit none
  integer, allocatable :: a(:)
  !$acc declare create(a)
end module declare_create_m

program declare_create_allocatable
  use openacc, only: acc_device_host, acc_on_device
  use declare_create_m
  implicit none
  integer, parameter :: n = 1000
  integer :: i, bad, host_flag
  allocate(a(n))
  a = -1
  !$acc parallel loop present(a)
  do i = 1, n
    a(i) = i
  end do
  !$acc update host(a)
  bad = 0
  do i = 1, n
    if (a(i) /= i) bad = bad + 1
  end do
  host_flag = 1
  !$acc parallel num_gangs(1) copy(host_flag)
  if (.not. acc_on_device(acc_device_host)) host_flag = 0
  !$acc end parallel
  print '(a,i0)', 'bad= ', bad
  print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', bad == 0)
  deallocate(a)
end program declare_create_allocatable

Source in the repository

Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page

Expected and observed

Expected

check=pass

Observed where it behaves incorrectly

ACC: libcrayacc/acc_hw_amd.c:1927 CRAY_ACC_ERROR -  hipStreamSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' from site-harvest--gcc-acc-declare-create-allocatab
ACC: libcrayacc/acc_hw_amd.c:1476 CRAY_ACC_ERROR -  hipDeviceSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered'

What the specification requires

The specification text this program relies on and the compiler is expected to support.

OpenACC 3.4 §2.13.2

memory will be allocated in both local memory as well as in the current device memory and the dynamic reference counter will be set to one

Evidence

Same compiler, one change

  • an enter data create after the allocate statement: not run yet
    Control program
    ! Control of harvest/gcc-acc-declare-create-allocatable-not-mapped.f90: the same program with an enter data create
    ! after the allocate statement (correct on gcc_g90 and gcc_sm80).
    module declare_create_m
      implicit none
      integer, allocatable :: a(:)
      !$acc declare create(a)
    end module declare_create_m
    
    program declare_create_allocatable
      use openacc, only: acc_device_host, acc_on_device
      use declare_create_m
      implicit none
      integer, parameter :: n = 1000
      integer :: i, bad, host_flag
      allocate(a(n))
      !$acc enter data create(a)
      a = -1
      !$acc parallel loop present(a)
      do i = 1, n
        a(i) = i
      end do
      !$acc update host(a)
      bad = 0
      do i = 1, n
        if (a(i) /= i) bad = bad + 1
      end do
      host_flag = 1
      !$acc parallel num_gangs(1) copy(host_flag)
      if (.not. acc_on_device(acc_device_host)) host_flag = 0
      !$acc end parallel
      print '(a,i0)', 'bad= ', bad
      print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', bad == 0)
      deallocate(a)
    end program declare_create_allocatable
    

Other compilers run it correctly

NVHPC 25.11, NVHPC 26.9