Observed issues › harvest/gcc-acc-declare-create-allocatable-not-mapped
OpenACC: declare create of a Fortran allocatable gives no device copy at the allocate statement
Legality: not independently confirmed. Independent readings are uncertain: Section 2.13.2 says a global-context declare sets the structured reference counter to one, and also says that for an allocatable the allocate statement raises a runtime error if that counter is not zero. Read literally, the program's allocate would be an error. How this is checked
OpenACC 2.6 and later make an allocate statement for an allocatable in a declare create directive allocate the device copy too. GCC 16.2 does not: the first compute construct that names the array stops with "is not mapped" (exit 1), on the MI250X and the A100. With an explicit enter data create after the allocate statement the same program runs correctly. CCE and NVHPC run the harvested programs correctly.
Affected variants
| Compiler | MI300A gfx942 unified memory (XNACK 1) | MI300A gfx942 copy (XNACK 0) | MI250X gfx90a copy (XNACK 0) | MI300X gfx942 copy (XNACK 0) | MI350X gfx950 copy (XNACK 0) | A100 sm_80 separate memory | A100 sm_80 managed memory |
|---|---|---|---|---|---|---|---|
| AFAR 24.3.0 | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| CCE 19.0.0 | fails another way | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 20.0.2 | fails another way | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.3 | behaves incorrectly | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.4 | behaves incorrectly | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| NVHPC 25.11 | n/a | n/a | n/a | n/a | n/a | correct | correct |
| NVHPC 26.9 | n/a | n/a | n/a | n/a | n/a | correct | correct |
| GCC 16.2.0 | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | n/a |
"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.
Messages
| Result | Compiler and GPU | Message |
|---|---|---|
| behaves incorrectly | CCE 21.0.3, MI300A unified CCE 21.0.4, MI300A unified | showACC: libcrayacc/acc_hw_amd.c:1927 CRAY_ACC_ERROR - hipStreamSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' from site-harvest--gcc-acc-declare-create-allocatab ACC: libcrayacc/acc_hw_amd.c:1476 CRAY_ACC_ERROR - hipDeviceSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' |
| behaves incorrectly | GCC 16.2.0, MI300A unified GCC 16.2.0, MI300A copy | showlibgomp: [0x8d02e0,4000] is not mapped |
| behaves incorrectly | GCC 16.2.0, MI250X | showlibgomp: [0xa74c80,4000] is not mapped |
| behaves incorrectly | GCC 16.2.0, MI300X | showlibgomp: [0x1a283d0,4000] is not mapped |
| behaves incorrectly | GCC 16.2.0, MI350X | showlibgomp: [0x101a040,4000] is not mapped |
| behaves incorrectly | GCC 16.2.0, A100 separate | showlibgomp: [0x139b1840,4000] is not mapped |
| fails another way | CCE 19.0.0, MI300A unified | showMemory access fault by GPU node-7 (Agent handle: 0x4c6a30) on address (nil). Reason: Write access to a read-only page. |
| fails another way | CCE 20.0.2, MI300A unified | showMemory access fault by GPU node-5 (Agent handle: 0x4c7f10) on address (nil). Reason: Write access to a read-only page. |
Reproduce
CCE 21.0.3, MI300A unified
module load cce/21.0.3 craype-accel-amd-gfx942 rocm/7.2.0 module unload cray-mpich ftn -h acc -O2 repro.f90 -o repro HSA_XNACK=1 ./repro
CCE 21.0.4, MI300A unified
module load cce/21.0.4 craype-accel-amd-gfx942 rocm/7.2.0 module unload cray-mpich ftn -h acc -O2 repro.f90 -o repro HSA_XNACK=1 ./repro
GCC 16.2.0, MI300A unified
export LD_LIBRARY_PATH=$GCC_16_2/lib64:/opt/rocm-7.2.1/lib $GCC_16_2/bin/gfortran -fopenacc -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx942 -O2 repro.f90 -o repro HSA_XNACK=1 ./repro
GCC 16.2.0, A100 separate
export PATH=/usr/local/cuda/bin:$PATH export LD_LIBRARY_PATH=$GCC_16_2/lib64 $GCC_16_2/bin/gfortran -fopenacc -foffload=nvptx-none -foffload-options=nvptx-none=-march=sm_80 -foffload-options=nvptx-none=-latomic -O2 repro.f90 -o repro ./repro
NVHPC 25.11, A100 separate (runs correctly)
export LD_LIBRARY_PATH=$NVHPC_25_11/compilers/lib nvfortran -acc=gpu -gpu=cc80 -gpu=mem:separate -O2 repro.f90 -o repro ./repro
Program (58 lines)
! # Finding (GCC 16.2, known-issues harvest): OpenACC declare create of a module allocatable: the allocate statement
! creates no device copy ("is not mapped")
!
! - toolchains: gcc_g90 (GCC 16.2.0, AMD Instinct MI250X gfx90a, Tioga) and gcc_sm80 (GCC 16.2.0, NVIDIA A100 sm_80, wingtip);
! separate memory only; -O2
! - build: gfortran -fopenacc -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx90a -O2 (gcc_g90);
! -foffload=nvptx-none -foffload-options=nvptx-none=-march=sm_80 -foffload-options=nvptx-none=-latomic -O2 (gcc_sm80)
! - failure: run_error: is not mapped
! - observed: `libgomp: [0x...,4000] is not mapped` and exit 1 at the first compute construct that names the array (present(a)), in
! every run (gcc_sm80: mandatory exit 5 of 5; unset exit 5 of 5; gcc_g90: mandatory exit 5 of 5; unset exit 5 of 5
! (results/parity/<machine>/<tc>/probes.jsonl))
! - expected: `bad= 0`, then `RESULT ran_on_host=0 check=pass`
! - rule: in Fortran, for an allocatable in a declare create directive and a non-shared memory device, at an allocate statement
! "memory will be allocated in both local memory as well as in the current device memory and the dynamic reference counter will be
! set to one" (OpenACC 3.4 §2.13.2 l.3056-3057). The behaviour is OpenACC 2.6's (the changelog of OpenACC 3.4, "Changes from
! Version 2.5 to 2.6", margin line 588: "The declare create directive with a Fortran allocatable has new behavior"), the version
! GCC documents (libgomp.texi, "Enabling OpenACC": OpenACC 2.6).
! - same-compiler control (correct, gcc_sm80: mandatory pass 5 of 5; unset pass 5 of 5; gcc_g90: mandatory pass 5 of 5; unset pass 5
! of 5 (results/parity/<machine>/<tc>/probes.jsonl)): the same program with `!$acc enter data create(a)` after the allocate
! statement (controls/gcc-acc-declare-create-allocatable-control.f90).
! - other families: CCE 19/20/21 (copy mode, MI300A and MI250X) and NVHPC 25.11/26.9 (separate and managed memory) run the harvested
! programs correctly (hk-007, hk-013, hk-087: results/harvest-known/main*). The site's finding matrix runs this file on every
! compiler.
! - harvested programs (6 cells (3 gcc_g90, 3 gcc_sm80); gate: confirmed 6; confirmer: needs_human (R22-expected-weak) 6): hk-007,
! hk-013, hk-087
! - legality: the pre-run check (offload.engine.prerun.check: the rules and AFAR's front end, two witnesses) accepts the harvested
! programs; this file is read against the rule above. GCC is a compiler under test only and never a legality witness.
module declare_create_m
implicit none
integer, allocatable :: a(:)
!$acc declare create(a)
end module declare_create_m
program declare_create_allocatable
use openacc, only: acc_device_host, acc_on_device
use declare_create_m
implicit none
integer, parameter :: n = 1000
integer :: i, bad, host_flag
allocate(a(n))
a = -1
!$acc parallel loop present(a)
do i = 1, n
a(i) = i
end do
!$acc update host(a)
bad = 0
do i = 1, n
if (a(i) /= i) bad = bad + 1
end do
host_flag = 1
!$acc parallel num_gangs(1) copy(host_flag)
if (.not. acc_on_device(acc_device_host)) host_flag = 0
!$acc end parallel
print '(a,i0)', 'bad= ', bad
print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', bad == 0)
deallocate(a)
end program declare_create_allocatable
Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page
Expected and observed
Expected
check=pass
Observed where it behaves incorrectly
ACC: libcrayacc/acc_hw_amd.c:1927 CRAY_ACC_ERROR - hipStreamSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' from site-harvest--gcc-acc-declare-create-allocatab ACC: libcrayacc/acc_hw_amd.c:1476 CRAY_ACC_ERROR - hipDeviceSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered'
What the specification requires
The specification text this program relies on and the compiler is expected to support.
OpenACC 3.4 §2.13.2
memory will be allocated in both local memory as well as in the current device memory and the dynamic reference counter will be set to one
Evidence
Same compiler, one change
- an enter data create after the allocate statement: not run yet
Control program
! Control of harvest/gcc-acc-declare-create-allocatable-not-mapped.f90: the same program with an enter data create ! after the allocate statement (correct on gcc_g90 and gcc_sm80). module declare_create_m implicit none integer, allocatable :: a(:) !$acc declare create(a) end module declare_create_m program declare_create_allocatable use openacc, only: acc_device_host, acc_on_device use declare_create_m implicit none integer, parameter :: n = 1000 integer :: i, bad, host_flag allocate(a(n)) !$acc enter data create(a) a = -1 !$acc parallel loop present(a) do i = 1, n a(i) = i end do !$acc update host(a) bad = 0 do i = 1, n if (a(i) /= i) bad = bad + 1 end do host_flag = 1 !$acc parallel num_gangs(1) copy(host_flag) if (.not. acc_on_device(acc_device_host)) host_flag = 0 !$acc end parallel print '(a,i0)', 'bad= ', bad print '(a,i0,a,a)', 'RESULT ran_on_host=', host_flag, ' check=', merge('pass', 'fail', bad == 0) deallocate(a) end program declare_create_allocatable
Other compilers run it correctly
NVHPC 25.11, NVHPC 26.9