Observed issues › harvest/cce-acc-declare-create-module-allocatable-xnack1-fault
Appears to behave incorrectlyCrashes or aborts at run time · Fortran · OpenACC · Open
ftn (OpenACC): a module allocatable in declare create faults at HSA_XNACK=1
Legality: independently confirmed. 2 independent readings find the program legal and the result required. How this is checked
Affected variants
| Compiler | MI300A gfx942 unified memory (XNACK 1) | MI300A gfx942 copy (XNACK 0) | MI250X gfx90a copy (XNACK 0) | MI300X gfx942 copy (XNACK 0) | MI350X gfx950 copy (XNACK 0) | A100 sm_80 separate memory | A100 sm_80 managed memory |
|---|---|---|---|---|---|---|---|
| AFAR 24.3.0 | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| CCE 19.0.0 | behaves incorrectly | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 20.0.2 | behaves incorrectly | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.3 | behaves incorrectly | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| CCE 21.0.4 | behaves incorrectly | correct | unknown not run | unknown not run | unknown not run | n/a | n/a |
| NVHPC 25.11 | n/a | n/a | n/a | n/a | n/a | correct | correct |
| NVHPC 26.9 | n/a | n/a | n/a | n/a | n/a | correct | correct |
| GCC 16.2.0 | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | behaves incorrectly | n/a |
"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.
Messages
| Result | Compiler and GPU | Message |
|---|---|---|
| behaves incorrectly | CCE 19.0.0, MI300A unified | showMemory access fault by GPU node-7 (Agent handle: 0x4d5c50) on address (nil). Reason: Write access to a read-only page. |
| behaves incorrectly | CCE 20.0.2, MI300A unified | showMemory access fault by GPU node-7 (Agent handle: 0x4d6eb0) on address 0x6000. Reason: Write access to a read-only page. |
| behaves incorrectly | CCE 21.0.3, MI300A unified CCE 21.0.4, MI300A unified | showACC: libcrayacc/acc_hw_amd.c:1927 CRAY_ACC_ERROR - hipStreamSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' from site-harvest--cce-acc-declare-create-module-al ACC: libcrayacc/acc_hw_amd.c:1476 CRAY_ACC_ERROR - hipDeviceSynchronize returned hipErrorIllegalAddress: 'an illegal memory access was encountered' |
| behaves incorrectly | GCC 16.2.0, MI300A unified GCC 16.2.0, MI300A copy | showlibgomp: [0x8dfdd0,32768] is not mapped |
| behaves incorrectly | GCC 16.2.0, MI250X | showlibgomp: [0xa84a70,32768] is not mapped |
| behaves incorrectly | GCC 16.2.0, MI300X | showlibgomp: [0x1098e80,32768] is not mapped |
| behaves incorrectly | GCC 16.2.0, MI350X | showlibgomp: [0x20d8ad0,32768] is not mapped |
| behaves incorrectly | GCC 16.2.0, A100 separate | showlibgomp: [0x12e37f40,32768] is not mapped |
Reproduce
CCE 19.0.0, MI300A unified
module load cce/19.0.0 craype-accel-amd-gfx942 rocm/6.4.3 module unload cray-mpich ftn -h acc -O2 repro.f90 -o repro HSA_XNACK=1 ./repro
CCE 20.0.2, MI300A unified
module load cce/20.0.2 craype-accel-amd-gfx942 rocm/6.4.3 module unload cray-mpich ftn -h acc -O2 repro.f90 -o repro HSA_XNACK=1 ./repro
CCE 21.0.3, MI300A unified
module load cce/21.0.3 craype-accel-amd-gfx942 rocm/7.2.0 module unload cray-mpich ftn -h acc -O2 repro.f90 -o repro HSA_XNACK=1 ./repro
CCE 21.0.4, MI300A unified
module load cce/21.0.4 craype-accel-amd-gfx942 rocm/7.2.0 module unload cray-mpich ftn -h acc -O2 repro.f90 -o repro HSA_XNACK=1 ./repro
GCC 16.2.0, MI300A unified
export LD_LIBRARY_PATH=$GCC_16_2/lib64:/opt/rocm-7.2.1/lib $GCC_16_2/bin/gfortran -fopenacc -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx942 -O2 repro.f90 -o repro HSA_XNACK=1 ./repro
GCC 16.2.0, A100 separate
export PATH=/usr/local/cuda/bin:$PATH export LD_LIBRARY_PATH=$GCC_16_2/lib64 $GCC_16_2/bin/gfortran -fopenacc -foffload=nvptx-none -foffload-options=nvptx-none=-march=sm_80 -foffload-options=nvptx-none=-latomic -O2 repro.f90 -o repro ./repro
NVHPC 25.11, A100 separate (runs correctly)
export LD_LIBRARY_PATH=$NVHPC_25_11/compilers/lib nvfortran -acc=gpu -gpu=cc80 -gpu=mem:separate -O2 repro.f90 -o repro ./repro
Program (55 lines)
! # Finding (known-issues harvest): CCE faults on a module allocatable in
! declare create at HSA_XNACK=1 (MI300A)
!
! - source: MFC's compiler-ticket list (PE-53844, "CCE >17 crashes on host_data use_device for
! module variables"), reconstructed as data/bench/harvest-known hk-008 (a declare create module
! allocatable, host_data use_device, a deviceptr routine); the fault is at the first compute
! construct, before host_data, so this file drops host_data
! (data/reports/harvest-known/v008-declare-create-alloc.f90)
! - compilers: CCE 19.0.0, 20.0.2, 21.0.3, ftn -h acc -O2, MI300A at HSA_XNACK=1 (THP always)
! - expected: bad = 0, RESULT ... check=pass. For an allocatable in declare create, "for an
! allocate statement for var ... memory will be allocated in both local memory as well as in the
! current device memory" on a non-shared memory device (OpenACC 3.4 §2.13.2 p.83 l.3053-3057); on
! a shared memory device the host memory is the device's; either way buf is present in the
! parallel loop and the update copies (or leaves) the values
! - actual: CCE 19 and 20: "Memory access fault by GPU node-4 ... on address 0x6000. Reason: Write
! access to a read-only page." (SIGABRT); CCE 21: "hipStreamSynchronize returned
! hipErrorIllegalAddress ... from v008-declare-create-alloc.f90:19" (the parallel loop), exit 1
! - control (same compiler): HSA_XNACK=0 passes; at HSA_XNACK=1 a local allocatable made present
! with enter data create passes (c008-local-enter-data.f90), and a fixed-size module array in
! declare create passes (c008-fixed-module.f90), both in data/reports/harvest-known/
! - other families: NVHPC 25.11 and 26.9 (OpenACC; separate and managed memory) pass; AFAR has no
! OpenACC
! - legality: the pre-run check (rules and the AFAR front end with -fopenacc) accepts it; present
! requires the data present, which declare create and the allocate statement provide
! - also: hk-007 (MFC ticket OLCFHELP-19298 / HPECOE-71) and hk-013 (thornado commit 2253976852,
! "another CCE bug") hold a module allocatable in declare create too, and fail the same way on CCE
! 19/20/21 at HSA_XNACK=1 only (19 and 20 fault, 21 hipErrorIllegalAddress; another node)
! - evidence: results/harvest-known/main (hk-007, hk-008, hk-013, CCE 19/20/21 at HSA_XNACK=1),
! results/harvest-known/probes.tuolumne.jsonl, probes.wingtip.jsonl
module v008_m
implicit none
real(8), allocatable :: buf(:)
!$acc declare create(buf)
end module v008_m
program v008
use openacc
use v008_m
implicit none
integer, parameter :: n = 4096
integer :: i, bad, hk_host
allocate(buf(n))
!$acc parallel loop present(buf)
do i = 1, n
buf(i) = real(i, 8)
end do
!$acc update host(buf)
bad = count(buf /= [(real(i, 8), i = 1, n)])
hk_host = 1
!$acc parallel num_gangs(1) copy(hk_host)
if (.not. acc_on_device(acc_device_host)) hk_host = 0
!$acc end parallel
print '(a,i0)', 'bad = ', bad
print '(a,i0,a,a)', 'RESULT ran_on_host=', hk_host, ' check=', merge('pass', 'fail', bad == 0)
end program v008
Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page
Expected and observed
Expected
check=pass
Observed where it behaves incorrectly
Memory access fault by GPU node-7 (Agent handle: 0x4d5c50) on address (nil). Reason: Write access to a read-only page.
What the specification requires
The specification text this program relies on and the compiler is expected to support.
OpenACC 3.4 §2.13.2 p.83 l.3053-3057
Evidence
Other compilers run it correctly
NVHPC 25.11, NVHPC 26.9