Observed issues › nested/gcc-acc-c-exit-data-of-a-pointer-array-element-target-hangs

Appears to behave incorrectlyHangs at run time · C · OpenACC · Open

GCC OpenACC: exit data of a pointer-array element target hangs

Legality: independently confirmed. One independent reading was uncertain; 2 adjudicating readings that weighed both find the program legal and the result required: Struct member r.nx with subarray r.nx[0:2] is allowed because the parent r is not a subarray; r.nx[0][0:1] is a subarray of a pointer element, not a member of a subarray. Copying r copies the pointer members without their targets, so the later member copyins attach them. Deletes run in reverse order. The serial region reads only r.v, so tot=6 is required; the UNHANDLED kind hang is a runtime defect. How this is checked

GCC prints "UNHANDLED kind 0x50" and hangs after exit data of an element target of a pointer array.

Affected variants

Compiler	MI300A gfx942 unified memory (XNACK 1)	MI300A gfx942 copy (XNACK 0)	MI250X gfx90a copy (XNACK 0)	MI300X gfx942 copy (XNACK 0)	MI350X gfx950 copy (XNACK 0)	A100 sm_80 separate memory	A100 sm_80 managed memory
AFAR 24.3.0	n/a	n/a	n/a	n/a	n/a	n/a	n/a
CCE 19.0.0	n/a	n/a	unknown not run	unknown not run	unknown not run	n/a	n/a
CCE 20.0.2	n/a	n/a	unknown not run	unknown not run	unknown not run	n/a	n/a
CCE 21.0.3	n/a	n/a	unknown not run	unknown not run	unknown not run	n/a	n/a
CCE 21.0.4	n/a	n/a	unknown not run	unknown not run	unknown not run	n/a	n/a
NVHPC 25.11	n/a	n/a	n/a	n/a	n/a	correct	correct
NVHPC 26.9	n/a	n/a	n/a	n/a	n/a	correct	correct
GCC 16.2.0	behaves incorrectly	behaves incorrectly	behaves incorrectly	behaves incorrectly	behaves incorrectly	behaves incorrectly	n/a

"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.

Messages

Result	Compiler and GPU	Message
behaves incorrectly	GCC 16.2.0, MI300A unified GCC 16.2.0, MI300A copy GCC 16.2.0, MI250X GCC 16.2.0, MI300X GCC 16.2.0, MI350X GCC 16.2.0, A100 separate	show libgomp: >>>> goacc_exit_data_internal UNHANDLED kind 0x50

Reproduce

GCC 16.2.0, MI300A unified

export LD_LIBRARY_PATH=$GCC_16_2/lib64:/opt/rocm-7.2.1/lib
$GCC_16_2/bin/gcc -fopenacc -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx942 -O2 repro.c -o repro
HSA_XNACK=1 ./repro

GCC 16.2.0, A100 separate

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=$GCC_16_2/lib64
$GCC_16_2/bin/gcc -fopenacc -foffload=nvptx-none -foffload-options=nvptx-none=-march=sm_80 -foffload-options=nvptx-none=-latomic -O2 repro.c -o repro
./repro

NVHPC 25.11, A100 separate (runs correctly)

export LD_LIBRARY_PATH=$NVHPC_25_11/compilers/lib
nvc -acc=gpu -gpu=cc80 -gpu=mem:separate -O2 repro.c -o repro
./repro

Program (75 lines)

// # Finding (nested-data): GCC OpenACC: exit data of the target of an element of a pointer array stops at run time:
// # "goacc_exit_data_internal UNHANDLED kind 0x50" (the program hangs)
//
// - toolchains and modes that fail: gcc_g90 (GCC 16.2.0), gcc_sm80 (GCC 16.2.0); modes: copy; -O2
// - build: gcc -fopenacc -foffload=amdgcn-amdhsa -foffload-options=amdgcn-amdhsa=-march=gfx90a -O2 (gcc_g90); gcc -fopenacc
//   -foffload=nvptx-none -foffload-options=nvptx-none=-march=sm_80 -foffload-options=nvptx-none=-latomic -O2 (gcc_sm80)
// - failure: timeout
// - observed: `RUN_TIMEOUT`; identical signature on 2 of 2 failing cells
// - expected: compiles and prints `tot= 6` with `check=pass`
// - rule: OpenACC 3.4 §2.7.1 p.49 l.1686-1688 describes how the pointers of dynamically allocated dimensions are filled
//   in; the text has no sentence that names a subarray of the target of an element of a pointer array (r.nx[0][0:1]) as
//   a var form, so the legality of the program rests on the rules' reading of var forms and on NVHPC accepting and
//   running it. Exit data with delete or copyout of a var that is present decrements its counters and, for a pointer
//   reference, its attachment counter (3.4 §2.7.12 p.58 l.2023-2024)
// - same-compiler control: controls/f3-pointer-array-only.c (one change from this file; correct on gcc_g90 (GCC 16.2.0), gcc_sm80
//   (GCC 16.2.0); rows results/nested/red/red10.jsonl, results/nested/red/wgs.jsonl, results/nested/red/wnv.jsonl);
//   controls/f5-ctl-copyout-exit.c (one change from this file; correct on gcc_g90 (GCC 16.2.0), gcc_sm80 (GCC 16.2.0); rows
//   results/nested/red/red10.jsonl, results/nested/red/wgs.jsonl, results/nested/red/wnv.jsonl);
//   controls/f6-ctl-pointer-to-struct-array.c (one change from this file; correct on gcc_g90 (GCC 16.2.0), gcc_sm80 (GCC 16.2.0);
//   rows results/nested/red/red10.jsonl, results/nested/red/wgs.jsonl, results/nested/red/wnv.jsonl)
// - other families: nvhpc (NVHPC 25.11), nvhpc269 (NVHPC 26.9) run the file correctly (rows results/nested/red/wnv.jsonl)
// - gate and confirmer: campaign gate of the programs below: 14 cells, 14 confirmed in the four environments (a-d; (c) on another
//   node or GPU; rows results/nested/big/gate_reruns.part.<host>.1371141.jsonl,
//   results/nested/big/gate_reruns.part.<host>.517665.jsonl,
//   results/nested/big/gate_reruns.part.wingtip.501872.jsonl,
//   results/nested/big/gate_reruns.part.wingtip.501873.jsonl,
//   results/nested/big/gate_reruns.part.wingtip.711615.jsonl,
//   results/nested/big/gate_reruns.part.wingtip.724122.jsonl, results/nested/big/rows.part.<host>.397000.jsonl,
//   results/nested/big/rows.part.wingtip.3886921.jsonl,
//   results/nested/main/gate_reruns.part.<host>.2711328.jsonl, results/nested/main/gate_reruns.part.<host>.351539.jsonl,
//   results/nested/main/gate_reruns.part.wingtip.3582436.jsonl,
//   results/nested/main/gate_reruns.part.wingtip.3582437.jsonl,
//   results/nested/main/gate_reruns.part.wingtip.3637854.jsonl,
//   results/nested/main/gate_reruns.part.wingtip.3639010.jsonl,
//   results/nested/main/rows.part.<host>.2691037.jsonl, results/nested/main/rows.part.wingtip.3363230.jsonl)
// - programs (7): nd-c-acc-enter-chain-d2-a882d869, nd-c-acc-enter-chain-d2-e7da4dab, nd-c-acc-enter-chain-d3-315d10d9,
//   nd-c-acc-enter-chain-d3-358c069c, nd-c-acc-enter-chain-d3-b9bb6619, nd-c-acc-enter-chain-d4-1fa7e914,
//   nd-c-acc-enter-chain-d4-658d9bcd
// - legality: the rules (offload.engine.rules) accept it; both NVHPC releases compile and run the reduction correctly (a
//   compiler of another family; the registry has no OpenACC C clause checker, so this is acceptance and execution, not a
//   clause check)
#include <stdio.h>
#include <stdlib.h>
#include <openacc.h>
struct s1 { int *a; };
struct s0 { int v; struct s1 **nx; };
int main(void)
{
  int hf = 1, i, tot = 0, ok;
  struct s0 r;
  r.v = 5;
  r.nx = malloc(2 * sizeof *r.nx);
  for (i = 0; i < 2; i++) {
    r.nx[i] = malloc(sizeof *r.nx[i]);
    r.nx[i]->a = malloc(4 * sizeof(int));
    for (int q = 0; q < 4; q++) r.nx[i]->a[q] = 10 * (i + 1) + q;
  }
  #pragma acc serial copy(hf)
  hf = acc_on_device(acc_device_host) ? 1 : 0;
  #pragma acc enter data copyin(r)
  #pragma acc enter data copyin(r.nx[0:2])
  #pragma acc enter data copyin(r.nx[0][0:1])
  #pragma acc serial copy(tot) present(r)
  {
    tot = r.v;
    tot += 1;
  }
  #pragma acc exit data delete(r.nx[0][0:1])
  #pragma acc exit data delete(r.nx[0:2])
  #pragma acc exit data delete(r)
  ok = tot == 6;
  printf("tot= %d\n", tot);
  printf("RESULT ran_on_host=%d check=%s\n", hf, ok ? "pass" : "fail");
  return 0;
}

Source in the repository

Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page

Expected and observed

Observed where it behaves incorrectly

libgomp: >>>> goacc_exit_data_internal UNHANDLED kind 0x50

What the specification requires

The specification text this program relies on and the compiler is expected to support.

OpenACC 3.4 §2.7.1 p.49 l.1686-1688

OpenACC 3.4 §2.7.12 p.58 l.2023-2024

Evidence

Same compiler, one change

one change from the finding; correct on gcc_g90, gcc_sm80: not run yet

Control program

// a pointer to a pointer array: copyin(r), copyin(r.nx[0:2]), then delete in reverse; the elements are not touched
#include <stdio.h>
#include <stdlib.h>
#include <openacc.h>
struct s1 { int *a; };
struct s0 { int v; struct s1 **nx; };
int main(void)
{
  int hf = 1, i, tot = 0, ok;
  struct s0 r;
  r.v = 5;
  r.nx = malloc(2 * sizeof *r.nx);
  for (i = 0; i < 2; i++) {
    r.nx[i] = malloc(sizeof *r.nx[i]);
    r.nx[i]->a = malloc(4 * sizeof(int));
    for (int q = 0; q < 4; q++) r.nx[i]->a[q] = 10 * (i + 1) + q;
  }
  #pragma acc serial copy(hf)
  hf = acc_on_device(acc_device_host) ? 1 : 0;
  #pragma acc enter data copyin(r)
  #pragma acc enter data copyin(r.nx[0:2])
  #pragma acc serial copy(tot) present(r)
  {
    tot = r.v;
    tot += 1;
  }
  #pragma acc exit data delete(r.nx[0:2])
  #pragma acc exit data delete(r)
  ok = tot == 6;
  printf("tot= %d\n", tot);
  printf("RESULT ran_on_host=%d check=%s\n", hf, ok ? "pass" : "fail");
  return 0;
}

one change from the finding; correct on gcc_g90, gcc_sm80: not run yet

Control program

// control: the pointer array leaves with copyout instead of delete
#include <stdio.h>
#include <stdlib.h>
#include <openacc.h>
struct s1 { int *a; };
struct s0 { int v; struct s1 **nx; };
int main(void)
{
  int hf = 1, i, tot = 0, ok;
  struct s0 r;
  r.v = 5;
  r.nx = malloc(2 * sizeof *r.nx);
  for (i = 0; i < 2; i++) {
    r.nx[i] = malloc(sizeof *r.nx[i]);
    r.nx[i]->a = malloc(4 * sizeof(int));
    for (int q = 0; q < 4; q++) r.nx[i]->a[q] = 10 * (i + 1) + q;
  }
  #pragma acc serial copy(hf)
  hf = acc_on_device(acc_device_host) ? 1 : 0;
  #pragma acc enter data copyin(r)
  #pragma acc enter data copyin(r.nx[0:2])
  #pragma acc serial copy(tot) present(r)
  {
    tot = r.v;
    tot += 1;
  }
  #pragma acc exit data copyout(r.nx[0:2])
  #pragma acc exit data delete(r)
  ok = tot == 6;
  printf("tot= %d\n", tot);
  printf("RESULT ran_on_host=%d check=%s\n", hf, ok ? "pass" : "fail");
  return 0;
}

one change from the finding; correct on gcc_g90, gcc_sm80: not run yet

Control program

// control: one pointer hop less: a pointer to an array of structures that hold the arrays
#include <stdio.h>
#include <stdlib.h>
#include <openacc.h>
struct s1 { int *a; };
struct s0 { int v; struct s1 *nx; };
int main(void)
{
  int hf = 1, tot = 0, ok;
  struct s0 r;
  r.v = 5;
  r.nx = malloc(2 * sizeof *r.nx);
  for (int i = 0; i < 2; i++) r.nx[i].a = malloc(4 * sizeof(int));
  #pragma acc serial copy(hf)
  hf = acc_on_device(acc_device_host) ? 1 : 0;
  #pragma acc enter data copyin(r)
  #pragma acc enter data copyin(r.nx[0:2])
  #pragma acc serial copy(tot) present(r)
  {
    tot = r.v + 1;
  }
  #pragma acc exit data delete(r.nx[0:2])
  #pragma acc exit data delete(r)
  ok = tot == 6;
  printf("tot= %d\n", tot);
  printf("RESULT ran_on_host=%d check=%s\n", hf, ok ? "pass" : "fail");
  return 0;
}

Other compilers run it correctly

NVHPC 25.11, NVHPC 26.9