ftn/craycc: constructs documented as unsupported in offload regions (single, masked, task, ordered, scan) give internal compiler errors or device link failures instead of a diagnostic

Legality: not independently confirmed. no independent review yet How this is checked

Same issue as cce19-target-parallel-single-ice; this program is another reproducer of it.

Affected variants

Compiler	MI300A gfx942 unified memory (XNACK 1)	MI300A gfx942 copy (XNACK 0)	MI250X gfx90a copy (XNACK 0)	MI300X gfx942 copy (XNACK 0)	MI350X gfx950 copy (XNACK 0)	A100 sm_80 separate memory	A100 sm_80 managed memory
AFAR 24.3.0	correct	correct	correct	correct	correct	n/a	n/a
CCE 19.0.0	behaves incorrectly	behaves incorrectly	behaves incorrectly	unknown not run	unknown not run	n/a	n/a
CCE 20.0.2	behaves incorrectly	behaves incorrectly	behaves incorrectly	unknown not run	unknown not run	n/a	n/a
CCE 21.0.3	behaves incorrectly	behaves incorrectly	behaves incorrectly	unknown not run	unknown not run	n/a	n/a
CCE 21.0.4	behaves incorrectly	behaves incorrectly	behaves incorrectly	unknown not run	unknown not run	n/a	n/a
NVHPC 25.11	n/a	n/a	n/a	n/a	n/a	fails another way	fails another way
NVHPC 26.9	n/a	n/a	n/a	n/a	n/a	fails another way	fails another way
GCC 16.2.0	correct	correct	correct	correct	correct	correct	n/a

"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.

Messages

Result	Compiler and GPU	Message
behaves incorrectly	CCE 19.0.0, MI300A unified CCE 19.0.0, MI300A copy CCE 19.0.0, MI250X	show unexpected reduction type UNREACHABLE executed at /workspace/clang/lib/CodeGen/Cray_CGOpenMPRuntimeGPU.cpp:749! PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace, preprocessed source, and associated run script. Stack dump: 0. Program arguments: /opt/cray/pe/cce/19.0.0/cce-clang/x86_64/bin/clang-19 -cc1 -triple amdgcn-amd-amdhsa -mllvm -cray-omp-opt-fork-call -mllvm -cray-omp-parallel-opt-call -mllvm -cray-openmp-rename- 1. <eof> parser at end of file 2. cce-c-reduction-inscan-ice.c:7:5: LLVM IR generation of declaration 'main' 3. cce-c-reduction-inscan-ice.c:22:3: LLVM IR generation of compound statement ('{}')
behaves incorrectly	CCE 20.0.2, MI300A unified CCE 20.0.2, MI300A copy CCE 20.0.2, MI250X	show unexpected reduction type UNREACHABLE executed at /workspace/clang/lib/CodeGen/Cray_CGOpenMPRuntimeGPU.cpp:766! PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace, preprocessed source, and associated run script. Stack dump: 0. Program arguments: /opt/cray/pe/cce/20.0.2/cce-clang/x86_64/bin/clang-20 -cc1 -triple amdgcn-amd-amdhsa -mllvm -cray-omp-opt-fork-call -mllvm -cray-omp-parallel-opt-call -mllvm -cray-openmp-rename- 1. <eof> parser at end of file 2. cce-c-reduction-inscan-ice.c:7:5: LLVM IR generation of declaration 'main' 3. cce-c-reduction-inscan-ice.c:22:3: LLVM IR generation of compound statement ('{}')
behaves incorrectly	CCE 21.0.3, MI300A unified CCE 21.0.3, MI300A copy CCE 21.0.3, MI250X	show unexpected reduction type UNREACHABLE executed at /workspace/clang/lib/CodeGen/Cray_CGOpenMPRuntimeGPU.cpp:766! PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace, preprocessed source, and associated run script. Stack dump: 0. Program arguments: /opt/cray/pe/cce/21.0.3/cce-clang/x86_64/bin/clang-21 -cc1 -triple amdgcn-amd-amdhsa -mllvm -cray-omp-opt-fork-call -mllvm -cray-omp-parallel-opt-call -mllvm -cray-openmp-rename- 1. <eof> parser at end of file 2. cce-c-reduction-inscan-ice.c:7:5: LLVM IR generation of declaration 'main' 3. cce-c-reduction-inscan-ice.c:22:3: LLVM IR generation of compound statement ('{}')
behaves incorrectly	CCE 21.0.4, MI300A unified CCE 21.0.4, MI300A copy CCE 21.0.4, MI250X	show unexpected reduction type UNREACHABLE executed at /workspace/clang/lib/CodeGen/Cray_CGOpenMPRuntimeGPU.cpp:766! PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace, preprocessed source, and associated run script. Stack dump: 0. Program arguments: /opt/cray/pe/cce/21.0.4/cce-clang/x86_64/bin/clang-21 -cc1 -triple amdgcn-amd-amdhsa -mllvm -cray-omp-opt-fork-call -mllvm -cray-omp-parallel-opt-call -mllvm -cray-openmp-rename- 1. <eof> parser at end of file 2. cce-c-reduction-inscan-ice.c:7:5: LLVM IR generation of declaration 'main' 3. cce-c-reduction-inscan-ice.c:22:3: LLVM IR generation of compound statement ('{}')
fails another way	NVHPC 25.11, A100 separate NVHPC 25.11, A100 managed NVHPC 26.9, A100 separate NVHPC 26.9, A100 managed	show "cce-c-reduction-inscan-ice.c", line 24: error: invalid text in pragma #pragma omp parallel for default(none) shared(a, b) reduction(inscan, +: s) ^ "cce-c-reduction-inscan-ice.c", line 24: error: identifier "inscan" is undefined #pragma omp parallel for default(none) shared(a, b) reduction(inscan, +: s) ^ "cce-c-reduction-inscan-ice.c", line 24: error: invalid text in pragma #pragma omp parallel for default(none) shared(a, b) reduction(inscan, +: s)

Reproduce

CCE 19.0.0, MI300A unified

module load cce/19.0.0 craype-accel-amd-gfx942 rocm/6.4.3
module unload cray-mpich
craycc -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx942 -fopenmp-version=52 -O2 repro.c -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro

CCE 20.0.2, MI300A unified

module load cce/20.0.2 craype-accel-amd-gfx942 rocm/6.4.3
module unload cray-mpich
craycc -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx942 -fopenmp-version=52 -O2 repro.c -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro

CCE 21.0.3, MI300A unified

module load cce/21.0.3 craype-accel-amd-gfx942 rocm/7.2.0
module unload cray-mpich
craycc -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx942 -fopenmp-version=52 -O2 repro.c -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro

CCE 21.0.4, MI300A unified

module load cce/21.0.4 craype-accel-amd-gfx942 rocm/7.2.0
module unload cray-mpich
craycc -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx942 -fopenmp-version=52 -O2 repro.c -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro

AFAR 24.3.0, MI300A unified (runs correctly)

$AFAR/bin/amdclang -fopenmp --offload-arch=gfx942 -fopenmp-version=52 -fopenmp-version=52 -O2 repro.c -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro

Program (60 lines)

// # Draft finding: omp52-c-clause-reduction-inscan on cce19
//
// - compiler: cce19
// - toolchain: cce19; arch: gfx942; machine: tuolumne; memory mode: unified; opt: -O2
// - program: catalog omp52-c-clause-reduction-inscan (c, version 52, sha256 09fe312de8f77fd1)
// - expected: None (None; )
// - actual: ice  (ice: PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace, preprocessed source, and associated run script.)
// - verdict: compiler_bug by rule R10-compile-ice: an internal compiler error (or the compiler rejecting code it generated) on a legal program: always the compiler's defect
// - reason: 
// - citations: RUL-COMPILE-TIME (rulings.md)
// - legality: legal (legal by construction: offload.engine.gen / c_gen / acc_gen check every program against the catalog rules and the offload scope as it is built (no compiler))
// - witnesses: none
// - withdrawn: none
//
// Generated by compliance.c_gen (OpenMP 5.2, C11): the C port (offload.engine.f2c) of omp52-clause-reduction-inscan. Do not edit.
// id: omp52-c-clause-reduction-inscan
// primary atom: omp.clause.reduction.inscan
// legal: yes
// observable: yes
// ran_on_host: inside the target region of interest
// evidence: construct
// needs: none
// std: c11
#include <stdio.h>
#include <stdbool.h>
#include <omp.h>

enum { n = 64 };

int main(void) {
  int a[n];
  int b[n];
  int ref[n];
  int host_flag;
  bool ok;
  int s;
  for (int i = 1; i <= n; i++) {
    a[i - 1] = i;
    b[i - 1] = 0;
    ref[i - 1] = i * (i + 1) / 2;
  }
  host_flag = 1;
  s = 0;
  #pragma omp target map(to: a) map(from: b) map(tofrom: s) map(tofrom: host_flag)
  {
    host_flag = (omp_is_initial_device() ? 1 : 0);
    #pragma omp parallel for default(none) shared(a, b) reduction(inscan, +: s)
    for (int i = 1; i <= n; i++) {
      s = s + a[i - 1];
      #pragma omp scan inclusive(s)
      b[i - 1] = s;
    }
  }
  bool t1_ = true;
  for (int k2_ = 0; k2_ < n; k2_++)
    t1_ = t1_ && (b[k2_] == ref[k2_]);
  ok = t1_;
  printf("RESULT ran_on_host=%d check=%s\n", host_flag, ok ? "pass" : "fail");
  return 0;
}

Source in the repository

Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page

Expected and observed

Observed where it behaves incorrectly

unexpected reduction type
UNREACHABLE executed at /workspace/clang/lib/CodeGen/Cray_CGOpenMPRuntimeGPU.cpp:749!
PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /opt/cray/pe/cce/19.0.0/cce-clang/x86_64/bin/clang-19 -cc1 -triple amdgcn-amd-amdhsa -mllvm -cray-omp-opt-fork-call -mllvm -cray-omp-parallel-opt-call -mllvm -cray-openmp-rename-
1.	<eof> parser at end of file
2.	cce-c-reduction-inscan-ice.c:7:5: LLVM IR generation of declaration 'main'
3.	cce-c-reduction-inscan-ice.c:22:3: LLVM IR generation of compound statement ('{}')

What the specification requires

The specification text this program relies on and the compiler is expected to support.

The same requirement as the issue cce19-target-parallel-single-ice.

OpenMP 5.2 §11.1 p.241 l.2-3

The single construct specifies that the associated structured block is executed by only one of the threads in the team

OpenMP 5.2 §11.1 p.241 l.5-6

An implicit barrier occurs at the end of a single region if the nowait clause is not specified.

Evidence

Same compiler, one change

the same scan on the host (the target directive removed): AFAR 24.3.0: correct; CCE 19.0.0: correct; CCE 20.0.2: correct; CCE 21.0.3: correct; CCE 21.0.4: correct

Control program

#include <stdio.h>
#include <stdbool.h>
#include <omp.h>

enum { n = 64 };

int main(void) {
  int a[n];
  int b[n];
  int ref[n];
  int host_flag;
  bool ok;
  int s;
  for (int i = 1; i <= n; i++) {
    a[i - 1] = i;
    b[i - 1] = 0;
    ref[i - 1] = i * (i + 1) / 2;
  }
  host_flag = 1;
  s = 0;
  {
    host_flag = (omp_is_initial_device() ? 1 : 0);
    #pragma omp parallel for default(none) shared(a, b) reduction(inscan, +: s)
    for (int i = 1; i <= n; i++) {
      s = s + a[i - 1];
      #pragma omp scan inclusive(s)
      b[i - 1] = s;
    }
  }
  bool t1_ = true;
  for (int k2_ = 0; k2_ < n; k2_++)
    t1_ = t1_ && (b[k2_] == ref[k2_]);
  ok = t1_;
  printf("RESULT ran_on_host=%d check=%s\n", host_flag, ok ? "pass" : "fail");
  return 0;
}

Other compilers run it correctly

AFAR 24.3.0, GCC 16.2.0