nvc/nvfortran -mp=gpu: compile succeeds but the kernel is missing at run time ("Failed to find device function") for function pointers, static locals, VLAs, long double, type-bound calls

Legality: not independently confirmed. no independent review yet How this is checked

Same issue as catalog/nvhpc-c-function-pointer-kernel-missing; this program is another reproducer of it.

Affected variants

Compiler	MI300A gfx942 unified memory (XNACK 1)	MI300A gfx942 copy (XNACK 0)	MI250X gfx90a copy (XNACK 0)	MI300X gfx942 copy (XNACK 0)	MI350X gfx950 copy (XNACK 0)	A100 sm_80 separate memory	A100 sm_80 managed memory
AFAR 24.3.0	fails another way	fails another way	unknown not run	fails another way	fails another way	n/a	n/a
CCE 19.0.0	fails another way	fails another way	unknown not run	unknown not run	unknown not run	n/a	n/a
CCE 20.0.2	fails another way	fails another way	unknown not run	unknown not run	unknown not run	n/a	n/a
CCE 21.0.3	fails another way	fails another way	unknown not run	unknown not run	unknown not run	n/a	n/a
CCE 21.0.4	fails another way	fails another way	unknown not run	unknown not run	unknown not run	n/a	n/a
NVHPC 25.11	n/a	n/a	n/a	n/a	n/a	behaves incorrectly	behaves incorrectly
NVHPC 26.9	n/a	n/a	n/a	n/a	n/a	behaves incorrectly	behaves incorrectly
GCC 16.2.0	fails another way	fails another way	fails another way	fails another way	fails another way	fails another way	n/a

"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.

Messages

Result	Compiler and GPU	Message
behaves incorrectly	NVHPC 25.11, A100 separate NVHPC 25.11, A100 managed	show Accelerator Fatal Error: Failed to find device function 'nvkernel_main_F1L16_2'! File was compiled with: -gpu=cc80 Rebuild this file with -gpu=cc80 to use NVIDIA Tesla GPU 0 File: <dir>/nvhpc-c-long-double-kernel-missing.c Function: main:7 Line: 16
behaves incorrectly	NVHPC 26.9, A100 separate NVHPC 26.9, A100 managed	show Accelerator Fatal Error: Failed to find device function 'nvkernel_main_F1L19_2'! File was compiled with: -gpu=cc80 Rebuild this file with -gpu=cc80 to use NVIDIA Tesla GPU 0 File: <dir>/nvhpc-c-long-double-kernel-missing.c Function: main:7 Line: 19
fails another way	AFAR 24.3.0, MI300A unified AFAR 24.3.0, MI300A copy AFAR 24.3.0, MI300X AFAR 24.3.0, MI350X	show ld.lld: error: no libcall available for sint_to_fp ld.lld: error: no libcall available for fadd ld.lld: error: no libcall available for fadd clang: error: ld.lld command failed with exit code 1 (use -v to see invocation) $AFAR/lib/llvm/bin/clang-linker-wrapper: error: 'clang' failed clang: error: linker command failed with exit code 1 (use -v to see invocation)
fails another way	CCE 19.0.0, MI300A unified CCE 19.0.0, MI300A copy	show llc: /workspace/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:1723: llvm::SDValue llvm::SelectionDAG::getConstant(const llvm::ConstantInt&, const llvm::SDLoc&, llvm::EVT, bool, bool): Assertion `Elt- PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace. Stack dump: 0. Program arguments: /opt/cray/pe/cce/19.0.0/cce-clang/x86_64/bin/llc -mtriple=amdgcn-amd-amdhsa -disable-promote-alloca-to-lds -mcpu=gfx942 <dir>/cooltmpdir-SPiNZL/repro-cce-openmp-pre-llc.bc -filet 1. Running pass 'CallGraph Pass Manager' on module '<dir>/cooltmpdir-SPiNZL/repro-cce-openmp-pre-llc.bc'. 2. Running pass 'AMDGPU DAG->DAG Pattern Instruction Selection' on function '@__omp_offloading_14_1c7a5562_main_l17' #0 0x0000000003f00137 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/cray/pe/cce/19.0.0/cce-clang/x86_64/bin/llc+0x3f00137) #1 0x0000000003efd02b SignalHandler(int) (/opt/cray/pe/cce/19.0.0/cce-clang/x86_64/bin/llc+0x3efd02b)
fails another way	CCE 20.0.2, MI300A unified CCE 20.0.2, MI300A copy	show lld: /workspace/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:1767: llvm::SDValue llvm::SelectionDAG::getConstant(const llvm::ConstantInt&, const llvm::SDLoc&, llvm::EVT, bool, bool): Assertion `Elt- PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace. Stack dump: 0. Program arguments: /opt/cray/pe/cce/20.0.2/cce-clang/x86_64/bin/lld -flavor gnu --no-undefined -shared -plugin-opt=mcpu=gfx942 -plugin-opt=-disable-promote-alloca-to-lds -plugin-opt=defaults=cray - 1. Running pass 'CallGraph Pass Manager' on module 'ld-temp.o'. 2. Running pass 'AMDGPU DAG->DAG Pattern Instruction Selection' on function '@__omp_offloading_14_1c799559_main_l17' #0 0x0000000000a4ac57 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/cray/pe/cce/20.0.2/cce-clang/x86_64/bin/lld+0xa4ac57) #1 0x0000000000a47f83 SignalHandler(int) (/opt/cray/pe/cce/20.0.2/cce-clang/x86_64/bin/lld+0xa47f83)
fails another way	CCE 21.0.3, MI300A unified CCE 21.0.3, MI300A copy	show lld: /workspace/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:1762: llvm::SDValue llvm::SelectionDAG::getConstant(const llvm::ConstantInt&, const llvm::SDLoc&, llvm::EVT, bool, bool): Assertion `Elt- PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace. Stack dump: 0. Program arguments: /opt/cray/pe/cce/21.0.3/cce-clang/x86_64/bin/lld -flavor gnu --no-undefined -shared -plugin-opt=mcpu=gfx942 -plugin-opt=-disable-promote-alloca-to-lds -plugin-opt=defaults=cray - 1. Running pass 'CallGraph Pass Manager' on module 'ld-temp.o'. 2. Running pass 'AMDGPU DAG->DAG Pattern Instruction Selection' on function '@__omp_offloading_14_1c6e1a1d_main_l17' #0 0x0000000000a04627 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/cray/pe/cce/21.0.3/cce-clang/x86_64/bin/lld+0xa04627) #1 0x0000000000a01264 SignalHandler(int, siginfo_t, void) (/opt/cray/pe/cce/21.0.3/cce-clang/x86_64/bin/lld+0xa01264)
fails another way	CCE 21.0.4, MI300A unified CCE 21.0.4, MI300A copy	show lld: /workspace/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:1762: llvm::SDValue llvm::SelectionDAG::getConstant(const llvm::ConstantInt&, const llvm::SDLoc&, llvm::EVT, bool, bool): Assertion `Elt- PLEASE submit a bug report to HPE at https://support.hpe.com/ and include the crash backtrace. Stack dump: 0. Program arguments: /opt/cray/pe/cce/21.0.4/cce-clang/x86_64/bin/lld -flavor gnu --no-undefined -shared -plugin-opt=mcpu=gfx942 -plugin-opt=-disable-promote-alloca-to-lds -plugin-opt=defaults=cray - 1. Running pass 'CallGraph Pass Manager' on module 'ld-temp.o'. 2. Running pass 'AMDGPU DAG->DAG Pattern Instruction Selection' on function '@__omp_offloading_14_1c69a2fd_main_l17' #0 0x0000000000a04627 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/cray/pe/cce/21.0.4/cce-clang/x86_64/bin/lld+0xa04627) #1 0x0000000000a01264 SignalHandler(int, siginfo_t, void) (/opt/cray/pe/cce/21.0.4/cce-clang/x86_64/bin/lld+0xa01264)
fails another way	GCC 16.2.0, MI300A unified GCC 16.2.0, MI300A copy GCC 16.2.0, MI250X GCC 16.2.0, MI300X GCC 16.2.0, MI350X GCC 16.2.0, A100 separate	show RESULT ran_on_host=0 check=fail

Reproduce

NVHPC 25.11, A100 separate

export LD_LIBRARY_PATH=$NVHPC_25_11/compilers/lib
nvc -mp=gpu -gpu=cc80 -gpu=mem:separate -O2 repro.c -o repro
OMP_TARGET_OFFLOAD=mandatory ./repro

NVHPC 26.9, A100 separate

export LD_LIBRARY_PATH=$NVHPC_26_9/compilers/lib
nvc -mp=gpu -gpu=cc80 -gpu=mem:separate -O2 repro.c -o repro
OMP_TARGET_OFFLOAD=mandatory ./repro

Program (58 lines)

// # Draft finding: omp52-c-w-long-double on nvhpc
//
// - compiler: nvhpc
// - toolchain: nvhpc; arch: sm_80; machine: wingtip; memory mode: copy; opt: -O2
// - program: catalog omp52-c-w-long-double (c, version 52, sha256 95c075422c35bf6b)
// - expected: check=pass (trusted; elision + model)
// - actual: run_error  (exit_status: 1; without mandatory: exit_status: 1) [the confirmer's class run_error; the matrix's class of the same run is wrong_answer: a run that fails]
// - observed: no diagnostic; the run ends with "Accelerator Fatal Error: Failed to find device function 'nvkernel_main_...'" (25.11 and 26.9, both memory modes): long double arithmetic in a target loop
// - verdict: compiler_bug by rule R20-runtime-bug: a legal program fails at run time, deterministically, against an expected value from two or more independent sources with an executed witness, and another toolchain, mode, arch or the elision gives the expected value
// - reason: expected trusted check=pass (elision + model): stored main verdict: elision + model agree; witnesses: elision
// - citations: RUL-DERIVED-NEEDS-EXECUTION (rulings.md); RUL-WITNESS-PER-MODE (rulings.md); RUL-SAME-COMPILER-RUNS (rulings.md); RUL-PRESENCE-WITNESS (rulings.md); RUL-METAMORPHIC-OTHER-FAMILY (rulings.md)
// - legality: legal: offload.engine.prerun.check accepts it (rerun: offload.engine.rules with the
//   race rule and the restriction checks, and the AFAR clang front end, not under test here); no
//   compiler under test, gfortran or gcc is a legality witness
// - witnesses: elision
// - withdrawn: none
//
// Generated by compliance.widen (OpenMP 5.2, C11; catalog widening). Do not edit.
// id: omp52-c-w-long-double
// primary atom: base.device.long_double
// atoms: base.device.long_double
// legal: yes
// observable: yes
// ran_on_host: inside the target region of interest
// evidence: construct
// needs: none
// std: c11
// feature: long double arithmetic on a mapped long double array (the representation of long double is implementation defined; a device that has no wider type than double may reject it)
// spec: OpenMP 5.2 §1.7, p.35: C11 is a normative base language
#include <stdio.h>
#include <stdbool.h>
#include <omp.h>

enum { n = 64 };

int main(void) {
  int a[n], b[n], ref[n];
  int host_flag = 1;
  long double ld[n];
  for (int i = 0; i < n; i++) {
    a[i] = i + 1;
    b[i] = 0;
    ref[i] = 2 * (i + 1) + 1;
  }
  for (int i = 0; i < n; i++) ld[i] = 0.0L;
  #pragma omp target teams distribute parallel for map(to: a) map(from: ld) map(tofrom: \
      host_flag)
  for (int i = 0; i < n; i++) {
    ld[i] = (long double)a[i] * 2.0L + 1.0L;
    if (i == 0) host_flag = omp_is_initial_device();
  }
  for (int i = 0; i < n; i++) b[i] = (int)ld[i];
  bool ok = true;
  for (int i = 0; i < n; i++)
    ok = ok && (b[i] == ref[i]);
  printf("RESULT ran_on_host=%d check=%s\n", host_flag, ok ? "pass" : "fail");
  return 0;
}

Source in the repository

Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page

Expected and observed

Expected

check=pass

Observed where it behaves incorrectly

Accelerator Fatal Error: Failed to find device function 'nvkernel_main_F1L16_2'! File was compiled with: -gpu=cc80
Rebuild this file with -gpu=cc80 to use NVIDIA Tesla GPU 0
 File: <dir>/nvhpc-c-long-double-kernel-missing.c
 Function: main:7
 Line: 16

What the specification requires

The specification text this program relies on and the compiler is expected to support.

OpenMP 5.2 §1.7 p.35

Evidence

Same compiler, one change

double instead of long double: AFAR 24.3.0: correct; CCE 19.0.0: correct; CCE 20.0.2: correct; CCE 21.0.3: correct; CCE 21.0.4: correct; NVHPC 25.11: correct; NVHPC 26.9: correct

Control program

#include <stdio.h>
#include <stdbool.h>
#include <omp.h>

enum { n = 64 };

int main(void) {
  int a[n], b[n], ref[n];
  int host_flag = 1;
  double ld[n];
  for (int i = 0; i < n; i++) {
    a[i] = i + 1;
    b[i] = 0;
    ref[i] = 2 * (i + 1) + 1;
  }
  for (int i = 0; i < n; i++) ld[i] = 0.0;
  #pragma omp target teams distribute parallel for map(to: a) map(from: ld) map(tofrom: \
      host_flag)
  for (int i = 0; i < n; i++) {
    ld[i] = (double)a[i] * 2.0 + 1.0;
    if (i == 0) host_flag = omp_is_initial_device();
  }
  for (int i = 0; i < n; i++) b[i] = (int)ld[i];
  bool ok = true;
  for (int i = 0; i < n; i++)
    ok = ok && (b[i] == ref[i]);
  printf("RESULT ran_on_host=%d check=%s\n", host_flag, ok ? "pass" : "fail");
  return 0;
}