OffTarget

Observed issues › afar-loop-bind-thread-in-teams

Appears not to supportRefused at compile time · Fortran · OpenMP · Open

amdflang: loop bind(thread) closely nested in teams rejected ("BIND(TEAMS) must be specified")

Legality: contested. An independent adjudicating reading finds the program not legal: Both readings are right that bind(thread) is allowed inside teams in 5.2, so the compiler's stated reason is stale. But the loop construct asserts its iterations may execute in any order, including concurrently, and every iteration does mine = mine + 1 on one variable with no reduction. That loop-carried dependence is a data race, so the program is non-conforming and no result is required. How this is checked

Affected variants

CompilerMI300A gfx942
unified memory (XNACK 1)
MI300A gfx942
copy (XNACK 0)
MI250X gfx90a
copy (XNACK 0)
MI300X gfx942
copy (XNACK 0)
MI350X gfx950
copy (XNACK 0)
A100 sm_80
separate memory
A100 sm_80
managed memory
AFAR 24.3.0not supportednot supportednot supportednot supportednot supportedn/an/a
CCE 19.0.0correctcorrectcorrectunknown
not run
unknown
not run
n/an/a
CCE 20.0.2correctcorrectcorrectunknown
not run
unknown
not run
n/an/a
CCE 21.0.3correctcorrectcorrectunknown
not run
unknown
not run
n/an/a
CCE 21.0.4correctcorrectcorrectunknown
not run
unknown
not run
n/an/a
NVHPC 25.11n/an/an/an/an/afails another wayfails another way
NVHPC 26.9n/an/an/an/an/afails another wayfails another way
GCC 16.2.0correctcorrectcorrectcorrectcorrectfails another wayn/a

"behaves incorrectly" / "not supported": this issue, confirmed in up to four environments (another build directory, another node, a repeat). "fails another way": a different failure, not counted as this issue. "unknown": not run there, or the runs disagree.

Messages

ResultCompiler and GPUMessage
not supportedAFAR 24.3.0, MI300A unified
AFAR 24.3.0, MI300A copy
AFAR 24.3.0, MI250X
AFAR 24.3.0, MI300X
AFAR 24.3.0, MI350X
show
error: Semantic errors in afar-loop-bind-thread-in-teams.f90
afar-loop-bind-thread-in-teams.f90:11:13: error: `BIND(TEAMS)` must be specified since the `LOOP` region is strictly nested inside a `TEAMS` region.
        !$omp loop bind(thread)
              ^^^^
fails another wayNVHPC 25.11, A100 separate
NVHPC 25.11, A100 managed
NVHPC 26.9, A100 separate
NVHPC 26.9, A100 managed
show
 tot=           64            0            0            0
fails another wayGCC 16.2.0, A100 separate
show
 tot=           2           2           2           2

Reproduce

AFAR 24.3.0, MI300A unified

$AFAR/bin/amdflang -fopenmp --offload-arch=gfx942 -O2 repro.f90 -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro

CCE 19.0.0, MI300A unified (runs correctly)

module load cce/19.0.0 craype-accel-amd-gfx942 rocm/6.4.3
module unload cray-mpich
ftn -fopenmp -O2 repro.f90 -o repro
HSA_XNACK=1 OMP_TARGET_OFFLOAD=mandatory ./repro
Program (38 lines)
! loop bind(thread) closely nested in teams (README.md). Expected: tot= 64 64 64 64,
! which CCE prints as `tot= 4*64` (a list-directed repeat count: the same values,
! a correction). Each team's mine is private (teams private(tm, mine)) and the loop's
! binding thread set is the encountering thread, so no race (offload.engine.prerun's race rule
! flags mine: a false positive).
program t
  use omp_lib, only: omp_get_team_num
  implicit none
  integer, parameter :: n = 64
  integer :: tot(4), tm, mine, i
  tot = 0
  !$omp target map(tofrom: tot)
    !$omp teams num_teams(4) private(tm, mine)
      tm = omp_get_team_num()
      mine = 0
      !$omp loop bind(thread)
        do i = 1, n
          mine = mine + 1
        end do
      !$omp end loop
      tot(tm + 1) = mine
    !$omp end teams
  !$omp end target
  print *, 'tot=', tot
end program
! spec (requirement): With bind(thread) the binding region of a loop construct is not defined.
! "if binding is thread then the binding region is not defined" (5.2 §11.7.1 p.258 l.24-25)
! spec (requirement): When the binding region is not defined, the binding thread set is the encountering thread, so
!   each thread executes all iterations of the loop it encounters.
! "If the binding region is not defined, then the binding thread set is the encountering thread." (5.2 §11.7.1 p.258
!   l.31-32)
! spec (requirement): The bind clause is required only when the loop construct is not nested inside another OpenMP
!   construct.
! "If a loop construct is not nested inside another OpenMP construct then the bind clause must be present." (5.2 §11.7
!   p.258 l.1-2)
! spec (this program): The loop bind(thread) is closely nested in teams, so each team's initial thread runs all 64
!   iterations on its private mine and tot is 64 for every team; amdflang rejects the construct with BIND(TEAMS) must
!   be specified.

Source in the repository

Something to add? Dispute its legality · Report it fixed in a newer release · Report a mistake on this page

Expected and observed

Observed where it behaves incorrectly

error: Semantic errors in afar-loop-bind-thread-in-teams.f90
afar-loop-bind-thread-in-teams.f90:11:13: error: `BIND(TEAMS)` must be specified since the `LOOP` region is strictly nested inside a `TEAMS` region.
        !$omp loop bind(thread)
              ^^^^

What the specification requires

The specification text this program relies on and the compiler is expected to support.

OpenMP 5.2 §11.7.1 p.258 l.24-25

if binding is thread then the binding region is not defined

OpenMP 5.2 §11.7.1 p.258 l.31-32

If the binding region is not defined, then the binding thread set is the encountering thread.

OpenMP 5.2 §11.7 p.258 l.1-2

If a loop construct is not nested inside another OpenMP construct then the bind clause must be present.

Evidence

Same compiler, one change

  • the loop directive removed (a plain DO loop in each team): AFAR 24.3.0: correct; CCE 19.0.0: correct; CCE 20.0.2: correct; CCE 21.0.3: correct; CCE 21.0.4: correct
    Control program
    program t
      use omp_lib, only: omp_get_team_num
      implicit none
      integer, parameter :: n = 64
      integer :: tot(4), tm, mine, i
      tot = 0
      !$omp target map(tofrom: tot)
        !$omp teams num_teams(4) private(tm, mine)
          tm = omp_get_team_num()
          mine = 0
            do i = 1, n
              mine = mine + 1
            end do
          tot(tm + 1) = mine
        !$omp end teams
      !$omp end target
      print *, 'tot=', tot
    end program
    

Other compilers run it correctly

CCE 19.0.0, CCE 20.0.2, CCE 21.0.3, CCE 21.0.4, GCC 16.2.0