[flang][flang-driver][mlir][OpenMP] atomic control support #143441
+289
−162
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
llvmbot
added
clang
|
@llvm/pr-subscribers-mlir-llvm @llvm/pr-subscribers-flang-fir-hlfir Author: Anchu Rajendran S (anchuraj) ChangesAtomic Control Options are used to specify architectural characteristics to help lowering of atomic operations. The options used are: Full diff: https://github.com/llvm/llvm-project/pull/143441.diff 11 Files Affected:
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index fd6deb22d404e..c6d8f9106341a 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -2299,21 +2299,21 @@ def fsymbol_partition_EQ : Joined<["-"], "fsymbol-partition=">, Group<f_Group>,
defm atomic_remote_memory : BoolFOption<"atomic-remote-memory",
LangOpts<"AtomicRemoteMemory">, DefaultFalse,
- PosFlag<SetTrue, [], [ClangOption, CC1Option], "May have">,
- NegFlag<SetFalse, [], [ClangOption], "Assume no">,
- BothFlags<[], [ClangOption], " atomic operations on remote memory">>;
+ PosFlag<SetTrue, [], [ClangOption, CC1Option, FlangOption, FC1Option], "May have">,
+ NegFlag<SetFalse, [], [ClangOption, FlangOption], "Assume no">,
+ BothFlags<[], [ClangOption, FlangOption], " atomic operations on remote memory">>;
defm atomic_fine_grained_memory : BoolFOption<"atomic-fine-grained-memory",
LangOpts<"AtomicFineGrainedMemory">, DefaultFalse,
- PosFlag<SetTrue, [], [ClangOption, CC1Option], "May have">,
- NegFlag<SetFalse, [], [ClangOption], "Assume no">,
- BothFlags<[], [ClangOption], " atomic operations on fine-grained memory">>;
+ PosFlag<SetTrue, [], [ClangOption, CC1Option, FlangOption, FC1Option], "May have">,
+ NegFlag<SetFalse, [], [ClangOption, FlangOption], "Assume no">,
+ BothFlags<[], [ClangOption, FlangOption], " atomic operations on fine-grained memory">>;
defm atomic_ignore_denormal_mode : BoolFOption<"atomic-ignore-denormal-mode",
LangOpts<"AtomicIgnoreDenormalMode">, DefaultFalse,
- PosFlag<SetTrue, [], [ClangOption, CC1Option], "Allow">,
- NegFlag<SetFalse, [], [ClangOption], "Disallow">,
- BothFlags<[], [ClangOption], " atomic operations to ignore denormal mode">>;
+ PosFlag<SetTrue, [], [ClangOption, CC1Option, FlangOption, FC1Option], "Allow">,
+ NegFlag<SetFalse, [], [ClangOption, FlangOption], "Disallow">,
+ BothFlags<[], [ClangOption, FlangOption], " atomic operations to ignore denormal mode">>;
defm memory_profile : OptInCC1FFlag<"memory-profile", "Enable", "Disable", " heap memory profiling">;
def fmemory_profile_EQ : Joined<["-"], "fmemory-profile=">,
@@ -5270,9 +5270,9 @@ defm amdgpu_precise_memory_op
" precise memory mode (AMDGPU only)">;
def munsafe_fp_atomics : Flag<["-"], "munsafe-fp-atomics">,
- Visibility<[ClangOption, CC1Option]>, Alias<fatomic_ignore_denormal_mode>;
+ Visibility<[ClangOption, CC1Option, FlangOption, FC1Option]>, Alias<fatomic_ignore_denormal_mode>;
def mno_unsafe_fp_atomics : Flag<["-"], "mno-unsafe-fp-atomics">,
- Visibility<[ClangOption]>, Alias<fno_atomic_ignore_denormal_mode>;
+ Visibility<[ClangOption, FlangOption]>, Alias<fno_atomic_ignore_denormal_mode>;
def faltivec : Flag<["-"], "faltivec">, Group<f_Group>;
def fno_altivec : Flag<["-"], "fno-altivec">, Group<f_Group>;
diff --git a/flang/include/flang/Frontend/TargetOptions.h b/flang/include/flang/Frontend/TargetOptions.h
index 002d8d158abd4..26256fd775f11 100644
--- a/flang/include/flang/Frontend/TargetOptions.h
+++ b/flang/include/flang/Frontend/TargetOptions.h
@@ -53,6 +53,11 @@ class TargetOptions {
/// Print verbose assembly
bool asmVerbose = false;
+
+ /// Atomic Control Options for AMD GPU
+ bool amdgpuIgnoreDenormalMode = false;
+ bool amdgpuRemoteMemory = false;
+ bool amdgpuFineGrainedMemory = false;
};
} // end namespace Fortran::frontend
diff --git a/flang/include/flang/Optimizer/Dialect/Support/FIRContext.h b/flang/include/flang/Optimizer/Dialect/Support/FIRContext.h
index 2df14f83c11e1..ac563bcc402c7 100644
--- a/flang/include/flang/Optimizer/Dialect/Support/FIRContext.h
+++ b/flang/include/flang/Optimizer/Dialect/Support/FIRContext.h
@@ -58,10 +58,24 @@ void setTargetCPU(mlir::ModuleOp mod, llvm::StringRef cpu);
/// Get the target CPU string from the Module or return a null reference.
llvm::StringRef getTargetCPU(mlir::ModuleOp mod);
+// Setters and Getters for atomic control options
+void setAmdgpuIgnoreDenormalMode(mlir::ModuleOp mod);
+bool getAmdgpuIgnoreDenormalMode(mlir::ModuleOp mod);
+void setAmdgpuFineGrainedMemory(mlir::ModuleOp mod);
+bool getAmdgpuFineGrainedMemory(mlir::ModuleOp mod);
+void setAmdgpuRemoteMemory(mlir::ModuleOp mod);
+bool getAmdgpuRemoteMemory(mlir::ModuleOp mod);
+
/// Set the tune CPU for the module. `cpu` must not be deallocated while
/// module `mod` is still live.
void setTuneCPU(mlir::ModuleOp mod, llvm::StringRef cpu);
+// set Atomic control options for amd gpu.
+void setAmdgpuAtomicControlOptions(mlir::ModuleOp mod,
+ bool amdgpuIgnoreDenormalMode,
+ bool amdgpuNoFineGrainedMemory,
+ bool amdgpuNoRemoteMemory);
+
/// Get the tune CPU string from the Module or return a null reference.
llvm::StringRef getTuneCPU(mlir::ModuleOp mod);
diff --git a/flang/lib/Frontend/CompilerInvocation.cpp b/flang/lib/Frontend/CompilerInvocation.cpp
index 89aaee9f13853..78d44d76744dc 100644
--- a/flang/lib/Frontend/CompilerInvocation.cpp
+++ b/flang/lib/Frontend/CompilerInvocation.cpp
@@ -494,6 +494,18 @@ static void parseTargetArgs(TargetOptions &opts, llvm::opt::ArgList &args) {
args.getLastArg(clang::driver::options::OPT_triple))
opts.triple = a->getValue();
+ if (llvm::Triple(opts.triple).isAMDGPU()) {
+ opts.amdgpuIgnoreDenormalMode = args.hasFlag(
+ clang::driver::options::OPT_fatomic_ignore_denormal_mode,
+ clang::driver::options::OPT_fno_atomic_ignore_denormal_mode, false);
+ opts.amdgpuFineGrainedMemory = args.hasFlag(
+ clang::driver::options::OPT_fatomic_fine_grained_memory,
+ clang::driver::options::OPT_fno_atomic_fine_grained_memory, false);
+ opts.amdgpuRemoteMemory = args.hasFlag(
+ clang::driver::options::OPT_fatomic_remote_memory,
+ clang::driver::options::OPT_fno_atomic_remote_memory, false);
+ }
+
if (const llvm::opt::Arg *a =
args.getLastArg(clang::driver::options::OPT_target_cpu))
opts.cpu = a->getValue();
diff --git a/flang/lib/Lower/Bridge.cpp b/flang/lib/Lower/Bridge.cpp
index 64b16b3abe991..b045556565613 100644
--- a/flang/lib/Lower/Bridge.cpp
+++ b/flang/lib/Lower/Bridge.cpp
@@ -6648,6 +6648,12 @@ Fortran::lower::LoweringBridge::LoweringBridge(
fir::setKindMapping(*module, kindMap);
fir::setTargetCPU(*module, targetMachine.getTargetCPU());
fir::setTuneCPU(*module, targetOpts.cpuToTuneFor);
+ if (targetOpts.amdgpuIgnoreDenormalMode)
+ fir::setAmdgpuIgnoreDenormalMode(*module);
+ if (targetOpts.amdgpuFineGrainedMemory)
+ fir::setAmdgpuFineGrainedMemory(*module);
+ if (targetOpts.amdgpuRemoteMemory)
+ fir::setAmdgpuRemoteMemory(*module);
fir::setTargetFeatures(*module, targetMachine.getTargetFeatureString());
fir::support::setMLIRDataLayout(*module, targetMachine.createDataLayout());
fir::setIdent(*module, Fortran::common::getFlangFullVersion());
diff --git a/flang/lib/Lower/OpenMP/OpenMP.cpp b/flang/lib/Lower/OpenMP/OpenMP.cpp
index 6892e571e62a3..bd09073100f1d 100644
--- a/flang/lib/Lower/OpenMP/OpenMP.cpp
+++ b/flang/lib/Lower/OpenMP/OpenMP.cpp
@@ -2911,8 +2911,13 @@ static void genAtomicUpdateStatement(
if (rightHandClauseList)
genOmpAtomicHintAndMemoryOrderClauses(converter, *rightHandClauseList, hint,
memoryOrder);
+ auto module = firOpBuilder.getModule();
+ auto atomicControlAttr = mlir::omp::AtomicControlAttr::get(
+ firOpBuilder.getContext(), fir::getAmdgpuIgnoreDenormalMode(module),
+ fir::getAmdgpuFineGrainedMemory(module),
+ fir::getAmdgpuRemoteMemory(module));
atomicUpdateOp = firOpBuilder.create<mlir::omp::AtomicUpdateOp>(
- currentLocation, lhsAddr, hint, memoryOrder);
+ currentLocation, lhsAddr, atomicControlAttr, hint, memoryOrder);
processOmpAtomicTODO(varType, loc);
diff --git a/flang/lib/Optimizer/Dialect/Support/FIRContext.cpp b/flang/lib/Optimizer/Dialect/Support/FIRContext.cpp
index 01c0be66d1ecc..b961793dbdfd5 100644
--- a/flang/lib/Optimizer/Dialect/Support/FIRContext.cpp
+++ b/flang/lib/Optimizer/Dialect/Support/FIRContext.cpp
@@ -88,6 +88,46 @@ void fir::setTuneCPU(mlir::ModuleOp mod, llvm::StringRef cpu) {
mod->setAttr(tuneCpuName, mlir::StringAttr::get(ctx, cpu));
}
+static constexpr const char *amdgpuIgnoreDenormalModeName =
+ "fir.amdgpu.ignore.denormal.mode";
+void fir::setAmdgpuIgnoreDenormalMode(mlir::ModuleOp mod) {
+ auto *ctx = mod.getContext();
+ mod->setAttr(amdgpuIgnoreDenormalModeName, mlir::UnitAttr::get(ctx));
+}
+
+bool fir::getAmdgpuIgnoreDenormalMode(mlir::ModuleOp mod) {
+ if (auto attr =
+ mod->getAttrOfType<mlir::UnitAttr>(amdgpuIgnoreDenormalModeName))
+ return true;
+ return false;
+}
+
+static constexpr const char *amdgpuFineGrainedMemoryName =
+ "fir.amdgpu.fine.grained.memory";
+void fir::setAmdgpuFineGrainedMemory(mlir::ModuleOp mod) {
+ auto *ctx = mod.getContext();
+ mod->setAttr(amdgpuFineGrainedMemoryName, mlir::UnitAttr::get(ctx));
+}
+
+bool fir::getAmdgpuFineGrainedMemory(mlir::ModuleOp mod) {
+ if (auto attr =
+ mod->getAttrOfType<mlir::UnitAttr>(amdgpuFineGrainedMemoryName))
+ return true;
+ return false;
+}
+static constexpr const char *amdgpuRemoteMemoryName =
+ "fir.amdgpu.remote.memory";
+void fir::setAmdgpuRemoteMemory(mlir::ModuleOp mod) {
+ auto *ctx = mod.getContext();
+ mod->setAttr(amdgpuRemoteMemoryName, mlir::UnitAttr::get(ctx));
+}
+
+bool fir::getAmdgpuRemoteMemory(mlir::ModuleOp mod) {
+ if (auto attr = mod->getAttrOfType<mlir::UnitAttr>(amdgpuRemoteMemoryName))
+ return true;
+ return false;
+}
+
llvm::StringRef fir::getTuneCPU(mlir::ModuleOp mod) {
if (auto attr = mod->getAttrOfType<mlir::StringAttr>(tuneCpuName))
return attr.getValue();
diff --git a/flang/test/Lower/OpenMP/atomic-control-options.f90 b/flang/test/Lower/OpenMP/atomic-control-options.f90
new file mode 100644
index 0000000000000..1eb0f617f365e
--- /dev/null
+++ b/flang/test/Lower/OpenMP/atomic-control-options.f90
@@ -0,0 +1,37 @@
+! RUN: %flang_fc1 -emit-hlfir -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-device -munsafe-fp-atomics %s -o - | FileCheck -check-prefix=UNSAFE-FP-ATOMICS %s
+! RUN: %flang_fc1 -emit-hlfir -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-device -fatomic-ignore-denormal-mode %s -o - | FileCheck -check-prefix=IGNORE-DENORMAL %s
+! RUN: %flang_fc1 -emit-hlfir -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-device -fatomic-fine-grained-memory %s -o - | FileCheck -check-prefix=FINE-GRAINED-MEMORY %s
+! RUN: %flang_fc1 -emit-hlfir -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-device -fatomic-remote-memory %s -o - | FileCheck -check-prefix=REMOTE-MEMORY %s
+program test
+ implicit none
+ integer :: A, B, threads
+ threads = 128
+ A = 0
+ B = 0
+ !UNSAFE-FP-ATOMICS: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !UNSAFE-FP-ATOMICS: } {atomic_control = #omp.atomic_control<amdgpu_ignore_denormal_mode = true>}
+ !IGNORE-DENORMAL: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !IGNORE-DENORMAL: } {atomic_control = #omp.atomic_control<amdgpu_ignore_denormal_mode = true>}
+ !FINE-GRAINED-MEMORY: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !FINE-GRAINED-MEMORY: } {atomic_control = #omp.atomic_control<amdgpu_fine_grained_memory = true>}
+ !REMOTE-MEMORY: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !REMOTE-MEMORY: } {atomic_control = #omp.atomic_control<amdgpu_remote_memory = true>}
+ !$omp target parallel num_threads(threads)
+ !$omp atomic
+ A = A + 1
+ !$omp end target parallel
+ !UNSAFE-FP-ATOMICS: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !UNSAFE-FP-ATOMICS: } {atomic_control = #omp.atomic_control<amdgpu_ignore_denormal_mode = true>}
+ !IGNORE-DENORMAL: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !IGNORE-DENORMAL: } {atomic_control = #omp.atomic_control<amdgpu_ignore_denormal_mode = true>}
+ !FINE-GRAINED-MEMORY: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !FINE-GRAINED-MEMORY: } {atomic_control = #omp.atomic_control<amdgpu_fine_grained_memory = true>}
+ !REMOTE-MEMORY: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !REMOTE-MEMORY: } {atomic_control = #omp.atomic_control<amdgpu_remote_memory = true>}
+ !$omp target parallel num_threads(threads)
+ !$omp atomic capture
+ A = A + B
+ B = A
+ !$omp end atomic
+ !$omp end target parallel
+end program test
diff --git a/mlir/include/mlir/Dialect/OpenMP/OpenMPAttrDefs.td b/mlir/include/mlir/Dialect/OpenMP/OpenMPAttrDefs.td
index 704d0b2220e8a..84887dfd2c6f9 100644
--- a/mlir/include/mlir/Dialect/OpenMP/OpenMPAttrDefs.td
+++ b/mlir/include/mlir/Dialect/OpenMP/OpenMPAttrDefs.td
@@ -54,6 +54,20 @@ def FlagsAttr : OpenMP_Attr<"Flags", "flags"> {
let assemblyFormat = "`<` struct(params) `>`";
}
+//===----------------------------------------------------------------------===//
+// AtomicControlAttr
+//===----------------------------------------------------------------------===//
+
+// Runtime library flags attribute that holds information for lowering to LLVM.
+def AtomicControlAttr : OpenMP_Attr<"AtomicControl", "atomic_control"> {
+ let parameters =
+ (ins DefaultValuedParameter<"bool", "false">:$amdgpu_ignore_denormal_mode,
+ DefaultValuedParameter<"bool", "false">:$amdgpu_fine_grained_memory,
+ DefaultValuedParameter<"bool", "false">:$amdgpu_remote_memory);
+
+ let assemblyFormat = "`<` struct(params) `>`";
+}
+
//===----------------------------------------------------------------------===//
// TaskDependArrayAttr
//===----------------------------------------------------------------------===//
diff --git a/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td b/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td
index 036c6a6e350a8..d7d14e96233ae 100644
--- a/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td
+++ b/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td
@@ -1543,9 +1543,11 @@ def AtomicUpdateOp : OpenMP_Op<"atomic.update", traits = [
operations.
}] # clausesDescription;
- let arguments = !con((ins Arg<OpenMP_PointerLikeType,
- "Address of variable to be updated",
- [MemRead, MemWrite]>:$x), clausesArgs);
+ let arguments = !con(
+ (ins Arg<OpenMP_PointerLikeType,
+ "Address of variable to be updated", [MemRead, MemWrite]>:$x,
+ OptionalAttr<AtomicControlAttr>:$atomic_control),
+ clausesArgs);
// Override region definition.
let regions = (region SizedRegion<1>:$region);
diff --git a/mlir/test/Dialect/OpenMP/ops.mlir b/mlir/test/Dialect/OpenMP/ops.mlir
index 47cfc5278a5d0..e377b7aab3e43 100644
--- a/mlir/test/Dialect/OpenMP/ops.mlir
+++ b/mlir/test/Dialect/OpenMP/ops.mlir
@@ -1562,6 +1562,15 @@ func.func @omp_atomic_update(%x : memref<i32>, %expr : i32, %xBool : memref<i1>,
omp.yield(%newval : i32)
}
+ // CHECK: omp.atomic.update %[[X]] : memref<i32> {
+ // CHECK-NEXT: (%[[XVAL:.*]]: i32):
+ // CHECK-NEXT: omp.yield(%{{.+}} : i32)
+ // CHECK-NEXT: } {atomic_control_attr = #omp.atomic_control<amdgpu_ignore_denormal_mode = true, amdgpu_fine_grained_memory = true, amdgpu_remote_memory = true>}
+ omp.atomic.update %x : memref<i32> {
+ ^bb0(%xval:i32):
+ omp.yield(%const:i32)
+ } {atomic_control_attr = #omp.atomic_control<amdgpu_ignore_denormal_mode = true, amdgpu_fine_grained_memory = true, amdgpu_remote_memory = true>}
+
return
}
|
|
@llvm/pr-subscribers-flang-driver Author: Anchu Rajendran S (anchuraj) ChangesAtomic Control Options are used to specify architectural characteristics to help lowering of atomic operations. The options used are: Full diff: https://github.com/llvm/llvm-project/pull/143441.diff 11 Files Affected:
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index fd6deb22d404e..c6d8f9106341a 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -2299,21 +2299,21 @@ def fsymbol_partition_EQ : Joined<["-"], "fsymbol-partition=">, Group<f_Group>,
defm atomic_remote_memory : BoolFOption<"atomic-remote-memory",
LangOpts<"AtomicRemoteMemory">, DefaultFalse,
- PosFlag<SetTrue, [], [ClangOption, CC1Option], "May have">,
- NegFlag<SetFalse, [], [ClangOption], "Assume no">,
- BothFlags<[], [ClangOption], " atomic operations on remote memory">>;
+ PosFlag<SetTrue, [], [ClangOption, CC1Option, FlangOption, FC1Option], "May have">,
+ NegFlag<SetFalse, [], [ClangOption, FlangOption], "Assume no">,
+ BothFlags<[], [ClangOption, FlangOption], " atomic operations on remote memory">>;
defm atomic_fine_grained_memory : BoolFOption<"atomic-fine-grained-memory",
LangOpts<"AtomicFineGrainedMemory">, DefaultFalse,
- PosFlag<SetTrue, [], [ClangOption, CC1Option], "May have">,
- NegFlag<SetFalse, [], [ClangOption], "Assume no">,
- BothFlags<[], [ClangOption], " atomic operations on fine-grained memory">>;
+ PosFlag<SetTrue, [], [ClangOption, CC1Option, FlangOption, FC1Option], "May have">,
+ NegFlag<SetFalse, [], [ClangOption, FlangOption], "Assume no">,
+ BothFlags<[], [ClangOption, FlangOption], " atomic operations on fine-grained memory">>;
defm atomic_ignore_denormal_mode : BoolFOption<"atomic-ignore-denormal-mode",
LangOpts<"AtomicIgnoreDenormalMode">, DefaultFalse,
- PosFlag<SetTrue, [], [ClangOption, CC1Option], "Allow">,
- NegFlag<SetFalse, [], [ClangOption], "Disallow">,
- BothFlags<[], [ClangOption], " atomic operations to ignore denormal mode">>;
+ PosFlag<SetTrue, [], [ClangOption, CC1Option, FlangOption, FC1Option], "Allow">,
+ NegFlag<SetFalse, [], [ClangOption, FlangOption], "Disallow">,
+ BothFlags<[], [ClangOption, FlangOption], " atomic operations to ignore denormal mode">>;
defm memory_profile : OptInCC1FFlag<"memory-profile", "Enable", "Disable", " heap memory profiling">;
def fmemory_profile_EQ : Joined<["-"], "fmemory-profile=">,
@@ -5270,9 +5270,9 @@ defm amdgpu_precise_memory_op
" precise memory mode (AMDGPU only)">;
def munsafe_fp_atomics : Flag<["-"], "munsafe-fp-atomics">,
- Visibility<[ClangOption, CC1Option]>, Alias<fatomic_ignore_denormal_mode>;
+ Visibility<[ClangOption, CC1Option, FlangOption, FC1Option]>, Alias<fatomic_ignore_denormal_mode>;
def mno_unsafe_fp_atomics : Flag<["-"], "mno-unsafe-fp-atomics">,
- Visibility<[ClangOption]>, Alias<fno_atomic_ignore_denormal_mode>;
+ Visibility<[ClangOption, FlangOption]>, Alias<fno_atomic_ignore_denormal_mode>;
def faltivec : Flag<["-"], "faltivec">, Group<f_Group>;
def fno_altivec : Flag<["-"], "fno-altivec">, Group<f_Group>;
diff --git a/flang/include/flang/Frontend/TargetOptions.h b/flang/include/flang/Frontend/TargetOptions.h
index 002d8d158abd4..26256fd775f11 100644
--- a/flang/include/flang/Frontend/TargetOptions.h
+++ b/flang/include/flang/Frontend/TargetOptions.h
@@ -53,6 +53,11 @@ class TargetOptions {
/// Print verbose assembly
bool asmVerbose = false;
+
+ /// Atomic Control Options for AMD GPU
+ bool amdgpuIgnoreDenormalMode = false;
+ bool amdgpuRemoteMemory = false;
+ bool amdgpuFineGrainedMemory = false;
};
} // end namespace Fortran::frontend
diff --git a/flang/include/flang/Optimizer/Dialect/Support/FIRContext.h b/flang/include/flang/Optimizer/Dialect/Support/FIRContext.h
index 2df14f83c11e1..ac563bcc402c7 100644
--- a/flang/include/flang/Optimizer/Dialect/Support/FIRContext.h
+++ b/flang/include/flang/Optimizer/Dialect/Support/FIRContext.h
@@ -58,10 +58,24 @@ void setTargetCPU(mlir::ModuleOp mod, llvm::StringRef cpu);
/// Get the target CPU string from the Module or return a null reference.
llvm::StringRef getTargetCPU(mlir::ModuleOp mod);
+// Setters and Getters for atomic control options
+void setAmdgpuIgnoreDenormalMode(mlir::ModuleOp mod);
+bool getAmdgpuIgnoreDenormalMode(mlir::ModuleOp mod);
+void setAmdgpuFineGrainedMemory(mlir::ModuleOp mod);
+bool getAmdgpuFineGrainedMemory(mlir::ModuleOp mod);
+void setAmdgpuRemoteMemory(mlir::ModuleOp mod);
+bool getAmdgpuRemoteMemory(mlir::ModuleOp mod);
+
/// Set the tune CPU for the module. `cpu` must not be deallocated while
/// module `mod` is still live.
void setTuneCPU(mlir::ModuleOp mod, llvm::StringRef cpu);
+// set Atomic control options for amd gpu.
+void setAmdgpuAtomicControlOptions(mlir::ModuleOp mod,
+ bool amdgpuIgnoreDenormalMode,
+ bool amdgpuNoFineGrainedMemory,
+ bool amdgpuNoRemoteMemory);
+
/// Get the tune CPU string from the Module or return a null reference.
llvm::StringRef getTuneCPU(mlir::ModuleOp mod);
diff --git a/flang/lib/Frontend/CompilerInvocation.cpp b/flang/lib/Frontend/CompilerInvocation.cpp
index 89aaee9f13853..78d44d76744dc 100644
--- a/flang/lib/Frontend/CompilerInvocation.cpp
+++ b/flang/lib/Frontend/CompilerInvocation.cpp
@@ -494,6 +494,18 @@ static void parseTargetArgs(TargetOptions &opts, llvm::opt::ArgList &args) {
args.getLastArg(clang::driver::options::OPT_triple))
opts.triple = a->getValue();
+ if (llvm::Triple(opts.triple).isAMDGPU()) {
+ opts.amdgpuIgnoreDenormalMode = args.hasFlag(
+ clang::driver::options::OPT_fatomic_ignore_denormal_mode,
+ clang::driver::options::OPT_fno_atomic_ignore_denormal_mode, false);
+ opts.amdgpuFineGrainedMemory = args.hasFlag(
+ clang::driver::options::OPT_fatomic_fine_grained_memory,
+ clang::driver::options::OPT_fno_atomic_fine_grained_memory, false);
+ opts.amdgpuRemoteMemory = args.hasFlag(
+ clang::driver::options::OPT_fatomic_remote_memory,
+ clang::driver::options::OPT_fno_atomic_remote_memory, false);
+ }
+
if (const llvm::opt::Arg *a =
args.getLastArg(clang::driver::options::OPT_target_cpu))
opts.cpu = a->getValue();
diff --git a/flang/lib/Lower/Bridge.cpp b/flang/lib/Lower/Bridge.cpp
index 64b16b3abe991..b045556565613 100644
--- a/flang/lib/Lower/Bridge.cpp
+++ b/flang/lib/Lower/Bridge.cpp
@@ -6648,6 +6648,12 @@ Fortran::lower::LoweringBridge::LoweringBridge(
fir::setKindMapping(*module, kindMap);
fir::setTargetCPU(*module, targetMachine.getTargetCPU());
fir::setTuneCPU(*module, targetOpts.cpuToTuneFor);
+ if (targetOpts.amdgpuIgnoreDenormalMode)
+ fir::setAmdgpuIgnoreDenormalMode(*module);
+ if (targetOpts.amdgpuFineGrainedMemory)
+ fir::setAmdgpuFineGrainedMemory(*module);
+ if (targetOpts.amdgpuRemoteMemory)
+ fir::setAmdgpuRemoteMemory(*module);
fir::setTargetFeatures(*module, targetMachine.getTargetFeatureString());
fir::support::setMLIRDataLayout(*module, targetMachine.createDataLayout());
fir::setIdent(*module, Fortran::common::getFlangFullVersion());
diff --git a/flang/lib/Lower/OpenMP/OpenMP.cpp b/flang/lib/Lower/OpenMP/OpenMP.cpp
index 6892e571e62a3..bd09073100f1d 100644
--- a/flang/lib/Lower/OpenMP/OpenMP.cpp
+++ b/flang/lib/Lower/OpenMP/OpenMP.cpp
@@ -2911,8 +2911,13 @@ static void genAtomicUpdateStatement(
if (rightHandClauseList)
genOmpAtomicHintAndMemoryOrderClauses(converter, *rightHandClauseList, hint,
memoryOrder);
+ auto module = firOpBuilder.getModule();
+ auto atomicControlAttr = mlir::omp::AtomicControlAttr::get(
+ firOpBuilder.getContext(), fir::getAmdgpuIgnoreDenormalMode(module),
+ fir::getAmdgpuFineGrainedMemory(module),
+ fir::getAmdgpuRemoteMemory(module));
atomicUpdateOp = firOpBuilder.create<mlir::omp::AtomicUpdateOp>(
- currentLocation, lhsAddr, hint, memoryOrder);
+ currentLocation, lhsAddr, atomicControlAttr, hint, memoryOrder);
processOmpAtomicTODO(varType, loc);
diff --git a/flang/lib/Optimizer/Dialect/Support/FIRContext.cpp b/flang/lib/Optimizer/Dialect/Support/FIRContext.cpp
index 01c0be66d1ecc..b961793dbdfd5 100644
--- a/flang/lib/Optimizer/Dialect/Support/FIRContext.cpp
+++ b/flang/lib/Optimizer/Dialect/Support/FIRContext.cpp
@@ -88,6 +88,46 @@ void fir::setTuneCPU(mlir::ModuleOp mod, llvm::StringRef cpu) {
mod->setAttr(tuneCpuName, mlir::StringAttr::get(ctx, cpu));
}
+static constexpr const char *amdgpuIgnoreDenormalModeName =
+ "fir.amdgpu.ignore.denormal.mode";
+void fir::setAmdgpuIgnoreDenormalMode(mlir::ModuleOp mod) {
+ auto *ctx = mod.getContext();
+ mod->setAttr(amdgpuIgnoreDenormalModeName, mlir::UnitAttr::get(ctx));
+}
+
+bool fir::getAmdgpuIgnoreDenormalMode(mlir::ModuleOp mod) {
+ if (auto attr =
+ mod->getAttrOfType<mlir::UnitAttr>(amdgpuIgnoreDenormalModeName))
+ return true;
+ return false;
+}
+
+static constexpr const char *amdgpuFineGrainedMemoryName =
+ "fir.amdgpu.fine.grained.memory";
+void fir::setAmdgpuFineGrainedMemory(mlir::ModuleOp mod) {
+ auto *ctx = mod.getContext();
+ mod->setAttr(amdgpuFineGrainedMemoryName, mlir::UnitAttr::get(ctx));
+}
+
+bool fir::getAmdgpuFineGrainedMemory(mlir::ModuleOp mod) {
+ if (auto attr =
+ mod->getAttrOfType<mlir::UnitAttr>(amdgpuFineGrainedMemoryName))
+ return true;
+ return false;
+}
+static constexpr const char *amdgpuRemoteMemoryName =
+ "fir.amdgpu.remote.memory";
+void fir::setAmdgpuRemoteMemory(mlir::ModuleOp mod) {
+ auto *ctx = mod.getContext();
+ mod->setAttr(amdgpuRemoteMemoryName, mlir::UnitAttr::get(ctx));
+}
+
+bool fir::getAmdgpuRemoteMemory(mlir::ModuleOp mod) {
+ if (auto attr = mod->getAttrOfType<mlir::UnitAttr>(amdgpuRemoteMemoryName))
+ return true;
+ return false;
+}
+
llvm::StringRef fir::getTuneCPU(mlir::ModuleOp mod) {
if (auto attr = mod->getAttrOfType<mlir::StringAttr>(tuneCpuName))
return attr.getValue();
diff --git a/flang/test/Lower/OpenMP/atomic-control-options.f90 b/flang/test/Lower/OpenMP/atomic-control-options.f90
new file mode 100644
index 0000000000000..1eb0f617f365e
--- /dev/null
+++ b/flang/test/Lower/OpenMP/atomic-control-options.f90
@@ -0,0 +1,37 @@
+! RUN: %flang_fc1 -emit-hlfir -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-device -munsafe-fp-atomics %s -o - | FileCheck -check-prefix=UNSAFE-FP-ATOMICS %s
+! RUN: %flang_fc1 -emit-hlfir -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-device -fatomic-ignore-denormal-mode %s -o - | FileCheck -check-prefix=IGNORE-DENORMAL %s
+! RUN: %flang_fc1 -emit-hlfir -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-device -fatomic-fine-grained-memory %s -o - | FileCheck -check-prefix=FINE-GRAINED-MEMORY %s
+! RUN: %flang_fc1 -emit-hlfir -triple amdgcn-amd-amdhsa -fopenmp -fopenmp-is-device -fatomic-remote-memory %s -o - | FileCheck -check-prefix=REMOTE-MEMORY %s
+program test
+ implicit none
+ integer :: A, B, threads
+ threads = 128
+ A = 0
+ B = 0
+ !UNSAFE-FP-ATOMICS: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !UNSAFE-FP-ATOMICS: } {atomic_control = #omp.atomic_control<amdgpu_ignore_denormal_mode = true>}
+ !IGNORE-DENORMAL: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !IGNORE-DENORMAL: } {atomic_control = #omp.atomic_control<amdgpu_ignore_denormal_mode = true>}
+ !FINE-GRAINED-MEMORY: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !FINE-GRAINED-MEMORY: } {atomic_control = #omp.atomic_control<amdgpu_fine_grained_memory = true>}
+ !REMOTE-MEMORY: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !REMOTE-MEMORY: } {atomic_control = #omp.atomic_control<amdgpu_remote_memory = true>}
+ !$omp target parallel num_threads(threads)
+ !$omp atomic
+ A = A + 1
+ !$omp end target parallel
+ !UNSAFE-FP-ATOMICS: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !UNSAFE-FP-ATOMICS: } {atomic_control = #omp.atomic_control<amdgpu_ignore_denormal_mode = true>}
+ !IGNORE-DENORMAL: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !IGNORE-DENORMAL: } {atomic_control = #omp.atomic_control<amdgpu_ignore_denormal_mode = true>}
+ !FINE-GRAINED-MEMORY: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !FINE-GRAINED-MEMORY: } {atomic_control = #omp.atomic_control<amdgpu_fine_grained_memory = true>}
+ !REMOTE-MEMORY: omp.atomic.update %{{.*}} : !fir.ref<i32> {
+ !REMOTE-MEMORY: } {atomic_control = #omp.atomic_control<amdgpu_remote_memory = true>}
+ !$omp target parallel num_threads(threads)
+ !$omp atomic capture
+ A = A + B
+ B = A
+ !$omp end atomic
+ !$omp end target parallel
+end program test
diff --git a/mlir/include/mlir/Dialect/OpenMP/OpenMPAttrDefs.td b/mlir/include/mlir/Dialect/OpenMP/OpenMPAttrDefs.td
index 704d0b2220e8a..84887dfd2c6f9 100644
--- a/mlir/include/mlir/Dialect/OpenMP/OpenMPAttrDefs.td
+++ b/mlir/include/mlir/Dialect/OpenMP/OpenMPAttrDefs.td
@@ -54,6 +54,20 @@ def FlagsAttr : OpenMP_Attr<"Flags", "flags"> {
let assemblyFormat = "`<` struct(params) `>`";
}
+//===----------------------------------------------------------------------===//
+// AtomicControlAttr
+//===----------------------------------------------------------------------===//
+
+// Runtime library flags attribute that holds information for lowering to LLVM.
+def AtomicControlAttr : OpenMP_Attr<"AtomicControl", "atomic_control"> {
+ let parameters =
+ (ins DefaultValuedParameter<"bool", "false">:$amdgpu_ignore_denormal_mode,
+ DefaultValuedParameter<"bool", "false">:$amdgpu_fine_grained_memory,
+ DefaultValuedParameter<"bool", "false">:$amdgpu_remote_memory);
+
+ let assemblyFormat = "`<` struct(params) `>`";
+}
+
//===----------------------------------------------------------------------===//
// TaskDependArrayAttr
//===----------------------------------------------------------------------===//
diff --git a/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td b/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td
index 036c6a6e350a8..d7d14e96233ae 100644
--- a/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td
+++ b/mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td
@@ -1543,9 +1543,11 @@ def AtomicUpdateOp : OpenMP_Op<"atomic.update", traits = [
operations.
}] # clausesDescription;
- let arguments = !con((ins Arg<OpenMP_PointerLikeType,
- "Address of variable to be updated",
- [MemRead, MemWrite]>:$x), clausesArgs);
+ let arguments = !con(
+ (ins Arg<OpenMP_PointerLikeType,
+ "Address of variable to be updated", [MemRead, MemWrite]>:$x,
+ OptionalAttr<AtomicControlAttr>:$atomic_control),
+ clausesArgs);
// Override region definition.
let regions = (region SizedRegion<1>:$region);
diff --git a/mlir/test/Dialect/OpenMP/ops.mlir b/mlir/test/Dialect/OpenMP/ops.mlir
index 47cfc5278a5d0..e377b7aab3e43 100644
--- a/mlir/test/Dialect/OpenMP/ops.mlir
+++ b/mlir/test/Dialect/OpenMP/ops.mlir
@@ -1562,6 +1562,15 @@ func.func @omp_atomic_update(%x : memref<i32>, %expr : i32, %xBool : memref<i1>,
omp.yield(%newval : i32)
}
+ // CHECK: omp.atomic.update %[[X]] : memref<i32> {
+ // CHECK-NEXT: (%[[XVAL:.*]]: i32):
+ // CHECK-NEXT: omp.yield(%{{.+}} : i32)
+ // CHECK-NEXT: } {atomic_control_attr = #omp.atomic_control<amdgpu_ignore_denormal_mode = true, amdgpu_fine_grained_memory = true, amdgpu_remote_memory = true>}
+ omp.atomic.update %x : memref<i32> {
+ ^bb0(%xval:i32):
+ omp.yield(%const:i32)
+ } {atomic_control_attr = #omp.atomic_control<amdgpu_ignore_denormal_mode = true, amdgpu_fine_grained_memory = true, amdgpu_remote_memory = true>}
+
return
}
|
tarunprabhu
reviewed
Jun 9, 2025
There was a problem hiding this comment.
Thanks for this.
Since you have added both FlangOption and FC1Option in Options.td, could you add a test that checks that the option is handled by the main driver as well as fc1. One that simply checks that the option gets passed down to fc1 from the main driver would be sufficient.
| @@ -53,6 +53,11 @@ class TargetOptions { | |||
|
|
|||
| /// Print verbose assembly | |||
| bool asmVerbose = false; | |||
|
|
|||
| /// Atomic Control Options for AMD GPU | |||
There was a problem hiding this comment.
Nit
Suggested change
| /// Atomic Control Options for AMD GPU | |
| /// Atomic control options for AMD GPU |
| @@ -58,10 +58,24 @@ void setTargetCPU(mlir::ModuleOp mod, llvm::StringRef cpu); | |||
| /// Get the target CPU string from the Module or return a null reference. | |||
| llvm::StringRef getTargetCPU(mlir::ModuleOp mod); | |||
|
|
|||
| // Setters and Getters for atomic control options | |||
There was a problem hiding this comment.
Nit:
Suggested change
| // Setters and Getters for atomic control options | |
| // Setters and getters for atomic control options |
| /// Set the tune CPU for the module. `cpu` must not be deallocated while | ||
| /// module `mod` is still live. | ||
| void setTuneCPU(mlir::ModuleOp mod, llvm::StringRef cpu); | ||
|
|
||
| // set Atomic control options for amd gpu. |
There was a problem hiding this comment.
Nit
Suggested change
| // set Atomic control options for amd gpu. | |
| // set atomic control options for amd gpu. |
flang/lib/Lower/OpenMP/OpenMP.cpp
Outdated
| @@ -2911,8 +2911,13 @@ static void genAtomicUpdateStatement( | |||
| if (rightHandClauseList) | |||
| genOmpAtomicHintAndMemoryOrderClauses(converter, *rightHandClauseList, hint, | |||
| memoryOrder); | |||
| auto module = firOpBuilder.getModule(); | |||
There was a problem hiding this comment.
Could we use a more concrete type instead of auto here
| } | ||
|
|
||
| bool fir::getAmdgpuFineGrainedMemory(mlir::ModuleOp mod) { | ||
| if (auto attr = |
There was a problem hiding this comment.
Would it be correct to use mod->hasAttrOfType here? If so, that might be clearer since, if I understand this correctly, you are returning true if the attribute is present and false otherwise anyway.
There was a problem hiding this comment.
Updated. Thank you!
| // AtomicControlAttr | ||
| //===----------------------------------------------------------------------===// | ||
|
|
||
| // Runtime library flags attribute that holds information for lowering to LLVM. |
There was a problem hiding this comment.
I don't understand what this comment is saying. Could this be made a bit clearer?
There was a problem hiding this comment.
That was copy-paste error. I have updated the comment
anchuraj
force-pushed
the
atomic-control-frontend
branch
from
c69cca2 to
26d8e64
Compare
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
anchuraj
force-pushed
the
atomic-control-frontend
branch
from
26d8e64 to
6a42a1d
Compare
Hi Thank you very much for reviewing. I have added the driver test case in #143769 (where it checks metadata is added) I will raise it for review after this is merged. I have also updated AtomicControl attribute as non-optional (Though input to the compiler is optional, in cases where no atomic control option is specified, we need to emit metadatas |
anchuraj
force-pushed
the
atomic-control-frontend
branch
2 times, most recently
from
3499c15 to
74ba59b
Compare
tarunprabhu
approved these changes
Jun 12, 2025
| } | ||
|
|
||
| bool fir::getAmdgpuIgnoreDenormalMode(mlir::ModuleOp mod) { | ||
| if (mod->hasAttrOfType<mlir::UnitAttr>(amdgpuIgnoreDenormalModeName)) |
There was a problem hiding this comment.
I think we can just return the result of mod->hasAttrOfType here
There was a problem hiding this comment.
True , missed it :) . I have updated
anchuraj
force-pushed
the
atomic-control-frontend
branch
from
74ba59b to
1b93740
Compare
anchuraj
changed the title
kparzysz
reviewed
Jun 18, 2025
anchuraj
force-pushed
the
atomic-control-frontend
branch
from
676d8bc to
6d54cfd
Compare
Thank you for the review. I have updated. |
anchuraj
force-pushed
the
atomic-control-frontend
branch
from
6d54cfd to
ccc83b2
Compare
anchuraj
force-pushed
the
atomic-control-frontend
branch
from
ccc83b2 to
1fdbf81
Compare
anchuraj
force-pushed
the
atomic-control-frontend
branch
from
1fdbf81 to
fe9a702
Compare
|
pinging for review |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Atomic Control Options are used to specify architectural characteristics to help lowering of atomic operations. The options used are:
-f[no-]atomic-remote-memory,-f[no-]atomic-fine-grained-memory,-f[no-]atomic-ignore-denormal-mode.Legacy option
-m[no-]unsafe-fp-atomicsis aliased to-f[no-]ignore-denormal-mode.More details can be found in #102569. This PR implements the frontend support for these options with OpenMP atomic in flang.
Backend changes are available in the draft PR: #143769 which will be raised after this merged.