ICODE-ADC/lammps
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update Kokkos library in LAMMPS to v4.2

Browse Source
This commit is contained in:
Stan Gerald Moore
2023-11-21 15:02:12 -07:00
parent 33dcfb5390
commit 3306b95589
493 changed files with 41548 additions and 15629 deletions
Download Patch File Download Diff File Expand all files Collapse all files

93
lib/kokkos/CHANGELOG.md
View File

@ -1,6 +1,97 @@
# CHANGELOG # CHANGELOG
## [4.1.00](https://github.com/kokkos/kokkos/tree/4.0.01) (2023-06-16) ## [4.2.00](https://github.com/kokkos/kokkos/tree/4.2.00) (2023-11-06)
[Full Changelog](https://github.com/kokkos/kokkos/compare/4.1.00...4.2.00)
### Features:
- SIMD: significant improvements to SIMD support and alignment with C++26 SIMD
- add `Kokkos::abs` overload for SIMD types [\#6069](https://github.com/kokkos/kokkos/pull/6069)
- add generator constructors [\#6347](https://github.com/kokkos/kokkos/pull/6347)
- convert binary operators to hidden friends [\#6320](https://github.com/kokkos/kokkos/pull/6320)
- add shift operators [\#6109](https://github.com/kokkos/kokkos/pull/6109)
- add `float` support [\#6177](https://github.com/kokkos/kokkos/pull/6177)
- add remaining `gather_from` and `scatter_to` overloads [\#6220](https://github.com/kokkos/kokkos/pull/6220)
- define simd math function overloads in the Kokkos namespace [\#6465](https://github.com/kokkos/kokkos/pull/6465), [\#6487](https://github.com/kokkos/kokkos/pull/6487)
- `Kokkos_ENABLE_NATIVE=ON` autodetects SIMD types supported [\#6188](https://github.com/kokkos/kokkos/pull/6188)
- fix AVX2 SIMD support for ZEN2 AMD CPU [\#6238](https://github.com/kokkos/kokkos/pull/6238)
- `Kokkos::printf` [\#6083](https://github.com/kokkos/kokkos/pull/6083)
- `Kokkos::sort`: support custom comparator [\#6253](https://github.com/kokkos/kokkos/pull/6253)
- `half_t` and `bhalf_t` numeric traits [\#5778](https://github.com/kokkos/kokkos/pull/5778)
- `half_t` and `bhalf_t` mixed comparisons [\#6407](https://github.com/kokkos/kokkos/pull/6407)
- `half_t` and `bhalf_t` mathematical functions [\#6124](https://github.com/kokkos/kokkos/pull/6124)
- `TeamThreadRange` `parallel_scan` with return value [\#6090](https://github.com/kokkos/kokkos/pull/6090), [\#6301](https://github.com/kokkos/kokkos/pull/6301), [\#6302](https://github.com/kokkos/kokkos/pull/6302), [\#6303](https://github.com/kokkos/kokkos/pull/6303), [\#6307](https://github.com/kokkos/kokkos/pull/6307)
- `ThreadVectorRange` `parallel_scan` with return value [\#6235](https://github.com/kokkos/kokkos/pull/6235), [\#6242](https://github.com/kokkos/kokkos/pull/6242), [\#6308](https://github.com/kokkos/kokkos/pull/6308), [\#6305](https://github.com/kokkos/kokkos/pull/6305), [\#6292](https://github.com/kokkos/kokkos/pull/6292)
- Add team-level std algorithms [\#6200](https://github.com/kokkos/kokkos/pull/6200), [\#6205](https://github.com/kokkos/kokkos/pull/6205), [\#6207](https://github.com/kokkos/kokkos/pull/6207), [\#6208](https://github.com/kokkos/kokkos/pull/6208), [\#6209](https://github.com/kokkos/kokkos/pull/6209), [\#6210](https://github.com/kokkos/kokkos/pull/6210), [\#6211](https://github.com/kokkos/kokkos/pull/6211), [\#6212](https://github.com/kokkos/kokkos/pull/6212), [\#6213](https://github.com/kokkos/kokkos/pull/6213), [\#6256](https://github.com/kokkos/kokkos/pull/6256), [\#6258](https://github.com/kokkos/kokkos/pull/6258), [\#6350](https://github.com/kokkos/kokkos/pull/6350), [\#6351](https://github.com/kokkos/kokkos/pull/6351)
- Serial: Allow for distinct execution space instances [\#6441](https://github.com/kokkos/kokkos/pull/6441)
### Backend and Architecture Enhancements:
#### CUDA:
- Fixed potential data race in Cuda `parallel_reduce` [\#6236](https://github.com/kokkos/kokkos/pull/6236)
- Use `cudaMallocAsync` by default [\#6402](https://github.com/kokkos/kokkos/pull/6402)
- Bugfix for using Kokkos from a thread of execution [\#6299](https://github.com/kokkos/kokkos/pull/6299)
#### HIP:
- New naming convention for AMD GPU: VEGA906, VEGA908, VEGA90A, NAVI1030 to AMD_GFX906, AMD_GFX908, AMD_GFX90A, AMD_GFX1030 [\#6266](https://github.com/kokkos/kokkos/pull/6266)
- Add initial support for gfx942: [\#6358](https://github.com/kokkos/kokkos/pull/6358)
- Improve reduction performance [\#6229](https://github.com/kokkos/kokkos/pull/6229)
- Deprecate `HIP(hipStream_t,bool)` constructor [\#6401](https://github.com/kokkos/kokkos/pull/6401)
- Add support for Graph [\#6370](https://github.com/kokkos/kokkos/pull/6370)
- Improve reduction performance when using Teams [\#6284](https://github.com/kokkos/kokkos/pull/6284)
- Fix concurrency calculation [\#6479](https://github.com/kokkos/kokkos/pull/6479)
- Fix potential data race in HIP `parallel_reduce` [\#6429](https://github.com/kokkos/kokkos/pull/6429)
#### SYCL:
- Enforce external `sycl::queues` to be in-order [\#6246](https://github.com/kokkos/kokkos/pull/6246)
- Improve reduction performance: [\#6272](https://github.com/kokkos/kokkos/pull/6272) [\#6271](https://github.com/kokkos/kokkos/pull/6271) [\#6270](https://github.com/kokkos/kokkos/pull/6270) [\#6264](https://github.com/kokkos/kokkos/pull/6264)
- Allow using the SYCL execution space on AMD GPUs [\#6321](https://github.com/kokkos/kokkos/pull/6321)
- Allow sorting via native oneDPL to support Views with stride=1 [\#6322](https://github.com/kokkos/kokkos/pull/6322)
- Make in-order queues the default via macro [\#6189](https://github.com/kokkos/kokkos/pull/6189)
#### OpenACC:
- Support Clacc compiler [\#6250](https://github.com/kokkos/kokkos/pull/6250)
### General Enhancements
- Add missing `is_*_view` traits and `is_*_view_v` helper variable templates for `DynRankView`, `DynamicView`, `OffsetView`, `ScatterView` containers [\#6195](https://github.com/kokkos/kokkos/pull/6195)
- Make `nvcc_wrapper` and `compiler_launcher` scripts more portable by switching to a `#!/usr/bin/env` shebang [\#6357](https://github.com/kokkos/kokkos/pull/6357)
- Add an improved `Kokkos::malloc` / `Kokkos::free` performance test [\#6377](https://github.com/kokkos/kokkos/pull/6377)
- Ensure `Views` with `size==0` can be used with `deep_copy` [\#6273](https://github.com/kokkos/kokkos/pull/6273)
- `Kokkos::abort` is moved to header `Kokkos_Abort.hpp` [\#6445](https://github.com/kokkos/kokkos/pull/6445)
- `KOKKOS_ASSERT`, `KOKKOS_EXPECTS`, `KOKKOS_ENSURES` are moved to header `Kokkos_Assert.hpp` [\#6445](https://github.com/kokkos/kokkos/pull/6445)
- Add a permuted-index mode to the gups benchmark [\#6378](https://github.com/kokkos/kokkos/pull/6378)
- Check for overflow during backend initialization [\#6159](https://github.com/kokkos/kokkos/pull/6159)
- Make constraints on `Kokkos::sort` more visible [\#6234](https://github.com/kokkos/kokkos/pull/6234) and cleanup API [\#6239](https://github.com/kokkos/kokkos/pull/6239)
- Add converting assignment to `DualView`: [\#6474](https://github.com/kokkos/kokkos/pull/6474)
### Build System Changes
- Export `Kokkos_CXX_COMPILER_VERSION` [\#6282](https://github.com/kokkos/kokkos/pull/6282)
- Disable default oneDPL support in Trilinos [\#6342](https://github.com/kokkos/kokkos/pull/6342)
### Incompatibilities (i.e. breaking changes)
- Ensure that `Kokkos::complex` only gets instantiated for cv-unqualified floating-point types [\#6251](https://github.com/kokkos/kokkos/pull/6251)
- Removed (deprecated-3) support for volatile join operators in reductions [\#6385](https://github.com/kokkos/kokkos/pull/6385)
- Enforce `ViewCtorArgs` restrictions for `create_mirror_view` [\#6304](https://github.com/kokkos/kokkos/pull/6304)
- SIMD types for ARM NEON are not autodetected anymore but need `Kokkos_ARCH_ARM_NEON` or `Kokkos_ARCH_NATIVE=ON` [\#6394](https://github.com/kokkos/kokkos/pull/6394)
- Remove `#include <iostream>` from headers where possible [\#6482](https://github.com/kokkos/kokkos/pull/6482)
### Deprecations
- Deprecated `Kokkos::vector` [\#6252](https://github.com/kokkos/kokkos/pull/6252)
- All host allocation mechanisms except for `STD_MALLOC` have been deprecated [\#6341](https://github.com/kokkos/kokkos/pull/6341)
### Bug Fixes
- Missing memory fence in `RandomPool::free_state` functions [\#6290](https://github.com/kokkos/kokkos/pull/6290)
- Fix for corner case in `Kokkos::Experimental::is_partitioned` algorithm [\#6257](https://github.com/kokkos/kokkos/pull/6257)
- Fix initialization of scratch lock variables in the `Cuda` backend [\#6433](https://github.com/kokkos/kokkos/pull/6433)
- Fixes for `Kokkos::Array` [\#6372](https://github.com/kokkos/kokkos/pull/6372)
- Fixed symlink configure issue for Windows [\#6241](https://github.com/kokkos/kokkos/pull/6241)
- OpenMPTarget init-join fix [\#6444](https://github.com/kokkos/kokkos/pull/6444)
- Fix atomic operations bug for Min and Max [\#6435](https://github.com/kokkos/kokkos/pull/6435)
- Fix implementation for `cyl_bessel_i0` [\#6484](https://github.com/kokkos/kokkos/pull/6484)
- Fix various NVCC warnings in `BinSort`, `Array`, and bit manipulation function templates [\#6483](https://github.com/kokkos/kokkos/pull/6483)
## [4.1.00](https://github.com/kokkos/kokkos/tree/4.1.00) (2023-06-16)
[Full Changelog](https://github.com/kokkos/kokkos/compare/4.0.01...4.1.00) [Full Changelog](https://github.com/kokkos/kokkos/compare/4.0.01...4.1.00)
### Features: ### Features:

5
lib/kokkos/CMakeLists.txt
View File

@ -150,8 +150,8 @@ ENDIF()
set(Kokkos_VERSION_MAJOR 4) set(Kokkos_VERSION_MAJOR 4)
set(Kokkos_VERSION_MINOR 1) set(Kokkos_VERSION_MINOR 2)
set(Kokkos_VERSION_PATCH 00) set(Kokkos_VERSION_PATCH 0)
set(Kokkos_VERSION "${Kokkos_VERSION_MAJOR}.${Kokkos_VERSION_MINOR}.${Kokkos_VERSION_PATCH}") set(Kokkos_VERSION "${Kokkos_VERSION_MAJOR}.${Kokkos_VERSION_MINOR}.${Kokkos_VERSION_PATCH}")
message(STATUS "Kokkos version: ${Kokkos_VERSION}") message(STATUS "Kokkos version: ${Kokkos_VERSION}")
math(EXPR KOKKOS_VERSION "${Kokkos_VERSION_MAJOR} * 10000 + ${Kokkos_VERSION_MINOR} * 100 + ${Kokkos_VERSION_PATCH}") math(EXPR KOKKOS_VERSION "${Kokkos_VERSION_MAJOR} * 10000 + ${Kokkos_VERSION_MINOR} * 100 + ${Kokkos_VERSION_PATCH}")
@ -314,7 +314,6 @@ KOKKOS_PROCESS_SUBPACKAGES()
# E) If Kokkos itself is enabled, process the Kokkos package # E) If Kokkos itself is enabled, process the Kokkos package
# #
KOKKOS_EXCLUDE_AUTOTOOLS_FILES()
KOKKOS_PACKAGE_POSTPROCESS() KOKKOS_PACKAGE_POSTPROCESS()
KOKKOS_CONFIGURE_CORE() KOKKOS_CONFIGURE_CORE()

70
lib/kokkos/Makefile.kokkos
View File

@ -11,8 +11,8 @@ CXXFLAGS += $(SHFLAGS)
endif endif
KOKKOS_VERSION_MAJOR = 4 KOKKOS_VERSION_MAJOR = 4
KOKKOS_VERSION_MINOR = 1 KOKKOS_VERSION_MINOR = 2
KOKKOS_VERSION_PATCH = 00 KOKKOS_VERSION_PATCH = 0
KOKKOS_VERSION = $(shell echo $(KOKKOS_VERSION_MAJOR)*10000+$(KOKKOS_VERSION_MINOR)*100+$(KOKKOS_VERSION_PATCH) | bc) KOKKOS_VERSION = $(shell echo $(KOKKOS_VERSION_MAJOR)*10000+$(KOKKOS_VERSION_MINOR)*100+$(KOKKOS_VERSION_PATCH) | bc)
# Options: Cuda,HIP,SYCL,OpenMPTarget,OpenMP,Threads,Serial # Options: Cuda,HIP,SYCL,OpenMPTarget,OpenMP,Threads,Serial
@ -23,7 +23,7 @@ KOKKOS_DEVICES ?= "OpenMP"
# NVIDIA: Kepler,Kepler30,Kepler32,Kepler35,Kepler37,Maxwell,Maxwell50,Maxwell52,Maxwell53,Pascal60,Pascal61,Volta70,Volta72,Turing75,Ampere80,Ampere86,Ada89,Hopper90 # NVIDIA: Kepler,Kepler30,Kepler32,Kepler35,Kepler37,Maxwell,Maxwell50,Maxwell52,Maxwell53,Pascal60,Pascal61,Volta70,Volta72,Turing75,Ampere80,Ampere86,Ada89,Hopper90
# ARM: ARMv80,ARMv81,ARMv8-ThunderX,ARMv8-TX2,A64FX # ARM: ARMv80,ARMv81,ARMv8-ThunderX,ARMv8-TX2,A64FX
# IBM: BGQ,Power7,Power8,Power9 # IBM: BGQ,Power7,Power8,Power9
# AMD-GPUS: Vega906,Vega908,Vega90A,Navi1030 # AMD-GPUS: GFX906,GFX908,GFX90A,GFX942,GFX1030,GFX1100
# AMD-CPUS: AMDAVX,Zen,Zen2,Zen3 # AMD-CPUS: AMDAVX,Zen,Zen2,Zen3
# Intel-GPUs: Gen9,Gen11,Gen12LP,DG1,XeHP,PVC # Intel-GPUs: Gen9,Gen11,Gen12LP,DG1,XeHP,PVC
KOKKOS_ARCH ?= "" KOKKOS_ARCH ?= ""
@ -40,7 +40,7 @@ KOKKOS_TRIBITS ?= "no"
KOKKOS_STANDALONE_CMAKE ?= "no" KOKKOS_STANDALONE_CMAKE ?= "no"
# Default settings specific options. # Default settings specific options.
# Options: force_uvm,use_ldg,rdc,enable_lambda,enable_constexpr # Options: force_uvm,use_ldg,rdc,enable_lambda,enable_constexpr,disable_malloc_async
KOKKOS_CUDA_OPTIONS ?= "enable_lambda" KOKKOS_CUDA_OPTIONS ?= "enable_lambda"
# Options: rdc # Options: rdc
@ -92,6 +92,7 @@ KOKKOS_INTERNAL_CUDA_USE_UVM := $(call kokkos_has_string,$(KOKKOS_CUDA_OPTIONS),
KOKKOS_INTERNAL_CUDA_USE_RELOC := $(call kokkos_has_string,$(KOKKOS_CUDA_OPTIONS),rdc) KOKKOS_INTERNAL_CUDA_USE_RELOC := $(call kokkos_has_string,$(KOKKOS_CUDA_OPTIONS),rdc)
KOKKOS_INTERNAL_CUDA_USE_LAMBDA := $(call kokkos_has_string,$(KOKKOS_CUDA_OPTIONS),enable_lambda) KOKKOS_INTERNAL_CUDA_USE_LAMBDA := $(call kokkos_has_string,$(KOKKOS_CUDA_OPTIONS),enable_lambda)
KOKKOS_INTERNAL_CUDA_USE_CONSTEXPR := $(call kokkos_has_string,$(KOKKOS_CUDA_OPTIONS),enable_constexpr) KOKKOS_INTERNAL_CUDA_USE_CONSTEXPR := $(call kokkos_has_string,$(KOKKOS_CUDA_OPTIONS),enable_constexpr)
KOKKOS_INTERNAL_CUDA_DISABLE_MALLOC_ASYNC := $(call kokkos_has_string,$(KOKKOS_CUDA_OPTIONS),disable_malloc_async)
KOKKOS_INTERNAL_HPX_ENABLE_ASYNC_DISPATCH := $(call kokkos_has_string,$(KOKKOS_HPX_OPTIONS),enable_async_dispatch) KOKKOS_INTERNAL_HPX_ENABLE_ASYNC_DISPATCH := $(call kokkos_has_string,$(KOKKOS_HPX_OPTIONS),enable_async_dispatch)
# deprecated # deprecated
KOKKOS_INTERNAL_ENABLE_DESUL_ATOMICS := $(call kokkos_has_string,$(KOKKOS_OPTIONS),enable_desul_atomics) KOKKOS_INTERNAL_ENABLE_DESUL_ATOMICS := $(call kokkos_has_string,$(KOKKOS_OPTIONS),enable_desul_atomics)
@ -412,10 +413,11 @@ ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ZEN3), 0)
KOKKOS_INTERNAL_USE_ARCH_ZEN := $(call kokkos_has_string,$(KOKKOS_ARCH),Zen) KOKKOS_INTERNAL_USE_ARCH_ZEN := $(call kokkos_has_string,$(KOKKOS_ARCH),Zen)
endif endif
endif endif
KOKKOS_INTERNAL_USE_ARCH_VEGA906 := $(call kokkos_has_string,$(KOKKOS_ARCH),Vega906) KOKKOS_INTERNAL_USE_ARCH_AMD_GFX906 := $(or $(call kokkos_has_string,$(KOKKOS_ARCH),VEGA906),$(call kokkos_has_string,$(KOKKOS_ARCH),AMD_GFX906))
KOKKOS_INTERNAL_USE_ARCH_VEGA908 := $(call kokkos_has_string,$(KOKKOS_ARCH),Vega908) KOKKOS_INTERNAL_USE_ARCH_AMD_GFX908 := $(or $(call kokkos_has_string,$(KOKKOS_ARCH),VEGA908),$(call kokkos_has_string,$(KOKKOS_ARCH),AMD_GFX908))
KOKKOS_INTERNAL_USE_ARCH_VEGA90A := $(call kokkos_has_string,$(KOKKOS_ARCH),Vega90A) KOKKOS_INTERNAL_USE_ARCH_AMD_GFX90A := $(or $(call kokkos_has_string,$(KOKKOS_ARCH),VEGA90A),$(call kokkos_has_string,$(KOKKOS_ARCH),AMD_GFX90A))
KOKKOS_INTERNAL_USE_ARCH_NAVI1030 := $(call kokkos_has_string,$(KOKKOS_ARCH),Navi1030) KOKKOS_INTERNAL_USE_ARCH_AMD_GFX1030 := $(or $(call kokkos_has_string,$(KOKKOS_ARCH),NAVI1030),$(call kokkos_has_string,$(KOKKOS_ARCH),AMD_GFX1030))
KOKKOS_INTERNAL_USE_ARCH_AMD_GFX1100 := $(or $(call kokkos_has_string,$(KOKKOS_ARCH),NAVI1100),$(call kokkos_has_string,$(KOKKOS_ARCH),AMD_GFX1100))
# Any AVX? # Any AVX?
KOKKOS_INTERNAL_USE_ARCH_SSE42 := $(shell expr $(KOKKOS_INTERNAL_USE_ARCH_WSM)) KOKKOS_INTERNAL_USE_ARCH_SSE42 := $(shell expr $(KOKKOS_INTERNAL_USE_ARCH_WSM))
@ -698,6 +700,12 @@ ifeq ($(KOKKOS_INTERNAL_USE_CUDA), 1)
ifeq ($(KOKKOS_INTERNAL_COMPILER_CLANG), 1) ifeq ($(KOKKOS_INTERNAL_COMPILER_CLANG), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_IMPL_CUDA_CLANG_WORKAROUND") tmp := $(call kokkos_append_header,"$H""define KOKKOS_IMPL_CUDA_CLANG_WORKAROUND")
endif endif
ifeq ($(KOKKOS_INTERNAL_CUDA_DISABLE_MALLOC_ASYNC), 0)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ENABLE_IMPL_CUDA_MALLOC_ASYNC")
else
tmp := $(call kokkos_append_header,"/* $H""undef KOKKOS_ENABLE_IMPL_CUDA_MALLOC_ASYNC */")
endif
endif endif
ifeq ($(KOKKOS_INTERNAL_USE_HPX), 1) ifeq ($(KOKKOS_INTERNAL_USE_HPX), 1)
@ -710,6 +718,7 @@ endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ARMV80), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ARMV80), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV80") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV80")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARM_NEON")
ifeq ($(KOKKOS_INTERNAL_COMPILER_CRAY), 1) ifeq ($(KOKKOS_INTERNAL_COMPILER_CRAY), 1)
KOKKOS_CXXFLAGS += KOKKOS_CXXFLAGS +=
@ -722,6 +731,7 @@ endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ARMV81), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ARMV81), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV81") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV81")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARM_NEON")
ifeq ($(KOKKOS_INTERNAL_COMPILER_CRAY), 1) ifeq ($(KOKKOS_INTERNAL_COMPILER_CRAY), 1)
KOKKOS_CXXFLAGS += KOKKOS_CXXFLAGS +=
@ -734,6 +744,7 @@ endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_A64FX), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_A64FX), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_A64FX") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_A64FX")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARM_NEON")
KOKKOS_CXXFLAGS += -march=armv8.2-a+sve KOKKOS_CXXFLAGS += -march=armv8.2-a+sve
KOKKOS_LDFLAGS += -march=armv8.2-a+sve KOKKOS_LDFLAGS += -march=armv8.2-a+sve
@ -749,7 +760,7 @@ endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ZEN), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ZEN), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_ZEN") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_ZEN")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_AVX2") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AVX2")
ifeq ($(KOKKOS_INTERNAL_COMPILER_INTEL), 1) ifeq ($(KOKKOS_INTERNAL_COMPILER_INTEL), 1)
KOKKOS_CXXFLAGS += -mavx2 KOKKOS_CXXFLAGS += -mavx2
@ -762,7 +773,7 @@ endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ZEN2), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ZEN2), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_ZEN2") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_ZEN2")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_AVX2") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AVX2")
ifeq ($(KOKKOS_INTERNAL_COMPILER_INTEL), 1) ifeq ($(KOKKOS_INTERNAL_COMPILER_INTEL), 1)
KOKKOS_CXXFLAGS += -mavx2 KOKKOS_CXXFLAGS += -mavx2
@ -775,7 +786,7 @@ endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ZEN3), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ZEN3), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_ZEN3") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_ZEN3")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_AVX2") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AVX2")
ifeq ($(KOKKOS_INTERNAL_COMPILER_INTEL), 1) ifeq ($(KOKKOS_INTERNAL_COMPILER_INTEL), 1)
KOKKOS_CXXFLAGS += -mavx2 KOKKOS_CXXFLAGS += -mavx2
@ -789,6 +800,7 @@ endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ARMV8_THUNDERX), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ARMV8_THUNDERX), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV80") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV80")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV8_THUNDERX") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV8_THUNDERX")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARM_NEON")
ifeq ($(KOKKOS_INTERNAL_COMPILER_CRAY), 1) ifeq ($(KOKKOS_INTERNAL_COMPILER_CRAY), 1)
KOKKOS_CXXFLAGS += KOKKOS_CXXFLAGS +=
@ -802,6 +814,7 @@ endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ARMV8_THUNDERX2), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_ARMV8_THUNDERX2), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV81") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV81")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV8_THUNDERX2") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARMV8_THUNDERX2")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_ARM_NEON")
ifeq ($(KOKKOS_INTERNAL_COMPILER_CRAY), 1) ifeq ($(KOKKOS_INTERNAL_COMPILER_CRAY), 1)
KOKKOS_CXXFLAGS += KOKKOS_CXXFLAGS +=
@ -1085,29 +1098,34 @@ endif
# Figure out the architecture flag for ROCm. # Figure out the architecture flag for ROCm.
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_VEGA906), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_AMD_GFX906), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_VEGA906") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GFX906")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_VEGA") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GPU")
KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx906 KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx906
endif endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_VEGA908), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_AMD_GFX908), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_VEGA908") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GFX908")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_VEGA") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GPU")
KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx908 KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx908
endif endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_VEGA90A), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_AMD_GFX90A), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_VEGA90A") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GFX90A")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_VEGA") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GPU")
KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx90a KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx90a
endif endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_NAVI1030), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_AMD_GFX942), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_NAVI1030") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GFX942")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_NAVI") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GPU")
KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx942
endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_AMD_GFX1030), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GFX1030")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GPU")
KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx1030 KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx1030
endif endif
ifeq ($(KOKKOS_INTERNAL_USE_ARCH_NAVI1100), 1) ifeq ($(KOKKOS_INTERNAL_USE_ARCH_AMD_GFX1100), 1)
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_NAVI1100") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GFX1100")
tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_NAVI") tmp := $(call kokkos_append_header,"$H""define KOKKOS_ARCH_AMD_GPU")
KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx1100 KOKKOS_INTERNAL_HIP_ARCH_FLAG := --offload-arch=gfx1100
endif endif

2
lib/kokkos/Makefile.targets
View File

@ -36,6 +36,8 @@ Kokkos_HostSpace_deepcopy.o: $(KOKKOS_CPP_DEPENDS) $(KOKKOS_PATH)/core/src/impl/
$(CXX) $(KOKKOS_CPPFLAGS) $(KOKKOS_CXXFLAGS) $(CXXFLAGS) -c $(KOKKOS_PATH)/core/src/impl/Kokkos_HostSpace_deepcopy.cpp $(CXX) $(KOKKOS_CPPFLAGS) $(KOKKOS_CXXFLAGS) $(CXXFLAGS) -c $(KOKKOS_PATH)/core/src/impl/Kokkos_HostSpace_deepcopy.cpp
Kokkos_NumericTraits.o: $(KOKKOS_CPP_DEPENDS) $(KOKKOS_PATH)/core/src/impl/Kokkos_NumericTraits.cpp Kokkos_NumericTraits.o: $(KOKKOS_CPP_DEPENDS) $(KOKKOS_PATH)/core/src/impl/Kokkos_NumericTraits.cpp
$(CXX) $(KOKKOS_CPPFLAGS) $(KOKKOS_CXXFLAGS) $(CXXFLAGS) -c $(KOKKOS_PATH)/core/src/impl/Kokkos_NumericTraits.cpp $(CXX) $(KOKKOS_CPPFLAGS) $(KOKKOS_CXXFLAGS) $(CXXFLAGS) -c $(KOKKOS_PATH)/core/src/impl/Kokkos_NumericTraits.cpp
Kokkos_Abort.o: $(KOKKOS_CPP_DEPENDS) $(KOKKOS_PATH)/core/src/impl/Kokkos_Abort.cpp
$(CXX) $(KOKKOS_CPPFLAGS) $(KOKKOS_CXXFLAGS) $(CXXFLAGS) -c $(KOKKOS_PATH)/core/src/impl/Kokkos_Abort.cpp
ifeq ($(KOKKOS_INTERNAL_USE_SERIAL), 1) ifeq ($(KOKKOS_INTERNAL_USE_SERIAL), 1)
Kokkos_Serial.o: $(KOKKOS_CPP_DEPENDS) $(KOKKOS_PATH)/core/src/Serial/Kokkos_Serial.cpp Kokkos_Serial.o: $(KOKKOS_CPP_DEPENDS) $(KOKKOS_PATH)/core/src/Serial/Kokkos_Serial.cpp

2
lib/kokkos/algorithms/CMakeLists.txt
View File

@ -2,6 +2,6 @@ IF (NOT Kokkos_INSTALL_TESTING)
ADD_SUBDIRECTORY(src) ADD_SUBDIRECTORY(src)
ENDIF() ENDIF()
# FIXME_OPENACC: temporarily disabled due to unimplemented features # FIXME_OPENACC: temporarily disabled due to unimplemented features
IF(NOT ((KOKKOS_ENABLE_OPENMPTARGET OR KOKKOS_ENABLE_OPENACC) AND KOKKOS_CXX_COMPILER_ID STREQUAL NVHPC)) IF(NOT ((KOKKOS_ENABLE_OPENMPTARGET AND KOKKOS_CXX_COMPILER_ID STREQUAL NVHPC) OR KOKKOS_ENABLE_OPENACC))
KOKKOS_ADD_TEST_DIRECTORIES(unit_tests) KOKKOS_ADD_TEST_DIRECTORIES(unit_tests)
ENDIF() ENDIF()

184
lib/kokkos/algorithms/src/Kokkos_NestedSort.hpp
View File

@ -14,175 +14,17 @@
// //
//@HEADER //@HEADER
#ifndef KOKKOS_NESTEDSORT_HPP_ #ifndef KOKKOS_NESTED_SORT_HPP_
#define KOKKOS_NESTEDSORT_HPP_ #define KOKKOS_NESTED_SORT_HPP_
#ifndef KOKKOS_IMPL_PUBLIC_INCLUDE
#include <Kokkos_Core.hpp> #define KOKKOS_IMPL_PUBLIC_INCLUDE
#include <std_algorithms/impl/Kokkos_HelperPredicates.hpp> #define KOKKOS_IMPL_PUBLIC_INCLUDE_NOTDEFINED_NESTED_SORT
#include <std_algorithms/Kokkos_Swap.hpp> #endif
namespace Kokkos { #include "sorting/Kokkos_NestedSortPublicAPI.hpp"
namespace Experimental {
namespace Impl { #ifdef KOKKOS_IMPL_PUBLIC_INCLUDE_NOTDEFINED_NESTED_SORT
#undef KOKKOS_IMPL_PUBLIC_INCLUDE
// true for TeamVectorRange, false for ThreadVectorRange #undef KOKKOS_IMPL_PUBLIC_INCLUDE_NOTDEFINED_NESTED_SORT
template <bool teamLevel> #endif
struct NestedRange {};
// Specialization for team-level
template <>
struct NestedRange<true> {
template <typename TeamMember, typename SizeType>
KOKKOS_FUNCTION static auto create(const TeamMember& t, SizeType len) {
return Kokkos::TeamVectorRange(t, len);
}
template <typename TeamMember>
KOKKOS_FUNCTION static void barrier(const TeamMember& t) {
t.team_barrier();
}
};
// Specialization for thread-level
template <>
struct NestedRange<false> {
template <typename TeamMember, typename SizeType>
KOKKOS_FUNCTION static auto create(const TeamMember& t, SizeType len) {
return Kokkos::ThreadVectorRange(t, len);
}
// Barrier is no-op, as vector lanes of a thread are implicitly synchronized
// after parallel region
template <typename TeamMember>
KOKKOS_FUNCTION static void barrier(const TeamMember&) {}
};
// When just doing sort (not sort_by_key), use nullptr_t for ValueViewType.
// This only takes the NestedRange instance for template arg deduction.
template <class TeamMember, class KeyViewType, class ValueViewType,
class Comparator, bool useTeamLevel>
KOKKOS_INLINE_FUNCTION void sort_nested_impl(
const TeamMember& t, const KeyViewType& keyView,
[[maybe_unused]] const ValueViewType& valueView, const Comparator& comp,
const NestedRange<useTeamLevel>) {
using SizeType = typename KeyViewType::size_type;
using KeyType = typename KeyViewType::non_const_value_type;
using Range = NestedRange<useTeamLevel>;
SizeType n = keyView.extent(0);
SizeType npot = 1;
SizeType levels = 0;
// FIXME: ceiling power-of-two is a common thing to need - make it a utility
while (npot < n) {
levels++;
npot <<= 1;
}
for (SizeType i = 0; i < levels; i++) {
for (SizeType j = 0; j <= i; j++) {
// n/2 pairs of items are compared in parallel
Kokkos::parallel_for(Range::create(t, npot / 2), [=](const SizeType k) {
// How big are the brown/pink boxes?
// (Terminology comes from Wikipedia diagram)
// https://commons.wikimedia.org/wiki/File:BitonicSort.svg#/media/File:BitonicSort.svg
SizeType boxSize = SizeType(2) << (i - j);
// Which box contains this thread?
SizeType boxID = k >> (i - j); // k * 2 / boxSize;
SizeType boxStart = boxID << (1 + i - j); // boxID * boxSize
SizeType boxOffset = k - (boxStart >> 1); // k - boxID * boxSize / 2;
SizeType elem1 = boxStart + boxOffset;
// In first phase (j == 0, brown box): within a box, compare with the
// opposite value in the box.
// In later phases (j > 0, pink box): within a box, compare with fixed
// distance (boxSize / 2) apart.
SizeType elem2 = (j == 0) ? (boxStart + boxSize - 1 - boxOffset)
: (elem1 + boxSize / 2);
if (elem2 < n) {
KeyType key1 = keyView(elem1);
KeyType key2 = keyView(elem2);
if (comp(key2, key1)) {
keyView(elem1) = key2;
keyView(elem2) = key1;
if constexpr (!std::is_same_v<ValueViewType, std::nullptr_t>) {
Kokkos::Experimental::swap(valueView(elem1), valueView(elem2));
}
}
}
});
Range::barrier(t);
}
}
}
} // namespace Impl
template <class TeamMember, class ViewType>
KOKKOS_INLINE_FUNCTION void sort_team(const TeamMember& t,
const ViewType& view) {
Impl::sort_nested_impl(t, view, nullptr,
Experimental::Impl::StdAlgoLessThanBinaryPredicate<
typename ViewType::non_const_value_type>(),
Impl::NestedRange<true>());
}
template <class TeamMember, class ViewType, class Comparator>
KOKKOS_INLINE_FUNCTION void sort_team(const TeamMember& t, const ViewType& view,
const Comparator& comp) {
Impl::sort_nested_impl(t, view, nullptr, comp, Impl::NestedRange<true>());
}
template <class TeamMember, class KeyViewType, class ValueViewType>
KOKKOS_INLINE_FUNCTION void sort_by_key_team(const TeamMember& t,
const KeyViewType& keyView,
const ValueViewType& valueView) {
Impl::sort_nested_impl(t, keyView, valueView,
Experimental::Impl::StdAlgoLessThanBinaryPredicate<
typename KeyViewType::non_const_value_type>(),
Impl::NestedRange<true>());
}
template <class TeamMember, class KeyViewType, class ValueViewType,
class Comparator>
KOKKOS_INLINE_FUNCTION void sort_by_key_team(const TeamMember& t,
const KeyViewType& keyView,
const ValueViewType& valueView,
const Comparator& comp) {
Impl::sort_nested_impl(t, keyView, valueView, comp,
Impl::NestedRange<true>());
}
template <class TeamMember, class ViewType>
KOKKOS_INLINE_FUNCTION void sort_thread(const TeamMember& t,
const ViewType& view) {
Impl::sort_nested_impl(t, view, nullptr,
Experimental::Impl::StdAlgoLessThanBinaryPredicate<
typename ViewType::non_const_value_type>(),
Impl::NestedRange<false>());
}
template <class TeamMember, class ViewType, class Comparator>
KOKKOS_INLINE_FUNCTION void sort_thread(const TeamMember& t,
const ViewType& view,
const Comparator& comp) {
Impl::sort_nested_impl(t, view, nullptr, comp, Impl::NestedRange<false>());
}
template <class TeamMember, class KeyViewType, class ValueViewType>
KOKKOS_INLINE_FUNCTION void sort_by_key_thread(const TeamMember& t,
const KeyViewType& keyView,
const ValueViewType& valueView) {
Impl::sort_nested_impl(t, keyView, valueView,
Experimental::Impl::StdAlgoLessThanBinaryPredicate<
typename KeyViewType::non_const_value_type>(),
Impl::NestedRange<false>());
}
template <class TeamMember, class KeyViewType, class ValueViewType,
class Comparator>
KOKKOS_INLINE_FUNCTION void sort_by_key_thread(const TeamMember& t,
const KeyViewType& keyView,
const ValueViewType& valueView,
const Comparator& comp) {
Impl::sort_nested_impl(t, keyView, valueView, comp,
Impl::NestedRange<false>());
}
} // namespace Experimental
} // namespace Kokkos
#endif #endif

6
lib/kokkos/algorithms/src/Kokkos_Random.hpp
View File

@ -956,6 +956,8 @@ class Random_XorShift64_Pool {
KOKKOS_INLINE_FUNCTION KOKKOS_INLINE_FUNCTION
void free_state(const Random_XorShift64<DeviceType>& state) const { void free_state(const Random_XorShift64<DeviceType>& state) const {
state_(state.state_idx_, 0) = state.state_; state_(state.state_idx_, 0) = state.state_;
// Release the lock only after the state has been updated in memory
Kokkos::memory_fence();
locks_(state.state_idx_, 0) = 0; locks_(state.state_idx_, 0) = 0;
} }
}; };
@ -1208,7 +1210,9 @@ class Random_XorShift1024_Pool {
KOKKOS_INLINE_FUNCTION KOKKOS_INLINE_FUNCTION
void free_state(const Random_XorShift1024<DeviceType>& state) const { void free_state(const Random_XorShift1024<DeviceType>& state) const {
for (int i = 0; i < 16; i++) state_(state.state_idx_, i) = state.state_[i]; for (int i = 0; i < 16; i++) state_(state.state_idx_, i) = state.state_[i];
p_(state.state_idx_, 0) = state.p_; p_(state.state_idx_, 0) = state.p_;
// Release the lock only after the state has been updated in memory
Kokkos::memory_fence();
locks_(state.state_idx_, 0) = 0; locks_(state.state_idx_, 0) = 0;
} }
}; };

759
lib/kokkos/algorithms/src/Kokkos_Sort.hpp
View File

@ -21,762 +21,9 @@
#define KOKKOS_IMPL_PUBLIC_INCLUDE_NOTDEFINED_SORT #define KOKKOS_IMPL_PUBLIC_INCLUDE_NOTDEFINED_SORT
#endif #endif
#include <Kokkos_Core.hpp> #include "sorting/Kokkos_BinSortPublicAPI.hpp"
#include <Kokkos_NestedSort.hpp> #include "sorting/Kokkos_SortPublicAPI.hpp"
#include <std_algorithms/Kokkos_BeginEnd.hpp> #include "sorting/Kokkos_NestedSortPublicAPI.hpp"
#include <algorithm>
#if defined(KOKKOS_ENABLE_CUDA)
// Workaround for `Instruction 'shfl' without '.sync' is not supported on
// .target sm_70 and higher from PTX ISA version 6.4`.
// Also see https://github.com/NVIDIA/cub/pull/170.
#if !defined(CUB_USE_COOPERATIVE_GROUPS)
#define CUB_USE_COOPERATIVE_GROUPS
#endif
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wshadow"
#if defined(KOKKOS_COMPILER_CLANG)
// Some versions of Clang fail to compile Thrust, failing with errors like
// this:
// <snip>/thrust/system/cuda/detail/core/agent_launcher.h:557:11:
// error: use of undeclared identifier 'va_printf'
// The exact combination of versions for Clang and Thrust (or CUDA) for this
// failure was not investigated, however even very recent version combination
// (Clang 10.0.0 and Cuda 10.0) demonstrated failure.
//
// Defining _CubLog here locally allows us to avoid that code path, however
// disabling some debugging diagnostics
#pragma push_macro("_CubLog")
#ifdef _CubLog
#undef _CubLog
#endif
#define _CubLog
#include <thrust/device_ptr.h>
#include <thrust/sort.h>
#pragma pop_macro("_CubLog")
#else
#include <thrust/device_ptr.h>
#include <thrust/sort.h>
#endif
#pragma GCC diagnostic pop
#endif
#if defined(KOKKOS_ENABLE_ONEDPL)
#include <oneapi/dpl/execution>
#include <oneapi/dpl/algorithm>
#endif
namespace Kokkos {
namespace Impl {
template <class DstViewType, class SrcViewType, int Rank = DstViewType::rank>
struct CopyOp;
template <class DstViewType, class SrcViewType>
struct CopyOp<DstViewType, SrcViewType, 1> {
KOKKOS_INLINE_FUNCTION
static void copy(DstViewType const& dst, size_t i_dst, SrcViewType const& src,
size_t i_src) {
dst(i_dst) = src(i_src);
}
};
template <class DstViewType, class SrcViewType>
struct CopyOp<DstViewType, SrcViewType, 2> {
KOKKOS_INLINE_FUNCTION
static void copy(DstViewType const& dst, size_t i_dst, SrcViewType const& src,
size_t i_src) {
for (int j = 0; j < (int)dst.extent(1); j++) dst(i_dst, j) = src(i_src, j);
}
};
template <class DstViewType, class SrcViewType>
struct CopyOp<DstViewType, SrcViewType, 3> {
KOKKOS_INLINE_FUNCTION
static void copy(DstViewType const& dst, size_t i_dst, SrcViewType const& src,
size_t i_src) {
for (int j = 0; j < dst.extent(1); j++)
for (int k = 0; k < dst.extent(2); k++)
dst(i_dst, j, k) = src(i_src, j, k);
}
};
} // namespace Impl
//----------------------------------------------------------------------------
template <class KeyViewType, class BinSortOp,
class Space = typename KeyViewType::device_type,
class SizeType = typename KeyViewType::memory_space::size_type>
class BinSort {
public:
template <class DstViewType, class SrcViewType>
struct copy_functor {
using src_view_type = typename SrcViewType::const_type;
using copy_op = Impl::CopyOp<DstViewType, src_view_type>;
DstViewType dst_values;
src_view_type src_values;
int dst_offset;
copy_functor(DstViewType const& dst_values_, int const& dst_offset_,
SrcViewType const& src_values_)
: dst_values(dst_values_),
src_values(src_values_),
dst_offset(dst_offset_) {}
KOKKOS_INLINE_FUNCTION
void operator()(const int& i) const {
copy_op::copy(dst_values, i + dst_offset, src_values, i);
}
};
template <class DstViewType, class PermuteViewType, class SrcViewType>
struct copy_permute_functor {
// If a Kokkos::View then can generate constant random access
// otherwise can only use the constant type.
using src_view_type = std::conditional_t<
Kokkos::is_view<SrcViewType>::value,
Kokkos::View<typename SrcViewType::const_data_type,
typename SrcViewType::array_layout,
typename SrcViewType::device_type
#if !defined(KOKKOS_COMPILER_NVHPC) // FIXME_NVHPC
,
Kokkos::MemoryTraits<Kokkos::RandomAccess>
#endif
>,
typename SrcViewType::const_type>;
using perm_view_type = typename PermuteViewType::const_type;
using copy_op = Impl::CopyOp<DstViewType, src_view_type>;
DstViewType dst_values;
perm_view_type sort_order;
src_view_type src_values;
int src_offset;
copy_permute_functor(DstViewType const& dst_values_,
PermuteViewType const& sort_order_,
SrcViewType const& src_values_, int const& src_offset_)
: dst_values(dst_values_),
sort_order(sort_order_),
src_values(src_values_),
src_offset(src_offset_) {}
KOKKOS_INLINE_FUNCTION
void operator()(const int& i) const {
copy_op::copy(dst_values, i, src_values, src_offset + sort_order(i));
}
};
// Naming this alias "execution_space" would be problematic since it would be
// considered as execution space for the various functors which might use
// another execution space through sort() or create_permute_vector().
using exec_space = typename Space::execution_space;
using bin_op_type = BinSortOp;
struct bin_count_tag {};
struct bin_offset_tag {};
struct bin_binning_tag {};
struct bin_sort_bins_tag {};
public:
using size_type = SizeType;
using value_type = size_type;
using offset_type = Kokkos::View<size_type*, Space>;
using bin_count_type = Kokkos::View<const int*, Space>;
using const_key_view_type = typename KeyViewType::const_type;
// If a Kokkos::View then can generate constant random access
// otherwise can only use the constant type.
using const_rnd_key_view_type = std::conditional_t<
Kokkos::is_view<KeyViewType>::value,
Kokkos::View<typename KeyViewType::const_data_type,
typename KeyViewType::array_layout,
typename KeyViewType::device_type,
Kokkos::MemoryTraits<Kokkos::RandomAccess> >,
const_key_view_type>;
using non_const_key_scalar = typename KeyViewType::non_const_value_type;
using const_key_scalar = typename KeyViewType::const_value_type;
using bin_count_atomic_type =
Kokkos::View<int*, Space, Kokkos::MemoryTraits<Kokkos::Atomic> >;
private:
const_key_view_type keys;
const_rnd_key_view_type keys_rnd;
public:
BinSortOp bin_op;
offset_type bin_offsets;
bin_count_atomic_type bin_count_atomic;
bin_count_type bin_count_const;
offset_type sort_order;
int range_begin;
int range_end;
bool sort_within_bins;
public:
#ifdef KOKKOS_ENABLE_DEPRECATED_CODE_4
KOKKOS_DEPRECATED BinSort() = default;
#else
BinSort() = delete;
#endif
//----------------------------------------
// Constructor: takes the keys, the binning_operator and optionally whether to
// sort within bins (default false)
template <typename ExecutionSpace>
BinSort(const ExecutionSpace& exec, const_key_view_type keys_,
int range_begin_, int range_end_, BinSortOp bin_op_,
bool sort_within_bins_ = false)
: keys(keys_),
keys_rnd(keys_),
bin_op(bin_op_),
bin_offsets(),
bin_count_atomic(),
bin_count_const(),
sort_order(),
range_begin(range_begin_),
range_end(range_end_),
sort_within_bins(sort_within_bins_) {
static_assert(
Kokkos::SpaceAccessibility<ExecutionSpace,
typename Space::memory_space>::accessible,
"The provided execution space must be able to access the memory space "
"BinSort was initialized with!");
if (bin_op.max_bins() <= 0)
Kokkos::abort(
"The number of bins in the BinSortOp object must be greater than 0!");
bin_count_atomic = Kokkos::View<int*, Space>(
"Kokkos::SortImpl::BinSortFunctor::bin_count", bin_op.max_bins());
bin_count_const = bin_count_atomic;
bin_offsets =
offset_type(view_alloc(exec, WithoutInitializing,
"Kokkos::SortImpl::BinSortFunctor::bin_offsets"),
bin_op.max_bins());
sort_order =
offset_type(view_alloc(exec, WithoutInitializing,
"Kokkos::SortImpl::BinSortFunctor::sort_order"),
range_end - range_begin);
}
BinSort(const_key_view_type keys_, int range_begin_, int range_end_,
BinSortOp bin_op_, bool sort_within_bins_ = false)
: BinSort(exec_space{}, keys_, range_begin_, range_end_, bin_op_,
sort_within_bins_) {}
template <typename ExecutionSpace>
BinSort(const ExecutionSpace& exec, const_key_view_type keys_,
BinSortOp bin_op_, bool sort_within_bins_ = false)
: BinSort(exec, keys_, 0, keys_.extent(0), bin_op_, sort_within_bins_) {}
BinSort(const_key_view_type keys_, BinSortOp bin_op_,
bool sort_within_bins_ = false)
: BinSort(exec_space{}, keys_, bin_op_, sort_within_bins_) {}
//----------------------------------------
// Create the permutation vector, the bin_offset array and the bin_count
// array. Can be called again if keys changed
template <class ExecutionSpace>
void create_permute_vector(const ExecutionSpace& exec) {
static_assert(
Kokkos::SpaceAccessibility<ExecutionSpace,
typename Space::memory_space>::accessible,
"The provided execution space must be able to access the memory space "
"BinSort was initialized with!");
const size_t len = range_end - range_begin;
Kokkos::parallel_for(
"Kokkos::Sort::BinCount",
Kokkos::RangePolicy<ExecutionSpace, bin_count_tag>(exec, 0, len),
*this);
Kokkos::parallel_scan("Kokkos::Sort::BinOffset",
Kokkos::RangePolicy<ExecutionSpace, bin_offset_tag>(
exec, 0, bin_op.max_bins()),
*this);
Kokkos::deep_copy(exec, bin_count_atomic, 0);
Kokkos::parallel_for(
"Kokkos::Sort::BinBinning",
Kokkos::RangePolicy<ExecutionSpace, bin_binning_tag>(exec, 0, len),
*this);
if (sort_within_bins)
Kokkos::parallel_for(
"Kokkos::Sort::BinSort",
Kokkos::RangePolicy<ExecutionSpace, bin_sort_bins_tag>(
exec, 0, bin_op.max_bins()),
*this);
}
// Create the permutation vector, the bin_offset array and the bin_count
// array. Can be called again if keys changed
void create_permute_vector() {
Kokkos::fence("Kokkos::Binsort::create_permute_vector: before");
exec_space e{};
create_permute_vector(e);
e.fence("Kokkos::Binsort::create_permute_vector: after");
}
// Sort a subset of a view with respect to the first dimension using the
// permutation array
template <class ExecutionSpace, class ValuesViewType>
void sort(const ExecutionSpace& exec, ValuesViewType const& values,
int values_range_begin, int values_range_end) const {
if (values.extent(0) == 0) {
return;
}
static_assert(
Kokkos::SpaceAccessibility<ExecutionSpace,
typename Space::memory_space>::accessible,
"The provided execution space must be able to access the memory space "
"BinSort was initialized with!");
static_assert(
Kokkos::SpaceAccessibility<
ExecutionSpace, typename ValuesViewType::memory_space>::accessible,
"The provided execution space must be able to access the memory space "
"of the View argument!");
const size_t len = range_end - range_begin;
const size_t values_len = values_range_end - values_range_begin;
if (len != values_len) {
Kokkos::abort(
"BinSort::sort: values range length != permutation vector length");
}
using scratch_view_type =
Kokkos::View<typename ValuesViewType::data_type,
typename ValuesViewType::device_type>;
scratch_view_type sorted_values(
view_alloc(exec, WithoutInitializing,
"Kokkos::SortImpl::BinSortFunctor::sorted_values"),
values.rank_dynamic > 0 ? len : KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 1 ? values.extent(1)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 2 ? values.extent(2)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 3 ? values.extent(3)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 4 ? values.extent(4)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 5 ? values.extent(5)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 6 ? values.extent(6)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 7 ? values.extent(7)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG);
{
copy_permute_functor<scratch_view_type /* DstViewType */
,
offset_type /* PermuteViewType */
,
ValuesViewType /* SrcViewType */
>
functor(sorted_values, sort_order, values,
values_range_begin - range_begin);
parallel_for("Kokkos::Sort::CopyPermute",
Kokkos::RangePolicy<ExecutionSpace>(exec, 0, len), functor);
}
{
copy_functor<ValuesViewType, scratch_view_type> functor(
values, range_begin, sorted_values);
parallel_for("Kokkos::Sort::Copy",
Kokkos::RangePolicy<ExecutionSpace>(exec, 0, len), functor);
}
}
// Sort a subset of a view with respect to the first dimension using the
// permutation array
template <class ValuesViewType>
void sort(ValuesViewType const& values, int values_range_begin,
int values_range_end) const {
Kokkos::fence("Kokkos::Binsort::sort: before");
exec_space exec;
sort(exec, values, values_range_begin, values_range_end);
exec.fence("Kokkos::BinSort:sort: after");
}
template <class ExecutionSpace, class ValuesViewType>
void sort(ExecutionSpace const& exec, ValuesViewType const& values) const {
this->sort(exec, values, 0, /*values.extent(0)*/ range_end - range_begin);
}
template <class ValuesViewType>
void sort(ValuesViewType const& values) const {
this->sort(values, 0, /*values.extent(0)*/ range_end - range_begin);
}
// Get the permutation vector
KOKKOS_INLINE_FUNCTION
offset_type get_permute_vector() const { return sort_order; }
// Get the start offsets for each bin
KOKKOS_INLINE_FUNCTION
offset_type get_bin_offsets() const { return bin_offsets; }
// Get the count for each bin
KOKKOS_INLINE_FUNCTION
bin_count_type get_bin_count() const { return bin_count_const; }
public:
KOKKOS_INLINE_FUNCTION
void operator()(const bin_count_tag& /*tag*/, const int i) const {
const int j = range_begin + i;
bin_count_atomic(bin_op.bin(keys, j))++;
}
KOKKOS_INLINE_FUNCTION
void operator()(const bin_offset_tag& /*tag*/, const int i,
value_type& offset, const bool& final) const {
if (final) {
bin_offsets(i) = offset;
}
offset += bin_count_const(i);
}
KOKKOS_INLINE_FUNCTION
void operator()(const bin_binning_tag& /*tag*/, const int i) const {
const int j = range_begin + i;
const int bin = bin_op.bin(keys, j);
const int count = bin_count_atomic(bin)++;
sort_order(bin_offsets(bin) + count) = j;
}
KOKKOS_INLINE_FUNCTION
void operator()(const bin_sort_bins_tag& /*tag*/, const int i) const {
auto bin_size = bin_count_const(i);
if (bin_size <= 1) return;
constexpr bool use_std_sort =
std::is_same_v<typename exec_space::memory_space, HostSpace>;
int lower_bound = bin_offsets(i);
int upper_bound = lower_bound + bin_size;
// Switching to std::sort for more than 10 elements has been found
// reasonable experimentally.
if (use_std_sort && bin_size > 10) {
if constexpr (use_std_sort) {
std::sort(&sort_order(lower_bound), &sort_order(upper_bound),
[this](int p, int q) { return bin_op(keys_rnd, p, q); });
}
} else {
for (int k = lower_bound + 1; k < upper_bound; ++k) {
int old_idx = sort_order(k);
int j = k - 1;
while (j >= lower_bound) {
int new_idx = sort_order(j);
if (!bin_op(keys_rnd, old_idx, new_idx)) break;
sort_order(j + 1) = new_idx;
--j;
}
sort_order(j + 1) = old_idx;
}
}
}
};
//----------------------------------------------------------------------------
template <class KeyViewType>
struct BinOp1D {
int max_bins_ = {};
double mul_ = {};
double min_ = {};
#ifdef KOKKOS_ENABLE_DEPRECATED_CODE_4
KOKKOS_DEPRECATED BinOp1D() = default;
#else
BinOp1D() = delete;
#endif
// Construct BinOp with number of bins, minimum value and maximum value
BinOp1D(int max_bins__, typename KeyViewType::const_value_type min,
typename KeyViewType::const_value_type max)
: max_bins_(max_bins__ + 1),
// Cast to double to avoid possible overflow when using integer
mul_(static_cast<double>(max_bins__) /
(static_cast<double>(max) - static_cast<double>(min))),
min_(static_cast<double>(min)) {
// For integral types the number of bins may be larger than the range
// in which case we can exactly have one unique value per bin
// and then don't need to sort bins.
if (std::is_integral<typename KeyViewType::const_value_type>::value &&
(static_cast<double>(max) - static_cast<double>(min)) <=
static_cast<double>(max_bins__)) {
mul_ = 1.;
}
}
// Determine bin index from key value
template <class ViewType>
KOKKOS_INLINE_FUNCTION int bin(ViewType& keys, const int& i) const {
return static_cast<int>(mul_ * (static_cast<double>(keys(i)) - min_));
}
// Return maximum bin index + 1
KOKKOS_INLINE_FUNCTION
int max_bins() const { return max_bins_; }
// Compare to keys within a bin if true new_val will be put before old_val
template <class ViewType, typename iType1, typename iType2>
KOKKOS_INLINE_FUNCTION bool operator()(ViewType& keys, iType1& i1,
iType2& i2) const {
return keys(i1) < keys(i2);
}
};
template <class KeyViewType>
struct BinOp3D {
int max_bins_[3] = {};
double mul_[3] = {};
double min_[3] = {};
#ifdef KOKKOS_ENABLE_DEPRECATED_CODE_4
KOKKOS_DEPRECATED BinOp3D() = default;
#else
BinOp3D() = delete;
#endif
BinOp3D(int max_bins__[], typename KeyViewType::const_value_type min[],
typename KeyViewType::const_value_type max[]) {
max_bins_[0] = max_bins__[0];
max_bins_[1] = max_bins__[1];
max_bins_[2] = max_bins__[2];
mul_[0] = static_cast<double>(max_bins__[0]) /
(static_cast<double>(max[0]) - static_cast<double>(min[0]));
mul_[1] = static_cast<double>(max_bins__[1]) /
(static_cast<double>(max[1]) - static_cast<double>(min[1]));
mul_[2] = static_cast<double>(max_bins__[2]) /
(static_cast<double>(max[2]) - static_cast<double>(min[2]));
min_[0] = static_cast<double>(min[0]);
min_[1] = static_cast<double>(min[1]);
min_[2] = static_cast<double>(min[2]);
}
template <class ViewType>
KOKKOS_INLINE_FUNCTION int bin(ViewType& keys, const int& i) const {
return int((((int(mul_[0] * (keys(i, 0) - min_[0])) * max_bins_[1]) +
int(mul_[1] * (keys(i, 1) - min_[1]))) *
max_bins_[2]) +
int(mul_[2] * (keys(i, 2) - min_[2])));
}
KOKKOS_INLINE_FUNCTION
int max_bins() const { return max_bins_[0] * max_bins_[1] * max_bins_[2]; }
template <class ViewType, typename iType1, typename iType2>
KOKKOS_INLINE_FUNCTION bool operator()(ViewType& keys, iType1& i1,
iType2& i2) const {
if (keys(i1, 0) > keys(i2, 0))
return true;
else if (keys(i1, 0) == keys(i2, 0)) {
if (keys(i1, 1) > keys(i2, 1))
return true;
else if (keys(i1, 1) == keys(i2, 1)) {
if (keys(i1, 2) > keys(i2, 2)) return true;
}
}
return false;
}
};
namespace Impl {
template <class ViewType>
struct min_max_functor {
using minmax_scalar =
Kokkos::MinMaxScalar<typename ViewType::non_const_value_type>;
ViewType view;
min_max_functor(const ViewType& view_) : view(view_) {}
KOKKOS_INLINE_FUNCTION
void operator()(const size_t& i, minmax_scalar& minmax) const {
if (view(i) < minmax.min_val) minmax.min_val = view(i);
if (view(i) > minmax.max_val) minmax.max_val = view(i);
}
};
} // namespace Impl
template <class ExecutionSpace, class DataType, class... Properties>
std::enable_if_t<(Kokkos::is_execution_space<ExecutionSpace>::value) &&
(!SpaceAccessibility<
HostSpace, typename Kokkos::View<DataType, Properties...>::
memory_space>::accessible)>
sort(const ExecutionSpace& exec,
const Kokkos::View<DataType, Properties...>& view) {
if (view.extent(0) == 0) {
return;
}
using ViewType = Kokkos::View<DataType, Properties...>;
using CompType = BinOp1D<ViewType>;
Kokkos::MinMaxScalar<typename ViewType::non_const_value_type> result;
Kokkos::MinMax<typename ViewType::non_const_value_type> reducer(result);
parallel_reduce("Kokkos::Sort::FindExtent",
Kokkos::RangePolicy<typename ViewType::execution_space>(
exec, 0, view.extent(0)),
Impl::min_max_functor<ViewType>(view), reducer);
if (result.min_val == result.max_val) return;
// For integral types the number of bins may be larger than the range
// in which case we can exactly have one unique value per bin
// and then don't need to sort bins.
bool sort_in_bins = true;
// TODO: figure out better max_bins then this ...
int64_t max_bins = view.extent(0) / 2;
if (std::is_integral<typename ViewType::non_const_value_type>::value) {
// Cast to double to avoid possible overflow when using integer
auto const max_val = static_cast<double>(result.max_val);
auto const min_val = static_cast<double>(result.min_val);
// using 10M as the cutoff for special behavior (roughly 40MB for the count
// array)
if ((max_val - min_val) < 10000000) {
max_bins = max_val - min_val + 1;
sort_in_bins = false;
}
}
if (std::is_floating_point<typename ViewType::non_const_value_type>::value) {
KOKKOS_ASSERT(std::isfinite(static_cast<double>(result.max_val) -
static_cast<double>(result.min_val)));
}
BinSort<ViewType, CompType> bin_sort(
view, CompType(max_bins, result.min_val, result.max_val), sort_in_bins);
bin_sort.create_permute_vector(exec);
bin_sort.sort(exec, view);
}
#if defined(KOKKOS_ENABLE_ONEDPL)
template <class DataType, class... Properties>
void sort(const Experimental::SYCL& space,
const Kokkos::View<DataType, Properties...>& view) {
if (view.extent(0) == 0) {
return;
}
using ViewType = Kokkos::View<DataType, Properties...>;
static_assert(SpaceAccessibility<Experimental::SYCL,
typename ViewType::memory_space>::accessible,
"SYCL execution space is not able to access the memory space "
"of the View argument!");
auto queue = space.sycl_queue();
auto policy = oneapi::dpl::execution::make_device_policy(queue);
// Can't use Experimental::begin/end here since the oneDPL then assumes that
// the data is on the host.
static_assert(
ViewType::rank == 1 &&
(std::is_same<typename ViewType::array_layout, LayoutRight>::value ||
std::is_same<typename ViewType::array_layout, LayoutLeft>::value),
"SYCL sort only supports contiguous 1D Views.");
const int n = view.extent(0);
oneapi::dpl::sort(policy, view.data(), view.data() + n);
}
#endif
template <class ExecutionSpace, class DataType, class... Properties>
std::enable_if_t<(Kokkos::is_execution_space<ExecutionSpace>::value) &&
(SpaceAccessibility<
HostSpace, typename Kokkos::View<DataType, Properties...>::
memory_space>::accessible)>
sort(const ExecutionSpace&, const Kokkos::View<DataType, Properties...>& view) {
if (view.extent(0) == 0) {
return;
}
auto first = Experimental::begin(view);
auto last = Experimental::end(view);
std::sort(first, last);
}
#if defined(KOKKOS_ENABLE_CUDA)
template <class DataType, class... Properties>
void sort(const Cuda& space,
const Kokkos::View<DataType, Properties...>& view) {
if (view.extent(0) == 0) {
return;
}
const auto exec = thrust::cuda::par.on(space.cuda_stream());
auto first = Experimental::begin(view);
auto last = Experimental::end(view);
thrust::sort(exec, first, last);
}
#endif
template <class ViewType>
void sort(ViewType const& view) {
Kokkos::fence("Kokkos::sort: before");
if (view.extent(0) == 0) {
return;
}
typename ViewType::execution_space exec;
sort(exec, view);
exec.fence("Kokkos::sort: fence after sorting");
}
template <class ExecutionSpace, class ViewType>
std::enable_if_t<Kokkos::is_execution_space<ExecutionSpace>::value> sort(
const ExecutionSpace& exec, ViewType view, size_t const begin,
size_t const end) {
if (view.extent(0) == 0) {
return;
}
using range_policy = Kokkos::RangePolicy<typename ViewType::execution_space>;
using CompType = BinOp1D<ViewType>;
Kokkos::MinMaxScalar<typename ViewType::non_const_value_type> result;
Kokkos::MinMax<typename ViewType::non_const_value_type> reducer(result);
parallel_reduce("Kokkos::Sort::FindExtent", range_policy(exec, begin, end),
Impl::min_max_functor<ViewType>(view), reducer);
if (result.min_val == result.max_val) return;
BinSort<ViewType, CompType> bin_sort(
exec, view, begin, end,
CompType((end - begin) / 2, result.min_val, result.max_val), true);
bin_sort.create_permute_vector(exec);
bin_sort.sort(exec, view, begin, end);
}
template <class ViewType>
void sort(ViewType view, size_t const begin, size_t const end) {
Kokkos::fence("Kokkos::sort: before");
if (view.extent(0) == 0) {
return;
}
typename ViewType::execution_space exec;
sort(exec, view, begin, end);
exec.fence("Kokkos::Sort: fence after sorting");
}
} // namespace Kokkos
#ifdef KOKKOS_IMPL_PUBLIC_INCLUDE_NOTDEFINED_SORT #ifdef KOKKOS_IMPL_PUBLIC_INCLUDE_NOTDEFINED_SORT
#undef KOKKOS_IMPL_PUBLIC_INCLUDE #undef KOKKOS_IMPL_PUBLIC_INCLUDE

129
lib/kokkos/algorithms/src/sorting/Kokkos_BinOpsPublicAPI.hpp Normal file
View File

@ -0,0 +1,129 @@
//@HEADER
// ************************************************************************
//
// Kokkos v. 4.0
// Copyright (2022) National Technology & Engineering
// Solutions of Sandia, LLC (NTESS).
//
// Under the terms of Contract DE-NA0003525 with NTESS,
// the U.S. Government retains certain rights in this software.
//
// Part of Kokkos, under the Apache License v2.0 with LLVM Exceptions.
// See https://kokkos.org/LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//@HEADER
#ifndef KOKKOS_BIN_OPS_PUBLIC_API_HPP_
#define KOKKOS_BIN_OPS_PUBLIC_API_HPP_
#include <Kokkos_Macros.hpp>
#include <type_traits>
namespace Kokkos {
template <class KeyViewType>
struct BinOp1D {
int max_bins_ = {};
double mul_ = {};
double min_ = {};
#ifdef KOKKOS_ENABLE_DEPRECATED_CODE_4
KOKKOS_DEPRECATED BinOp1D() = default;
#else
BinOp1D() = delete;
#endif
// Construct BinOp with number of bins, minimum value and maximum value
BinOp1D(int max_bins__, typename KeyViewType::const_value_type min,
typename KeyViewType::const_value_type max)
: max_bins_(max_bins__ + 1),
// Cast to double to avoid possible overflow when using integer
mul_(static_cast<double>(max_bins__) /
(static_cast<double>(max) - static_cast<double>(min))),
min_(static_cast<double>(min)) {
// For integral types the number of bins may be larger than the range
// in which case we can exactly have one unique value per bin
// and then don't need to sort bins.
if (std::is_integral<typename KeyViewType::const_value_type>::value &&
(static_cast<double>(max) - static_cast<double>(min)) <=
static_cast<double>(max_bins__)) {
mul_ = 1.;
}
}
// Determine bin index from key value
template <class ViewType>
KOKKOS_INLINE_FUNCTION int bin(ViewType& keys, const int& i) const {
return static_cast<int>(mul_ * (static_cast<double>(keys(i)) - min_));
}
// Return maximum bin index + 1
KOKKOS_INLINE_FUNCTION
int max_bins() const { return max_bins_; }
// Compare to keys within a bin if true new_val will be put before old_val
template <class ViewType, typename iType1, typename iType2>
KOKKOS_INLINE_FUNCTION bool operator()(ViewType& keys, iType1& i1,
iType2& i2) const {
return keys(i1) < keys(i2);
}
};
template <class KeyViewType>
struct BinOp3D {
int max_bins_[3] = {};
double mul_[3] = {};
double min_[3] = {};
#ifdef KOKKOS_ENABLE_DEPRECATED_CODE_4
KOKKOS_DEPRECATED BinOp3D() = default;
#else
BinOp3D() = delete;
#endif
BinOp3D(int max_bins__[], typename KeyViewType::const_value_type min[],
typename KeyViewType::const_value_type max[]) {
max_bins_[0] = max_bins__[0];
max_bins_[1] = max_bins__[1];
max_bins_[2] = max_bins__[2];
mul_[0] = static_cast<double>(max_bins__[0]) /
(static_cast<double>(max[0]) - static_cast<double>(min[0]));
mul_[1] = static_cast<double>(max_bins__[1]) /
(static_cast<double>(max[1]) - static_cast<double>(min[1]));
mul_[2] = static_cast<double>(max_bins__[2]) /
(static_cast<double>(max[2]) - static_cast<double>(min[2]));
min_[0] = static_cast<double>(min[0]);
min_[1] = static_cast<double>(min[1]);
min_[2] = static_cast<double>(min[2]);
}
template <class ViewType>
KOKKOS_INLINE_FUNCTION int bin(ViewType& keys, const int& i) const {
return int((((int(mul_[0] * (keys(i, 0) - min_[0])) * max_bins_[1]) +
int(mul_[1] * (keys(i, 1) - min_[1]))) *
max_bins_[2]) +
int(mul_[2] * (keys(i, 2) - min_[2])));
}
KOKKOS_INLINE_FUNCTION
int max_bins() const { return max_bins_[0] * max_bins_[1] * max_bins_[2]; }
template <class ViewType, typename iType1, typename iType2>
KOKKOS_INLINE_FUNCTION bool operator()(ViewType& keys, iType1& i1,
iType2& i2) const {
if (keys(i1, 0) > keys(i2, 0))
return true;
else if (keys(i1, 0) == keys(i2, 0)) {
if (keys(i1, 1) > keys(i2, 1))
return true;
else if (keys(i1, 1) == keys(i2, 1)) {
if (keys(i1, 2) > keys(i2, 2)) return true;
}
}
return false;
}
};
} // namespace Kokkos
#endif

410
lib/kokkos/algorithms/src/sorting/Kokkos_BinSortPublicAPI.hpp Normal file
View File

@ -0,0 +1,410 @@
//@HEADER
// ************************************************************************
//
// Kokkos v. 4.0
// Copyright (2022) National Technology & Engineering
// Solutions of Sandia, LLC (NTESS).
//
// Under the terms of Contract DE-NA0003525 with NTESS,
// the U.S. Government retains certain rights in this software.
//
// Part of Kokkos, under the Apache License v2.0 with LLVM Exceptions.
// See https://kokkos.org/LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//@HEADER
#ifndef KOKKOS_BIN_SORT_PUBLIC_API_HPP_
#define KOKKOS_BIN_SORT_PUBLIC_API_HPP_
#include "Kokkos_BinOpsPublicAPI.hpp"
#include "impl/Kokkos_CopyOpsForBinSortImpl.hpp"
#include <Kokkos_Core.hpp>
#include <algorithm>
namespace Kokkos {
template <class KeyViewType, class BinSortOp,
class Space = typename KeyViewType::device_type,
class SizeType = typename KeyViewType::memory_space::size_type>
class BinSort {
public:
template <class DstViewType, class SrcViewType>
struct copy_functor {
using src_view_type = typename SrcViewType::const_type;
using copy_op = Impl::CopyOp<DstViewType, src_view_type>;
DstViewType dst_values;
src_view_type src_values;
int dst_offset;
copy_functor(DstViewType const& dst_values_, int const& dst_offset_,
SrcViewType const& src_values_)
: dst_values(dst_values_),
src_values(src_values_),
dst_offset(dst_offset_) {}
KOKKOS_INLINE_FUNCTION
void operator()(const int& i) const {
copy_op::copy(dst_values, i + dst_offset, src_values, i);
}
};
template <class DstViewType, class PermuteViewType, class SrcViewType>
struct copy_permute_functor {
// If a Kokkos::View then can generate constant random access
// otherwise can only use the constant type.
using src_view_type = std::conditional_t<
Kokkos::is_view<SrcViewType>::value,
Kokkos::View<typename SrcViewType::const_data_type,
typename SrcViewType::array_layout,
typename SrcViewType::device_type
#if !defined(KOKKOS_COMPILER_NVHPC) || (KOKKOS_COMPILER_NVHPC >= 230700)
,
Kokkos::MemoryTraits<Kokkos::RandomAccess>
#endif
>,
typename SrcViewType::const_type>;
using perm_view_type = typename PermuteViewType::const_type;
using copy_op = Impl::CopyOp<DstViewType, src_view_type>;
DstViewType dst_values;
perm_view_type sort_order;
src_view_type src_values;
int src_offset;
copy_permute_functor(DstViewType const& dst_values_,
PermuteViewType const& sort_order_,
SrcViewType const& src_values_, int const& src_offset_)
: dst_values(dst_values_),
sort_order(sort_order_),
src_values(src_values_),
src_offset(src_offset_) {}
KOKKOS_INLINE_FUNCTION
void operator()(const int& i) const {
copy_op::copy(dst_values, i, src_values, src_offset + sort_order(i));
}
};
// Naming this alias "execution_space" would be problematic since it would be
// considered as execution space for the various functors which might use
// another execution space through sort() or create_permute_vector().
using exec_space = typename Space::execution_space;
using bin_op_type = BinSortOp;
struct bin_count_tag {};
struct bin_offset_tag {};
struct bin_binning_tag {};
struct bin_sort_bins_tag {};
public:
using size_type = SizeType;
using value_type = size_type;
using offset_type = Kokkos::View<size_type*, Space>;
using bin_count_type = Kokkos::View<const int*, Space>;
using const_key_view_type = typename KeyViewType::const_type;
// If a Kokkos::View then can generate constant random access
// otherwise can only use the constant type.
using const_rnd_key_view_type = std::conditional_t<
Kokkos::is_view<KeyViewType>::value,
Kokkos::View<typename KeyViewType::const_data_type,
typename KeyViewType::array_layout,
typename KeyViewType::device_type,
Kokkos::MemoryTraits<Kokkos::RandomAccess> >,
const_key_view_type>;
using non_const_key_scalar = typename KeyViewType::non_const_value_type;
using const_key_scalar = typename KeyViewType::const_value_type;
using bin_count_atomic_type =
Kokkos::View<int*, Space, Kokkos::MemoryTraits<Kokkos::Atomic> >;
private:
const_key_view_type keys;
const_rnd_key_view_type keys_rnd;
public:
BinSortOp bin_op;
offset_type bin_offsets;
bin_count_atomic_type bin_count_atomic;
bin_count_type bin_count_const;
offset_type sort_order;
int range_begin;
int range_end;
bool sort_within_bins;
public:
#ifdef KOKKOS_ENABLE_DEPRECATED_CODE_4
KOKKOS_DEPRECATED BinSort() = default;
#else
BinSort() = delete;
#endif
//----------------------------------------
// Constructor: takes the keys, the binning_operator and optionally whether to
// sort within bins (default false)
template <typename ExecutionSpace>
BinSort(const ExecutionSpace& exec, const_key_view_type keys_,
int range_begin_, int range_end_, BinSortOp bin_op_,
bool sort_within_bins_ = false)
: keys(keys_),
keys_rnd(keys_),
bin_op(bin_op_),
bin_offsets(),
bin_count_atomic(),
bin_count_const(),
sort_order(),
range_begin(range_begin_),
range_end(range_end_),
sort_within_bins(sort_within_bins_) {
static_assert(
Kokkos::SpaceAccessibility<ExecutionSpace,
typename Space::memory_space>::accessible,
"The provided execution space must be able to access the memory space "
"BinSort was initialized with!");
if (bin_op.max_bins() <= 0)
Kokkos::abort(
"The number of bins in the BinSortOp object must be greater than 0!");
bin_count_atomic = Kokkos::View<int*, Space>(
"Kokkos::SortImpl::BinSortFunctor::bin_count", bin_op.max_bins());
bin_count_const = bin_count_atomic;
bin_offsets =
offset_type(view_alloc(exec, WithoutInitializing,
"Kokkos::SortImpl::BinSortFunctor::bin_offsets"),
bin_op.max_bins());
sort_order =
offset_type(view_alloc(exec, WithoutInitializing,
"Kokkos::SortImpl::BinSortFunctor::sort_order"),
range_end - range_begin);
}
BinSort(const_key_view_type keys_, int range_begin_, int range_end_,
BinSortOp bin_op_, bool sort_within_bins_ = false)
: BinSort(exec_space{}, keys_, range_begin_, range_end_, bin_op_,
sort_within_bins_) {}
template <typename ExecutionSpace>
BinSort(const ExecutionSpace& exec, const_key_view_type keys_,
BinSortOp bin_op_, bool sort_within_bins_ = false)
: BinSort(exec, keys_, 0, keys_.extent(0), bin_op_, sort_within_bins_) {}
BinSort(const_key_view_type keys_, BinSortOp bin_op_,
bool sort_within_bins_ = false)
: BinSort(exec_space{}, keys_, bin_op_, sort_within_bins_) {}
//----------------------------------------
// Create the permutation vector, the bin_offset array and the bin_count
// array. Can be called again if keys changed
template <class ExecutionSpace>
void create_permute_vector(const ExecutionSpace& exec) {
static_assert(
Kokkos::SpaceAccessibility<ExecutionSpace,
typename Space::memory_space>::accessible,
"The provided execution space must be able to access the memory space "
"BinSort was initialized with!");
const size_t len = range_end - range_begin;
Kokkos::parallel_for(
"Kokkos::Sort::BinCount",
Kokkos::RangePolicy<ExecutionSpace, bin_count_tag>(exec, 0, len),
*this);
Kokkos::parallel_scan("Kokkos::Sort::BinOffset",
Kokkos::RangePolicy<ExecutionSpace, bin_offset_tag>(
exec, 0, bin_op.max_bins()),
*this);
Kokkos::deep_copy(exec, bin_count_atomic, 0);
Kokkos::parallel_for(
"Kokkos::Sort::BinBinning",
Kokkos::RangePolicy<ExecutionSpace, bin_binning_tag>(exec, 0, len),
*this);
if (sort_within_bins)
Kokkos::parallel_for(
"Kokkos::Sort::BinSort",
Kokkos::RangePolicy<ExecutionSpace, bin_sort_bins_tag>(
exec, 0, bin_op.max_bins()),
*this);
}
// Create the permutation vector, the bin_offset array and the bin_count
// array. Can be called again if keys changed
void create_permute_vector() {
Kokkos::fence("Kokkos::Binsort::create_permute_vector: before");
exec_space e{};
create_permute_vector(e);
e.fence("Kokkos::Binsort::create_permute_vector: after");
}
// Sort a subset of a view with respect to the first dimension using the
// permutation array
template <class ExecutionSpace, class ValuesViewType>
void sort(const ExecutionSpace& exec, ValuesViewType const& values,
int values_range_begin, int values_range_end) const {
if (values.extent(0) == 0) {
return;
}
static_assert(
Kokkos::SpaceAccessibility<ExecutionSpace,
typename Space::memory_space>::accessible,
"The provided execution space must be able to access the memory space "
"BinSort was initialized with!");
static_assert(
Kokkos::SpaceAccessibility<
ExecutionSpace, typename ValuesViewType::memory_space>::accessible,
"The provided execution space must be able to access the memory space "
"of the View argument!");
const size_t len = range_end - range_begin;
const size_t values_len = values_range_end - values_range_begin;
if (len != values_len) {
Kokkos::abort(
"BinSort::sort: values range length != permutation vector length");
}
using scratch_view_type =
Kokkos::View<typename ValuesViewType::data_type,
typename ValuesViewType::device_type>;
scratch_view_type sorted_values(
view_alloc(exec, WithoutInitializing,
"Kokkos::SortImpl::BinSortFunctor::sorted_values"),
values.rank_dynamic > 0 ? len : KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 1 ? values.extent(1)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 2 ? values.extent(2)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 3 ? values.extent(3)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 4 ? values.extent(4)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 5 ? values.extent(5)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 6 ? values.extent(6)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG,
values.rank_dynamic > 7 ? values.extent(7)
: KOKKOS_IMPL_CTOR_DEFAULT_ARG);
{
copy_permute_functor<scratch_view_type /* DstViewType */
,
offset_type /* PermuteViewType */
,
ValuesViewType /* SrcViewType */
>
functor(sorted_values, sort_order, values,
values_range_begin - range_begin);
parallel_for("Kokkos::Sort::CopyPermute",
Kokkos::RangePolicy<ExecutionSpace>(exec, 0, len), functor);
}
{
copy_functor<ValuesViewType, scratch_view_type> functor(
values, range_begin, sorted_values);
parallel_for("Kokkos::Sort::Copy",
Kokkos::RangePolicy<ExecutionSpace>(exec, 0, len), functor);
}
}
// Sort a subset of a view with respect to the first dimension using the
// permutation array
template <class ValuesViewType>
void sort(ValuesViewType const& values, int values_range_begin,
int values_range_end) const {
Kokkos::fence("Kokkos::Binsort::sort: before");
exec_space exec;
sort(exec, values, values_range_begin, values_range_end);
exec.fence("Kokkos::BinSort:sort: after");
}
template <class ExecutionSpace, class ValuesViewType>
void sort(ExecutionSpace const& exec, ValuesViewType const& values) const {
this->sort(exec, values, 0, /*values.extent(0)*/ range_end - range_begin);
}
template <class ValuesViewType>
void sort(ValuesViewType const& values) const {
this->sort(values, 0, /*values.extent(0)*/ range_end - range_begin);
}
// Get the permutation vector
KOKKOS_INLINE_FUNCTION
offset_type get_permute_vector() const { return sort_order; }
// Get the start offsets for each bin
KOKKOS_INLINE_FUNCTION
offset_type get_bin_offsets() const { return bin_offsets; }
// Get the count for each bin
KOKKOS_INLINE_FUNCTION
bin_count_type get_bin_count() const { return bin_count_const; }
public:
KOKKOS_INLINE_FUNCTION
void operator()(const bin_count_tag& /*tag*/, const int i) const {
const int j = range_begin + i;
bin_count_atomic(bin_op.bin(keys, j))++;
}
KOKKOS_INLINE_FUNCTION
void operator()(const bin_offset_tag& /*tag*/, const int i,
value_type& offset, const bool& final) const {
if (final) {
bin_offsets(i) = offset;
}
offset += bin_count_const(i);
}
KOKKOS_INLINE_FUNCTION
void operator()(const bin_binning_tag& /*tag*/, const int i) const {
const int j = range_begin + i;
const int bin = bin_op.bin(keys, j);
const int count = bin_count_atomic(bin)++;
sort_order(bin_offsets(bin) + count) = j;
}
KOKKOS_INLINE_FUNCTION
void operator()(const bin_sort_bins_tag& /*tag*/, const int i) const {
auto bin_size = bin_count_const(i);
if (bin_size <= 1) return;
constexpr bool use_std_sort =
std::is_same_v<typename exec_space::memory_space, HostSpace>;
int lower_bound = bin_offsets(i);
int upper_bound = lower_bound + bin_size;
// Switching to std::sort for more than 10 elements has been found
// reasonable experimentally.
if (use_std_sort && bin_size > 10) {
KOKKOS_IF_ON_HOST(
(std::sort(&sort_order(lower_bound), &sort_order(upper_bound),
[this](int p, int q) { return bin_op(keys_rnd, p, q); });))
} else {
for (int k = lower_bound + 1; k < upper_bound; ++k) {
int old_idx = sort_order(k);
int j = k - 1;
while (j >= lower_bound) {
int new_idx = sort_order(j);
if (!bin_op(keys_rnd, old_idx, new_idx)) break;
sort_order(j + 1) = new_idx;
--j;
}
sort_order(j + 1) = old_idx;
}
}
}
};
} // namespace Kokkos
#endif

100
lib/kokkos/algorithms/src/sorting/Kokkos_NestedSortPublicAPI.hpp Normal file
View File

@ -0,0 +1,100 @@
//@HEADER
// ************************************************************************
//
// Kokkos v. 4.0
// Copyright (2022) National Technology & Engineering
// Solutions of Sandia, LLC (NTESS).
//
// Under the terms of Contract DE-NA0003525 with NTESS,
// the U.S. Government retains certain rights in this software.
//
// Part of Kokkos, under the Apache License v2.0 with LLVM Exceptions.
// See https://kokkos.org/LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//@HEADER
#ifndef KOKKOS_NESTED_SORT_PUBLIC_API_HPP_
#define KOKKOS_NESTED_SORT_PUBLIC_API_HPP_
#include "impl/Kokkos_NestedSortImpl.hpp"
#include <Kokkos_Core.hpp>
#include <std_algorithms/impl/Kokkos_HelperPredicates.hpp>
namespace Kokkos {
namespace Experimental {
template <class TeamMember, class ViewType>
KOKKOS_INLINE_FUNCTION void sort_team(const TeamMember& t,
const ViewType& view) {
Impl::sort_nested_impl(t, view, nullptr,
Experimental::Impl::StdAlgoLessThanBinaryPredicate<
typename ViewType::non_const_value_type>(),
Impl::NestedRange<true>());
}
template <class TeamMember, class ViewType, class Comparator>
KOKKOS_INLINE_FUNCTION void sort_team(const TeamMember& t, const ViewType& view,
const Comparator& comp) {
Impl::sort_nested_impl(t, view, nullptr, comp, Impl::NestedRange<true>());
}
template <class TeamMember, class KeyViewType, class ValueViewType>
KOKKOS_INLINE_FUNCTION void sort_by_key_team(const TeamMember& t,
const KeyViewType& keyView,
const ValueViewType& valueView) {
Impl::sort_nested_impl(t, keyView, valueView,
Experimental::Impl::StdAlgoLessThanBinaryPredicate<
typename KeyViewType::non_const_value_type>(),
Impl::NestedRange<true>());
}
template <class TeamMember, class KeyViewType, class ValueViewType,
class Comparator>
KOKKOS_INLINE_FUNCTION void sort_by_key_team(const TeamMember& t,
const KeyViewType& keyView,
const ValueViewType& valueView,
const Comparator& comp) {
Impl::sort_nested_impl(t, keyView, valueView, comp,
Impl::NestedRange<true>());
}
template <class TeamMember, class ViewType>
KOKKOS_INLINE_FUNCTION void sort_thread(const TeamMember& t,
const ViewType& view) {
Impl::sort_nested_impl(t, view, nullptr,
Experimental::Impl::StdAlgoLessThanBinaryPredicate<
typename ViewType::non_const_value_type>(),
Impl::NestedRange<false>());
}
template <class TeamMember, class ViewType, class Comparator>
KOKKOS_INLINE_FUNCTION void sort_thread(const TeamMember& t,
const ViewType& view,
const Comparator& comp) {
Impl::sort_nested_impl(t, view, nullptr, comp, Impl::NestedRange<false>());
}
template <class TeamMember, class KeyViewType, class ValueViewType>
KOKKOS_INLINE_FUNCTION void sort_by_key_thread(const TeamMember& t,
const KeyViewType& keyView,
const ValueViewType& valueView) {
Impl::sort_nested_impl(t, keyView, valueView,
Experimental::Impl::StdAlgoLessThanBinaryPredicate<
typename KeyViewType::non_const_value_type>(),
Impl::NestedRange<false>());
}
template <class TeamMember, class KeyViewType, class ValueViewType,
class Comparator>
KOKKOS_INLINE_FUNCTION void sort_by_key_thread(const TeamMember& t,
const KeyViewType& keyView,
const ValueViewType& valueView,
const Comparator& comp) {
Impl::sort_nested_impl(t, keyView, valueView, comp,
Impl::NestedRange<false>());
}
} // namespace Experimental
} // namespace Kokkos
#endif

194
lib/kokkos/algorithms/src/sorting/Kokkos_SortPublicAPI.hpp Normal file
View File

@ -0,0 +1,194 @@
//@HEADER
// ************************************************************************
//
// Kokkos v. 4.0
// Copyright (2022) National Technology & Engineering
// Solutions of Sandia, LLC (NTESS).
//
// Under the terms of Contract DE-NA0003525 with NTESS,
// the U.S. Government retains certain rights in this software.
//
// Part of Kokkos, under the Apache License v2.0 with LLVM Exceptions.
// See https://kokkos.org/LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//@HEADER
#ifndef KOKKOS_SORT_PUBLIC_API_HPP_
#define KOKKOS_SORT_PUBLIC_API_HPP_
#include "./impl/Kokkos_SortImpl.hpp"
#include <std_algorithms/Kokkos_BeginEnd.hpp>
#include <Kokkos_Core.hpp>
#include <algorithm>
namespace Kokkos {
// ---------------------------------------------------------------
// basic overloads
// ---------------------------------------------------------------
template <class ExecutionSpace, class DataType, class... Properties>
void sort([[maybe_unused]] const ExecutionSpace& exec,
const Kokkos::View<DataType, Properties...>& view) {
// constraints
using ViewType = Kokkos::View<DataType, Properties...>;
using MemSpace = typename ViewType::memory_space;
static_assert(
ViewType::rank == 1 &&
(std::is_same_v<typename ViewType::array_layout, LayoutRight> ||
std::is_same_v<typename ViewType::array_layout, LayoutLeft> ||
std::is_same_v<typename ViewType::array_layout, LayoutStride>),
"Kokkos::sort without comparator: supports 1D Views with LayoutRight, "
"LayoutLeft or LayoutStride.");
static_assert(SpaceAccessibility<ExecutionSpace, MemSpace>::accessible,
"Kokkos::sort: execution space instance is not able to access "
"the memory space of the "
"View argument!");
if (view.extent(0) <= 1) {
return;
}
if constexpr (Impl::better_off_calling_std_sort_v<ExecutionSpace>) {
auto first = ::Kokkos::Experimental::begin(view);
auto last = ::Kokkos::Experimental::end(view);
std::sort(first, last);
} else {
Impl::sort_device_view_without_comparator(exec, view);
}
}
template <class DataType, class... Properties>
void sort(const Kokkos::View<DataType, Properties...>& view) {
using ViewType = Kokkos::View<DataType, Properties...>;
static_assert(ViewType::rank == 1,
"Kokkos::sort: currently only supports rank-1 Views.");
Kokkos::fence("Kokkos::sort: before");
if (view.extent(0) <= 1) {
return;
}
typename ViewType::execution_space exec;
sort(exec, view);
exec.fence("Kokkos::sort: fence after sorting");
}
// ---------------------------------------------------------------
// overloads supporting a custom comparator
// ---------------------------------------------------------------
template <class ExecutionSpace, class ComparatorType, class DataType,
class... Properties>
void sort([[maybe_unused]] const ExecutionSpace& exec,
const Kokkos::View<DataType, Properties...>& view,
const ComparatorType& comparator) {
// constraints
using ViewType = Kokkos::View<DataType, Properties...>;
using MemSpace = typename ViewType::memory_space;
static_assert(
ViewType::rank == 1 &&
(std::is_same_v<typename ViewType::array_layout, LayoutRight> ||
std::is_same_v<typename ViewType::array_layout, LayoutLeft> ||
std::is_same_v<typename ViewType::array_layout, LayoutStride>),
"Kokkos::sort with comparator: supports 1D Views with LayoutRight, "
"LayoutLeft or LayoutStride.");
static_assert(SpaceAccessibility<ExecutionSpace, MemSpace>::accessible,
"Kokkos::sort: execution space instance is not able to access "
"the memory space of the View argument!");
if (view.extent(0) <= 1) {
return;
}
if constexpr (Impl::better_off_calling_std_sort_v<ExecutionSpace>) {
auto first = ::Kokkos::Experimental::begin(view);
auto last = ::Kokkos::Experimental::end(view);
std::sort(first, last, comparator);
} else {
Impl::sort_device_view_with_comparator(exec, view, comparator);
}
}
template <class ComparatorType, class DataType, class... Properties>
void sort(const Kokkos::View<DataType, Properties...>& view,
const ComparatorType& comparator) {
using ViewType = Kokkos::View<DataType, Properties...>;
static_assert(
ViewType::rank == 1 &&
(std::is_same_v<typename ViewType::array_layout, LayoutRight> ||
std::is_same_v<typename ViewType::array_layout, LayoutLeft> ||
std::is_same_v<typename ViewType::array_layout, LayoutStride>),
"Kokkos::sort with comparator: supports 1D Views with LayoutRight, "
"LayoutLeft or LayoutStride.");
Kokkos::fence("Kokkos::sort with comparator: before");
if (view.extent(0) <= 1) {
return;
}
typename ViewType::execution_space exec;
sort(exec, view, comparator);
exec.fence("Kokkos::sort with comparator: fence after sorting");
}
// ---------------------------------------------------------------
// overloads for sorting a view with a subrange
// specified via integers begin, end
// ---------------------------------------------------------------
template <class ExecutionSpace, class ViewType>
std::enable_if_t<Kokkos::is_execution_space<ExecutionSpace>::value> sort(
const ExecutionSpace& exec, ViewType view, size_t const begin,
size_t const end) {
// view must be rank-1 because the Impl::min_max_functor
// used below only works for rank-1 views for now
static_assert(ViewType::rank == 1,
"Kokkos::sort: currently only supports rank-1 Views.");
if (view.extent(0) <= 1) {
return;
}
using range_policy = Kokkos::RangePolicy<typename ViewType::execution_space>;
using CompType = BinOp1D<ViewType>;
Kokkos::MinMaxScalar<typename ViewType::non_const_value_type> result;
Kokkos::MinMax<typename ViewType::non_const_value_type> reducer(result);
parallel_reduce("Kokkos::Sort::FindExtent", range_policy(exec, begin, end),
Impl::min_max_functor<ViewType>(view), reducer);
if (result.min_val == result.max_val) return;
BinSort<ViewType, CompType> bin_sort(
exec, view, begin, end,
CompType((end - begin) / 2, result.min_val, result.max_val), true);
bin_sort.create_permute_vector(exec);
bin_sort.sort(exec, view, begin, end);
}
template <class ViewType>
void sort(ViewType view, size_t const begin, size_t const end) {
// same constraints as the overload above which this gets dispatched to
static_assert(ViewType::rank == 1,
"Kokkos::sort: currently only supports rank-1 Views.");
Kokkos::fence("Kokkos::sort: before");
if (view.extent(0) <= 1) {
return;
}
typename ViewType::execution_space exec;
sort(exec, view, begin, end);
exec.fence("Kokkos::Sort: fence after sorting");
}
} // namespace Kokkos
#endif

61
lib/kokkos/algorithms/src/sorting/impl/Kokkos_CopyOpsForBinSortImpl.hpp Normal file
View File

@ -0,0 +1,61 @@
//@HEADER
// ************************************************************************
//
// Kokkos v. 4.0
// Copyright (2022) National Technology & Engineering
// Solutions of Sandia, LLC (NTESS).
//
// Under the terms of Contract DE-NA0003525 with NTESS,
// the U.S. Government retains certain rights in this software.
//
// Part of Kokkos, under the Apache License v2.0 with LLVM Exceptions.
// See https://kokkos.org/LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//@HEADER
#ifndef KOKKOS_COPY_OPS_FOR_BINSORT_IMPL_HPP_
#define KOKKOS_COPY_OPS_FOR_BINSORT_IMPL_HPP_
#include <Kokkos_Macros.hpp>
#include <cstddef>
namespace Kokkos {
namespace Impl {
template <class DstViewType, class SrcViewType, int Rank = DstViewType::rank>
struct CopyOp;
template <class DstViewType, class SrcViewType>
struct CopyOp<DstViewType, SrcViewType, 1> {
KOKKOS_INLINE_FUNCTION
static void copy(DstViewType const& dst, size_t i_dst, SrcViewType const& src,
size_t i_src) {
dst(i_dst) = src(i_src);
}
};
template <class DstViewType, class SrcViewType>
struct CopyOp<DstViewType, SrcViewType, 2> {
KOKKOS_INLINE_FUNCTION
static void copy(DstViewType const& dst, size_t i_dst, SrcViewType const& src,
size_t i_src) {
for (int j = 0; j < (int)dst.extent(1); j++) dst(i_dst, j) = src(i_src, j);
}
};
template <class DstViewType, class SrcViewType>
struct CopyOp<DstViewType, SrcViewType, 3> {
KOKKOS_INLINE_FUNCTION
static void copy(DstViewType const& dst, size_t i_dst, SrcViewType const& src,
size_t i_src) {
for (int j = 0; j < dst.extent(1); j++)
for (int k = 0; k < dst.extent(2); k++)
dst(i_dst, j, k) = src(i_src, j, k);
}
};
} // namespace Impl
} // namespace Kokkos
#endif

115
lib/kokkos/algorithms/src/sorting/impl/Kokkos_NestedSortImpl.hpp Normal file
View File

@ -0,0 +1,115 @@
//@HEADER
// ************************************************************************
//
// Kokkos v. 4.0
// Copyright (2022) National Technology & Engineering
// Solutions of Sandia, LLC (NTESS).
//
// Under the terms of Contract DE-NA0003525 with NTESS,
// the U.S. Government retains certain rights in this software.
//
// Part of Kokkos, under the Apache License v2.0 with LLVM Exceptions.
// See https://kokkos.org/LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//@HEADER
#ifndef KOKKOS_NESTED_SORT_IMPL_HPP_
#define KOKKOS_NESTED_SORT_IMPL_HPP_
#include <Kokkos_Core.hpp>
#include <std_algorithms/Kokkos_Swap.hpp>
namespace Kokkos {
namespace Experimental {
namespace Impl {
// true for TeamVectorRange, false for ThreadVectorRange
template <bool teamLevel>
struct NestedRange {};
// Specialization for team-level
template <>
struct NestedRange<true> {
template <typename TeamMember, typename SizeType>
KOKKOS_FUNCTION static auto create(const TeamMember& t, SizeType len) {
return Kokkos::TeamVectorRange(t, len);
}
template <typename TeamMember>
KOKKOS_FUNCTION static void barrier(const TeamMember& t) {
t.team_barrier();
}
};
// Specialization for thread-level
template <>
struct NestedRange<false> {
template <typename TeamMember, typename SizeType>
KOKKOS_FUNCTION static auto create(const TeamMember& t, SizeType len) {
return Kokkos::ThreadVectorRange(t, len);
}
// Barrier is no-op, as vector lanes of a thread are implicitly synchronized
// after parallel region
template <typename TeamMember>
KOKKOS_FUNCTION static void barrier(const TeamMember&) {}
};
// When just doing sort (not sort_by_key), use nullptr_t for ValueViewType.
// This only takes the NestedRange instance for template arg deduction.
template <class TeamMember, class KeyViewType, class ValueViewType,
class Comparator, bool useTeamLevel>
KOKKOS_INLINE_FUNCTION void sort_nested_impl(
const TeamMember& t, const KeyViewType& keyView,
[[maybe_unused]] const ValueViewType& valueView, const Comparator& comp,
const NestedRange<useTeamLevel>) {
using SizeType = typename KeyViewType::size_type;
using KeyType = typename KeyViewType::non_const_value_type;
using Range = NestedRange<useTeamLevel>;
SizeType n = keyView.extent(0);
SizeType npot = 1;
SizeType levels = 0;
// FIXME: ceiling power-of-two is a common thing to need - make it a utility
while (npot < n) {
levels++;
npot <<= 1;
}
for (SizeType i = 0; i < levels; i++) {
for (SizeType j = 0; j <= i; j++) {
// n/2 pairs of items are compared in parallel
Kokkos::parallel_for(Range::create(t, npot / 2), [=](const SizeType k) {
// How big are the brown/pink boxes?
// (Terminology comes from Wikipedia diagram)
// https://commons.wikimedia.org/wiki/File:BitonicSort.svg#/media/File:BitonicSort.svg
SizeType boxSize = SizeType(2) << (i - j);
// Which box contains this thread?
SizeType boxID = k >> (i - j); // k * 2 / boxSize;
SizeType boxStart = boxID << (1 + i - j); // boxID * boxSize
SizeType boxOffset = k - (boxStart >> 1); // k - boxID * boxSize / 2;
SizeType elem1 = boxStart + boxOffset;
// In first phase (j == 0, brown box): within a box, compare with the
// opposite value in the box.
// In later phases (j > 0, pink box): within a box, compare with fixed
// distance (boxSize / 2) apart.
SizeType elem2 = (j == 0) ? (boxStart + boxSize - 1 - boxOffset)
: (elem1 + boxSize / 2);
if (elem2 < n) {
KeyType key1 = keyView(elem1);
KeyType key2 = keyView(elem2);
if (comp(key2, key1)) {
keyView(elem1) = key2;
keyView(elem2) = key1;
if constexpr (!std::is_same_v<ValueViewType, std::nullptr_t>) {
Kokkos::Experimental::swap(valueView(elem1), valueView(elem2));
}
}
}
});
Range::barrier(t);
}
}
}
} // namespace Impl
} // namespace Experimental
} // namespace Kokkos
#endif

369
lib/kokkos/algorithms/src/sorting/impl/Kokkos_SortImpl.hpp Normal file
View File

@ -0,0 +1,369 @@
//@HEADER
// ************************************************************************
//
// Kokkos v. 4.0
// Copyright (2022) National Technology & Engineering
// Solutions of Sandia, LLC (NTESS).
//
// Under the terms of Contract DE-NA0003525 with NTESS,
// the U.S. Government retains certain rights in this software.
//
// Part of Kokkos, under the Apache License v2.0 with LLVM Exceptions.
// See https://kokkos.org/LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//@HEADER
#ifndef KOKKOS_SORT_FREE_FUNCS_IMPL_HPP_
#define KOKKOS_SORT_FREE_FUNCS_IMPL_HPP_
#include "../Kokkos_BinOpsPublicAPI.hpp"
#include "../Kokkos_BinSortPublicAPI.hpp"
#include <std_algorithms/Kokkos_BeginEnd.hpp>
#include <std_algorithms/Kokkos_Copy.hpp>
#include <Kokkos_Core.hpp>
#if defined(KOKKOS_ENABLE_CUDA)
// Workaround for `Instruction 'shfl' without '.sync' is not supported on
// .target sm_70 and higher from PTX ISA version 6.4`.
// Also see https://github.com/NVIDIA/cub/pull/170.
#if !defined(CUB_USE_COOPERATIVE_GROUPS)
#define CUB_USE_COOPERATIVE_GROUPS
#endif
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wshadow"
#if defined(KOKKOS_COMPILER_CLANG)
// Some versions of Clang fail to compile Thrust, failing with errors like
// this:
// <snip>/thrust/system/cuda/detail/core/agent_launcher.h:557:11:
// error: use of undeclared identifier 'va_printf'
// The exact combination of versions for Clang and Thrust (or CUDA) for this
// failure was not investigated, however even very recent version combination
// (Clang 10.0.0 and Cuda 10.0) demonstrated failure.
//
// Defining _CubLog here locally allows us to avoid that code path, however
// disabling some debugging diagnostics
#pragma push_macro("_CubLog")
#ifdef _CubLog
#undef _CubLog
#endif
#define _CubLog
#include <thrust/device_ptr.h>
#include <thrust/sort.h>
#pragma pop_macro("_CubLog")
#else
#include <thrust/device_ptr.h>
#include <thrust/sort.h>
#endif
#pragma GCC diagnostic pop
#endif
#if defined(KOKKOS_ENABLE_ONEDPL)
#include <oneapi/dpl/execution>
#include <oneapi/dpl/algorithm>
#endif
namespace Kokkos {
namespace Impl {
template <class ExecutionSpace>
struct better_off_calling_std_sort : std::false_type {};
#if defined KOKKOS_ENABLE_SERIAL
template <>
struct better_off_calling_std_sort<Kokkos::Serial> : std::true_type {};
#endif
#if defined KOKKOS_ENABLE_OPENMP
template <>
struct better_off_calling_std_sort<Kokkos::OpenMP> : std::true_type {};
#endif
#if defined KOKKOS_ENABLE_THREADS
template <>
struct better_off_calling_std_sort<Kokkos::Threads> : std::true_type {};
#endif
#if defined KOKKOS_ENABLE_HPX
template <>
struct better_off_calling_std_sort<Kokkos::Experimental::HPX> : std::true_type {
};
#endif
template <class T>
inline constexpr bool better_off_calling_std_sort_v =
better_off_calling_std_sort<T>::value;
template <class ViewType>
struct min_max_functor {
using minmax_scalar =
Kokkos::MinMaxScalar<typename ViewType::non_const_value_type>;
ViewType view;
min_max_functor(const ViewType& view_) : view(view_) {}
KOKKOS_INLINE_FUNCTION
void operator()(const size_t& i, minmax_scalar& minmax) const {
if (view(i) < minmax.min_val) minmax.min_val = view(i);
if (view(i) > minmax.max_val) minmax.max_val = view(i);
}
};
template <class ExecutionSpace, class DataType, class... Properties>
void sort_via_binsort(const ExecutionSpace& exec,
const Kokkos::View<DataType, Properties...>& view) {
// Although we are using BinSort below, which could work on rank-2 views,
// for now view must be rank-1 because the min_max_functor
// used below only works for rank-1 views
using ViewType = Kokkos::View<DataType, Properties...>;
static_assert(ViewType::rank == 1,
"Kokkos::sort: currently only supports rank-1 Views.");
if (view.extent(0) <= 1) {
return;
}
Kokkos::MinMaxScalar<typename ViewType::non_const_value_type> result;
Kokkos::MinMax<typename ViewType::non_const_value_type> reducer(result);
parallel_reduce("Kokkos::Sort::FindExtent",
Kokkos::RangePolicy<typename ViewType::execution_space>(
exec, 0, view.extent(0)),
min_max_functor<ViewType>(view), reducer);
if (result.min_val == result.max_val) return;
// For integral types the number of bins may be larger than the range
// in which case we can exactly have one unique value per bin
// and then don't need to sort bins.
bool sort_in_bins = true;
// TODO: figure out better max_bins then this ...
int64_t max_bins = view.extent(0) / 2;
if (std::is_integral<typename ViewType::non_const_value_type>::value) {
// Cast to double to avoid possible overflow when using integer
auto const max_val = static_cast<double>(result.max_val);
auto const min_val = static_cast<double>(result.min_val);
// using 10M as the cutoff for special behavior (roughly 40MB for the count
// array)
if ((max_val - min_val) < 10000000) {
max_bins = max_val - min_val + 1;
sort_in_bins = false;
}
}
if (std::is_floating_point<typename ViewType::non_const_value_type>::value) {
KOKKOS_ASSERT(std::isfinite(static_cast<double>(result.max_val) -
static_cast<double>(result.min_val)));
}
using CompType = BinOp1D<ViewType>;
BinSort<ViewType, CompType> bin_sort(
view, CompType(max_bins, result.min_val, result.max_val), sort_in_bins);
bin_sort.create_permute_vector(exec);
bin_sort.sort(exec, view);
}
#if defined(KOKKOS_ENABLE_CUDA)
template <class DataType, class... Properties, class... MaybeComparator>
void sort_cudathrust(const Cuda& space,
const Kokkos::View<DataType, Properties...>& view,
MaybeComparator&&... maybeComparator) {
using ViewType = Kokkos::View<DataType, Properties...>;
static_assert(ViewType::rank == 1,
"Kokkos::sort: currently only supports rank-1 Views.");
if (view.extent(0) <= 1) {
return;
}
const auto exec = thrust::cuda::par.on(space.cuda_stream());
auto first = ::Kokkos::Experimental::begin(view);
auto last = ::Kokkos::Experimental::end(view);
thrust::sort(exec, first, last,
std::forward<MaybeComparator>(maybeComparator)...);
}
#endif
#if defined(KOKKOS_ENABLE_ONEDPL)
template <class DataType, class... Properties, class... MaybeComparator>
void sort_onedpl(const Kokkos::Experimental::SYCL& space,
const Kokkos::View<DataType, Properties...>& view,
MaybeComparator&&... maybeComparator) {
using ViewType = Kokkos::View<DataType, Properties...>;
static_assert(SpaceAccessibility<Kokkos::Experimental::SYCL,
typename ViewType::memory_space>::accessible,
"SYCL execution space is not able to access the memory space "
"of the View argument!");
static_assert(
(ViewType::rank == 1) &&
(std::is_same_v<typename ViewType::array_layout, LayoutRight> ||
std::is_same_v<typename ViewType::array_layout, LayoutLeft> ||
std::is_same_v<typename ViewType::array_layout, LayoutStride>),
"SYCL sort only supports contiguous rank-1 Views with LayoutLeft, "
"LayoutRight or LayoutStride"
"For the latter, this means the View must have stride(0) = 1, enforced "
"at runtime.");
if (view.stride(0) != 1) {
Kokkos::abort("SYCL sort only supports rank-1 Views with stride(0) = 1.");
}
if (view.extent(0) <= 1) {
return;
}
// Can't use Experimental::begin/end here since the oneDPL then assumes that
// the data is on the host.
auto queue = space.sycl_queue();
auto policy = oneapi::dpl::execution::make_device_policy(queue);
const int n = view.extent(0);
oneapi::dpl::sort(policy, view.data(), view.data() + n,
std::forward<MaybeComparator>(maybeComparator)...);
}
#endif
template <class ExecutionSpace, class DataType, class... Properties,
class... MaybeComparator>
void copy_to_host_run_stdsort_copy_back(
const ExecutionSpace& exec,
const Kokkos::View<DataType, Properties...>& view,
MaybeComparator&&... maybeComparator) {
namespace KE = ::Kokkos::Experimental;
using ViewType = Kokkos::View<DataType, Properties...>;
using layout = typename ViewType::array_layout;
if constexpr (std::is_same_v<LayoutStride, layout>) {
// for strided views we cannot just deep_copy from device to host,
// so we need to do a few more jumps
using view_value_type = typename ViewType::non_const_value_type;
using view_exespace = typename ViewType::execution_space;
using view_deep_copyable_t = Kokkos::View<view_value_type*, view_exespace>;
view_deep_copyable_t view_dc("view_dc", view.extent(0));
KE::copy(exec, view, view_dc);
// run sort on the mirror of view_dc
auto mv_h = create_mirror_view_and_copy(Kokkos::HostSpace(), view_dc);
auto first = KE::begin(mv_h);
auto last = KE::end(mv_h);
std::sort(first, last, std::forward<MaybeComparator>(maybeComparator)...);
Kokkos::deep_copy(exec, view_dc, mv_h);
// copy back to argument view
KE::copy(exec, KE::cbegin(view_dc), KE::cend(view_dc), KE::begin(view));
} else {
auto view_h = create_mirror_view_and_copy(Kokkos::HostSpace(), view);
auto first = KE::begin(view_h);
auto last = KE::end(view_h);
std::sort(first, last, std::forward<MaybeComparator>(maybeComparator)...);
Kokkos::deep_copy(exec, view, view_h);
}
}
// --------------------------------------------------
//
// specialize cases for sorting without comparator
//
// --------------------------------------------------
#if defined(KOKKOS_ENABLE_CUDA)
template <class DataType, class... Properties>
void sort_device_view_without_comparator(
const Cuda& exec, const Kokkos::View<DataType, Properties...>& view) {
sort_cudathrust(exec, view);
}
#endif
#if defined(KOKKOS_ENABLE_ONEDPL)
template <class DataType, class... Properties>
void sort_device_view_without_comparator(
const Kokkos::Experimental::SYCL& exec,
const Kokkos::View<DataType, Properties...>& view) {
using ViewType = Kokkos::View<DataType, Properties...>;
static_assert(
(ViewType::rank == 1) &&
(std::is_same_v<typename ViewType::array_layout, LayoutRight> ||
std::is_same_v<typename ViewType::array_layout, LayoutLeft> ||
std::is_same_v<typename ViewType::array_layout, LayoutStride>),
"sort_device_view_without_comparator: supports rank-1 Views "
"with LayoutLeft, LayoutRight or LayoutStride");
if (view.stride(0) == 1) {
sort_onedpl(exec, view);
} else {
copy_to_host_run_stdsort_copy_back(exec, view);
}
}
#endif
// fallback case
template <class ExecutionSpace, class DataType, class... Properties>
std::enable_if_t<Kokkos::is_execution_space<ExecutionSpace>::value>
sort_device_view_without_comparator(
const ExecutionSpace& exec,
const Kokkos::View<DataType, Properties...>& view) {
sort_via_binsort(exec, view);
}
// --------------------------------------------------
//
// specialize cases for sorting with comparator
//
// --------------------------------------------------
#if defined(KOKKOS_ENABLE_CUDA)
template <class ComparatorType, class DataType, class... Properties>
void sort_device_view_with_comparator(
const Cuda& exec, const Kokkos::View<DataType, Properties...>& view,
const ComparatorType& comparator) {
sort_cudathrust(exec, view, comparator);
}
#endif
#if defined(KOKKOS_ENABLE_ONEDPL)
template <class ComparatorType, class DataType, class... Properties>
void sort_device_view_with_comparator(
const Kokkos::Experimental::SYCL& exec,
const Kokkos::View<DataType, Properties...>& view,
const ComparatorType& comparator) {
using ViewType = Kokkos::View<DataType, Properties...>;
static_assert(
(ViewType::rank == 1) &&
(std::is_same_v<typename ViewType::array_layout, LayoutRight> ||
std::is_same_v<typename ViewType::array_layout, LayoutLeft> ||
std::is_same_v<typename ViewType::array_layout, LayoutStride>),
"sort_device_view_with_comparator: supports rank-1 Views "
"with LayoutLeft, LayoutRight or LayoutStride");
if (view.stride(0) == 1) {
sort_onedpl(exec, view, comparator);
} else {
copy_to_host_run_stdsort_copy_back(exec, view, comparator);
}
}
#endif
template <class ExecutionSpace, class ComparatorType, class DataType,
class... Properties>
std::enable_if_t<Kokkos::is_execution_space<ExecutionSpace>::value>
sort_device_view_with_comparator(
const ExecutionSpace& exec,
const Kokkos::View<DataType, Properties...>& view,
const ComparatorType& comparator) {
// This is a fallback case if a more specialized overload does not exist:
// for now, this fallback copies data to host, runs std::sort
// and then copies data back. Potentially, this can later be changed
// with a better solution like our own quicksort on device or similar.
using ViewType = Kokkos::View<DataType, Properties...>;
using MemSpace = typename ViewType::memory_space;
static_assert(!SpaceAccessibility<HostSpace, MemSpace>::accessible,
"Impl::sort_device_view_with_comparator: should not be called "
"on a view that is already accessible on the host");
copy_to_host_run_stdsort_copy_back(exec, view, comparator);
}
} // namespace Impl
} // namespace Kokkos
#endif

203
lib/kokkos/algorithms/src/std_algorithms/Kokkos_AdjacentDifference.hpp
View File

@ -23,64 +23,85 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIteratorType, //
class OutputIteratorType> // overload set accepting execution space
std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value, //
OutputIteratorType> template <
adjacent_difference(const ExecutionSpace& ex, InputIteratorType first_from, typename ExecutionSpace, typename InputIteratorType,
InputIteratorType last_from, typename OutputIteratorType,
OutputIteratorType first_dest) { std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value &&
::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
OutputIteratorType adjacent_difference(const ExecutionSpace& ex,
InputIteratorType first_from,
InputIteratorType last_from,
OutputIteratorType first_dest) {
using value_type1 = typename InputIteratorType::value_type; using value_type1 = typename InputIteratorType::value_type;
using value_type2 = typename OutputIteratorType::value_type; using value_type2 = typename OutputIteratorType::value_type;
using binary_op = using binary_op =
Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1, Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1,
value_type2>; value_type2>;
return Impl::adjacent_difference_impl( return Impl::adjacent_difference_exespace_impl(
"Kokkos::adjacent_difference_iterator_api", ex, first_from, last_from, "Kokkos::adjacent_difference_iterator_api", ex, first_from, last_from,
first_dest, binary_op()); first_dest, binary_op());
} }
template <class ExecutionSpace, class InputIteratorType, template <
class OutputIteratorType, class BinaryOp> typename ExecutionSpace, typename InputIteratorType,
std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value, typename OutputIteratorType, typename BinaryOp,
OutputIteratorType> std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value &&
adjacent_difference(const ExecutionSpace& ex, InputIteratorType first_from, ::Kokkos::is_execution_space<ExecutionSpace>::value,
InputIteratorType last_from, OutputIteratorType first_dest, int> = 0>
BinaryOp bin_op) { OutputIteratorType adjacent_difference(const ExecutionSpace& ex,
return Impl::adjacent_difference_impl( InputIteratorType first_from,
InputIteratorType last_from,
OutputIteratorType first_dest,
BinaryOp bin_op) {
return Impl::adjacent_difference_exespace_impl(
"Kokkos::adjacent_difference_iterator_api", ex, first_from, last_from, "Kokkos::adjacent_difference_iterator_api", ex, first_from, last_from,
first_dest, bin_op); first_dest, bin_op);
} }
template <class ExecutionSpace, class InputIteratorType, template <
class OutputIteratorType> typename ExecutionSpace, typename InputIteratorType,
std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value, typename OutputIteratorType,
OutputIteratorType> std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value &&
adjacent_difference(const std::string& label, const ExecutionSpace& ex, ::Kokkos::is_execution_space<ExecutionSpace>::value,
InputIteratorType first_from, InputIteratorType last_from, int> = 0>
OutputIteratorType first_dest) { OutputIteratorType adjacent_difference(const std::string& label,
const ExecutionSpace& ex,
InputIteratorType first_from,
InputIteratorType last_from,
OutputIteratorType first_dest) {
using value_type1 = typename InputIteratorType::value_type; using value_type1 = typename InputIteratorType::value_type;
using value_type2 = typename OutputIteratorType::value_type; using value_type2 = typename OutputIteratorType::value_type;
using binary_op = using binary_op =
Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1, Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1,
value_type2>; value_type2>;
return Impl::adjacent_difference_impl(label, ex, first_from, last_from, return Impl::adjacent_difference_exespace_impl(
first_dest, binary_op()); label, ex, first_from, last_from, first_dest, binary_op());
} }
template <class ExecutionSpace, class InputIteratorType, template <
class OutputIteratorType, class BinaryOp> typename ExecutionSpace, typename InputIteratorType,
std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value, typename OutputIteratorType, typename BinaryOp,
OutputIteratorType> std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value &&
adjacent_difference(const std::string& label, const ExecutionSpace& ex, ::Kokkos::is_execution_space<ExecutionSpace>::value,
InputIteratorType first_from, InputIteratorType last_from, int> = 0>
OutputIteratorType first_dest, BinaryOp bin_op) { OutputIteratorType adjacent_difference(const std::string& label,
return Impl::adjacent_difference_impl(label, ex, first_from, last_from, const ExecutionSpace& ex,
first_dest, bin_op); InputIteratorType first_from,
InputIteratorType last_from,
OutputIteratorType first_dest,
BinaryOp bin_op) {
return Impl::adjacent_difference_exespace_impl(label, ex, first_from,
last_from, first_dest, bin_op);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2> typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto adjacent_difference( auto adjacent_difference(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -96,13 +117,15 @@ auto adjacent_difference(
using binary_op = using binary_op =
Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1, Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1,
value_type2>; value_type2>;
return Impl::adjacent_difference_impl( return Impl::adjacent_difference_exespace_impl(
"Kokkos::adjacent_difference_view_api", ex, KE::cbegin(view_from), "Kokkos::adjacent_difference_view_api", ex, KE::cbegin(view_from),
KE::cend(view_from), KE::begin(view_dest), binary_op()); KE::cend(view_from), KE::begin(view_dest), binary_op());
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class BinaryOp> typename DataType2, typename... Properties2, typename BinaryOp,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto adjacent_difference( auto adjacent_difference(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -111,13 +134,15 @@ auto adjacent_difference(
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
return Impl::adjacent_difference_impl( return Impl::adjacent_difference_exespace_impl(
"Kokkos::adjacent_difference_view_api", ex, KE::cbegin(view_from), "Kokkos::adjacent_difference_view_api", ex, KE::cbegin(view_from),
KE::cend(view_from), KE::begin(view_dest), bin_op); KE::cend(view_from), KE::begin(view_dest), bin_op);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2> typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto adjacent_difference( auto adjacent_difference(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -134,13 +159,15 @@ auto adjacent_difference(
Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1, Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1,
value_type2>; value_type2>;
return Impl::adjacent_difference_impl(label, ex, KE::cbegin(view_from), return Impl::adjacent_difference_exespace_impl(
KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), binary_op()); KE::begin(view_dest), binary_op());
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class BinaryOp> typename DataType2, typename... Properties2, typename BinaryOp,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto adjacent_difference( auto adjacent_difference(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -149,9 +176,85 @@ auto adjacent_difference(
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
return Impl::adjacent_difference_impl(label, ex, KE::cbegin(view_from), return Impl::adjacent_difference_exespace_impl(
KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), bin_op); KE::begin(view_dest), bin_op);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType,
std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value &&
::Kokkos::is_team_handle<TeamHandleType>::value,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType adjacent_difference(
const TeamHandleType& teamHandle, InputIteratorType first_from,
InputIteratorType last_from, OutputIteratorType first_dest) {
using value_type1 = typename InputIteratorType::value_type;
using value_type2 = typename OutputIteratorType::value_type;
using binary_op =
Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1,
value_type2>;
return Impl::adjacent_difference_team_impl(teamHandle, first_from, last_from,
first_dest, binary_op());
}
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType, typename BinaryOp,
std::enable_if_t<!::Kokkos::is_view<InputIteratorType>::value &&
::Kokkos::is_team_handle<TeamHandleType>::value,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType
adjacent_difference(const TeamHandleType& teamHandle,
InputIteratorType first_from, InputIteratorType last_from,
OutputIteratorType first_dest, BinaryOp bin_op) {
return Impl::adjacent_difference_team_impl(teamHandle, first_from, last_from,
first_dest, bin_op);
}
template <
typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION auto adjacent_difference(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest) {
namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
using view_type1 = ::Kokkos::View<DataType1, Properties1...>;
using view_type2 = ::Kokkos::View<DataType2, Properties2...>;
using value_type1 = typename view_type1::value_type;
using value_type2 = typename view_type2::value_type;
using binary_op =
Impl::StdAdjacentDifferenceDefaultBinaryOpFunctor<value_type1,
value_type2>;
return Impl::adjacent_difference_team_impl(teamHandle, KE::cbegin(view_from),
KE::cend(view_from),
KE::begin(view_dest), binary_op());
}
template <
typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryOp,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION auto adjacent_difference(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
BinaryOp bin_op) {
namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
return Impl::adjacent_difference_team_impl(teamHandle, KE::cbegin(view_from),
KE::cend(view_from),
KE::begin(view_dest), bin_op);
} }
} // namespace Experimental } // namespace Experimental

117
lib/kokkos/algorithms/src/std_algorithms/Kokkos_AdjacentFind.hpp
View File

@ -23,71 +23,144 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// overload set1 // overload set1
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType adjacent_find(const ExecutionSpace& ex, IteratorType first, IteratorType adjacent_find(const ExecutionSpace& ex, IteratorType first,
IteratorType last) { IteratorType last) {
return Impl::adjacent_find_impl("Kokkos::adjacent_find_iterator_api_default", return Impl::adjacent_find_exespace_impl(
ex, first, last); "Kokkos::adjacent_find_iterator_api_default", ex, first, last);
} }
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType adjacent_find(const std::string& label, const ExecutionSpace& ex, IteratorType adjacent_find(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last) { IteratorType first, IteratorType last) {
return Impl::adjacent_find_impl(label, ex, first, last); return Impl::adjacent_find_exespace_impl(label, ex, first, last);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto adjacent_find(const ExecutionSpace& ex, auto adjacent_find(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v) { const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::adjacent_find_impl("Kokkos::adjacent_find_view_api_default", ex, return Impl::adjacent_find_exespace_impl(
KE::begin(v), KE::end(v)); "Kokkos::adjacent_find_view_api_default", ex, KE::begin(v), KE::end(v));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto adjacent_find(const std::string& label, const ExecutionSpace& ex, auto adjacent_find(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v) { const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::adjacent_find_impl(label, ex, KE::begin(v), KE::end(v)); return Impl::adjacent_find_exespace_impl(label, ex, KE::begin(v), KE::end(v));
} }
// overload set2 // overload set2
template <class ExecutionSpace, class IteratorType, class BinaryPredicateType> template <
typename ExecutionSpace, typename IteratorType,
typename BinaryPredicateType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType adjacent_find(const ExecutionSpace& ex, IteratorType first, IteratorType adjacent_find(const ExecutionSpace& ex, IteratorType first,
IteratorType last, BinaryPredicateType pred) { IteratorType last, BinaryPredicateType pred) {
return Impl::adjacent_find_impl("Kokkos::adjacent_find_iterator_api_default", return Impl::adjacent_find_exespace_impl(
ex, first, last, pred); "Kokkos::adjacent_find_iterator_api_default", ex, first, last, pred);
} }
template <class ExecutionSpace, class IteratorType, class BinaryPredicateType> template <
typename ExecutionSpace, typename IteratorType,
typename BinaryPredicateType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType adjacent_find(const std::string& label, const ExecutionSpace& ex, IteratorType adjacent_find(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
BinaryPredicateType pred) { BinaryPredicateType pred) {
return Impl::adjacent_find_impl(label, ex, first, last, pred); return Impl::adjacent_find_exespace_impl(label, ex, first, last, pred);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class BinaryPredicateType> typename ExecutionSpace, typename DataType, typename... Properties,
typename BinaryPredicateType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto adjacent_find(const ExecutionSpace& ex, auto adjacent_find(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
BinaryPredicateType pred) { BinaryPredicateType pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::adjacent_find_impl("Kokkos::adjacent_find_view_api_default", ex, return Impl::adjacent_find_exespace_impl(
KE::begin(v), KE::end(v), pred); "Kokkos::adjacent_find_view_api_default", ex, KE::begin(v), KE::end(v),
pred);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class BinaryPredicateType> typename ExecutionSpace, typename DataType, typename... Properties,
typename BinaryPredicateType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto adjacent_find(const std::string& label, const ExecutionSpace& ex, auto adjacent_find(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
BinaryPredicateType pred) { BinaryPredicateType pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::adjacent_find_impl(label, ex, KE::begin(v), KE::end(v), pred); return Impl::adjacent_find_exespace_impl(label, ex, KE::begin(v), KE::end(v),
pred);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
// overload set1
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType adjacent_find(const TeamHandleType& teamHandle,
IteratorType first,
IteratorType last) {
return Impl::adjacent_find_team_impl(teamHandle, first, last);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto adjacent_find(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::adjacent_find_team_impl(teamHandle, KE::begin(v), KE::end(v));
}
// overload set2
template <typename TeamHandleType, typename IteratorType,
typename BinaryPredicateType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType adjacent_find(const TeamHandleType& teamHandle,
IteratorType first,
IteratorType last,
BinaryPredicateType pred) {
return Impl::adjacent_find_team_impl(teamHandle, first, last, pred);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename BinaryPredicateType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto adjacent_find(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v,
BinaryPredicateType pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::adjacent_find_team_impl(teamHandle, KE::begin(v), KE::end(v),
pred);
} }
} // namespace Experimental } // namespace Experimental

64
lib/kokkos/algorithms/src/std_algorithms/Kokkos_AllOf.hpp
View File

@ -23,41 +23,79 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class Predicate> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool all_of(const ExecutionSpace& ex, InputIterator first, InputIterator last, bool all_of(const ExecutionSpace& ex, InputIterator first, InputIterator last,
Predicate predicate) { Predicate predicate) {
return Impl::all_of_impl("Kokkos::all_of_iterator_api_default", ex, first, return Impl::all_of_exespace_impl("Kokkos::all_of_iterator_api_default", ex,
last, predicate); first, last, predicate);
} }
template <class ExecutionSpace, class InputIterator, class Predicate> template <
typename ExecutionSpace, typename InputIterator, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool all_of(const std::string& label, const ExecutionSpace& ex, bool all_of(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, Predicate predicate) { InputIterator first, InputIterator last, Predicate predicate) {
return Impl::all_of_impl(label, ex, first, last, predicate); return Impl::all_of_exespace_impl(label, ex, first, last, predicate);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class Predicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool all_of(const ExecutionSpace& ex, bool all_of(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::all_of_impl("Kokkos::all_of_view_api_default", ex, KE::cbegin(v), return Impl::all_of_exespace_impl("Kokkos::all_of_view_api_default", ex,
KE::cend(v), std::move(predicate)); KE::cbegin(v), KE::cend(v),
std::move(predicate));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class Predicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool all_of(const std::string& label, const ExecutionSpace& ex, bool all_of(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::all_of_impl(label, ex, KE::cbegin(v), KE::cend(v), return Impl::all_of_exespace_impl(label, ex, KE::cbegin(v), KE::cend(v),
std::move(predicate)); std::move(predicate));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator, typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool all_of(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last,
Predicate predicate) {
return Impl::all_of_team_impl(teamHandle, first, last, predicate);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool all_of(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::all_of_team_impl(teamHandle, KE::cbegin(v), KE::cend(v),
std::move(predicate));
} }
} // namespace Experimental } // namespace Experimental

64
lib/kokkos/algorithms/src/std_algorithms/Kokkos_AnyOf.hpp
View File

@ -23,41 +23,79 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class Predicate> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool any_of(const ExecutionSpace& ex, InputIterator first, InputIterator last, bool any_of(const ExecutionSpace& ex, InputIterator first, InputIterator last,
Predicate predicate) { Predicate predicate) {
return Impl::any_of_impl("Kokkos::any_of_view_api_default", ex, first, last, return Impl::any_of_exespace_impl("Kokkos::any_of_view_api_default", ex,
predicate); first, last, predicate);
} }
template <class ExecutionSpace, class InputIterator, class Predicate> template <
typename ExecutionSpace, typename InputIterator, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool any_of(const std::string& label, const ExecutionSpace& ex, bool any_of(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, Predicate predicate) { InputIterator first, InputIterator last, Predicate predicate) {
return Impl::any_of_impl(label, ex, first, last, predicate); return Impl::any_of_exespace_impl(label, ex, first, last, predicate);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class Predicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool any_of(const ExecutionSpace& ex, bool any_of(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::any_of_impl("Kokkos::any_of_view_api_default", ex, KE::cbegin(v), return Impl::any_of_exespace_impl("Kokkos::any_of_view_api_default", ex,
KE::cend(v), std::move(predicate)); KE::cbegin(v), KE::cend(v),
std::move(predicate));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class Predicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool any_of(const std::string& label, const ExecutionSpace& ex, bool any_of(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::any_of_impl(label, ex, KE::cbegin(v), KE::cend(v), return Impl::any_of_exespace_impl(label, ex, KE::cbegin(v), KE::cend(v),
std::move(predicate)); std::move(predicate));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator, typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool any_of(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last,
Predicate predicate) {
return Impl::any_of_team_impl(teamHandle, first, last, predicate);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool any_of(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::any_of_team_impl(teamHandle, KE::cbegin(v), KE::cend(v),
std::move(predicate));
} }
} // namespace Experimental } // namespace Experimental

65
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Copy.hpp
View File

@ -23,22 +23,31 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator copy(const ExecutionSpace& ex, InputIterator first, OutputIterator copy(const ExecutionSpace& ex, InputIterator first,
InputIterator last, OutputIterator d_first) { InputIterator last, OutputIterator d_first) {
return Impl::copy_impl("Kokkos::copy_iterator_api_default", ex, first, last, return Impl::copy_exespace_impl("Kokkos::copy_iterator_api_default", ex,
d_first); first, last, d_first);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator> template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator copy(const std::string& label, const ExecutionSpace& ex, OutputIterator copy(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, InputIterator first, InputIterator last,
OutputIterator d_first) { OutputIterator d_first) {
return Impl::copy_impl(label, ex, first, last, d_first); return Impl::copy_exespace_impl(label, ex, first, last, d_first);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto copy(const ExecutionSpace& ex, auto copy(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
@ -46,12 +55,15 @@ auto copy(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::copy_impl("Kokkos::copy_view_api_default", ex, return Impl::copy_exespace_impl("Kokkos::copy_view_api_default", ex,
KE::cbegin(source), KE::cend(source), KE::begin(dest)); KE::cbegin(source), KE::cend(source),
KE::begin(dest));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto copy(const std::string& label, const ExecutionSpace& ex, auto copy(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
@ -59,8 +71,35 @@ auto copy(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::copy_impl(label, ex, KE::cbegin(source), KE::cend(source), return Impl::copy_exespace_impl(label, ex, KE::cbegin(source),
KE::begin(dest)); KE::cend(source), KE::begin(dest));
}
//
// overload set accepting team handle
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator copy(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last,
OutputIterator d_first) {
return Impl::copy_team_impl(teamHandle, first, last, d_first);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto copy(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
namespace KE = ::Kokkos::Experimental;
return Impl::copy_team_impl(teamHandle, KE::cbegin(source), KE::cend(source),
KE::begin(dest));
} }
} // namespace Experimental } // namespace Experimental

65
lib/kokkos/algorithms/src/std_algorithms/Kokkos_CopyBackward.hpp
View File

@ -23,42 +23,81 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType1, class IteratorType2> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType2 copy_backward(const ExecutionSpace& ex, IteratorType1 first, IteratorType2 copy_backward(const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 d_last) { IteratorType1 last, IteratorType2 d_last) {
return Impl::copy_backward_impl("Kokkos::copy_backward_iterator_api_default", return Impl::copy_backward_exespace_impl(
ex, first, last, d_last); "Kokkos::copy_backward_iterator_api_default", ex, first, last, d_last);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType2 copy_backward(const std::string& label, const ExecutionSpace& ex, IteratorType2 copy_backward(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first, IteratorType1 last, IteratorType1 first, IteratorType1 last,
IteratorType2 d_last) { IteratorType2 d_last) {
return Impl::copy_backward_impl(label, ex, first, last, d_last); return Impl::copy_backward_exespace_impl(label, ex, first, last, d_last);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto copy_backward(const ExecutionSpace& ex, auto copy_backward(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::copy_backward_impl("Kokkos::copy_backward_view_api_default", ex, return Impl::copy_backward_exespace_impl(
cbegin(source), cend(source), end(dest)); "Kokkos::copy_backward_view_api_default", ex, cbegin(source),
cend(source), end(dest));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto copy_backward(const std::string& label, const ExecutionSpace& ex, auto copy_backward(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::copy_backward_impl(label, ex, cbegin(source), cend(source), return Impl::copy_backward_exespace_impl(label, ex, cbegin(source),
end(dest)); cend(source), end(dest));
}
//
// overload set accepting team handle
//
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType2 copy_backward(const TeamHandleType& teamHandle,
IteratorType1 first,
IteratorType1 last,
IteratorType2 d_last) {
return Impl::copy_backward_team_impl(teamHandle, first, last, d_last);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto copy_backward(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::copy_backward_team_impl(teamHandle, cbegin(source), cend(source),
end(dest));
} }
} // namespace Experimental } // namespace Experimental

71
lib/kokkos/algorithms/src/std_algorithms/Kokkos_CopyIf.hpp
View File

@ -23,46 +23,85 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator, //
class Predicate> // overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator copy_if(const ExecutionSpace& ex, InputIterator first, OutputIterator copy_if(const ExecutionSpace& ex, InputIterator first,
InputIterator last, OutputIterator d_first, InputIterator last, OutputIterator d_first,
Predicate pred) { Predicate pred) {
return Impl::copy_if_impl("Kokkos::copy_if_iterator_api_default", ex, first, return Impl::copy_if_exespace_impl("Kokkos::copy_if_iterator_api_default", ex,
last, d_first, std::move(pred)); first, last, d_first, std::move(pred));
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator, template <
class Predicate> typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator copy_if(const std::string& label, const ExecutionSpace& ex, OutputIterator copy_if(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, InputIterator first, InputIterator last,
OutputIterator d_first, Predicate pred) { OutputIterator d_first, Predicate pred) {
return Impl::copy_if_impl(label, ex, first, last, d_first, std::move(pred)); return Impl::copy_if_exespace_impl(label, ex, first, last, d_first,
std::move(pred));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class Predicate> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto copy_if(const ExecutionSpace& ex, auto copy_if(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest, Predicate pred) { ::Kokkos::View<DataType2, Properties2...>& dest, Predicate pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::copy_if_impl("Kokkos::copy_if_view_api_default", ex, return Impl::copy_if_exespace_impl("Kokkos::copy_if_view_api_default", ex,
cbegin(source), cend(source), begin(dest), cbegin(source), cend(source), begin(dest),
std::move(pred)); std::move(pred));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class Predicate> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto copy_if(const std::string& label, const ExecutionSpace& ex, auto copy_if(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest, Predicate pred) { ::Kokkos::View<DataType2, Properties2...>& dest, Predicate pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::copy_if_impl(label, ex, cbegin(source), cend(source), return Impl::copy_if_exespace_impl(label, ex, cbegin(source), cend(source),
begin(dest), std::move(pred)); begin(dest), std::move(pred));
}
//
// overload set accepting team handle
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator, typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator copy_if(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last,
OutputIterator d_first, Predicate pred) {
return Impl::copy_if_team_impl(teamHandle, first, last, d_first,
std::move(pred));
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto copy_if(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest, Predicate pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::copy_if_team_impl(teamHandle, cbegin(source), cend(source),
begin(dest), std::move(pred));
} }
} // namespace Experimental } // namespace Experimental

68
lib/kokkos/algorithms/src/std_algorithms/Kokkos_CopyN.hpp
View File

@ -23,23 +23,32 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class Size, //
class OutputIterator> // overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename Size,
typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator copy_n(const ExecutionSpace& ex, InputIterator first, Size count, OutputIterator copy_n(const ExecutionSpace& ex, InputIterator first, Size count,
OutputIterator result) { OutputIterator result) {
return Impl::copy_n_impl("Kokkos::copy_n_iterator_api_default", ex, first, return Impl::copy_n_exespace_impl("Kokkos::copy_n_iterator_api_default", ex,
count, result); first, count, result);
} }
template <class ExecutionSpace, class InputIterator, class Size, template <
class OutputIterator> typename ExecutionSpace, typename InputIterator, typename Size,
typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator copy_n(const std::string& label, const ExecutionSpace& ex, OutputIterator copy_n(const std::string& label, const ExecutionSpace& ex,
InputIterator first, Size count, OutputIterator result) { InputIterator first, Size count, OutputIterator result) {
return Impl::copy_n_impl(label, ex, first, count, result); return Impl::copy_n_exespace_impl(label, ex, first, count, result);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class Size, class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename Size, typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto copy_n(const ExecutionSpace& ex, auto copy_n(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, Size count, const ::Kokkos::View<DataType1, Properties1...>& source, Size count,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
@ -47,12 +56,14 @@ auto copy_n(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::copy_n_impl("Kokkos::copy_n_view_api_default", ex, return Impl::copy_n_exespace_impl("Kokkos::copy_n_view_api_default", ex,
KE::cbegin(source), count, KE::begin(dest)); KE::cbegin(source), count, KE::begin(dest));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class Size, class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename Size, typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto copy_n(const std::string& label, const ExecutionSpace& ex, auto copy_n(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, Size count, const ::Kokkos::View<DataType1, Properties1...>& source, Size count,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
@ -60,8 +71,35 @@ auto copy_n(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::copy_n_impl(label, ex, KE::cbegin(source), count, return Impl::copy_n_exespace_impl(label, ex, KE::cbegin(source), count,
KE::begin(dest)); KE::begin(dest));
}
//
// overload set accepting team handle
//
template <typename TeamHandleType, typename InputIterator, typename Size,
typename OutputIterator,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator copy_n(const TeamHandleType& teamHandle,
InputIterator first, Size count,
OutputIterator result) {
return Impl::copy_n_team_impl(teamHandle, first, count, result);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename Size, typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto copy_n(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source, Size count,
::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
namespace KE = ::Kokkos::Experimental;
return Impl::copy_n_team_impl(teamHandle, KE::cbegin(source), count,
KE::begin(dest));
} }
} // namespace Experimental } // namespace Experimental

60
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Count.hpp
View File

@ -23,41 +23,81 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class T> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType, typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
typename IteratorType::difference_type count(const ExecutionSpace& ex, typename IteratorType::difference_type count(const ExecutionSpace& ex,
IteratorType first, IteratorType first,
IteratorType last, IteratorType last,
const T& value) { const T& value) {
return Impl::count_impl("Kokkos::count_iterator_api_default", ex, first, last, return Impl::count_exespace_impl("Kokkos::count_iterator_api_default", ex,
value); first, last, value);
} }
template <class ExecutionSpace, class IteratorType, class T> template <
typename ExecutionSpace, typename IteratorType, typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
typename IteratorType::difference_type count(const std::string& label, typename IteratorType::difference_type count(const std::string& label,
const ExecutionSpace& ex, const ExecutionSpace& ex,
IteratorType first, IteratorType first,
IteratorType last, IteratorType last,
const T& value) { const T& value) {
return Impl::count_impl(label, ex, first, last, value); return Impl::count_exespace_impl(label, ex, first, last, value);
} }
template <class ExecutionSpace, class DataType, class... Properties, class T> template <
typename ExecutionSpace, typename DataType, typename... Properties,
typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto count(const ExecutionSpace& ex, auto count(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const T& value) { const ::Kokkos::View<DataType, Properties...>& v, const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::count_impl("Kokkos::count_view_api_default", ex, KE::cbegin(v), return Impl::count_exespace_impl("Kokkos::count_view_api_default", ex,
KE::cend(v), value); KE::cbegin(v), KE::cend(v), value);
} }
template <class ExecutionSpace, class DataType, class... Properties, class T> template <
typename ExecutionSpace, typename DataType, typename... Properties,
typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto count(const std::string& label, const ExecutionSpace& ex, auto count(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const T& value) { const ::Kokkos::View<DataType, Properties...>& v, const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::count_impl(label, ex, KE::cbegin(v), KE::cend(v), value); return Impl::count_exespace_impl(label, ex, KE::cbegin(v), KE::cend(v),
value);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType, typename T,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION typename IteratorType::difference_type count(
const TeamHandleType& teamHandle, IteratorType first, IteratorType last,
const T& value) {
return Impl::count_team_impl(teamHandle, first, last, value);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename T,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto count(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v,
const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::count_team_impl(teamHandle, KE::cbegin(v), KE::cend(v), value);
} }
} // namespace Experimental } // namespace Experimental

64
lib/kokkos/algorithms/src/std_algorithms/Kokkos_CountIf.hpp
View File

@ -23,46 +23,84 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class Predicate> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
typename IteratorType::difference_type count_if(const ExecutionSpace& ex, typename IteratorType::difference_type count_if(const ExecutionSpace& ex,
IteratorType first, IteratorType first,
IteratorType last, IteratorType last,
Predicate predicate) { Predicate predicate) {
return Impl::count_if_impl("Kokkos::count_if_iterator_api_default", ex, first, return Impl::count_if_exespace_impl("Kokkos::count_if_iterator_api_default",
last, std::move(predicate)); ex, first, last, std::move(predicate));
} }
template <class ExecutionSpace, class IteratorType, class Predicate> template <
typename ExecutionSpace, typename IteratorType, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
typename IteratorType::difference_type count_if(const std::string& label, typename IteratorType::difference_type count_if(const std::string& label,
const ExecutionSpace& ex, const ExecutionSpace& ex,
IteratorType first, IteratorType first,
IteratorType last, IteratorType last,
Predicate predicate) { Predicate predicate) {
return Impl::count_if_impl(label, ex, first, last, std::move(predicate)); return Impl::count_if_exespace_impl(label, ex, first, last,
std::move(predicate));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class Predicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto count_if(const ExecutionSpace& ex, auto count_if(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::count_if_impl("Kokkos::count_if_view_api_default", ex, return Impl::count_if_exespace_impl("Kokkos::count_if_view_api_default", ex,
KE::cbegin(v), KE::cend(v), std::move(predicate)); KE::cbegin(v), KE::cend(v),
std::move(predicate));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class Predicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto count_if(const std::string& label, const ExecutionSpace& ex, auto count_if(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::count_if_impl(label, ex, KE::cbegin(v), KE::cend(v), return Impl::count_if_exespace_impl(label, ex, KE::cbegin(v), KE::cend(v),
std::move(predicate)); std::move(predicate));
}
//
// overload set accepting team handle
//
template <typename TeamHandleType, typename IteratorType, typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION typename IteratorType::difference_type count_if(
const TeamHandleType& teamHandle, IteratorType first, IteratorType last,
Predicate predicate) {
return Impl::count_if_team_impl(teamHandle, first, last,
std::move(predicate));
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto count_if(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::count_if_team_impl(teamHandle, KE::cbegin(v), KE::cend(v),
std::move(predicate));
} }
} // namespace Experimental } // namespace Experimental

289
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Equal.hpp
View File

@ -23,50 +23,61 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType1, class IteratorType2> //
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< // overload set accepting execution space
IteratorType1, IteratorType2>::value, //
bool> template <typename ExecutionSpace, typename IteratorType1,
equal(const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1, typename IteratorType2,
IteratorType2 first2) { std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
return Impl::equal_impl("Kokkos::equal_iterator_api_default", ex, first1, IteratorType1, IteratorType2> &&
last1, first2); Kokkos::is_execution_space_v<ExecutionSpace>,
int> = 0>
bool equal(const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2) {
return Impl::equal_exespace_impl("Kokkos::equal_iterator_api_default", ex,
first1, last1, first2);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <typename ExecutionSpace, typename IteratorType1,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< typename IteratorType2,
IteratorType1, IteratorType2>::value, std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
bool> IteratorType1, IteratorType2>&& ::Kokkos::
equal(const std::string& label, const ExecutionSpace& ex, IteratorType1 first1, is_execution_space_v<ExecutionSpace>,
IteratorType1 last1, IteratorType2 first2) { int> = 0>
return Impl::equal_impl(label, ex, first1, last1, first2); bool equal(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType2 first2) {
return Impl::equal_exespace_impl(label, ex, first1, last1, first2);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <typename ExecutionSpace, typename IteratorType1,
class BinaryPredicateType> typename IteratorType2, typename BinaryPredicateType,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
IteratorType1, IteratorType2>::value, IteratorType1, IteratorType2>&& ::Kokkos::
bool> is_execution_space_v<ExecutionSpace>,
equal(const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1, int> = 0>
IteratorType2 first2, BinaryPredicateType predicate) { bool equal(const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1,
return Impl::equal_impl("Kokkos::equal_iterator_api_default", ex, first1, IteratorType2 first2, BinaryPredicateType predicate) {
last1, first2, std::move(predicate)); return Impl::equal_exespace_impl("Kokkos::equal_iterator_api_default", ex,
first1, last1, first2, std::move(predicate));
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <typename ExecutionSpace, typename IteratorType1,
class BinaryPredicateType> typename IteratorType2, typename BinaryPredicateType,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
IteratorType1, IteratorType2>::value, IteratorType1, IteratorType2>&& ::Kokkos::
bool> is_execution_space_v<ExecutionSpace>,
equal(const std::string& label, const ExecutionSpace& ex, IteratorType1 first1, int> = 0>
IteratorType1 last1, IteratorType2 first2, bool equal(const std::string& label, const ExecutionSpace& ex,
BinaryPredicateType predicate) { IteratorType1 first1, IteratorType1 last1, IteratorType2 first2,
return Impl::equal_impl(label, ex, first1, last1, first2, BinaryPredicateType predicate) {
std::move(predicate)); return Impl::equal_exespace_impl(label, ex, first1, last1, first2,
std::move(predicate));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool equal(const ExecutionSpace& ex, bool equal(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
::Kokkos::View<DataType2, Properties2...>& view2) { ::Kokkos::View<DataType2, Properties2...>& view2) {
@ -74,13 +85,15 @@ bool equal(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::equal_impl("Kokkos::equal_view_api_default", ex, return Impl::equal_exespace_impl("Kokkos::equal_view_api_default", ex,
KE::cbegin(view1), KE::cend(view1), KE::cbegin(view1), KE::cend(view1),
KE::cbegin(view2)); KE::cbegin(view2));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool equal(const std::string& label, const ExecutionSpace& ex, bool equal(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
::Kokkos::View<DataType2, Properties2...>& view2) { ::Kokkos::View<DataType2, Properties2...>& view2) {
@ -88,12 +101,14 @@ bool equal(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::equal_impl(label, ex, KE::cbegin(view1), KE::cend(view1), return Impl::equal_exespace_impl(label, ex, KE::cbegin(view1),
KE::cbegin(view2)); KE::cend(view1), KE::cbegin(view2));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool equal(const ExecutionSpace& ex, bool equal(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
::Kokkos::View<DataType2, Properties2...>& view2, ::Kokkos::View<DataType2, Properties2...>& view2,
@ -102,13 +117,15 @@ bool equal(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::equal_impl("Kokkos::equal_view_api_default", ex, return Impl::equal_exespace_impl("Kokkos::equal_view_api_default", ex,
KE::cbegin(view1), KE::cend(view1), KE::cbegin(view2), KE::cbegin(view1), KE::cend(view1),
std::move(predicate)); KE::cbegin(view2), std::move(predicate));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool equal(const std::string& label, const ExecutionSpace& ex, bool equal(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
::Kokkos::View<DataType2, Properties2...>& view2, ::Kokkos::View<DataType2, Properties2...>& view2,
@ -117,51 +134,149 @@ bool equal(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::equal_impl(label, ex, KE::cbegin(view1), KE::cend(view1), return Impl::equal_exespace_impl(label, ex, KE::cbegin(view1),
KE::cbegin(view2), std::move(predicate)); KE::cend(view1), KE::cbegin(view2),
std::move(predicate));
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <typename ExecutionSpace, typename IteratorType1,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< typename IteratorType2,
IteratorType1, IteratorType2>::value, std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
bool> IteratorType1, IteratorType2>&& ::Kokkos::
equal(const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1, is_execution_space_v<ExecutionSpace>,
IteratorType2 first2, IteratorType2 last2) { int> = 0>
return Impl::equal_impl("Kokkos::equal_iterator_api_default", ex, first1, bool equal(const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1,
last1, first2, last2); IteratorType2 first2, IteratorType2 last2) {
return Impl::equal_exespace_impl("Kokkos::equal_iterator_api_default", ex,
first1, last1, first2, last2);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <typename ExecutionSpace, typename IteratorType1,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< typename IteratorType2,
IteratorType1, IteratorType2>::value, std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
bool> IteratorType1, IteratorType2>&& ::Kokkos::
equal(const std::string& label, const ExecutionSpace& ex, IteratorType1 first1, is_execution_space_v<ExecutionSpace>,
IteratorType1 last1, IteratorType2 first2, IteratorType2 last2) { int> = 0>
return Impl::equal_impl(label, ex, first1, last1, first2, last2); bool equal(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType2 first2,
IteratorType2 last2) {
return Impl::equal_exespace_impl(label, ex, first1, last1, first2, last2);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <typename ExecutionSpace, typename IteratorType1,
class BinaryPredicateType> typename IteratorType2, typename BinaryPredicateType,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
IteratorType1, IteratorType2>::value, IteratorType1, IteratorType2>&& ::Kokkos::
bool> is_execution_space_v<ExecutionSpace>,
equal(const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1, int> = 0>
IteratorType2 first2, IteratorType2 last2, bool equal(const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1,
BinaryPredicateType predicate) { IteratorType2 first2, IteratorType2 last2,
return Impl::equal_impl("Kokkos::equal_iterator_api_default", ex, first1, BinaryPredicateType predicate) {
last1, first2, last2, std::move(predicate)); return Impl::equal_exespace_impl("Kokkos::equal_iterator_api_default", ex,
first1, last1, first2, last2,
std::move(predicate));
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <typename ExecutionSpace, typename IteratorType1,
class BinaryPredicateType> typename IteratorType2, typename BinaryPredicateType,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
IteratorType1, IteratorType2>::value, IteratorType1, IteratorType2>&& ::Kokkos::
bool> is_execution_space_v<ExecutionSpace>,
equal(const std::string& label, const ExecutionSpace& ex, IteratorType1 first1, int> = 0>
IteratorType1 last1, IteratorType2 first2, IteratorType2 last2, bool equal(const std::string& label, const ExecutionSpace& ex,
BinaryPredicateType predicate) { IteratorType1 first1, IteratorType1 last1, IteratorType2 first2,
return Impl::equal_impl(label, ex, first1, last1, first2, last2, IteratorType2 last2, BinaryPredicateType predicate) {
std::move(predicate)); return Impl::equal_exespace_impl(label, ex, first1, last1, first2, last2,
std::move(predicate));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
IteratorType1, IteratorType2>&& ::Kokkos::
is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION bool equal(const TeamHandleType& teamHandle,
IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2) {
return Impl::equal_team_impl(teamHandle, first1, last1, first2);
}
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
IteratorType1, IteratorType2>&& ::Kokkos::
is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION bool equal(const TeamHandleType& teamHandle,
IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2,
BinaryPredicateType predicate) {
return Impl::equal_team_impl(teamHandle, first1, last1, first2,
std::move(predicate));
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool equal(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view1,
::Kokkos::View<DataType2, Properties2...>& view2) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view1);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental;
return Impl::equal_team_impl(teamHandle, KE::cbegin(view1), KE::cend(view1),
KE::cbegin(view2));
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool equal(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view1,
::Kokkos::View<DataType2, Properties2...>& view2,
BinaryPredicateType predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view1);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental;
return Impl::equal_team_impl(teamHandle, KE::cbegin(view1), KE::cend(view1),
KE::cbegin(view2), std::move(predicate));
}
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
IteratorType1, IteratorType2>&& ::Kokkos::
is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION bool equal(const TeamHandleType& teamHandle,
IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2) {
return Impl::equal_team_impl(teamHandle, first1, last1, first2, last2);
}
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators_v<
IteratorType1, IteratorType2>&& ::Kokkos::
is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION bool equal(const TeamHandleType& teamHandle,
IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2,
BinaryPredicateType predicate) {
return Impl::equal_team_impl(teamHandle, first1, last1, first2, last2,
std::move(predicate));
} }
} // namespace Experimental } // namespace Experimental

243
lib/kokkos/algorithms/src/std_algorithms/Kokkos_ExclusiveScan.hpp
View File

@ -23,105 +23,130 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// overload set 1 // overload set 1
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class ValueType> typename OutputIteratorType, typename ValueType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<
InputIteratorType, OutputIteratorType>::value, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
OutputIteratorType> Kokkos::is_execution_space_v<ExecutionSpace>,
exclusive_scan(const ExecutionSpace& ex, InputIteratorType first, int> = 0>
InputIteratorType last, OutputIteratorType first_dest, OutputIteratorType exclusive_scan(const ExecutionSpace& ex,
ValueType init_value) { InputIteratorType first,
static_assert(std::is_move_constructible<ValueType>::value, InputIteratorType last,
OutputIteratorType first_dest,
ValueType init_value) {
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::exclusive_scan_default_op_impl( return Impl::exclusive_scan_default_op_exespace_impl(
"Kokkos::exclusive_scan_default_functors_iterator_api", ex, first, last, "Kokkos::exclusive_scan_default_functors_iterator_api", ex, first, last,
first_dest, init_value); first_dest, std::move(init_value));
} }
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class ValueType> typename OutputIteratorType, typename ValueType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<
InputIteratorType, OutputIteratorType>::value, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
OutputIteratorType> Kokkos::is_execution_space_v<ExecutionSpace>,
exclusive_scan(const std::string& label, const ExecutionSpace& ex, int> = 0>
InputIteratorType first, InputIteratorType last, OutputIteratorType exclusive_scan(const std::string& label,
OutputIteratorType first_dest, ValueType init_value) { const ExecutionSpace& ex,
static_assert(std::is_move_constructible<ValueType>::value, InputIteratorType first,
InputIteratorType last,
OutputIteratorType first_dest,
ValueType init_value) {
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::exclusive_scan_default_op_impl(label, ex, first, last, return Impl::exclusive_scan_default_op_exespace_impl(
first_dest, init_value); label, ex, first, last, first_dest, std::move(init_value));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto exclusive_scan(const ExecutionSpace& ex, auto exclusive_scan(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
ValueType init_value) { ValueType init_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::exclusive_scan_default_op_impl( return Impl::exclusive_scan_default_op_exespace_impl(
"Kokkos::exclusive_scan_default_functors_view_api", ex, "Kokkos::exclusive_scan_default_functors_view_api", ex,
KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest), KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest),
init_value); std::move(init_value));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto exclusive_scan(const std::string& label, const ExecutionSpace& ex, auto exclusive_scan(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
ValueType init_value) { ValueType init_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::exclusive_scan_default_op_impl(label, ex, KE::cbegin(view_from), return Impl::exclusive_scan_default_op_exespace_impl(
KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), init_value); KE::begin(view_dest), std::move(init_value));
} }
// overload set 2 // overload set 2
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class ValueType, class BinaryOpType> typename OutputIteratorType, typename ValueType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< typename BinaryOpType,
InputIteratorType, OutputIteratorType>::value, std::enable_if_t<
OutputIteratorType> Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
exclusive_scan(const ExecutionSpace& ex, InputIteratorType first, Kokkos::is_execution_space_v<ExecutionSpace>,
InputIteratorType last, OutputIteratorType first_dest, int> = 0>
ValueType init_value, BinaryOpType bop) { OutputIteratorType exclusive_scan(const ExecutionSpace& ex,
InputIteratorType first,
InputIteratorType last,
OutputIteratorType first_dest,
ValueType init_value, BinaryOpType bop) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::exclusive_scan_custom_op_impl( return Impl::exclusive_scan_custom_op_exespace_impl(
"Kokkos::exclusive_scan_custom_functors_iterator_api", ex, first, last, "Kokkos::exclusive_scan_custom_functors_iterator_api", ex, first, last,
first_dest, init_value, bop); first_dest, std::move(init_value), bop);
} }
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class ValueType, class BinaryOpType> typename OutputIteratorType, typename ValueType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< typename BinaryOpType,
InputIteratorType, OutputIteratorType>::value, std::enable_if_t<
OutputIteratorType> Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
exclusive_scan(const std::string& label, const ExecutionSpace& ex, Kokkos::is_execution_space_v<ExecutionSpace>,
InputIteratorType first, InputIteratorType last, int> = 0>
OutputIteratorType first_dest, ValueType init_value, OutputIteratorType exclusive_scan(const std::string& label,
BinaryOpType bop) { const ExecutionSpace& ex,
InputIteratorType first,
InputIteratorType last,
OutputIteratorType first_dest,
ValueType init_value, BinaryOpType bop) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::exclusive_scan_custom_op_impl(label, ex, first, last, first_dest, return Impl::exclusive_scan_custom_op_exespace_impl(
init_value, bop); label, ex, first, last, first_dest, std::move(init_value), bop);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class BinaryOpType> typename DataType2, typename... Properties2, typename ValueType,
typename BinaryOpType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto exclusive_scan(const ExecutionSpace& ex, auto exclusive_scan(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -129,18 +154,20 @@ auto exclusive_scan(const ExecutionSpace& ex,
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::exclusive_scan_custom_op_impl( return Impl::exclusive_scan_custom_op_exespace_impl(
"Kokkos::exclusive_scan_custom_functors_view_api", ex, "Kokkos::exclusive_scan_custom_functors_view_api", ex,
KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest), KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest),
init_value, bop); std::move(init_value), bop);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class BinaryOpType> typename DataType2, typename... Properties2, typename ValueType,
typename BinaryOpType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto exclusive_scan(const std::string& label, const ExecutionSpace& ex, auto exclusive_scan(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -148,12 +175,92 @@ auto exclusive_scan(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::exclusive_scan_custom_op_impl( return Impl::exclusive_scan_custom_op_exespace_impl(
label, ex, KE::cbegin(view_from), KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), init_value, bop); KE::begin(view_dest), std::move(init_value), bop);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
// overload set 1
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType, typename ValueType,
std::enable_if_t<
Impl::are_iterators_v<InputIteratorType, OutputIteratorType> &&
Kokkos::is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType
exclusive_scan(const TeamHandleType& teamHandle, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest,
ValueType init_value) {
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
return Impl::exclusive_scan_default_op_team_impl(
teamHandle, first, last, first_dest, std::move(init_value));
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto exclusive_scan(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
ValueType init_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental;
return Impl::exclusive_scan_default_op_team_impl(
teamHandle, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), std::move(init_value));
}
// overload set 2
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType, typename ValueType,
typename BinaryOpType,
std::enable_if_t<
Impl::are_iterators_v<InputIteratorType, OutputIteratorType> &&
Kokkos::is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType
exclusive_scan(const TeamHandleType& teamHandle, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest,
ValueType init_value, BinaryOpType bop) {
Impl::static_assert_is_not_openmptarget(teamHandle);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
return Impl::exclusive_scan_custom_op_team_impl(
teamHandle, first, last, first_dest, std::move(init_value), bop);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
typename BinaryOpType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto exclusive_scan(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
ValueType init_value, BinaryOpType bop) {
Impl::static_assert_is_not_openmptarget(teamHandle);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental;
return Impl::exclusive_scan_custom_op_team_impl(
teamHandle, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), std::move(init_value), bop);
} }
} // namespace Experimental } // namespace Experimental

54
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Fill.hpp
View File

@ -23,33 +23,67 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class T> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType, typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void fill(const ExecutionSpace& ex, IteratorType first, IteratorType last, void fill(const ExecutionSpace& ex, IteratorType first, IteratorType last,
const T& value) { const T& value) {
Impl::fill_impl("Kokkos::fill_iterator_api_default", ex, first, last, value); Impl::fill_exespace_impl("Kokkos::fill_iterator_api_default", ex, first, last,
value);
} }
template <class ExecutionSpace, class IteratorType, class T> template <
typename ExecutionSpace, typename IteratorType, typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void fill(const std::string& label, const ExecutionSpace& ex, void fill(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, const T& value) { IteratorType first, IteratorType last, const T& value) {
Impl::fill_impl(label, ex, first, last, value); Impl::fill_exespace_impl(label, ex, first, last, value);
} }
template <class ExecutionSpace, class DataType, class... Properties, class T> template <
typename ExecutionSpace, typename DataType, typename... Properties,
typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void fill(const ExecutionSpace& ex, void fill(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const T& value) { const ::Kokkos::View<DataType, Properties...>& view, const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::fill_exespace_impl("Kokkos::fill_view_api_default", ex, begin(view),
Impl::fill_impl("Kokkos::fill_view_api_default", ex, begin(view), end(view), end(view), value);
value);
} }
template <class ExecutionSpace, class DataType, class... Properties, class T> template <
typename ExecutionSpace, typename DataType, typename... Properties,
typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void fill(const std::string& label, const ExecutionSpace& ex, void fill(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const T& value) { const ::Kokkos::View<DataType, Properties...>& view, const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::fill_exespace_impl(label, ex, begin(view), end(view), value);
}
Impl::fill_impl(label, ex, begin(view), end(view), value); //
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType, typename T,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void fill(const TeamHandleType& th, IteratorType first,
IteratorType last, const T& value) {
Impl::fill_team_impl(th, first, last, value);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename T,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void fill(const TeamHandleType& th,
const ::Kokkos::View<DataType, Properties...>& view,
const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::fill_team_impl(th, begin(view), end(view), value);
} }
} // namespace Experimental } // namespace Experimental

58
lib/kokkos/algorithms/src/std_algorithms/Kokkos_FillN.hpp
View File

@ -23,38 +23,72 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class SizeType, class T> template <
typename ExecutionSpace, typename IteratorType, typename SizeType,
typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType fill_n(const ExecutionSpace& ex, IteratorType first, SizeType n, IteratorType fill_n(const ExecutionSpace& ex, IteratorType first, SizeType n,
const T& value) { const T& value) {
return Impl::fill_n_impl("Kokkos::fill_n_iterator_api_default", ex, first, n, return Impl::fill_n_exespace_impl("Kokkos::fill_n_iterator_api_default", ex,
value); first, n, value);
} }
template <class ExecutionSpace, class IteratorType, class SizeType, class T> template <
typename ExecutionSpace, typename IteratorType, typename SizeType,
typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType fill_n(const std::string& label, const ExecutionSpace& ex, IteratorType fill_n(const std::string& label, const ExecutionSpace& ex,
IteratorType first, SizeType n, const T& value) { IteratorType first, SizeType n, const T& value) {
return Impl::fill_n_impl(label, ex, first, n, value); return Impl::fill_n_exespace_impl(label, ex, first, n, value);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class SizeType, class T> typename ExecutionSpace, typename DataType, typename... Properties,
typename SizeType, typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto fill_n(const ExecutionSpace& ex, auto fill_n(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, SizeType n, const ::Kokkos::View<DataType, Properties...>& view, SizeType n,
const T& value) { const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::fill_n_impl("Kokkos::fill_n_view_api_default", ex, begin(view), return Impl::fill_n_exespace_impl("Kokkos::fill_n_view_api_default", ex,
n, value); begin(view), n, value);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class SizeType, class T> typename ExecutionSpace, typename DataType, typename... Properties,
typename SizeType, typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto fill_n(const std::string& label, const ExecutionSpace& ex, auto fill_n(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, SizeType n, const ::Kokkos::View<DataType, Properties...>& view, SizeType n,
const T& value) { const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::fill_n_impl(label, ex, begin(view), n, value); return Impl::fill_n_exespace_impl(label, ex, begin(view), n, value);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType, typename SizeType,
typename T,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType fill_n(const TeamHandleType& th,
IteratorType first, SizeType n,
const T& value) {
return Impl::fill_n_team_impl(th, first, n, value);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename SizeType, typename T,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto fill_n(const TeamHandleType& th,
const ::Kokkos::View<DataType, Properties...>& view,
SizeType n, const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::fill_n_team_impl(th, begin(view), n, value);
} }
} // namespace Experimental } // namespace Experimental

60
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Find.hpp
View File

@ -23,36 +23,76 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class T> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
InputIterator find(const ExecutionSpace& ex, InputIterator first, InputIterator find(const ExecutionSpace& ex, InputIterator first,
InputIterator last, const T& value) { InputIterator last, const T& value) {
return Impl::find_impl("Kokkos::find_iterator_api_default", ex, first, last, return Impl::find_exespace_impl("Kokkos::find_iterator_api_default", ex,
value); first, last, value);
} }
template <class ExecutionSpace, class InputIterator, class T> template <
typename ExecutionSpace, typename InputIterator, typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
InputIterator find(const std::string& label, const ExecutionSpace& ex, InputIterator find(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, const T& value) { InputIterator first, InputIterator last, const T& value) {
return Impl::find_impl(label, ex, first, last, value); return Impl::find_exespace_impl(label, ex, first, last, value);
} }
template <class ExecutionSpace, class DataType, class... Properties, class T> template <
typename ExecutionSpace, typename DataType, typename... Properties,
typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find(const ExecutionSpace& ex, auto find(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const T& value) { const ::Kokkos::View<DataType, Properties...>& view, const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_impl("Kokkos::find_view_api_default", ex, KE::begin(view), return Impl::find_exespace_impl("Kokkos::find_view_api_default", ex,
KE::end(view), value); KE::begin(view), KE::end(view), value);
} }
template <class ExecutionSpace, class DataType, class... Properties, class T> template <
typename ExecutionSpace, typename DataType, typename... Properties,
typename T,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find(const std::string& label, const ExecutionSpace& ex, auto find(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const T& value) { const ::Kokkos::View<DataType, Properties...>& view, const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_impl(label, ex, KE::begin(view), KE::end(view), value); return Impl::find_exespace_impl(label, ex, KE::begin(view), KE::end(view),
value);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator, typename T,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION InputIterator find(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last,
const T& value) {
return Impl::find_team_impl(teamHandle, first, last, value);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename T,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto find(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view,
const T& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental;
return Impl::find_team_impl(teamHandle, KE::begin(view), KE::end(view),
value);
} }
} // namespace Experimental } // namespace Experimental

144
lib/kokkos/algorithms/src/std_algorithms/Kokkos_FindEnd.hpp
View File

@ -24,24 +24,34 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// overload set 1: no binary predicate passed // overload set 1: no binary predicate passed
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 find_end(const ExecutionSpace& ex, IteratorType1 first, IteratorType1 find_end(const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first, IteratorType1 last, IteratorType2 s_first,
IteratorType2 s_last) { IteratorType2 s_last) {
return Impl::find_end_impl("Kokkos::find_end_iterator_api_default", ex, first, return Impl::find_end_exespace_impl("Kokkos::find_end_iterator_api_default",
last, s_first, s_last); ex, first, last, s_first, s_last);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 find_end(const std::string& label, const ExecutionSpace& ex, IteratorType1 find_end(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first, IteratorType1 last, IteratorType1 first, IteratorType1 last,
IteratorType2 s_first, IteratorType2 s_last) { IteratorType2 s_first, IteratorType2 s_last) {
return Impl::find_end_impl(label, ex, first, last, s_first, s_last); return Impl::find_end_exespace_impl(label, ex, first, last, s_first, s_last);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find_end(const ExecutionSpace& ex, auto find_end(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view) { const ::Kokkos::View<DataType2, Properties2...>& s_view) {
@ -49,13 +59,15 @@ auto find_end(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_end_impl("Kokkos::find_end_view_api_default", ex, return Impl::find_end_exespace_impl("Kokkos::find_end_view_api_default", ex,
KE::begin(view), KE::end(view), KE::begin(s_view), KE::begin(view), KE::end(view),
KE::end(s_view)); KE::begin(s_view), KE::end(s_view));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find_end(const std::string& label, const ExecutionSpace& ex, auto find_end(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view) { const ::Kokkos::View<DataType2, Properties2...>& s_view) {
@ -63,31 +75,38 @@ auto find_end(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_end_impl(label, ex, KE::begin(view), KE::end(view), return Impl::find_end_exespace_impl(label, ex, KE::begin(view), KE::end(view),
KE::begin(s_view), KE::end(s_view)); KE::begin(s_view), KE::end(s_view));
} }
// overload set 2: binary predicate passed // overload set 2: binary predicate passed
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class BinaryPredicateType> typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 find_end(const ExecutionSpace& ex, IteratorType1 first, IteratorType1 find_end(const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first, IteratorType1 last, IteratorType2 s_first,
IteratorType2 s_last, const BinaryPredicateType& pred) { IteratorType2 s_last, const BinaryPredicateType& pred) {
return Impl::find_end_impl("Kokkos::find_end_iterator_api_default", ex, first, return Impl::find_end_exespace_impl("Kokkos::find_end_iterator_api_default",
last, s_first, s_last, pred); ex, first, last, s_first, s_last, pred);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class BinaryPredicateType> typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 find_end(const std::string& label, const ExecutionSpace& ex, IteratorType1 find_end(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first, IteratorType1 last, IteratorType1 first, IteratorType1 last,
IteratorType2 s_first, IteratorType2 s_last, IteratorType2 s_first, IteratorType2 s_last,
const BinaryPredicateType& pred) { const BinaryPredicateType& pred) {
return Impl::find_end_impl(label, ex, first, last, s_first, s_last, pred); return Impl::find_end_exespace_impl(label, ex, first, last, s_first, s_last,
pred);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find_end(const ExecutionSpace& ex, auto find_end(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view, const ::Kokkos::View<DataType2, Properties2...>& s_view,
@ -96,13 +115,15 @@ auto find_end(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_end_impl("Kokkos::find_end_view_api_default", ex, return Impl::find_end_exespace_impl("Kokkos::find_end_view_api_default", ex,
KE::begin(view), KE::end(view), KE::begin(s_view), KE::begin(view), KE::end(view),
KE::end(s_view), pred); KE::begin(s_view), KE::end(s_view), pred);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find_end(const std::string& label, const ExecutionSpace& ex, auto find_end(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view, const ::Kokkos::View<DataType2, Properties2...>& s_view,
@ -111,8 +132,71 @@ auto find_end(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_end_impl(label, ex, KE::begin(view), KE::end(view), return Impl::find_end_exespace_impl(label, ex, KE::begin(view), KE::end(view),
KE::begin(s_view), KE::end(s_view), pred); KE::begin(s_view), KE::end(s_view), pred);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
// overload set 1: no binary predicate passed
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType1 find_end(const TeamHandleType& teamHandle,
IteratorType1 first, IteratorType1 last,
IteratorType2 s_first,
IteratorType2 s_last) {
return Impl::find_end_team_impl(teamHandle, first, last, s_first, s_last);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto find_end(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental;
return Impl::find_end_team_impl(teamHandle, KE::begin(view), KE::end(view),
KE::begin(s_view), KE::end(s_view));
}
// overload set 2: binary predicate passed
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType1 find_end(const TeamHandleType& teamHandle,
IteratorType1 first, IteratorType1 last,
IteratorType2 s_first,
IteratorType2 s_last,
const BinaryPredicateType& pred) {
return Impl::find_end_team_impl(teamHandle, first, last, s_first, s_last,
pred);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto find_end(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view,
const BinaryPredicateType& pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental;
return Impl::find_end_team_impl(teamHandle, KE::begin(view), KE::end(view),
KE::begin(s_view), KE::end(s_view), pred);
} }
} // namespace Experimental } // namespace Experimental

155
lib/kokkos/algorithms/src/std_algorithms/Kokkos_FindFirstOf.hpp
View File

@ -23,24 +23,36 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// overload set 1: no binary predicate passed // overload set 1: no binary predicate passed
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 find_first_of(const ExecutionSpace& ex, IteratorType1 first, IteratorType1 find_first_of(const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first, IteratorType1 last, IteratorType2 s_first,
IteratorType2 s_last) { IteratorType2 s_last) {
return Impl::find_first_of_impl("Kokkos::find_first_of_iterator_api_default", return Impl::find_first_of_exespace_impl(
ex, first, last, s_first, s_last); "Kokkos::find_first_of_iterator_api_default", ex, first, last, s_first,
s_last);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 find_first_of(const std::string& label, const ExecutionSpace& ex, IteratorType1 find_first_of(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first, IteratorType1 last, IteratorType1 first, IteratorType1 last,
IteratorType2 s_first, IteratorType2 s_last) { IteratorType2 s_first, IteratorType2 s_last) {
return Impl::find_first_of_impl(label, ex, first, last, s_first, s_last); return Impl::find_first_of_exespace_impl(label, ex, first, last, s_first,
s_last);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find_first_of(const ExecutionSpace& ex, auto find_first_of(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view) { const ::Kokkos::View<DataType2, Properties2...>& s_view) {
@ -48,13 +60,15 @@ auto find_first_of(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_first_of_impl("Kokkos::find_first_of_view_api_default", ex, return Impl::find_first_of_exespace_impl(
KE::begin(view), KE::end(view), "Kokkos::find_first_of_view_api_default", ex, KE::begin(view),
KE::begin(s_view), KE::end(s_view)); KE::end(view), KE::begin(s_view), KE::end(s_view));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find_first_of(const std::string& label, const ExecutionSpace& ex, auto find_first_of(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view) { const ::Kokkos::View<DataType2, Properties2...>& s_view) {
@ -62,33 +76,41 @@ auto find_first_of(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_first_of_impl(label, ex, KE::begin(view), KE::end(view), return Impl::find_first_of_exespace_impl(label, ex, KE::begin(view),
KE::begin(s_view), KE::end(s_view)); KE::end(view), KE::begin(s_view),
KE::end(s_view));
} }
// overload set 2: binary predicate passed // overload set 2: binary predicate passed
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class BinaryPredicateType> typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 find_first_of(const ExecutionSpace& ex, IteratorType1 first, IteratorType1 find_first_of(const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first, IteratorType1 last, IteratorType2 s_first,
IteratorType2 s_last, IteratorType2 s_last,
const BinaryPredicateType& pred) { const BinaryPredicateType& pred) {
return Impl::find_first_of_impl("Kokkos::find_first_of_iterator_api_default", return Impl::find_first_of_exespace_impl(
ex, first, last, s_first, s_last, pred); "Kokkos::find_first_of_iterator_api_default", ex, first, last, s_first,
s_last, pred);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class BinaryPredicateType> typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 find_first_of(const std::string& label, const ExecutionSpace& ex, IteratorType1 find_first_of(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first, IteratorType1 last, IteratorType1 first, IteratorType1 last,
IteratorType2 s_first, IteratorType2 s_last, IteratorType2 s_first, IteratorType2 s_last,
const BinaryPredicateType& pred) { const BinaryPredicateType& pred) {
return Impl::find_first_of_impl(label, ex, first, last, s_first, s_last, return Impl::find_first_of_exespace_impl(label, ex, first, last, s_first,
pred); s_last, pred);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find_first_of(const ExecutionSpace& ex, auto find_first_of(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view, const ::Kokkos::View<DataType2, Properties2...>& s_view,
@ -97,13 +119,15 @@ auto find_first_of(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_first_of_impl("Kokkos::find_first_of_view_api_default", ex, return Impl::find_first_of_exespace_impl(
KE::begin(view), KE::end(view), "Kokkos::find_first_of_view_api_default", ex, KE::begin(view),
KE::begin(s_view), KE::end(s_view), pred); KE::end(view), KE::begin(s_view), KE::end(s_view), pred);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto find_first_of(const std::string& label, const ExecutionSpace& ex, auto find_first_of(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view, const ::Kokkos::View<DataType2, Properties2...>& s_view,
@ -112,8 +136,77 @@ auto find_first_of(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_first_of_impl(label, ex, KE::begin(view), KE::end(view), return Impl::find_first_of_exespace_impl(label, ex, KE::begin(view),
KE::begin(s_view), KE::end(s_view), pred); KE::end(view), KE::begin(s_view),
KE::end(s_view), pred);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
// overload set 1: no binary predicate passed
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType1 find_first_of(const TeamHandleType& teamHandle,
IteratorType1 first,
IteratorType1 last,
IteratorType2 s_first,
IteratorType2 s_last) {
return Impl::find_first_of_team_impl(teamHandle, first, last, s_first,
s_last);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto find_first_of(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental;
return Impl::find_first_of_team_impl(teamHandle, KE::begin(view),
KE::end(view), KE::begin(s_view),
KE::end(s_view));
}
// overload set 2: binary predicate passed
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType1 find_first_of(const TeamHandleType& teamHandle,
IteratorType1 first,
IteratorType1 last,
IteratorType2 s_first,
IteratorType2 s_last,
const BinaryPredicateType& pred) {
return Impl::find_first_of_team_impl(teamHandle, first, last, s_first, s_last,
pred);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto find_first_of(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view,
const BinaryPredicateType& pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental;
return Impl::find_first_of_team_impl(teamHandle, KE::begin(view),
KE::end(view), KE::begin(s_view),
KE::end(s_view), pred);
} }
} // namespace Experimental } // namespace Experimental

70
lib/kokkos/algorithms/src/std_algorithms/Kokkos_FindIf.hpp
View File

@ -23,42 +23,82 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class PredicateType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType, typename PredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType find_if(const ExecutionSpace& ex, IteratorType first, IteratorType find_if(const ExecutionSpace& ex, IteratorType first,
IteratorType last, PredicateType predicate) { IteratorType last, PredicateType predicate) {
return Impl::find_if_or_not_impl<true>("Kokkos::find_if_iterator_api_default", return Impl::find_if_or_not_exespace_impl<true>(
ex, first, last, std::move(predicate)); "Kokkos::find_if_iterator_api_default", ex, first, last,
std::move(predicate));
} }
template <class ExecutionSpace, class IteratorType, class PredicateType> template <
typename ExecutionSpace, typename IteratorType, typename PredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType find_if(const std::string& label, const ExecutionSpace& ex, IteratorType find_if(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
PredicateType predicate) { PredicateType predicate) {
return Impl::find_if_or_not_impl<true>(label, ex, first, last, return Impl::find_if_or_not_exespace_impl<true>(label, ex, first, last,
std::move(predicate)); std::move(predicate));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class Predicate> typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto find_if(const ExecutionSpace& ex, auto find_if(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_if_or_not_impl<true>("Kokkos::find_if_view_api_default", ex, return Impl::find_if_or_not_exespace_impl<true>(
KE::begin(v), KE::end(v), "Kokkos::find_if_view_api_default", ex, KE::begin(v), KE::end(v),
std::move(predicate)); std::move(predicate));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class Predicate> typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto find_if(const std::string& label, const ExecutionSpace& ex, auto find_if(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_if_or_not_impl<true>(label, ex, KE::begin(v), KE::end(v), return Impl::find_if_or_not_exespace_impl<true>(
std::move(predicate)); label, ex, KE::begin(v), KE::end(v), std::move(predicate));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
typename PredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType find_if(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
PredicateType predicate) {
return Impl::find_if_or_not_team_impl<true>(teamHandle, first, last,
std::move(predicate));
}
template <
typename TeamHandleType, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION auto find_if(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::find_if_or_not_team_impl<true>(teamHandle, KE::begin(v),
KE::end(v), std::move(predicate));
} }
} // namespace Experimental } // namespace Experimental

63
lib/kokkos/algorithms/src/std_algorithms/Kokkos_FindIfNot.hpp
View File

@ -23,45 +23,84 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class Predicate> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType find_if_not(const ExecutionSpace& ex, IteratorType first, IteratorType find_if_not(const ExecutionSpace& ex, IteratorType first,
IteratorType last, Predicate predicate) { IteratorType last, Predicate predicate) {
return Impl::find_if_or_not_impl<false>( return Impl::find_if_or_not_exespace_impl<false>(
"Kokkos::find_if_not_iterator_api_default", ex, first, last, "Kokkos::find_if_not_iterator_api_default", ex, first, last,
std::move(predicate)); std::move(predicate));
} }
template <class ExecutionSpace, class IteratorType, class Predicate> template <
typename ExecutionSpace, typename IteratorType, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType find_if_not(const std::string& label, const ExecutionSpace& ex, IteratorType find_if_not(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
Predicate predicate) { Predicate predicate) {
return Impl::find_if_or_not_impl<false>(label, ex, first, last, return Impl::find_if_or_not_exespace_impl<false>(label, ex, first, last,
std::move(predicate)); std::move(predicate));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class Predicate> typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto find_if_not(const ExecutionSpace& ex, auto find_if_not(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_if_or_not_impl<false>( return Impl::find_if_or_not_exespace_impl<false>(
"Kokkos::find_if_not_view_api_default", ex, KE::begin(v), KE::end(v), "Kokkos::find_if_not_view_api_default", ex, KE::begin(v), KE::end(v),
std::move(predicate)); std::move(predicate));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class Predicate> typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto find_if_not(const std::string& label, const ExecutionSpace& ex, auto find_if_not(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::find_if_or_not_impl<false>(label, ex, KE::begin(v), KE::end(v), return Impl::find_if_or_not_exespace_impl<false>(
std::move(predicate)); label, ex, KE::begin(v), KE::end(v), std::move(predicate));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType, typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType find_if_not(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
Predicate predicate) {
return Impl::find_if_or_not_team_impl<false>(teamHandle, first, last,
std::move(predicate));
}
template <
typename TeamHandleType, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION auto find_if_not(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v, Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::find_if_or_not_team_impl<false>(
teamHandle, KE::begin(v), KE::end(v), std::move(predicate));
} }
} // namespace Experimental } // namespace Experimental

67
lib/kokkos/algorithms/src/std_algorithms/Kokkos_ForEach.hpp
View File

@ -23,42 +23,83 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class UnaryFunctorType> //
// overload set accepting execution space
//
template <
class ExecutionSpace, class IteratorType, class UnaryFunctorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
UnaryFunctorType for_each(const std::string& label, const ExecutionSpace& ex, UnaryFunctorType for_each(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
UnaryFunctorType functor) { UnaryFunctorType functor) {
return Impl::for_each_impl(label, ex, first, last, std::move(functor)); return Impl::for_each_exespace_impl(label, ex, first, last,
std::move(functor));
} }
template <class ExecutionSpace, class IteratorType, class UnaryFunctorType> template <
class ExecutionSpace, class IteratorType, class UnaryFunctorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
UnaryFunctorType for_each(const ExecutionSpace& ex, IteratorType first, UnaryFunctorType for_each(const ExecutionSpace& ex, IteratorType first,
IteratorType last, UnaryFunctorType functor) { IteratorType last, UnaryFunctorType functor) {
return Impl::for_each_impl("Kokkos::for_each_iterator_api_default", ex, first, return Impl::for_each_exespace_impl("Kokkos::for_each_iterator_api_default",
last, std::move(functor)); ex, first, last, std::move(functor));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class UnaryFunctorType> class ExecutionSpace, class DataType, class... Properties,
class UnaryFunctorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
UnaryFunctorType for_each(const std::string& label, const ExecutionSpace& ex, UnaryFunctorType for_each(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
UnaryFunctorType functor) { UnaryFunctorType functor) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::for_each_impl(label, ex, KE::begin(v), KE::end(v), return Impl::for_each_exespace_impl(label, ex, KE::begin(v), KE::end(v),
std::move(functor)); std::move(functor));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class UnaryFunctorType> class ExecutionSpace, class DataType, class... Properties,
class UnaryFunctorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
UnaryFunctorType for_each(const ExecutionSpace& ex, UnaryFunctorType for_each(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
UnaryFunctorType functor) { UnaryFunctorType functor) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::for_each_impl("Kokkos::for_each_view_api_default", ex, return Impl::for_each_exespace_impl("Kokkos::for_each_view_api_default", ex,
KE::begin(v), KE::end(v), std::move(functor)); KE::begin(v), KE::end(v),
std::move(functor));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <class TeamHandleType, class IteratorType, class UnaryFunctorType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION UnaryFunctorType for_each(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
UnaryFunctorType functor) {
return Impl::for_each_team_impl(teamHandle, first, last, std::move(functor));
}
template <class TeamHandleType, class DataType, class... Properties,
class UnaryFunctorType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION UnaryFunctorType
for_each(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v,
UnaryFunctorType functor) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::for_each_team_impl(teamHandle, KE::begin(v), KE::end(v),
std::move(functor));
} }
} // namespace Experimental } // namespace Experimental

72
lib/kokkos/algorithms/src/std_algorithms/Kokkos_ForEachN.hpp
View File

@ -23,43 +23,87 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class SizeType, //
class UnaryFunctorType> // overload set accepting execution space
//
template <
class ExecutionSpace, class IteratorType, class SizeType,
class UnaryFunctorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType for_each_n(const std::string& label, const ExecutionSpace& ex, IteratorType for_each_n(const std::string& label, const ExecutionSpace& ex,
IteratorType first, SizeType n, IteratorType first, SizeType n,
UnaryFunctorType functor) { UnaryFunctorType functor) {
return Impl::for_each_n_impl(label, ex, first, n, std::move(functor)); return Impl::for_each_n_exespace_impl(label, ex, first, n,
std::move(functor));
} }
template <class ExecutionSpace, class IteratorType, class SizeType, template <
class UnaryFunctorType> class ExecutionSpace, class IteratorType, class SizeType,
class UnaryFunctorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType for_each_n(const ExecutionSpace& ex, IteratorType first, IteratorType for_each_n(const ExecutionSpace& ex, IteratorType first,
SizeType n, UnaryFunctorType functor) { SizeType n, UnaryFunctorType functor) {
return Impl::for_each_n_impl("Kokkos::for_each_n_iterator_api_default", ex, return Impl::for_each_n_exespace_impl(
first, n, std::move(functor)); "Kokkos::for_each_n_iterator_api_default", ex, first, n,
std::move(functor));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class SizeType, class UnaryFunctorType> class ExecutionSpace, class DataType, class... Properties, class SizeType,
class UnaryFunctorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto for_each_n(const std::string& label, const ExecutionSpace& ex, auto for_each_n(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, SizeType n, const ::Kokkos::View<DataType, Properties...>& v, SizeType n,
UnaryFunctorType functor) { UnaryFunctorType functor) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::for_each_n_impl(label, ex, KE::begin(v), n, std::move(functor)); return Impl::for_each_n_exespace_impl(label, ex, KE::begin(v), n,
std::move(functor));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class SizeType, class UnaryFunctorType> class ExecutionSpace, class DataType, class... Properties, class SizeType,
class UnaryFunctorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto for_each_n(const ExecutionSpace& ex, auto for_each_n(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, SizeType n, const ::Kokkos::View<DataType, Properties...>& v, SizeType n,
UnaryFunctorType functor) { UnaryFunctorType functor) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::for_each_n_impl("Kokkos::for_each_n_view_api_default", ex, return Impl::for_each_n_exespace_impl("Kokkos::for_each_n_view_api_default",
KE::begin(v), n, std::move(functor)); ex, KE::begin(v), n,
std::move(functor));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <class TeamHandleType, class IteratorType, class SizeType,
class UnaryFunctorType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType for_each_n(const TeamHandleType& teamHandle,
IteratorType first, SizeType n,
UnaryFunctorType functor) {
return Impl::for_each_n_team_impl(teamHandle, first, n, std::move(functor));
}
template <class TeamHandleType, class DataType, class... Properties,
class SizeType, class UnaryFunctorType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto for_each_n(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v, SizeType n,
UnaryFunctorType functor) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::for_each_n_team_impl(teamHandle, KE::begin(v), n,
std::move(functor));
} }
} // namespace Experimental } // namespace Experimental

54
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Generate.hpp
View File

@ -23,38 +23,68 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class Generator> //
// overload set accepting execution space
//
template <typename ExecutionSpace, typename IteratorType, typename Generator,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
void generate(const ExecutionSpace& ex, IteratorType first, IteratorType last, void generate(const ExecutionSpace& ex, IteratorType first, IteratorType last,
Generator g) { Generator g) {
Impl::generate_impl("Kokkos::generate_iterator_api_default", ex, first, last, Impl::generate_exespace_impl("Kokkos::generate_iterator_api_default", ex,
std::move(g)); first, last, std::move(g));
} }
template <class ExecutionSpace, class IteratorType, class Generator> template <typename ExecutionSpace, typename IteratorType, typename Generator,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
void generate(const std::string& label, const ExecutionSpace& ex, void generate(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, Generator g) { IteratorType first, IteratorType last, Generator g) {
Impl::generate_impl(label, ex, first, last, std::move(g)); Impl::generate_exespace_impl(label, ex, first, last, std::move(g));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class Generator> typename Generator,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
void generate(const ExecutionSpace& ex, void generate(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
Generator g) { Generator g) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::generate_impl("Kokkos::generate_view_api_default", ex, begin(view), Impl::generate_exespace_impl("Kokkos::generate_view_api_default", ex,
end(view), std::move(g)); begin(view), end(view), std::move(g));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class Generator> typename Generator,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
void generate(const std::string& label, const ExecutionSpace& ex, void generate(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
Generator g) { Generator g) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::generate_impl(label, ex, begin(view), end(view), std::move(g)); Impl::generate_exespace_impl(label, ex, begin(view), end(view), std::move(g));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType, typename Generator,
std::enable_if_t<is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void generate(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
Generator g) {
Impl::generate_team_impl(teamHandle, first, last, std::move(g));
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename Generator,
std::enable_if_t<is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void generate(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view, Generator g) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::generate_team_impl(teamHandle, begin(view), end(view), std::move(g));
} }
} // namespace Experimental } // namespace Experimental

63
lib/kokkos/algorithms/src/std_algorithms/Kokkos_GenerateN.hpp
View File

@ -23,40 +23,75 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class Size, class Generator> //
// overload set accepting execution space
//
template <typename ExecutionSpace, typename IteratorType, typename Size,
typename Generator,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType generate_n(const ExecutionSpace& ex, IteratorType first, IteratorType generate_n(const ExecutionSpace& ex, IteratorType first,
Size count, Generator g) { Size count, Generator g) {
Impl::generate_n_impl("Kokkos::generate_n_iterator_api_default", ex, first, return Impl::generate_n_exespace_impl(
count, std::move(g)); "Kokkos::generate_n_iterator_api_default", ex, first, count,
return first + count; std::move(g));
} }
template <class ExecutionSpace, class IteratorType, class Size, class Generator> template <typename ExecutionSpace, typename IteratorType, typename Size,
typename Generator,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType generate_n(const std::string& label, const ExecutionSpace& ex, IteratorType generate_n(const std::string& label, const ExecutionSpace& ex,
IteratorType first, Size count, Generator g) { IteratorType first, Size count, Generator g) {
Impl::generate_n_impl(label, ex, first, count, std::move(g)); return Impl::generate_n_exespace_impl(label, ex, first, count, std::move(g));
return first + count;
} }
template <class ExecutionSpace, class DataType, class... Properties, class Size, template <typename ExecutionSpace, typename DataType, typename... Properties,
class Generator> typename Size, typename Generator,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
auto generate_n(const ExecutionSpace& ex, auto generate_n(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, Size count, const ::Kokkos::View<DataType, Properties...>& view, Size count,
Generator g) { Generator g) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::generate_n_impl("Kokkos::generate_n_view_api_default", ex, return Impl::generate_n_exespace_impl("Kokkos::generate_n_view_api_default",
begin(view), count, std::move(g)); ex, begin(view), count, std::move(g));
} }
template <class ExecutionSpace, class DataType, class... Properties, class Size, template <typename ExecutionSpace, typename DataType, typename... Properties,
class Generator> typename Size, typename Generator,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
auto generate_n(const std::string& label, const ExecutionSpace& ex, auto generate_n(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, Size count, const ::Kokkos::View<DataType, Properties...>& view, Size count,
Generator g) { Generator g) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::generate_n_impl(label, ex, begin(view), count, std::move(g)); return Impl::generate_n_exespace_impl(label, ex, begin(view), count,
std::move(g));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType, typename Size,
typename Generator,
std::enable_if_t<is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType generate_n(const TeamHandleType& teamHandle,
IteratorType first, Size count,
Generator g) {
return Impl::generate_n_team_impl(teamHandle, first, count, std::move(g));
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename Size, typename Generator,
std::enable_if_t<is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto generate_n(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view, Size count,
Generator g) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::generate_n_team_impl(teamHandle, begin(view), count,
std::move(g));
} }
} // namespace Experimental } // namespace Experimental

311
lib/kokkos/algorithms/src/std_algorithms/Kokkos_InclusiveScan.hpp
View File

@ -23,33 +23,45 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// overload set 1 // overload set 1
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType> typename OutputIteratorType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<
InputIteratorType, OutputIteratorType>::value, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
OutputIteratorType> Kokkos::is_execution_space_v<ExecutionSpace>,
inclusive_scan(const ExecutionSpace& ex, InputIteratorType first, int> = 0>
InputIteratorType last, OutputIteratorType first_dest) { OutputIteratorType inclusive_scan(const ExecutionSpace& ex,
return Impl::inclusive_scan_default_op_impl( InputIteratorType first,
InputIteratorType last,
OutputIteratorType first_dest) {
return Impl::inclusive_scan_default_op_exespace_impl(
"Kokkos::inclusive_scan_default_functors_iterator_api", ex, first, last, "Kokkos::inclusive_scan_default_functors_iterator_api", ex, first, last,
first_dest); first_dest);
} }
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType> typename OutputIteratorType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<
InputIteratorType, OutputIteratorType>::value, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
OutputIteratorType> Kokkos::is_execution_space_v<ExecutionSpace>,
inclusive_scan(const std::string& label, const ExecutionSpace& ex, int> = 0>
InputIteratorType first, InputIteratorType last, OutputIteratorType inclusive_scan(const std::string& label,
OutputIteratorType first_dest) { const ExecutionSpace& ex,
return Impl::inclusive_scan_default_op_impl(label, ex, first, last, InputIteratorType first,
first_dest); InputIteratorType last,
OutputIteratorType first_dest) {
return Impl::inclusive_scan_default_op_exespace_impl(label, ex, first, last,
first_dest);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto inclusive_scan( auto inclusive_scan(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -57,13 +69,15 @@ auto inclusive_scan(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::inclusive_scan_default_op_impl( return Impl::inclusive_scan_default_op_exespace_impl(
"Kokkos::inclusive_scan_default_functors_view_api", ex, "Kokkos::inclusive_scan_default_functors_view_api", ex,
KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest)); KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto inclusive_scan( auto inclusive_scan(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -71,39 +85,45 @@ auto inclusive_scan(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::inclusive_scan_default_op_impl(label, ex, KE::cbegin(view_from), return Impl::inclusive_scan_default_op_exespace_impl(
KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest)); KE::begin(view_dest));
} }
// overload set 2 (accepting custom binary op) // overload set 2 (accepting custom binary op)
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class BinaryOp> typename OutputIteratorType, typename BinaryOp,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<
InputIteratorType, OutputIteratorType>::value, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
OutputIteratorType> Kokkos::is_execution_space_v<ExecutionSpace>,
inclusive_scan(const ExecutionSpace& ex, InputIteratorType first, int> = 0>
InputIteratorType last, OutputIteratorType first_dest, OutputIteratorType inclusive_scan(const ExecutionSpace& ex,
BinaryOp binary_op) { InputIteratorType first,
return Impl::inclusive_scan_custom_binary_op_impl( InputIteratorType last,
OutputIteratorType first_dest,
BinaryOp binary_op) {
return Impl::inclusive_scan_custom_binary_op_exespace_impl(
"Kokkos::inclusive_scan_custom_functors_iterator_api", ex, first, last, "Kokkos::inclusive_scan_custom_functors_iterator_api", ex, first, last,
first_dest, binary_op); first_dest, binary_op);
} }
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class BinaryOp> typename OutputIteratorType, typename BinaryOp,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<
InputIteratorType, OutputIteratorType>::value, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
OutputIteratorType> Kokkos::is_execution_space_v<ExecutionSpace>,
inclusive_scan(const std::string& label, const ExecutionSpace& ex, int> = 0>
InputIteratorType first, InputIteratorType last, OutputIteratorType inclusive_scan(
OutputIteratorType first_dest, BinaryOp binary_op) { const std::string& label, const ExecutionSpace& ex, InputIteratorType first,
return Impl::inclusive_scan_custom_binary_op_impl(label, ex, first, last, InputIteratorType last, OutputIteratorType first_dest, BinaryOp binary_op) {
first_dest, binary_op); return Impl::inclusive_scan_custom_binary_op_exespace_impl(
label, ex, first, last, first_dest, binary_op);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryOp> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryOp,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto inclusive_scan(const ExecutionSpace& ex, auto inclusive_scan(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -111,14 +131,16 @@ auto inclusive_scan(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::inclusive_scan_custom_binary_op_impl( return Impl::inclusive_scan_custom_binary_op_exespace_impl(
"Kokkos::inclusive_scan_custom_functors_view_api", ex, "Kokkos::inclusive_scan_custom_functors_view_api", ex,
KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest), KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest),
binary_op); binary_op);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryOp> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryOp,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto inclusive_scan(const std::string& label, const ExecutionSpace& ex, auto inclusive_scan(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -126,67 +148,192 @@ auto inclusive_scan(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::inclusive_scan_custom_binary_op_impl( return Impl::inclusive_scan_custom_binary_op_exespace_impl(
label, ex, KE::cbegin(view_from), KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), binary_op); KE::begin(view_dest), binary_op);
} }
// overload set 3 // overload set 3
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class BinaryOp, class ValueType> typename OutputIteratorType, typename BinaryOp, typename ValueType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<
InputIteratorType, OutputIteratorType>::value, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
OutputIteratorType> Kokkos::is_execution_space_v<ExecutionSpace>,
inclusive_scan(const ExecutionSpace& ex, InputIteratorType first, int> = 0>
InputIteratorType last, OutputIteratorType first_dest, OutputIteratorType inclusive_scan(const ExecutionSpace& ex,
BinaryOp binary_op, ValueType init_value) { InputIteratorType first,
return Impl::inclusive_scan_custom_binary_op_impl( InputIteratorType last,
OutputIteratorType first_dest,
BinaryOp binary_op, ValueType init_value) {
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
return Impl::inclusive_scan_custom_binary_op_exespace_impl(
"Kokkos::inclusive_scan_custom_functors_iterator_api", ex, first, last, "Kokkos::inclusive_scan_custom_functors_iterator_api", ex, first, last,
first_dest, binary_op, init_value); first_dest, binary_op, std::move(init_value));
} }
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class BinaryOp, class ValueType> typename OutputIteratorType, typename BinaryOp, typename ValueType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<
InputIteratorType, OutputIteratorType>::value, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
OutputIteratorType> Kokkos::is_execution_space_v<ExecutionSpace>,
inclusive_scan(const std::string& label, const ExecutionSpace& ex, int> = 0>
InputIteratorType first, InputIteratorType last, OutputIteratorType inclusive_scan(const std::string& label,
OutputIteratorType first_dest, BinaryOp binary_op, const ExecutionSpace& ex,
ValueType init_value) { InputIteratorType first,
return Impl::inclusive_scan_custom_binary_op_impl( InputIteratorType last,
label, ex, first, last, first_dest, binary_op, init_value); OutputIteratorType first_dest,
BinaryOp binary_op, ValueType init_value) {
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
return Impl::inclusive_scan_custom_binary_op_exespace_impl(
label, ex, first, last, first_dest, binary_op, std::move(init_value));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryOp, typename ExecutionSpace, typename DataType1, typename... Properties1,
class ValueType> typename DataType2, typename... Properties2, typename BinaryOp,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto inclusive_scan(const ExecutionSpace& ex, auto inclusive_scan(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
BinaryOp binary_op, ValueType init_value) { BinaryOp binary_op, ValueType init_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::inclusive_scan_custom_binary_op_impl( return Impl::inclusive_scan_custom_binary_op_exespace_impl(
"Kokkos::inclusive_scan_custom_functors_view_api", ex, "Kokkos::inclusive_scan_custom_functors_view_api", ex,
KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest), KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest),
binary_op, init_value); binary_op, std::move(init_value));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryOp, typename ExecutionSpace, typename DataType1, typename... Properties1,
class ValueType> typename DataType2, typename... Properties2, typename BinaryOp,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto inclusive_scan(const std::string& label, const ExecutionSpace& ex, auto inclusive_scan(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
BinaryOp binary_op, ValueType init_value) { BinaryOp binary_op, ValueType init_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::inclusive_scan_custom_binary_op_impl( return Impl::inclusive_scan_custom_binary_op_exespace_impl(
label, ex, KE::cbegin(view_from), KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), binary_op, init_value); KE::begin(view_dest), binary_op, std::move(init_value));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
// overload set 1
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType,
std::enable_if_t<
Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
Kokkos::is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType
inclusive_scan(const TeamHandleType& teamHandle, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest) {
return Impl::inclusive_scan_default_op_team_impl(teamHandle, first, last,
first_dest);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto inclusive_scan(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental;
return Impl::inclusive_scan_default_op_team_impl(
teamHandle, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest));
}
// overload set 2 (accepting custom binary op)
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType, typename BinaryOp,
std::enable_if_t<
Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
Kokkos::is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType inclusive_scan(
const TeamHandleType& teamHandle, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest, BinaryOp binary_op) {
return Impl::inclusive_scan_custom_binary_op_team_impl(
teamHandle, first, last, first_dest, binary_op);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryOp,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto inclusive_scan(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
BinaryOp binary_op) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental;
return Impl::inclusive_scan_custom_binary_op_team_impl(
teamHandle, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), binary_op);
}
// overload set 3
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType, typename BinaryOp, typename ValueType,
std::enable_if_t<
Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
Kokkos::is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType
inclusive_scan(const TeamHandleType& teamHandle, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest,
BinaryOp binary_op, ValueType init_value) {
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
return Impl::inclusive_scan_custom_binary_op_team_impl(
teamHandle, first, last, first_dest, binary_op, std::move(init_value));
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryOp,
typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto inclusive_scan(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
BinaryOp binary_op, ValueType init_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental;
return Impl::inclusive_scan_custom_binary_op_team_impl(
teamHandle, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), binary_op, std::move(init_value));
} }
} // namespace Experimental } // namespace Experimental

61
lib/kokkos/algorithms/src/std_algorithms/Kokkos_IsPartitioned.hpp
View File

@ -23,39 +23,78 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class PredicateType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType, typename PredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_partitioned(const ExecutionSpace& ex, IteratorType first, bool is_partitioned(const ExecutionSpace& ex, IteratorType first,
IteratorType last, PredicateType p) { IteratorType last, PredicateType p) {
return Impl::is_partitioned_impl( return Impl::is_partitioned_exespace_impl(
"Kokkos::is_partitioned_iterator_api_default", ex, first, last, "Kokkos::is_partitioned_iterator_api_default", ex, first, last,
std::move(p)); std::move(p));
} }
template <class ExecutionSpace, class IteratorType, class PredicateType> template <
typename ExecutionSpace, typename IteratorType, typename PredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_partitioned(const std::string& label, const ExecutionSpace& ex, bool is_partitioned(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, PredicateType p) { IteratorType first, IteratorType last, PredicateType p) {
return Impl::is_partitioned_impl(label, ex, first, last, std::move(p)); return Impl::is_partitioned_exespace_impl(label, ex, first, last,
std::move(p));
} }
template <class ExecutionSpace, class PredicateType, class DataType, template <
class... Properties> typename ExecutionSpace, typename PredicateType, typename DataType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_partitioned(const ExecutionSpace& ex, bool is_partitioned(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
PredicateType p) { PredicateType p) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::is_partitioned_impl("Kokkos::is_partitioned_view_api_default", return Impl::is_partitioned_exespace_impl(
ex, cbegin(v), cend(v), std::move(p)); "Kokkos::is_partitioned_view_api_default", ex, cbegin(v), cend(v),
std::move(p));
} }
template <class ExecutionSpace, class PredicateType, class DataType, template <
class... Properties> typename ExecutionSpace, typename PredicateType, typename DataType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_partitioned(const std::string& label, const ExecutionSpace& ex, bool is_partitioned(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
PredicateType p) { PredicateType p) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::is_partitioned_impl(label, ex, cbegin(v), cend(v), std::move(p)); return Impl::is_partitioned_exespace_impl(label, ex, cbegin(v), cend(v),
std::move(p));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
typename PredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool is_partitioned(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
PredicateType p) {
return Impl::is_partitioned_team_impl(teamHandle, first, last, std::move(p));
}
template <typename TeamHandleType, typename PredicateType, typename DataType,
typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool is_partitioned(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v, PredicateType p) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::is_partitioned_team_impl(teamHandle, cbegin(v), cend(v),
std::move(p));
} }
} // namespace Experimental } // namespace Experimental

116
lib/kokkos/algorithms/src/std_algorithms/Kokkos_IsSorted.hpp
View File

@ -23,55 +23,73 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_sorted(const ExecutionSpace& ex, IteratorType first, bool is_sorted(const ExecutionSpace& ex, IteratorType first,
IteratorType last) { IteratorType last) {
return Impl::is_sorted_impl("Kokkos::is_sorted_iterator_api_default", ex, return Impl::is_sorted_exespace_impl("Kokkos::is_sorted_iterator_api_default",
first, last); ex, first, last);
} }
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_sorted(const std::string& label, const ExecutionSpace& ex, bool is_sorted(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last) { IteratorType first, IteratorType last) {
return Impl::is_sorted_impl(label, ex, first, last); return Impl::is_sorted_exespace_impl(label, ex, first, last);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_sorted(const ExecutionSpace& ex, bool is_sorted(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_impl("Kokkos::is_sorted_view_api_default", ex, return Impl::is_sorted_exespace_impl("Kokkos::is_sorted_view_api_default", ex,
KE::cbegin(view), KE::cend(view)); KE::cbegin(view), KE::cend(view));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_sorted(const std::string& label, const ExecutionSpace& ex, bool is_sorted(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_impl(label, ex, KE::cbegin(view), KE::cend(view)); return Impl::is_sorted_exespace_impl(label, ex, KE::cbegin(view),
KE::cend(view));
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_sorted(const ExecutionSpace& ex, IteratorType first, IteratorType last, bool is_sorted(const ExecutionSpace& ex, IteratorType first, IteratorType last,
ComparatorType comp) { ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::is_sorted_impl("Kokkos::is_sorted_iterator_api_default", ex, return Impl::is_sorted_exespace_impl("Kokkos::is_sorted_iterator_api_default",
first, last, std::move(comp)); ex, first, last, std::move(comp));
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_sorted(const std::string& label, const ExecutionSpace& ex, bool is_sorted(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, ComparatorType comp) { IteratorType first, IteratorType last, ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::is_sorted_impl(label, ex, first, last, std::move(comp)); return Impl::is_sorted_exespace_impl(label, ex, first, last, std::move(comp));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class ComparatorType> typename ExecutionSpace, typename DataType, typename... Properties,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_sorted(const ExecutionSpace& ex, bool is_sorted(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ComparatorType comp) { ComparatorType comp) {
@ -79,13 +97,15 @@ bool is_sorted(const ExecutionSpace& ex,
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_impl("Kokkos::is_sorted_view_api_default", ex, return Impl::is_sorted_exespace_impl("Kokkos::is_sorted_view_api_default", ex,
KE::cbegin(view), KE::cend(view), KE::cbegin(view), KE::cend(view),
std::move(comp)); std::move(comp));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class ComparatorType> typename ExecutionSpace, typename DataType, typename... Properties,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool is_sorted(const std::string& label, const ExecutionSpace& ex, bool is_sorted(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ComparatorType comp) { ComparatorType comp) {
@ -93,8 +113,56 @@ bool is_sorted(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_impl(label, ex, KE::cbegin(view), KE::cend(view), return Impl::is_sorted_exespace_impl(label, ex, KE::cbegin(view),
std::move(comp)); KE::cend(view), std::move(comp));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool is_sorted(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last) {
return Impl::is_sorted_team_impl(teamHandle, first, last);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool is_sorted(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_team_impl(teamHandle, KE::cbegin(view),
KE::cend(view));
}
template <typename TeamHandleType, typename IteratorType,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool is_sorted(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(teamHandle);
return Impl::is_sorted_team_impl(teamHandle, first, last, std::move(comp));
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool is_sorted(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view, ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::static_assert_is_not_openmptarget(teamHandle);
namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_team_impl(teamHandle, KE::cbegin(view), KE::cend(view),
std::move(comp));
} }
} // namespace Experimental } // namespace Experimental

115
lib/kokkos/algorithms/src/std_algorithms/Kokkos_IsSortedUntil.hpp
View File

@ -23,58 +23,78 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType is_sorted_until(const ExecutionSpace& ex, IteratorType first, IteratorType is_sorted_until(const ExecutionSpace& ex, IteratorType first,
IteratorType last) { IteratorType last) {
return Impl::is_sorted_until_impl( return Impl::is_sorted_until_exespace_impl(
"Kokkos::is_sorted_until_iterator_api_default", ex, first, last); "Kokkos::is_sorted_until_iterator_api_default", ex, first, last);
} }
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType is_sorted_until(const std::string& label, const ExecutionSpace& ex, IteratorType is_sorted_until(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last) { IteratorType first, IteratorType last) {
return Impl::is_sorted_until_impl(label, ex, first, last); return Impl::is_sorted_until_exespace_impl(label, ex, first, last);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto is_sorted_until(const ExecutionSpace& ex, auto is_sorted_until(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_until_impl("Kokkos::is_sorted_until_view_api_default", return Impl::is_sorted_until_exespace_impl(
ex, KE::begin(view), KE::end(view)); "Kokkos::is_sorted_until_view_api_default", ex, KE::begin(view),
KE::end(view));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto is_sorted_until(const std::string& label, const ExecutionSpace& ex, auto is_sorted_until(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_until_impl(label, ex, KE::begin(view), KE::end(view)); return Impl::is_sorted_until_exespace_impl(label, ex, KE::begin(view),
KE::end(view));
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType is_sorted_until(const ExecutionSpace& ex, IteratorType first, IteratorType is_sorted_until(const ExecutionSpace& ex, IteratorType first,
IteratorType last, ComparatorType comp) { IteratorType last, ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::is_sorted_until_impl( return Impl::is_sorted_until_exespace_impl(
"Kokkos::is_sorted_until_iterator_api_default", ex, first, last, "Kokkos::is_sorted_until_iterator_api_default", ex, first, last,
std::move(comp)); std::move(comp));
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType is_sorted_until(const std::string& label, const ExecutionSpace& ex, IteratorType is_sorted_until(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
ComparatorType comp) { ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::is_sorted_until_impl(label, ex, first, last, std::move(comp)); return Impl::is_sorted_until_exespace_impl(label, ex, first, last,
std::move(comp));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class ComparatorType> typename ExecutionSpace, typename DataType, typename... Properties,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto is_sorted_until(const ExecutionSpace& ex, auto is_sorted_until(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ComparatorType comp) { ComparatorType comp) {
@ -82,13 +102,15 @@ auto is_sorted_until(const ExecutionSpace& ex,
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_until_impl("Kokkos::is_sorted_until_view_api_default", return Impl::is_sorted_until_exespace_impl(
ex, KE::begin(view), KE::end(view), "Kokkos::is_sorted_until_view_api_default", ex, KE::begin(view),
std::move(comp)); KE::end(view), std::move(comp));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class ComparatorType> typename ExecutionSpace, typename DataType, typename... Properties,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto is_sorted_until(const std::string& label, const ExecutionSpace& ex, auto is_sorted_until(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ComparatorType comp) { ComparatorType comp) {
@ -96,8 +118,57 @@ auto is_sorted_until(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_until_impl(label, ex, KE::begin(view), KE::end(view), return Impl::is_sorted_until_exespace_impl(label, ex, KE::begin(view),
std::move(comp)); KE::end(view), std::move(comp));
}
//
// overload set accepting team handle
//
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType is_sorted_until(const TeamHandleType& teamHandle,
IteratorType first,
IteratorType last) {
return Impl::is_sorted_until_team_impl(teamHandle, first, last);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto is_sorted_until(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_until_team_impl(teamHandle, KE::begin(view),
KE::end(view));
}
template <typename TeamHandleType, typename IteratorType,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType is_sorted_until(const TeamHandleType& teamHandle,
IteratorType first,
IteratorType last,
ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(teamHandle);
return Impl::is_sorted_until_team_impl(teamHandle, first, last,
std::move(comp));
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto is_sorted_until(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view, ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::static_assert_is_not_openmptarget(teamHandle);
namespace KE = ::Kokkos::Experimental;
return Impl::is_sorted_until_team_impl(teamHandle, KE::begin(view),
KE::end(view), std::move(comp));
} }
} // namespace Experimental } // namespace Experimental

133
lib/kokkos/algorithms/src/std_algorithms/Kokkos_LexicographicalCompare.hpp
View File

@ -23,25 +23,34 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType1, class IteratorType2> //
// overload set accepting execution space
//
template <
class ExecutionSpace, class IteratorType1, class IteratorType2,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool lexicographical_compare(const ExecutionSpace& ex, IteratorType1 first1, bool lexicographical_compare(const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 last1, IteratorType2 first2, IteratorType1 last1, IteratorType2 first2,
IteratorType2 last2) { IteratorType2 last2) {
return Impl::lexicographical_compare_impl( return Impl::lexicographical_compare_exespace_impl(
"Kokkos::lexicographical_compare_iterator_api_default", ex, first1, last1, "Kokkos::lexicographical_compare_iterator_api_default", ex, first1, last1,
first2, last2); first2, last2);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
class ExecutionSpace, class IteratorType1, class IteratorType2,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool lexicographical_compare(const std::string& label, const ExecutionSpace& ex, bool lexicographical_compare(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2) { IteratorType2 first2, IteratorType2 last2) {
return Impl::lexicographical_compare_impl(label, ex, first1, last1, first2, return Impl::lexicographical_compare_exespace_impl(label, ex, first1, last1,
last2); first2, last2);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> class ExecutionSpace, class DataType1, class... Properties1,
class DataType2, class... Properties2,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool lexicographical_compare( bool lexicographical_compare(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
@ -50,13 +59,15 @@ bool lexicographical_compare(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::lexicographical_compare_impl( return Impl::lexicographical_compare_exespace_impl(
"Kokkos::lexicographical_compare_view_api_default", ex, KE::cbegin(view1), "Kokkos::lexicographical_compare_view_api_default", ex, KE::cbegin(view1),
KE::cend(view1), KE::cbegin(view2), KE::cend(view2)); KE::cend(view1), KE::cbegin(view2), KE::cend(view2));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> class ExecutionSpace, class DataType1, class... Properties1,
class DataType2, class... Properties2,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool lexicographical_compare( bool lexicographical_compare(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
@ -65,33 +76,39 @@ bool lexicographical_compare(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::lexicographical_compare_impl(label, ex, KE::cbegin(view1), return Impl::lexicographical_compare_exespace_impl(
KE::cend(view1), KE::cbegin(view2), label, ex, KE::cbegin(view1), KE::cend(view1), KE::cbegin(view2),
KE::cend(view2)); KE::cend(view2));
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class ComparatorType> class ExecutionSpace, class IteratorType1, class IteratorType2,
class ComparatorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool lexicographical_compare(const ExecutionSpace& ex, IteratorType1 first1, bool lexicographical_compare(const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 last1, IteratorType2 first2, IteratorType1 last1, IteratorType2 first2,
IteratorType2 last2, ComparatorType comp) { IteratorType2 last2, ComparatorType comp) {
return Impl::lexicographical_compare_impl( return Impl::lexicographical_compare_exespace_impl(
"Kokkos::lexicographical_compare_iterator_api_default", ex, first1, last1, "Kokkos::lexicographical_compare_iterator_api_default", ex, first1, last1,
first2, last2, comp); first2, last2, comp);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class ComparatorType> class ExecutionSpace, class IteratorType1, class IteratorType2,
class ComparatorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool lexicographical_compare(const std::string& label, const ExecutionSpace& ex, bool lexicographical_compare(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2, IteratorType2 first2, IteratorType2 last2,
ComparatorType comp) { ComparatorType comp) {
return Impl::lexicographical_compare_impl(label, ex, first1, last1, first2, return Impl::lexicographical_compare_exespace_impl(label, ex, first1, last1,
last2, comp); first2, last2, comp);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ComparatorType> class ExecutionSpace, class DataType1, class... Properties1,
class DataType2, class... Properties2, class ComparatorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool lexicographical_compare( bool lexicographical_compare(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
@ -100,13 +117,15 @@ bool lexicographical_compare(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::lexicographical_compare_impl( return Impl::lexicographical_compare_exespace_impl(
"Kokkos::lexicographical_compare_view_api_default", ex, KE::cbegin(view1), "Kokkos::lexicographical_compare_view_api_default", ex, KE::cbegin(view1),
KE::cend(view1), KE::cbegin(view2), KE::cend(view2), comp); KE::cend(view1), KE::cbegin(view2), KE::cend(view2), comp);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ComparatorType> class ExecutionSpace, class DataType1, class... Properties1,
class DataType2, class... Properties2, class ComparatorType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool lexicographical_compare( bool lexicographical_compare(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
@ -115,9 +134,67 @@ bool lexicographical_compare(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::lexicographical_compare_impl(label, ex, KE::cbegin(view1), return Impl::lexicographical_compare_exespace_impl(
KE::cend(view1), KE::cbegin(view2), label, ex, KE::cbegin(view1), KE::cend(view1), KE::cbegin(view2),
KE::cend(view2), comp); KE::cend(view2), comp);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <class TeamHandleType, class IteratorType1, class IteratorType2,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool lexicographical_compare(const TeamHandleType& teamHandle,
IteratorType1 first1,
IteratorType1 last1,
IteratorType2 first2,
IteratorType2 last2) {
return Impl::lexicographical_compare_team_impl(teamHandle, first1, last1,
first2, last2);
}
template <class TeamHandleType, class DataType1, class... Properties1,
class DataType2, class... Properties2,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool lexicographical_compare(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view1,
::Kokkos::View<DataType2, Properties2...>& view2) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view1);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental;
return Impl::lexicographical_compare_team_impl(
teamHandle, KE::cbegin(view1), KE::cend(view1), KE::cbegin(view2),
KE::cend(view2));
}
template <class TeamHandleType, class IteratorType1, class IteratorType2,
class ComparatorType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool lexicographical_compare(
const TeamHandleType& teamHandle, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2, ComparatorType comp) {
return Impl::lexicographical_compare_team_impl(teamHandle, first1, last1,
first2, last2, comp);
}
template <class TeamHandleType, class DataType1, class... Properties1,
class DataType2, class... Properties2, class ComparatorType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION bool lexicographical_compare(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view1,
::Kokkos::View<DataType2, Properties2...>& view2, ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view1);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental;
return Impl::lexicographical_compare_team_impl(
teamHandle, KE::cbegin(view1), KE::cend(view1), KE::cbegin(view2),
KE::cend(view2), comp);
} }
} // namespace Experimental } // namespace Experimental

105
lib/kokkos/algorithms/src/std_algorithms/Kokkos_MaxElement.hpp
View File

@ -23,81 +23,148 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto max_element(const ExecutionSpace& ex, IteratorType first, auto max_element(const ExecutionSpace& ex, IteratorType first,
IteratorType last) { IteratorType last) {
return Impl::min_or_max_element_impl<MaxFirstLoc>( return Impl::min_or_max_element_exespace_impl<MaxFirstLoc>(
"Kokkos::max_element_iterator_api_default", ex, first, last); "Kokkos::max_element_iterator_api_default", ex, first, last);
} }
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto max_element(const std::string& label, const ExecutionSpace& ex, auto max_element(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last) { IteratorType first, IteratorType last) {
return Impl::min_or_max_element_impl<MaxFirstLoc>(label, ex, first, last); return Impl::min_or_max_element_exespace_impl<MaxFirstLoc>(label, ex, first,
last);
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto max_element(const ExecutionSpace& ex, IteratorType first, auto max_element(const ExecutionSpace& ex, IteratorType first,
IteratorType last, ComparatorType comp) { IteratorType last, ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::min_or_max_element_impl<MaxFirstLocCustomComparator>( return Impl::min_or_max_element_exespace_impl<MaxFirstLocCustomComparator>(
"Kokkos::max_element_iterator_api_default", ex, first, last, "Kokkos::max_element_iterator_api_default", ex, first, last,
std::move(comp)); std::move(comp));
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto max_element(const std::string& label, const ExecutionSpace& ex, auto max_element(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, ComparatorType comp) { IteratorType first, IteratorType last, ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::min_or_max_element_impl<MaxFirstLocCustomComparator>( return Impl::min_or_max_element_exespace_impl<MaxFirstLocCustomComparator>(
label, ex, first, last, std::move(comp)); label, ex, first, last, std::move(comp));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto max_element(const ExecutionSpace& ex, auto max_element(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v) { const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::min_or_max_element_impl<MaxFirstLoc>( return Impl::min_or_max_element_exespace_impl<MaxFirstLoc>(
"Kokkos::max_element_view_api_default", ex, begin(v), end(v)); "Kokkos::max_element_view_api_default", ex, begin(v), end(v));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto max_element(const std::string& label, const ExecutionSpace& ex, auto max_element(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v) { const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::min_or_max_element_impl<MaxFirstLoc>(label, ex, begin(v), return Impl::min_or_max_element_exespace_impl<MaxFirstLoc>(label, ex,
end(v)); begin(v), end(v));
} }
template <class ExecutionSpace, class DataType, class ComparatorType, template <
class... Properties> typename ExecutionSpace, typename DataType, typename ComparatorType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto max_element(const ExecutionSpace& ex, auto max_element(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
ComparatorType comp) { ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::min_or_max_element_impl<MaxFirstLocCustomComparator>( return Impl::min_or_max_element_exespace_impl<MaxFirstLocCustomComparator>(
"Kokkos::max_element_view_api_default", ex, begin(v), end(v), "Kokkos::max_element_view_api_default", ex, begin(v), end(v),
std::move(comp)); std::move(comp));
} }
template <class ExecutionSpace, class DataType, class ComparatorType, template <
class... Properties> typename ExecutionSpace, typename DataType, typename ComparatorType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto max_element(const std::string& label, const ExecutionSpace& ex, auto max_element(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
ComparatorType comp) { ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::min_or_max_element_impl<MaxFirstLocCustomComparator>( return Impl::min_or_max_element_exespace_impl<MaxFirstLocCustomComparator>(
label, ex, begin(v), end(v), std::move(comp)); label, ex, begin(v), end(v), std::move(comp));
} }
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto max_element(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last) {
return Impl::min_or_max_element_team_impl<MaxFirstLoc>(teamHandle, first,
last);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto max_element(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::min_or_max_element_team_impl<MaxFirstLoc>(teamHandle, begin(v),
end(v));
}
template <typename TeamHandleType, typename IteratorType,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto max_element(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(teamHandle);
return Impl::min_or_max_element_team_impl<MaxFirstLocCustomComparator>(
teamHandle, first, last, std::move(comp));
}
template <typename TeamHandleType, typename DataType, typename ComparatorType,
typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto max_element(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v, ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
Impl::static_assert_is_not_openmptarget(teamHandle);
return Impl::min_or_max_element_team_impl<MaxFirstLocCustomComparator>(
teamHandle, begin(v), end(v), std::move(comp));
}
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

105
lib/kokkos/algorithms/src/std_algorithms/Kokkos_MinElement.hpp
View File

@ -23,81 +23,148 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto min_element(const ExecutionSpace& ex, IteratorType first, auto min_element(const ExecutionSpace& ex, IteratorType first,
IteratorType last) { IteratorType last) {
return Impl::min_or_max_element_impl<MinFirstLoc>( return Impl::min_or_max_element_exespace_impl<MinFirstLoc>(
"Kokkos::min_element_iterator_api_default", ex, first, last); "Kokkos::min_element_iterator_api_default", ex, first, last);
} }
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto min_element(const std::string& label, const ExecutionSpace& ex, auto min_element(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last) { IteratorType first, IteratorType last) {
return Impl::min_or_max_element_impl<MinFirstLoc>(label, ex, first, last); return Impl::min_or_max_element_exespace_impl<MinFirstLoc>(label, ex, first,
last);
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto min_element(const ExecutionSpace& ex, IteratorType first, auto min_element(const ExecutionSpace& ex, IteratorType first,
IteratorType last, ComparatorType comp) { IteratorType last, ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::min_or_max_element_impl<MinFirstLocCustomComparator>( return Impl::min_or_max_element_exespace_impl<MinFirstLocCustomComparator>(
"Kokkos::min_element_iterator_api_default", ex, first, last, "Kokkos::min_element_iterator_api_default", ex, first, last,
std::move(comp)); std::move(comp));
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto min_element(const std::string& label, const ExecutionSpace& ex, auto min_element(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, ComparatorType comp) { IteratorType first, IteratorType last, ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::min_or_max_element_impl<MinFirstLocCustomComparator>( return Impl::min_or_max_element_exespace_impl<MinFirstLocCustomComparator>(
label, ex, first, last, std::move(comp)); label, ex, first, last, std::move(comp));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto min_element(const ExecutionSpace& ex, auto min_element(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v) { const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::min_or_max_element_impl<MinFirstLoc>( return Impl::min_or_max_element_exespace_impl<MinFirstLoc>(
"Kokkos::min_element_view_api_default", ex, begin(v), end(v)); "Kokkos::min_element_view_api_default", ex, begin(v), end(v));
} }
template <class ExecutionSpace, class DataType, class ComparatorType, template <
class... Properties> typename ExecutionSpace, typename DataType, typename ComparatorType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto min_element(const ExecutionSpace& ex, auto min_element(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
ComparatorType comp) { ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::min_or_max_element_impl<MinFirstLocCustomComparator>( return Impl::min_or_max_element_exespace_impl<MinFirstLocCustomComparator>(
"Kokkos::min_element_view_api_default", ex, begin(v), end(v), "Kokkos::min_element_view_api_default", ex, begin(v), end(v),
std::move(comp)); std::move(comp));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto min_element(const std::string& label, const ExecutionSpace& ex, auto min_element(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v) { const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::min_or_max_element_impl<MinFirstLoc>(label, ex, begin(v), return Impl::min_or_max_element_exespace_impl<MinFirstLoc>(label, ex,
end(v)); begin(v), end(v));
} }
template <class ExecutionSpace, class DataType, class ComparatorType, template <
class... Properties> typename ExecutionSpace, typename DataType, typename ComparatorType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto min_element(const std::string& label, const ExecutionSpace& ex, auto min_element(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
ComparatorType comp) { ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::min_or_max_element_impl<MinFirstLocCustomComparator>( return Impl::min_or_max_element_exespace_impl<MinFirstLocCustomComparator>(
label, ex, begin(v), end(v), std::move(comp)); label, ex, begin(v), end(v), std::move(comp));
} }
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto min_element(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last) {
return Impl::min_or_max_element_team_impl<MinFirstLoc>(teamHandle, first,
last);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto min_element(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::min_or_max_element_team_impl<MinFirstLoc>(teamHandle, begin(v),
end(v));
}
template <typename TeamHandleType, typename IteratorType,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto min_element(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(teamHandle);
return Impl::min_or_max_element_team_impl<MinFirstLocCustomComparator>(
teamHandle, first, last, std::move(comp));
}
template <typename TeamHandleType, typename DataType, typename ComparatorType,
typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto min_element(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v, ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(teamHandle);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::min_or_max_element_team_impl<MinFirstLocCustomComparator>(
teamHandle, begin(v), end(v), std::move(comp));
}
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

107
lib/kokkos/algorithms/src/std_algorithms/Kokkos_MinMaxElement.hpp
View File

@ -23,82 +23,151 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto minmax_element(const ExecutionSpace& ex, IteratorType first, auto minmax_element(const ExecutionSpace& ex, IteratorType first,
IteratorType last) { IteratorType last) {
return Impl::minmax_element_impl<MinMaxFirstLastLoc>( return Impl::minmax_element_exespace_impl<MinMaxFirstLastLoc>(
"Kokkos::minmax_element_iterator_api_default", ex, first, last); "Kokkos::minmax_element_iterator_api_default", ex, first, last);
} }
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto minmax_element(const std::string& label, const ExecutionSpace& ex, auto minmax_element(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last) { IteratorType first, IteratorType last) {
return Impl::minmax_element_impl<MinMaxFirstLastLoc>(label, ex, first, last); return Impl::minmax_element_exespace_impl<MinMaxFirstLastLoc>(label, ex,
first, last);
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto minmax_element(const ExecutionSpace& ex, IteratorType first, auto minmax_element(const ExecutionSpace& ex, IteratorType first,
IteratorType last, ComparatorType comp) { IteratorType last, ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::minmax_element_impl<MinMaxFirstLastLocCustomComparator>( return Impl::minmax_element_exespace_impl<MinMaxFirstLastLocCustomComparator>(
"Kokkos::minmax_element_iterator_api_default", ex, first, last, "Kokkos::minmax_element_iterator_api_default", ex, first, last,
std::move(comp)); std::move(comp));
} }
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <
typename ExecutionSpace, typename IteratorType, typename ComparatorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto minmax_element(const std::string& label, const ExecutionSpace& ex, auto minmax_element(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
ComparatorType comp) { ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::minmax_element_impl<MinMaxFirstLastLocCustomComparator>( return Impl::minmax_element_exespace_impl<MinMaxFirstLastLocCustomComparator>(
label, ex, first, last, std::move(comp)); label, ex, first, last, std::move(comp));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto minmax_element(const ExecutionSpace& ex, auto minmax_element(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v) { const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::minmax_element_impl<MinMaxFirstLastLoc>( return Impl::minmax_element_exespace_impl<MinMaxFirstLastLoc>(
"Kokkos::minmax_element_view_api_default", ex, begin(v), end(v)); "Kokkos::minmax_element_view_api_default", ex, begin(v), end(v));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto minmax_element(const std::string& label, const ExecutionSpace& ex, auto minmax_element(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v) { const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::minmax_element_impl<MinMaxFirstLastLoc>(label, ex, begin(v), return Impl::minmax_element_exespace_impl<MinMaxFirstLastLoc>(
end(v)); label, ex, begin(v), end(v));
} }
template <class ExecutionSpace, class DataType, class ComparatorType, template <
class... Properties> typename ExecutionSpace, typename DataType, typename ComparatorType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto minmax_element(const ExecutionSpace& ex, auto minmax_element(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
ComparatorType comp) { ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::minmax_element_impl<MinMaxFirstLastLocCustomComparator>( return Impl::minmax_element_exespace_impl<MinMaxFirstLastLocCustomComparator>(
"Kokkos::minmax_element_view_api_default", ex, begin(v), end(v), "Kokkos::minmax_element_view_api_default", ex, begin(v), end(v),
std::move(comp)); std::move(comp));
} }
template <class ExecutionSpace, class DataType, class ComparatorType, template <
class... Properties> typename ExecutionSpace, typename DataType, typename ComparatorType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto minmax_element(const std::string& label, const ExecutionSpace& ex, auto minmax_element(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
ComparatorType comp) { ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::minmax_element_impl<MinMaxFirstLastLocCustomComparator>( return Impl::minmax_element_exespace_impl<MinMaxFirstLastLocCustomComparator>(
label, ex, begin(v), end(v), std::move(comp)); label, ex, begin(v), end(v), std::move(comp));
} }
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto minmax_element(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last) {
return Impl::minmax_element_team_impl<MinMaxFirstLastLoc>(teamHandle, first,
last);
}
template <typename TeamHandleType, typename IteratorType,
typename ComparatorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto minmax_element(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
ComparatorType comp) {
Impl::static_assert_is_not_openmptarget(teamHandle);
return Impl::minmax_element_team_impl<MinMaxFirstLastLocCustomComparator>(
teamHandle, first, last, std::move(comp));
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto minmax_element(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::minmax_element_team_impl<MinMaxFirstLastLoc>(teamHandle,
begin(v), end(v));
}
template <typename TeamHandleType, typename DataType, typename ComparatorType,
typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto minmax_element(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v, ComparatorType comp) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
Impl::static_assert_is_not_openmptarget(teamHandle);
return Impl::minmax_element_team_impl<MinMaxFirstLastLocCustomComparator>(
teamHandle, begin(v), end(v), std::move(comp));
}
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

145
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Mismatch.hpp
View File

@ -30,46 +30,60 @@ namespace Experimental {
// //
// makes API ambiguous (with the overload accepting views). // makes API ambiguous (with the overload accepting views).
template <class ExecutionSpace, class IteratorType1, class IteratorType2> //
// overload set accepting execution space
//
template <
class ExecutionSpace, class IteratorType1, class IteratorType2,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
::Kokkos::pair<IteratorType1, IteratorType2> mismatch(const ExecutionSpace& ex, ::Kokkos::pair<IteratorType1, IteratorType2> mismatch(const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 first1,
IteratorType1 last1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 first2,
IteratorType2 last2) { IteratorType2 last2) {
return Impl::mismatch_impl("Kokkos::mismatch_iterator_api_default", ex, return Impl::mismatch_exespace_impl("Kokkos::mismatch_iterator_api_default",
first1, last1, first2, last2); ex, first1, last1, first2, last2);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class BinaryPredicateType> class ExecutionSpace, class IteratorType1, class IteratorType2,
class BinaryPredicateType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
::Kokkos::pair<IteratorType1, IteratorType2> mismatch( ::Kokkos::pair<IteratorType1, IteratorType2> mismatch(
const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1, const ExecutionSpace& ex, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2, IteratorType2 first2, IteratorType2 last2,
BinaryPredicateType&& predicate) { BinaryPredicateType&& predicate) {
return Impl::mismatch_impl("Kokkos::mismatch_iterator_api_default", ex, return Impl::mismatch_exespace_impl(
first1, last1, first2, last2, "Kokkos::mismatch_iterator_api_default", ex, first1, last1, first2, last2,
std::forward<BinaryPredicateType>(predicate)); std::forward<BinaryPredicateType>(predicate));
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
class ExecutionSpace, class IteratorType1, class IteratorType2,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
::Kokkos::pair<IteratorType1, IteratorType2> mismatch( ::Kokkos::pair<IteratorType1, IteratorType2> mismatch(
const std::string& label, const ExecutionSpace& ex, IteratorType1 first1, const std::string& label, const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 last1, IteratorType2 first2, IteratorType2 last2) { IteratorType1 last1, IteratorType2 first2, IteratorType2 last2) {
return Impl::mismatch_impl(label, ex, first1, last1, first2, last2); return Impl::mismatch_exespace_impl(label, ex, first1, last1, first2, last2);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class BinaryPredicateType> class ExecutionSpace, class IteratorType1, class IteratorType2,
class BinaryPredicateType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
::Kokkos::pair<IteratorType1, IteratorType2> mismatch( ::Kokkos::pair<IteratorType1, IteratorType2> mismatch(
const std::string& label, const ExecutionSpace& ex, IteratorType1 first1, const std::string& label, const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 last1, IteratorType2 first2, IteratorType2 last2, IteratorType1 last1, IteratorType2 first2, IteratorType2 last2,
BinaryPredicateType&& predicate) { BinaryPredicateType&& predicate) {
return Impl::mismatch_impl(label, ex, first1, last1, first2, last2, return Impl::mismatch_exespace_impl(
std::forward<BinaryPredicateType>(predicate)); label, ex, first1, last1, first2, last2,
std::forward<BinaryPredicateType>(predicate));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> class ExecutionSpace, class DataType1, class... Properties1,
class DataType2, class... Properties2,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto mismatch(const ExecutionSpace& ex, auto mismatch(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
const ::Kokkos::View<DataType2, Properties2...>& view2) { const ::Kokkos::View<DataType2, Properties2...>& view2) {
@ -77,13 +91,15 @@ auto mismatch(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::mismatch_impl("Kokkos::mismatch_view_api_default", ex, return Impl::mismatch_exespace_impl("Kokkos::mismatch_view_api_default", ex,
KE::begin(view1), KE::end(view1), KE::begin(view2), KE::begin(view1), KE::end(view1),
KE::end(view2)); KE::begin(view2), KE::end(view2));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> class ExecutionSpace, class DataType1, class... Properties1,
class DataType2, class... Properties2, class BinaryPredicateType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto mismatch(const ExecutionSpace& ex, auto mismatch(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
const ::Kokkos::View<DataType2, Properties2...>& view2, const ::Kokkos::View<DataType2, Properties2...>& view2,
@ -92,14 +108,16 @@ auto mismatch(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::mismatch_impl("Kokkos::mismatch_view_api_default", ex, return Impl::mismatch_exespace_impl(
KE::begin(view1), KE::end(view1), KE::begin(view2), "Kokkos::mismatch_view_api_default", ex, KE::begin(view1), KE::end(view1),
KE::end(view2), KE::begin(view2), KE::end(view2),
std::forward<BinaryPredicateType>(predicate)); std::forward<BinaryPredicateType>(predicate));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> class ExecutionSpace, class DataType1, class... Properties1,
class DataType2, class... Properties2,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto mismatch(const std::string& label, const ExecutionSpace& ex, auto mismatch(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
const ::Kokkos::View<DataType2, Properties2...>& view2) { const ::Kokkos::View<DataType2, Properties2...>& view2) {
@ -107,12 +125,15 @@ auto mismatch(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::mismatch_impl(label, ex, KE::begin(view1), KE::end(view1), return Impl::mismatch_exespace_impl(label, ex, KE::begin(view1),
KE::begin(view2), KE::end(view2)); KE::end(view1), KE::begin(view2),
KE::end(view2));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> class ExecutionSpace, class DataType1, class... Properties1,
class DataType2, class... Properties2, class BinaryPredicateType,
std::enable_if_t<Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto mismatch(const std::string& label, const ExecutionSpace& ex, auto mismatch(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view1, const ::Kokkos::View<DataType1, Properties1...>& view1,
const ::Kokkos::View<DataType2, Properties2...>& view2, const ::Kokkos::View<DataType2, Properties2...>& view2,
@ -121,9 +142,65 @@ auto mismatch(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::mismatch_impl(label, ex, KE::begin(view1), KE::end(view1), return Impl::mismatch_exespace_impl(
KE::begin(view2), KE::end(view2), label, ex, KE::begin(view1), KE::end(view1), KE::begin(view2),
std::forward<BinaryPredicateType>(predicate)); KE::end(view2), std::forward<BinaryPredicateType>(predicate));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <class TeamHandleType, class IteratorType1, class IteratorType2,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION ::Kokkos::pair<IteratorType1, IteratorType2> mismatch(
const TeamHandleType& teamHandle, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2) {
return Impl::mismatch_team_impl(teamHandle, first1, last1, first2, last2);
}
template <class TeamHandleType, class IteratorType1, class IteratorType2,
class BinaryPredicateType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION ::Kokkos::pair<IteratorType1, IteratorType2> mismatch(
const TeamHandleType& teamHandle, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2,
BinaryPredicateType&& predicate) {
return Impl::mismatch_team_impl(teamHandle, first1, last1, first2, last2,
std::forward<BinaryPredicateType>(predicate));
}
template <class TeamHandleType, class DataType1, class... Properties1,
class DataType2, class... Properties2,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto mismatch(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view1,
const ::Kokkos::View<DataType2, Properties2...>& view2) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view1);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental;
return Impl::mismatch_team_impl(teamHandle, KE::begin(view1), KE::end(view1),
KE::begin(view2), KE::end(view2));
}
template <class TeamHandleType, class DataType1, class... Properties1,
class DataType2, class... Properties2, class BinaryPredicateType,
std::enable_if_t<Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto mismatch(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view1,
const ::Kokkos::View<DataType2, Properties2...>& view2,
BinaryPredicateType&& predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view1);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view2);
namespace KE = ::Kokkos::Experimental;
return Impl::mismatch_team_impl(teamHandle, KE::begin(view1), KE::end(view1),
KE::begin(view2), KE::end(view2),
std::forward<BinaryPredicateType>(predicate));
} }
} // namespace Experimental } // namespace Experimental

64
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Move.hpp
View File

@ -23,41 +23,81 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator move(const ExecutionSpace& ex, InputIterator first, OutputIterator move(const ExecutionSpace& ex, InputIterator first,
InputIterator last, OutputIterator d_first) { InputIterator last, OutputIterator d_first) {
return Impl::move_impl("Kokkos::move_iterator_api_default", ex, first, last, return Impl::move_exespace_impl("Kokkos::move_iterator_api_default", ex,
d_first); first, last, d_first);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator> template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator move(const std::string& label, const ExecutionSpace& ex, OutputIterator move(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, InputIterator first, InputIterator last,
OutputIterator d_first) { OutputIterator d_first) {
return Impl::move_impl(label, ex, first, last, d_first); return Impl::move_exespace_impl(label, ex, first, last, d_first);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto move(const ExecutionSpace& ex, auto move(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::move_impl("Kokkos::move_view_api_default", ex, begin(source), return Impl::move_exespace_impl("Kokkos::move_view_api_default", ex,
end(source), begin(dest)); begin(source), end(source), begin(dest));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto move(const std::string& label, const ExecutionSpace& ex, auto move(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::move_impl(label, ex, begin(source), end(source), begin(dest)); return Impl::move_exespace_impl(label, ex, begin(source), end(source),
begin(dest));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator move(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last,
OutputIterator d_first) {
return Impl::move_team_impl(teamHandle, first, last, d_first);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto move(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::move_team_impl(teamHandle, begin(source), end(source),
begin(dest));
} }
} // namespace Experimental } // namespace Experimental

67
lib/kokkos/algorithms/src/std_algorithms/Kokkos_MoveBackward.hpp
View File

@ -23,42 +23,83 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType1, class IteratorType2> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType2 move_backward(const ExecutionSpace& ex, IteratorType1 first, IteratorType2 move_backward(const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 d_last) { IteratorType1 last, IteratorType2 d_last) {
return Impl::move_backward_impl("Kokkos::move_backward_iterator_api_default", return Impl::move_backward_exespace_impl(
ex, first, last, d_last); "Kokkos::move_backward_iterator_api_default", ex, first, last, d_last);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto move_backward(const ExecutionSpace& ex, auto move_backward(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::move_backward_impl("Kokkos::move_backward_view_api_default", ex, return Impl::move_backward_exespace_impl(
begin(source), end(source), end(dest)); "Kokkos::move_backward_view_api_default", ex, begin(source), end(source),
end(dest));
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType2 move_backward(const std::string& label, const ExecutionSpace& ex, IteratorType2 move_backward(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first, IteratorType1 last, IteratorType1 first, IteratorType1 last,
IteratorType2 d_last) { IteratorType2 d_last) {
return Impl::move_backward_impl(label, ex, first, last, d_last); return Impl::move_backward_exespace_impl(label, ex, first, last, d_last);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto move_backward(const std::string& label, const ExecutionSpace& ex, auto move_backward(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::move_backward_impl(label, ex, begin(source), end(source), return Impl::move_backward_exespace_impl(label, ex, begin(source),
end(dest)); end(source), end(dest));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType2 move_backward(const TeamHandleType& teamHandle,
IteratorType1 first,
IteratorType1 last,
IteratorType2 d_last) {
return Impl::move_backward_team_impl(teamHandle, first, last, d_last);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto move_backward(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::move_backward_team_impl(teamHandle, begin(source), end(source),
end(dest));
} }
} // namespace Experimental } // namespace Experimental

65
lib/kokkos/algorithms/src/std_algorithms/Kokkos_NoneOf.hpp
View File

@ -23,41 +23,80 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class Predicate> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool none_of(const ExecutionSpace& ex, IteratorType first, IteratorType last, bool none_of(const ExecutionSpace& ex, IteratorType first, IteratorType last,
Predicate predicate) { Predicate predicate) {
return Impl::none_of_impl("Kokkos::none_of_iterator_api_default", ex, first, return Impl::none_of_exespace_impl("Kokkos::none_of_iterator_api_default", ex,
last, predicate); first, last, predicate);
} }
template <class ExecutionSpace, class IteratorType, class Predicate> template <
typename ExecutionSpace, typename IteratorType, typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool none_of(const std::string& label, const ExecutionSpace& ex, bool none_of(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, Predicate predicate) { IteratorType first, IteratorType last, Predicate predicate) {
return Impl::none_of_impl(label, ex, first, last, predicate); return Impl::none_of_exespace_impl(label, ex, first, last, predicate);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class Predicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool none_of(const ExecutionSpace& ex, bool none_of(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::none_of_impl("Kokkos::none_of_view_api_default", ex, return Impl::none_of_exespace_impl("Kokkos::none_of_view_api_default", ex,
KE::cbegin(v), KE::cend(v), std::move(predicate)); KE::cbegin(v), KE::cend(v),
std::move(predicate));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class Predicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename Predicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
bool none_of(const std::string& label, const ExecutionSpace& ex, bool none_of(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) { Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::none_of_impl(label, ex, KE::cbegin(v), KE::cend(v), return Impl::none_of_exespace_impl(label, ex, KE::cbegin(v), KE::cend(v),
std::move(predicate)); std::move(predicate));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType, typename Predicate>
KOKKOS_FUNCTION
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, bool>
none_of(const TeamHandleType& teamHandle, IteratorType first,
IteratorType last, Predicate predicate) {
return Impl::none_of_team_impl(teamHandle, first, last, predicate);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename Predicate>
KOKKOS_FUNCTION
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, bool>
none_of(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v,
Predicate predicate) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
namespace KE = ::Kokkos::Experimental;
return Impl::none_of_team_impl(teamHandle, KE::cbegin(v), KE::cend(v),
std::move(predicate));
} }
} // namespace Experimental } // namespace Experimental

90
lib/kokkos/algorithms/src/std_algorithms/Kokkos_PartitionCopy.hpp
View File

@ -23,57 +23,103 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIteratorType, //
class OutputIteratorTrueType, class OutputIteratorFalseType, // overload set accepting execution space
class PredicateType> //
template <
typename ExecutionSpace, typename InputIteratorType,
typename OutputIteratorTrueType, typename OutputIteratorFalseType,
typename PredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
::Kokkos::pair<OutputIteratorTrueType, OutputIteratorFalseType> partition_copy( ::Kokkos::pair<OutputIteratorTrueType, OutputIteratorFalseType> partition_copy(
const ExecutionSpace& ex, InputIteratorType from_first, const ExecutionSpace& ex, InputIteratorType from_first,
InputIteratorType from_last, OutputIteratorTrueType to_first_true, InputIteratorType from_last, OutputIteratorTrueType to_first_true,
OutputIteratorFalseType to_first_false, PredicateType p) { OutputIteratorFalseType to_first_false, PredicateType p) {
return Impl::partition_copy_impl( return Impl::partition_copy_exespace_impl(
"Kokkos::partition_copy_iterator_api_default", ex, from_first, from_last, "Kokkos::partition_copy_iterator_api_default", ex, from_first, from_last,
to_first_true, to_first_false, std::move(p)); to_first_true, to_first_false, std::move(p));
} }
template <class ExecutionSpace, class InputIteratorType, template <
class OutputIteratorTrueType, class OutputIteratorFalseType, typename ExecutionSpace, typename InputIteratorType,
class PredicateType> typename OutputIteratorTrueType, typename OutputIteratorFalseType,
typename PredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
::Kokkos::pair<OutputIteratorTrueType, OutputIteratorFalseType> partition_copy( ::Kokkos::pair<OutputIteratorTrueType, OutputIteratorFalseType> partition_copy(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
InputIteratorType from_first, InputIteratorType from_last, InputIteratorType from_first, InputIteratorType from_last,
OutputIteratorTrueType to_first_true, OutputIteratorTrueType to_first_true,
OutputIteratorFalseType to_first_false, PredicateType p) { OutputIteratorFalseType to_first_false, PredicateType p) {
return Impl::partition_copy_impl(label, ex, from_first, from_last, return Impl::partition_copy_exespace_impl(label, ex, from_first, from_last,
to_first_true, to_first_false, std::move(p)); to_first_true, to_first_false,
std::move(p));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class DataType3, typename ExecutionSpace, typename DataType1, typename... Properties1,
class... Properties3, class PredicateType> typename DataType2, typename... Properties2, typename DataType3,
typename... Properties3, typename PredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto partition_copy( auto partition_copy(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest_true, const ::Kokkos::View<DataType2, Properties2...>& view_dest_true,
const ::Kokkos::View<DataType3, Properties3...>& view_dest_false, const ::Kokkos::View<DataType3, Properties3...>& view_dest_false,
PredicateType p) { PredicateType p) {
return Impl::partition_copy_impl("Kokkos::partition_copy_view_api_default", return Impl::partition_copy_exespace_impl(
ex, cbegin(view_from), cend(view_from), "Kokkos::partition_copy_view_api_default", ex, cbegin(view_from),
begin(view_dest_true), cend(view_from), begin(view_dest_true), begin(view_dest_false),
begin(view_dest_false), std::move(p)); std::move(p));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class DataType3, typename ExecutionSpace, typename DataType1, typename... Properties1,
class... Properties3, class PredicateType> typename DataType2, typename... Properties2, typename DataType3,
typename... Properties3, typename PredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto partition_copy( auto partition_copy(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest_true, const ::Kokkos::View<DataType2, Properties2...>& view_dest_true,
const ::Kokkos::View<DataType3, Properties3...>& view_dest_false, const ::Kokkos::View<DataType3, Properties3...>& view_dest_false,
PredicateType p) { PredicateType p) {
return Impl::partition_copy_impl(label, ex, cbegin(view_from), return Impl::partition_copy_exespace_impl(
cend(view_from), begin(view_dest_true), label, ex, cbegin(view_from), cend(view_from), begin(view_dest_true),
begin(view_dest_false), std::move(p)); begin(view_dest_false), std::move(p));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorTrueType, typename OutputIteratorFalseType,
typename PredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION ::Kokkos::pair<OutputIteratorTrueType, OutputIteratorFalseType>
partition_copy(const TeamHandleType& teamHandle, InputIteratorType from_first,
InputIteratorType from_last,
OutputIteratorTrueType to_first_true,
OutputIteratorFalseType to_first_false, PredicateType p) {
return Impl::partition_copy_team_impl(teamHandle, from_first, from_last,
to_first_true, to_first_false,
std::move(p));
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename DataType3,
typename... Properties3, typename PredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto partition_copy(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest_true,
const ::Kokkos::View<DataType3, Properties3...>& view_dest_false,
PredicateType p) {
return Impl::partition_copy_team_impl(teamHandle, cbegin(view_from),
cend(view_from), begin(view_dest_true),
begin(view_dest_false), std::move(p));
} }
} // namespace Experimental } // namespace Experimental

62
lib/kokkos/algorithms/src/std_algorithms/Kokkos_PartitionPoint.hpp
View File

@ -23,38 +23,78 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType, class UnaryPredicate> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType, typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType partition_point(const ExecutionSpace& ex, IteratorType first, IteratorType partition_point(const ExecutionSpace& ex, IteratorType first,
IteratorType last, UnaryPredicate p) { IteratorType last, UnaryPredicate p) {
return Impl::partition_point_impl( return Impl::partition_point_exespace_impl(
"Kokkos::partitioned_point_iterator_api_default", ex, first, last, "Kokkos::partitioned_point_iterator_api_default", ex, first, last,
std::move(p)); std::move(p));
} }
template <class ExecutionSpace, class IteratorType, class UnaryPredicate> template <
typename ExecutionSpace, typename IteratorType, typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType partition_point(const std::string& label, const ExecutionSpace& ex, IteratorType partition_point(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
UnaryPredicate p) { UnaryPredicate p) {
return Impl::partition_point_impl(label, ex, first, last, std::move(p)); return Impl::partition_point_exespace_impl(label, ex, first, last,
std::move(p));
} }
template <class ExecutionSpace, class UnaryPredicate, class DataType, template <
class... Properties> typename ExecutionSpace, typename UnaryPredicate, typename DataType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto partition_point(const std::string& label, const ExecutionSpace& ex, auto partition_point(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
UnaryPredicate p) { UnaryPredicate p) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::partition_point_impl(label, ex, begin(v), end(v), std::move(p)); return Impl::partition_point_exespace_impl(label, ex, begin(v), end(v),
std::move(p));
} }
template <class ExecutionSpace, class UnaryPredicate, class DataType, template <
class... Properties> typename ExecutionSpace, typename UnaryPredicate, typename DataType,
typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto partition_point(const ExecutionSpace& ex, auto partition_point(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& v, const ::Kokkos::View<DataType, Properties...>& v,
UnaryPredicate p) { UnaryPredicate p) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::partition_point_impl("Kokkos::partition_point_view_api_default", return Impl::partition_point_exespace_impl(
ex, begin(v), end(v), std::move(p)); "Kokkos::partition_point_view_api_default", ex, begin(v), end(v),
std::move(p));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType partition_point(const TeamHandleType& teamHandle,
IteratorType first,
IteratorType last,
UnaryPredicate p) {
return Impl::partition_point_team_impl(teamHandle, first, last, std::move(p));
}
template <typename TeamHandleType, typename UnaryPredicate, typename DataType,
typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto partition_point(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& v, UnaryPredicate p) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(v);
return Impl::partition_point_team_impl(teamHandle, begin(v), end(v),
std::move(p));
} }
} // namespace Experimental } // namespace Experimental

203
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Reduce.hpp
View File

@ -23,28 +23,38 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// //
// overload set 1 // overload set 1
// //
template <class ExecutionSpace, class IteratorType> template <typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
typename IteratorType::value_type reduce(const ExecutionSpace& ex, typename IteratorType::value_type reduce(const ExecutionSpace& ex,
IteratorType first, IteratorType first,
IteratorType last) { IteratorType last) {
return Impl::reduce_default_functors_impl( return Impl::reduce_default_functors_exespace_impl(
"Kokkos::reduce_default_functors_iterator_api", ex, first, last, "Kokkos::reduce_default_functors_iterator_api", ex, first, last,
typename IteratorType::value_type()); typename IteratorType::value_type());
} }
template <class ExecutionSpace, class IteratorType> template <typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
typename IteratorType::value_type reduce(const std::string& label, typename IteratorType::value_type reduce(const std::string& label,
const ExecutionSpace& ex, const ExecutionSpace& ex,
IteratorType first, IteratorType first,
IteratorType last) { IteratorType last) {
return Impl::reduce_default_functors_impl( return Impl::reduce_default_functors_exespace_impl(
label, ex, first, last, typename IteratorType::value_type()); label, ex, first, last, typename IteratorType::value_type());
} }
template <class ExecutionSpace, class DataType, class... Properties> template <typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto reduce(const ExecutionSpace& ex, auto reduce(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
@ -53,12 +63,14 @@ auto reduce(const ExecutionSpace& ex,
using view_type = ::Kokkos::View<DataType, Properties...>; using view_type = ::Kokkos::View<DataType, Properties...>;
using value_type = typename view_type::value_type; using value_type = typename view_type::value_type;
return Impl::reduce_default_functors_impl( return Impl::reduce_default_functors_exespace_impl(
"Kokkos::reduce_default_functors_view_api", ex, KE::cbegin(view), "Kokkos::reduce_default_functors_view_api", ex, KE::cbegin(view),
KE::cend(view), value_type()); KE::cend(view), value_type());
} }
template <class ExecutionSpace, class DataType, class... Properties> template <typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto reduce(const std::string& label, const ExecutionSpace& ex, auto reduce(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
@ -67,37 +79,43 @@ auto reduce(const std::string& label, const ExecutionSpace& ex,
using view_type = ::Kokkos::View<DataType, Properties...>; using view_type = ::Kokkos::View<DataType, Properties...>;
using value_type = typename view_type::value_type; using value_type = typename view_type::value_type;
return Impl::reduce_default_functors_impl(label, ex, KE::cbegin(view), return Impl::reduce_default_functors_exespace_impl(
KE::cend(view), value_type()); label, ex, KE::cbegin(view), KE::cend(view), value_type());
} }
// //
// overload set2: // overload set2:
// //
template <class ExecutionSpace, class IteratorType, class ValueType> template <typename ExecutionSpace, typename IteratorType, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType reduce(const ExecutionSpace& ex, IteratorType first, ValueType reduce(const ExecutionSpace& ex, IteratorType first,
IteratorType last, ValueType init_reduction_value) { IteratorType last, ValueType init_reduction_value) {
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::reduce_default_functors_impl( return Impl::reduce_default_functors_exespace_impl(
"Kokkos::reduce_default_functors_iterator_api", ex, first, last, "Kokkos::reduce_default_functors_iterator_api", ex, first, last,
init_reduction_value); init_reduction_value);
} }
template <class ExecutionSpace, class IteratorType, class ValueType> template <typename ExecutionSpace, typename IteratorType, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType reduce(const std::string& label, const ExecutionSpace& ex, ValueType reduce(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
ValueType init_reduction_value) { ValueType init_reduction_value) {
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::reduce_default_functors_impl(label, ex, first, last, return Impl::reduce_default_functors_exespace_impl(label, ex, first, last,
init_reduction_value); init_reduction_value);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class ValueType> typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType reduce(const ExecutionSpace& ex, ValueType reduce(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ValueType init_reduction_value) { ValueType init_reduction_value) {
@ -107,13 +125,15 @@ ValueType reduce(const ExecutionSpace& ex,
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::reduce_default_functors_impl( return Impl::reduce_default_functors_exespace_impl(
"Kokkos::reduce_default_functors_view_api", ex, KE::cbegin(view), "Kokkos::reduce_default_functors_view_api", ex, KE::cbegin(view),
KE::cend(view), init_reduction_value); KE::cend(view), init_reduction_value);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class ValueType> typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType reduce(const std::string& label, const ExecutionSpace& ex, ValueType reduce(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ValueType init_reduction_value) { ValueType init_reduction_value) {
@ -123,40 +143,46 @@ ValueType reduce(const std::string& label, const ExecutionSpace& ex,
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::reduce_default_functors_impl( return Impl::reduce_default_functors_exespace_impl(
label, ex, KE::cbegin(view), KE::cend(view), init_reduction_value); label, ex, KE::cbegin(view), KE::cend(view), init_reduction_value);
} }
// //
// overload set 3 // overload set 3
// //
template <class ExecutionSpace, class IteratorType, class ValueType, template <typename ExecutionSpace, typename IteratorType, typename ValueType,
class BinaryOp> typename BinaryOp,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType reduce(const ExecutionSpace& ex, IteratorType first, ValueType reduce(const ExecutionSpace& ex, IteratorType first,
IteratorType last, ValueType init_reduction_value, IteratorType last, ValueType init_reduction_value,
BinaryOp joiner) { BinaryOp joiner) {
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::reduce_custom_functors_impl( return Impl::reduce_custom_functors_exespace_impl(
"Kokkos::reduce_default_functors_iterator_api", ex, first, last, "Kokkos::reduce_default_functors_iterator_api", ex, first, last,
init_reduction_value, joiner); init_reduction_value, joiner);
} }
template <class ExecutionSpace, class IteratorType, class ValueType, template <typename ExecutionSpace, typename IteratorType, typename ValueType,
class BinaryOp> typename BinaryOp,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType reduce(const std::string& label, const ExecutionSpace& ex, ValueType reduce(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
ValueType init_reduction_value, BinaryOp joiner) { ValueType init_reduction_value, BinaryOp joiner) {
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::reduce_custom_functors_impl(label, ex, first, last, return Impl::reduce_custom_functors_exespace_impl(
init_reduction_value, joiner); label, ex, first, last, init_reduction_value, joiner);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class ValueType, class BinaryOp> typename ValueType, typename BinaryOp,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType reduce(const ExecutionSpace& ex, ValueType reduce(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ValueType init_reduction_value, BinaryOp joiner) { ValueType init_reduction_value, BinaryOp joiner) {
@ -166,13 +192,15 @@ ValueType reduce(const ExecutionSpace& ex,
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::reduce_custom_functors_impl( return Impl::reduce_custom_functors_exespace_impl(
"Kokkos::reduce_custom_functors_view_api", ex, KE::cbegin(view), "Kokkos::reduce_custom_functors_view_api", ex, KE::cbegin(view),
KE::cend(view), init_reduction_value, joiner); KE::cend(view), init_reduction_value, joiner);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class ValueType, class BinaryOp> typename ValueType, typename BinaryOp,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType reduce(const std::string& label, const ExecutionSpace& ex, ValueType reduce(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ValueType init_reduction_value, BinaryOp joiner) { ValueType init_reduction_value, BinaryOp joiner) {
@ -182,9 +210,114 @@ ValueType reduce(const std::string& label, const ExecutionSpace& ex,
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::reduce_custom_functors_impl(label, ex, KE::cbegin(view), return Impl::reduce_custom_functors_exespace_impl(
KE::cend(view), init_reduction_value, label, ex, KE::cbegin(view), KE::cend(view), init_reduction_value,
joiner); joiner);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
//
// overload set 1
//
template <
typename TeamHandleType, typename IteratorType,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION typename IteratorType::value_type reduce(
const TeamHandleType& teamHandle, IteratorType first, IteratorType last) {
return Impl::reduce_default_functors_team_impl(
teamHandle, first, last, typename IteratorType::value_type());
}
template <
typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION auto reduce(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view) {
namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
using view_type = ::Kokkos::View<DataType, Properties...>;
using value_type = typename view_type::value_type;
return Impl::reduce_default_functors_team_impl(teamHandle, KE::cbegin(view),
KE::cend(view), value_type());
}
//
// overload set2:
//
template <
typename TeamHandleType, typename IteratorType, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION ValueType reduce(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
ValueType init_reduction_value) {
static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible.");
return Impl::reduce_default_functors_team_impl(teamHandle, first, last,
init_reduction_value);
}
template <
typename TeamHandleType, typename DataType, typename... Properties,
typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION ValueType
reduce(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view,
ValueType init_reduction_value) {
static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::reduce_default_functors_team_impl(
teamHandle, KE::cbegin(view), KE::cend(view), init_reduction_value);
}
//
// overload set 3
//
template <
typename TeamHandleType, typename IteratorType, typename ValueType,
typename BinaryOp,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION ValueType reduce(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
ValueType init_reduction_value,
BinaryOp joiner) {
static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible.");
return Impl::reduce_custom_functors_team_impl(teamHandle, first, last,
init_reduction_value, joiner);
}
template <
typename TeamHandleType, typename DataType, typename... Properties,
typename ValueType, typename BinaryOp,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION ValueType
reduce(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view,
ValueType init_reduction_value, BinaryOp joiner) {
static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::reduce_custom_functors_team_impl(teamHandle, KE::cbegin(view),
KE::cend(view),
init_reduction_value, joiner);
} }
} // namespace Experimental } // namespace Experimental

64
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Remove.hpp
View File

@ -23,38 +23,74 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class Iterator, class ValueType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename Iterator, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
Iterator remove(const ExecutionSpace& ex, Iterator first, Iterator last, Iterator remove(const ExecutionSpace& ex, Iterator first, Iterator last,
const ValueType& value) { const ValueType& value) {
return Impl::remove_impl("Kokkos::remove_iterator_api_default", ex, first, return Impl::remove_exespace_impl("Kokkos::remove_iterator_api_default", ex,
last, value); first, last, value);
} }
template <class ExecutionSpace, class Iterator, class ValueType> template <
typename ExecutionSpace, typename Iterator, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
Iterator remove(const std::string& label, const ExecutionSpace& ex, Iterator remove(const std::string& label, const ExecutionSpace& ex,
Iterator first, Iterator last, const ValueType& value) { Iterator first, Iterator last, const ValueType& value) {
return Impl::remove_impl(label, ex, first, last, value); return Impl::remove_exespace_impl(label, ex, first, last, value);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class ValueType> typename ExecutionSpace, typename DataType, typename... Properties,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto remove(const ExecutionSpace& ex, auto remove(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
const ValueType& value) { const ValueType& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::remove_impl("Kokkos::remove_iterator_api_default", ex, return Impl::remove_exespace_impl("Kokkos::remove_iterator_api_default", ex,
::Kokkos::Experimental::begin(view), ::Kokkos::Experimental::begin(view),
::Kokkos::Experimental::end(view), value); ::Kokkos::Experimental::end(view), value);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class ValueType> typename ExecutionSpace, typename DataType, typename... Properties,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto remove(const std::string& label, const ExecutionSpace& ex, auto remove(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
const ValueType& value) { const ValueType& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::remove_impl(label, ex, ::Kokkos::Experimental::begin(view), return Impl::remove_exespace_impl(label, ex,
::Kokkos::Experimental::end(view), value); ::Kokkos::Experimental::begin(view),
::Kokkos::Experimental::end(view), value);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename Iterator, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION Iterator remove(const TeamHandleType& teamHandle,
Iterator first, Iterator last,
const ValueType& value) {
return Impl::remove_team_impl(teamHandle, first, last, value);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto remove(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view,
const ValueType& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::remove_team_impl(teamHandle, ::Kokkos::Experimental::begin(view),
::Kokkos::Experimental::end(view), value);
} }
} // namespace Experimental } // namespace Experimental

82
lib/kokkos/algorithms/src/std_algorithms/Kokkos_RemoveCopy.hpp
View File

@ -23,26 +23,36 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator, //
class ValueType> // overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator remove_copy(const ExecutionSpace& ex, InputIterator first_from, OutputIterator remove_copy(const ExecutionSpace& ex, InputIterator first_from,
InputIterator last_from, OutputIterator first_dest, InputIterator last_from, OutputIterator first_dest,
const ValueType& value) { const ValueType& value) {
return Impl::remove_copy_impl("Kokkos::remove_copy_iterator_api_default", ex, return Impl::remove_copy_exespace_impl(
first_from, last_from, first_dest, value); "Kokkos::remove_copy_iterator_api_default", ex, first_from, last_from,
first_dest, value);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator, template <
class ValueType> typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator remove_copy(const std::string& label, const ExecutionSpace& ex, OutputIterator remove_copy(const std::string& label, const ExecutionSpace& ex,
InputIterator first_from, InputIterator last_from, InputIterator first_from, InputIterator last_from,
OutputIterator first_dest, const ValueType& value) { OutputIterator first_dest, const ValueType& value) {
return Impl::remove_copy_impl(label, ex, first_from, last_from, first_dest, return Impl::remove_copy_exespace_impl(label, ex, first_from, last_from,
value); first_dest, value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto remove_copy(const ExecutionSpace& ex, auto remove_copy(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -50,15 +60,17 @@ auto remove_copy(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
return Impl::remove_copy_impl("Kokkos::remove_copy_iterator_api_default", ex, return Impl::remove_copy_exespace_impl(
::Kokkos::Experimental::cbegin(view_from), "Kokkos::remove_copy_iterator_api_default", ex,
::Kokkos::Experimental::cend(view_from), ::Kokkos::Experimental::cbegin(view_from),
::Kokkos::Experimental::begin(view_dest), ::Kokkos::Experimental::cend(view_from),
value); ::Kokkos::Experimental::begin(view_dest), value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto remove_copy(const std::string& label, const ExecutionSpace& ex, auto remove_copy(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -66,12 +78,46 @@ auto remove_copy(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
return Impl::remove_copy_impl( return Impl::remove_copy_exespace_impl(
label, ex, ::Kokkos::Experimental::cbegin(view_from), label, ex, ::Kokkos::Experimental::cbegin(view_from),
::Kokkos::Experimental::cend(view_from), ::Kokkos::Experimental::cend(view_from),
::Kokkos::Experimental::begin(view_dest), value); ::Kokkos::Experimental::begin(view_dest), value);
} }
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator remove_copy(const TeamHandleType& teamHandle,
InputIterator first_from,
InputIterator last_from,
OutputIterator first_dest,
const ValueType& value) {
return Impl::remove_copy_team_impl(teamHandle, first_from, last_from,
first_dest, value);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto remove_copy(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
const ValueType& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
return Impl::remove_copy_team_impl(
teamHandle, ::Kokkos::Experimental::cbegin(view_from),
::Kokkos::Experimental::cend(view_from),
::Kokkos::Experimental::begin(view_dest), value);
}
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

71
lib/kokkos/algorithms/src/std_algorithms/Kokkos_RemoveCopyIf.hpp
View File

@ -23,30 +23,39 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator, //
class UnaryPredicate> // overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator remove_copy_if(const ExecutionSpace& ex, OutputIterator remove_copy_if(const ExecutionSpace& ex,
InputIterator first_from, InputIterator last_from, InputIterator first_from, InputIterator last_from,
OutputIterator first_dest, OutputIterator first_dest,
const UnaryPredicate& pred) { const UnaryPredicate& pred) {
return Impl::remove_copy_if_impl( return Impl::remove_copy_if_exespace_impl(
"Kokkos::remove_copy_if_iterator_api_default", ex, first_from, last_from, "Kokkos::remove_copy_if_iterator_api_default", ex, first_from, last_from,
first_dest, pred); first_dest, pred);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator, template <
class UnaryPredicate> typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator remove_copy_if(const std::string& label, OutputIterator remove_copy_if(const std::string& label,
const ExecutionSpace& ex, const ExecutionSpace& ex,
InputIterator first_from, InputIterator last_from, InputIterator first_from, InputIterator last_from,
OutputIterator first_dest, OutputIterator first_dest,
const UnaryPredicate& pred) { const UnaryPredicate& pred) {
return Impl::remove_copy_if_impl(label, ex, first_from, last_from, first_dest, return Impl::remove_copy_if_exespace_impl(label, ex, first_from, last_from,
pred); first_dest, pred);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class UnaryPredicate> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto remove_copy_if(const ExecutionSpace& ex, auto remove_copy_if(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -54,15 +63,17 @@ auto remove_copy_if(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
return Impl::remove_copy_if_impl( return Impl::remove_copy_if_exespace_impl(
"Kokkos::remove_copy_if_iterator_api_default", ex, "Kokkos::remove_copy_if_iterator_api_default", ex,
::Kokkos::Experimental::cbegin(view_from), ::Kokkos::Experimental::cbegin(view_from),
::Kokkos::Experimental::cend(view_from), ::Kokkos::Experimental::cend(view_from),
::Kokkos::Experimental::begin(view_dest), pred); ::Kokkos::Experimental::begin(view_dest), pred);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class UnaryPredicate> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto remove_copy_if(const std::string& label, const ExecutionSpace& ex, auto remove_copy_if(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -70,12 +81,46 @@ auto remove_copy_if(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
return Impl::remove_copy_if_impl( return Impl::remove_copy_if_exespace_impl(
label, ex, ::Kokkos::Experimental::cbegin(view_from), label, ex, ::Kokkos::Experimental::cbegin(view_from),
::Kokkos::Experimental::cend(view_from), ::Kokkos::Experimental::cend(view_from),
::Kokkos::Experimental::begin(view_dest), pred); ::Kokkos::Experimental::begin(view_dest), pred);
} }
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator, typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator remove_copy_if(const TeamHandleType& teamHandle,
InputIterator first_from,
InputIterator last_from,
OutputIterator first_dest,
const UnaryPredicate& pred) {
return Impl::remove_copy_if_team_impl(teamHandle, first_from, last_from,
first_dest, pred);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto remove_copy_if(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
const UnaryPredicate& pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
return Impl::remove_copy_if_team_impl(
teamHandle, ::Kokkos::Experimental::cbegin(view_from),
::Kokkos::Experimental::cend(view_from),
::Kokkos::Experimental::begin(view_dest), pred);
}
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

66
lib/kokkos/algorithms/src/std_algorithms/Kokkos_RemoveIf.hpp
View File

@ -23,39 +23,77 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class Iterator, class UnaryPredicate> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename Iterator, typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
Iterator remove_if(const ExecutionSpace& ex, Iterator first, Iterator last, Iterator remove_if(const ExecutionSpace& ex, Iterator first, Iterator last,
UnaryPredicate pred) { UnaryPredicate pred) {
return Impl::remove_if_impl("Kokkos::remove_if_iterator_api_default", ex, return Impl::remove_if_exespace_impl("Kokkos::remove_if_iterator_api_default",
first, last, pred); ex, first, last, pred);
} }
template <class ExecutionSpace, class Iterator, class UnaryPredicate> template <
typename ExecutionSpace, typename Iterator, typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
Iterator remove_if(const std::string& label, const ExecutionSpace& ex, Iterator remove_if(const std::string& label, const ExecutionSpace& ex,
Iterator first, Iterator last, UnaryPredicate pred) { Iterator first, Iterator last, UnaryPredicate pred) {
return Impl::remove_if_impl(label, ex, first, last, pred); return Impl::remove_if_exespace_impl(label, ex, first, last, pred);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class UnaryPredicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto remove_if(const ExecutionSpace& ex, auto remove_if(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
UnaryPredicate pred) { UnaryPredicate pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::remove_if_impl("Kokkos::remove_if_iterator_api_default", ex, return Impl::remove_if_exespace_impl("Kokkos::remove_if_iterator_api_default",
::Kokkos::Experimental::begin(view), ex, ::Kokkos::Experimental::begin(view),
::Kokkos::Experimental::end(view), pred); ::Kokkos::Experimental::end(view), pred);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class UnaryPredicate> typename ExecutionSpace, typename DataType, typename... Properties,
typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto remove_if(const std::string& label, const ExecutionSpace& ex, auto remove_if(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
UnaryPredicate pred) { UnaryPredicate pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::remove_if_impl(label, ex, ::Kokkos::Experimental::begin(view), return Impl::remove_if_exespace_impl(label, ex,
::Kokkos::Experimental::end(view), pred); ::Kokkos::Experimental::begin(view),
::Kokkos::Experimental::end(view), pred);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename Iterator, typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION Iterator remove_if(const TeamHandleType& teamHandle,
Iterator first, Iterator last,
UnaryPredicate pred) {
return Impl::remove_if_team_impl(teamHandle, first, last, pred);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename UnaryPredicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto remove_if(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view, UnaryPredicate pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::remove_if_team_impl(teamHandle,
::Kokkos::Experimental::begin(view),
::Kokkos::Experimental::end(view), pred);
} }
} // namespace Experimental } // namespace Experimental

63
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Replace.hpp
View File

@ -23,40 +23,77 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class Iterator, class ValueType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename Iterator, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void replace(const ExecutionSpace& ex, Iterator first, Iterator last, void replace(const ExecutionSpace& ex, Iterator first, Iterator last,
const ValueType& old_value, const ValueType& new_value) { const ValueType& old_value, const ValueType& new_value) {
return Impl::replace_impl("Kokkos::replace_iterator_api", ex, first, last, Impl::replace_exespace_impl("Kokkos::replace_iterator_api", ex, first, last,
old_value, new_value); old_value, new_value);
} }
template <class ExecutionSpace, class Iterator, class ValueType> template <
typename ExecutionSpace, typename Iterator, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void replace(const std::string& label, const ExecutionSpace& ex, Iterator first, void replace(const std::string& label, const ExecutionSpace& ex, Iterator first,
Iterator last, const ValueType& old_value, Iterator last, const ValueType& old_value,
const ValueType& new_value) { const ValueType& new_value) {
return Impl::replace_impl(label, ex, first, last, old_value, new_value); Impl::replace_exespace_impl(label, ex, first, last, old_value, new_value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void replace(const ExecutionSpace& ex, void replace(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ValueType& old_value, const ValueType& new_value) { const ValueType& old_value, const ValueType& new_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::replace_impl("Kokkos::replace_view_api", ex, KE::begin(view), Impl::replace_exespace_impl("Kokkos::replace_view_api", ex, KE::begin(view),
KE::end(view), old_value, new_value); KE::end(view), old_value, new_value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void replace(const std::string& label, const ExecutionSpace& ex, void replace(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ValueType& old_value, const ValueType& new_value) { const ValueType& old_value, const ValueType& new_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::replace_impl(label, ex, KE::begin(view), KE::end(view), Impl::replace_exespace_impl(label, ex, KE::begin(view), KE::end(view),
old_value, new_value); old_value, new_value);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename Iterator, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void replace(const TeamHandleType& teamHandle, Iterator first,
Iterator last, const ValueType& old_value,
const ValueType& new_value) {
Impl::replace_team_impl(teamHandle, first, last, old_value, new_value);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void replace(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view,
const ValueType& old_value, const ValueType& new_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental;
Impl::replace_team_impl(teamHandle, KE::begin(view), KE::end(view), old_value,
new_value);
} }
} // namespace Experimental } // namespace Experimental

83
lib/kokkos/algorithms/src/std_algorithms/Kokkos_ReplaceCopy.hpp
View File

@ -23,30 +23,39 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator, //
class ValueType> // overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator replace_copy(const ExecutionSpace& ex, InputIterator first_from, OutputIterator replace_copy(const ExecutionSpace& ex, InputIterator first_from,
InputIterator last_from, OutputIterator first_dest, InputIterator last_from, OutputIterator first_dest,
const ValueType& old_value, const ValueType& old_value,
const ValueType& new_value) { const ValueType& new_value) {
return Impl::replace_copy_impl("Kokkos::replace_copy_iterator_api", ex, return Impl::replace_copy_exespace_impl("Kokkos::replace_copy_iterator_api",
first_from, last_from, first_dest, old_value, ex, first_from, last_from, first_dest,
new_value); old_value, new_value);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator, template <
class ValueType> typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator replace_copy(const std::string& label, const ExecutionSpace& ex, OutputIterator replace_copy(const std::string& label, const ExecutionSpace& ex,
InputIterator first_from, InputIterator last_from, InputIterator first_from, InputIterator last_from,
OutputIterator first_dest, OutputIterator first_dest,
const ValueType& old_value, const ValueType& old_value,
const ValueType& new_value) { const ValueType& new_value) {
return Impl::replace_copy_impl(label, ex, first_from, last_from, first_dest, return Impl::replace_copy_exespace_impl(label, ex, first_from, last_from,
old_value, new_value); first_dest, old_value, new_value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto replace_copy(const ExecutionSpace& ex, auto replace_copy(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -54,13 +63,15 @@ auto replace_copy(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::replace_copy_impl("Kokkos::replace_copy_view_api", ex, return Impl::replace_copy_exespace_impl(
KE::cbegin(view_from), KE::cend(view_from), "Kokkos::replace_copy_view_api", ex, KE::cbegin(view_from),
KE::begin(view_dest), old_value, new_value); KE::cend(view_from), KE::begin(view_dest), old_value, new_value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto replace_copy(const std::string& label, const ExecutionSpace& ex, auto replace_copy(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -68,9 +79,43 @@ auto replace_copy(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::replace_copy_impl(label, ex, KE::cbegin(view_from), return Impl::replace_copy_exespace_impl(
KE::cend(view_from), KE::begin(view_dest), label, ex, KE::cbegin(view_from), KE::cend(view_from),
old_value, new_value); KE::begin(view_dest), old_value, new_value);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator replace_copy(const TeamHandleType& teamHandle,
InputIterator first_from,
InputIterator last_from,
OutputIterator first_dest,
const ValueType& old_value,
const ValueType& new_value) {
return Impl::replace_copy_team_impl(teamHandle, first_from, last_from,
first_dest, old_value, new_value);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto replace_copy(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
const ValueType& old_value, const ValueType& new_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental;
return Impl::replace_copy_team_impl(teamHandle, KE::cbegin(view_from),
KE::cend(view_from), KE::begin(view_dest),
old_value, new_value);
} }
} // namespace Experimental } // namespace Experimental

88
lib/kokkos/algorithms/src/std_algorithms/Kokkos_ReplaceCopyIf.hpp
View File

@ -23,33 +23,42 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator, //
class PredicateType, class ValueType> // overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename PredicateType, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator replace_copy_if(const ExecutionSpace& ex, OutputIterator replace_copy_if(const ExecutionSpace& ex,
InputIterator first_from, InputIterator first_from,
InputIterator last_from, InputIterator last_from,
OutputIterator first_dest, PredicateType pred, OutputIterator first_dest, PredicateType pred,
const ValueType& new_value) { const ValueType& new_value) {
return Impl::replace_copy_if_impl("Kokkos::replace_copy_if_iterator_api", ex, return Impl::replace_copy_if_exespace_impl(
first_from, last_from, first_dest, pred, "Kokkos::replace_copy_if_iterator_api", ex, first_from, last_from,
new_value); first_dest, pred, new_value);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator, template <
class PredicateType, class ValueType> typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename PredicateType, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator replace_copy_if(const std::string& label, OutputIterator replace_copy_if(const std::string& label,
const ExecutionSpace& ex, const ExecutionSpace& ex,
InputIterator first_from, InputIterator first_from,
InputIterator last_from, InputIterator last_from,
OutputIterator first_dest, PredicateType pred, OutputIterator first_dest, PredicateType pred,
const ValueType& new_value) { const ValueType& new_value) {
return Impl::replace_copy_if_impl(label, ex, first_from, last_from, return Impl::replace_copy_if_exespace_impl(label, ex, first_from, last_from,
first_dest, pred, new_value); first_dest, pred, new_value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class PredicateType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class ValueType> typename DataType2, typename... Properties2, typename PredicateType,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto replace_copy_if(const ExecutionSpace& ex, auto replace_copy_if(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -57,14 +66,16 @@ auto replace_copy_if(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::replace_copy_if_impl("Kokkos::replace_copy_if_view_api", ex, return Impl::replace_copy_if_exespace_impl(
KE::cbegin(view_from), KE::cend(view_from), "Kokkos::replace_copy_if_view_api", ex, KE::cbegin(view_from),
KE::begin(view_dest), pred, new_value); KE::cend(view_from), KE::begin(view_dest), pred, new_value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class PredicateType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class ValueType> typename DataType2, typename... Properties2, typename PredicateType,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto replace_copy_if(const std::string& label, const ExecutionSpace& ex, auto replace_copy_if(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest, const ::Kokkos::View<DataType2, Properties2...>& view_dest,
@ -72,9 +83,44 @@ auto replace_copy_if(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::replace_copy_if_impl(label, ex, KE::cbegin(view_from), return Impl::replace_copy_if_exespace_impl(
KE::cend(view_from), KE::begin(view_dest), label, ex, KE::cbegin(view_from), KE::cend(view_from),
pred, new_value); KE::begin(view_dest), pred, new_value);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator, typename PredicateType, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator replace_copy_if(const TeamHandleType& teamHandle,
InputIterator first_from,
InputIterator last_from,
OutputIterator first_dest,
PredicateType pred,
const ValueType& new_value) {
return Impl::replace_copy_if_team_impl(teamHandle, first_from, last_from,
first_dest, pred, new_value);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename PredicateType,
typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto replace_copy_if(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
PredicateType pred, const ValueType& new_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental;
return Impl::replace_copy_if_team_impl(teamHandle, KE::cbegin(view_from),
KE::cend(view_from),
KE::begin(view_dest), pred, new_value);
} }
} // namespace Experimental } // namespace Experimental

69
lib/kokkos/algorithms/src/std_algorithms/Kokkos_ReplaceIf.hpp
View File

@ -23,43 +23,82 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class Predicate, //
class ValueType> // overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename Predicate,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void replace_if(const ExecutionSpace& ex, InputIterator first, void replace_if(const ExecutionSpace& ex, InputIterator first,
InputIterator last, Predicate pred, InputIterator last, Predicate pred,
const ValueType& new_value) { const ValueType& new_value) {
return Impl::replace_if_impl("Kokkos::replace_if_iterator_api", ex, first, Impl::replace_if_exespace_impl("Kokkos::replace_if_iterator_api", ex, first,
last, pred, new_value); last, pred, new_value);
} }
template <class ExecutionSpace, class InputIterator, class Predicate, template <
class ValueType> typename ExecutionSpace, typename InputIterator, typename Predicate,
typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void replace_if(const std::string& label, const ExecutionSpace& ex, void replace_if(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, Predicate pred, InputIterator first, InputIterator last, Predicate pred,
const ValueType& new_value) { const ValueType& new_value) {
return Impl::replace_if_impl(label, ex, first, last, pred, new_value); Impl::replace_if_exespace_impl(label, ex, first, last, pred, new_value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class Predicate, class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename Predicate, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void replace_if(const ExecutionSpace& ex, void replace_if(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
Predicate pred, const ValueType& new_value) { Predicate pred, const ValueType& new_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::replace_if_impl("Kokkos::replace_if_view_api", ex, Impl::replace_if_exespace_impl("Kokkos::replace_if_view_api", ex,
KE::begin(view), KE::end(view), pred, new_value); KE::begin(view), KE::end(view), pred,
new_value);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class Predicate, class ValueType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename Predicate, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void replace_if(const std::string& label, const ExecutionSpace& ex, void replace_if(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
Predicate pred, const ValueType& new_value) { Predicate pred, const ValueType& new_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::replace_if_impl(label, ex, KE::begin(view), KE::end(view), pred, Impl::replace_if_exespace_impl(label, ex, KE::begin(view), KE::end(view),
new_value); pred, new_value);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator, typename Predicate,
typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void replace_if(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last,
Predicate pred, const ValueType& new_value) {
Impl::replace_if_team_impl(teamHandle, first, last, pred, new_value);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename Predicate, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void replace_if(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view, Predicate pred,
const ValueType& new_value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental;
Impl::replace_if_team_impl(teamHandle, KE::begin(view), KE::end(view), pred,
new_value);
} }
} // namespace Experimental } // namespace Experimental

53
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Reverse.hpp
View File

@ -23,34 +23,67 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void reverse(const ExecutionSpace& ex, InputIterator first, void reverse(const ExecutionSpace& ex, InputIterator first,
InputIterator last) { InputIterator last) {
return Impl::reverse_impl("Kokkos::reverse_iterator_api_default", ex, first, return Impl::reverse_exespace_impl("Kokkos::reverse_iterator_api_default", ex,
last); first, last);
} }
template <class ExecutionSpace, class InputIterator> template <
typename ExecutionSpace, typename InputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void reverse(const std::string& label, const ExecutionSpace& ex, void reverse(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last) { InputIterator first, InputIterator last) {
return Impl::reverse_impl(label, ex, first, last); return Impl::reverse_exespace_impl(label, ex, first, last);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void reverse(const ExecutionSpace& ex, void reverse(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::reverse_impl("Kokkos::reverse_view_api_default", ex, return Impl::reverse_exespace_impl("Kokkos::reverse_view_api_default", ex,
KE::begin(view), KE::end(view)); KE::begin(view), KE::end(view));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
void reverse(const std::string& label, const ExecutionSpace& ex, void reverse(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::reverse_impl(label, ex, KE::begin(view), KE::end(view)); return Impl::reverse_exespace_impl(label, ex, KE::begin(view), KE::end(view));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void reverse(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last) {
return Impl::reverse_team_impl(teamHandle, first, last);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION void reverse(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental;
return Impl::reverse_team_impl(teamHandle, KE::begin(view), KE::end(view));
} }
} // namespace Experimental } // namespace Experimental

67
lib/kokkos/algorithms/src/std_algorithms/Kokkos_ReverseCopy.hpp
View File

@ -23,42 +23,83 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator reverse_copy(const ExecutionSpace& ex, InputIterator first, OutputIterator reverse_copy(const ExecutionSpace& ex, InputIterator first,
InputIterator last, OutputIterator d_first) { InputIterator last, OutputIterator d_first) {
return Impl::reverse_copy_impl("Kokkos::reverse_copy_iterator_api_default", return Impl::reverse_copy_exespace_impl(
ex, first, last, d_first); "Kokkos::reverse_copy_iterator_api_default", ex, first, last, d_first);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator> template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator reverse_copy(const std::string& label, const ExecutionSpace& ex, OutputIterator reverse_copy(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, InputIterator first, InputIterator last,
OutputIterator d_first) { OutputIterator d_first) {
return Impl::reverse_copy_impl(label, ex, first, last, d_first); return Impl::reverse_copy_exespace_impl(label, ex, first, last, d_first);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto reverse_copy(const ExecutionSpace& ex, auto reverse_copy(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::reverse_copy_impl("Kokkos::reverse_copy_view_api_default", ex, return Impl::reverse_copy_exespace_impl(
cbegin(source), cend(source), begin(dest)); "Kokkos::reverse_copy_view_api_default", ex, cbegin(source), cend(source),
begin(dest));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto reverse_copy(const std::string& label, const ExecutionSpace& ex, auto reverse_copy(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::reverse_copy_impl(label, ex, cbegin(source), cend(source), return Impl::reverse_copy_exespace_impl(label, ex, cbegin(source),
begin(dest)); cend(source), begin(dest));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator reverse_copy(const TeamHandleType& teamHandle,
InputIterator first,
InputIterator last,
OutputIterator d_first) {
return Impl::reverse_copy_team_impl(teamHandle, first, last, d_first);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto reverse_copy(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::reverse_copy_team_impl(teamHandle, cbegin(source), cend(source),
begin(dest));
} }
} // namespace Experimental } // namespace Experimental

57
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Rotate.hpp
View File

@ -23,36 +23,71 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType rotate(const ExecutionSpace& ex, IteratorType first, IteratorType rotate(const ExecutionSpace& ex, IteratorType first,
IteratorType n_first, IteratorType last) { IteratorType n_first, IteratorType last) {
return Impl::rotate_impl("Kokkos::rotate_iterator_api_default", ex, first, return Impl::rotate_exespace_impl("Kokkos::rotate_iterator_api_default", ex,
n_first, last); first, n_first, last);
} }
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType rotate(const std::string& label, const ExecutionSpace& ex, IteratorType rotate(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType n_first, IteratorType first, IteratorType n_first,
IteratorType last) { IteratorType last) {
return Impl::rotate_impl(label, ex, first, n_first, last); return Impl::rotate_exespace_impl(label, ex, first, n_first, last);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto rotate(const ExecutionSpace& ex, auto rotate(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
std::size_t n_location) { std::size_t n_location) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::rotate_impl("Kokkos::rotate_view_api_default", ex, begin(view), return Impl::rotate_exespace_impl("Kokkos::rotate_view_api_default", ex,
begin(view) + n_location, end(view)); begin(view), begin(view) + n_location,
end(view));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto rotate(const std::string& label, const ExecutionSpace& ex, auto rotate(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
std::size_t n_location) { std::size_t n_location) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::rotate_impl(label, ex, begin(view), begin(view) + n_location, return Impl::rotate_exespace_impl(label, ex, begin(view),
end(view)); begin(view) + n_location, end(view));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType rotate(const TeamHandleType& teamHandle,
IteratorType first, IteratorType n_first,
IteratorType last) {
return Impl::rotate_team_impl(teamHandle, first, n_first, last);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto rotate(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view,
std::size_t n_location) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::rotate_team_impl(teamHandle, begin(view),
begin(view) + n_location, end(view));
} }
} // namespace Experimental } // namespace Experimental

75
lib/kokkos/algorithms/src/std_algorithms/Kokkos_RotateCopy.hpp
View File

@ -23,23 +23,34 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator rotate_copy(const ExecutionSpace& ex, InputIterator first, OutputIterator rotate_copy(const ExecutionSpace& ex, InputIterator first,
InputIterator n_first, InputIterator last, InputIterator n_first, InputIterator last,
OutputIterator d_first) { OutputIterator d_first) {
return Impl::rotate_copy_impl("Kokkos::rotate_copy_iterator_api_default", ex, return Impl::rotate_copy_exespace_impl(
first, n_first, last, d_first); "Kokkos::rotate_copy_iterator_api_default", ex, first, n_first, last,
d_first);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator> template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator rotate_copy(const std::string& label, const ExecutionSpace& ex, OutputIterator rotate_copy(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator n_first, InputIterator first, InputIterator n_first,
InputIterator last, OutputIterator d_first) { InputIterator last, OutputIterator d_first) {
return Impl::rotate_copy_impl(label, ex, first, n_first, last, d_first); return Impl::rotate_copy_exespace_impl(label, ex, first, n_first, last,
d_first);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto rotate_copy(const ExecutionSpace& ex, auto rotate_copy(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
std::size_t n_location, std::size_t n_location,
@ -47,13 +58,15 @@ auto rotate_copy(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::rotate_copy_impl("Kokkos::rotate_copy_view_api_default", ex, return Impl::rotate_copy_exespace_impl(
cbegin(source), cbegin(source) + n_location, "Kokkos::rotate_copy_view_api_default", ex, cbegin(source),
cend(source), begin(dest)); cbegin(source) + n_location, cend(source), begin(dest));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto rotate_copy(const std::string& label, const ExecutionSpace& ex, auto rotate_copy(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
std::size_t n_location, std::size_t n_location,
@ -61,9 +74,41 @@ auto rotate_copy(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::rotate_copy_impl(label, ex, cbegin(source), return Impl::rotate_copy_exespace_impl(label, ex, cbegin(source),
cbegin(source) + n_location, cend(source), cbegin(source) + n_location,
begin(dest)); cend(source), begin(dest));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator rotate_copy(const TeamHandleType& teamHandle,
InputIterator first,
InputIterator n_first,
InputIterator last,
OutputIterator d_first) {
return Impl::rotate_copy_team_impl(teamHandle, first, n_first, last, d_first);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto rotate_copy(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
std::size_t n_location,
const ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::rotate_copy_team_impl(teamHandle, cbegin(source),
cbegin(source) + n_location, cend(source),
begin(dest));
} }
} // namespace Experimental } // namespace Experimental

143
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Search.hpp
View File

@ -23,24 +23,34 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// overload set 1: no binary predicate passed // overload set 1: no binary predicate passed
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 search(const ExecutionSpace& ex, IteratorType1 first, IteratorType1 search(const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first, IteratorType1 last, IteratorType2 s_first,
IteratorType2 s_last) { IteratorType2 s_last) {
return Impl::search_impl("Kokkos::search_iterator_api_default", ex, first, return Impl::search_exespace_impl("Kokkos::search_iterator_api_default", ex,
last, s_first, s_last); first, last, s_first, s_last);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <
typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 search(const std::string& label, const ExecutionSpace& ex, IteratorType1 search(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first, IteratorType1 last, IteratorType1 first, IteratorType1 last,
IteratorType2 s_first, IteratorType2 s_last) { IteratorType2 s_first, IteratorType2 s_last) {
return Impl::search_impl(label, ex, first, last, s_first, s_last); return Impl::search_exespace_impl(label, ex, first, last, s_first, s_last);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto search(const ExecutionSpace& ex, auto search(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view) { const ::Kokkos::View<DataType2, Properties2...>& s_view) {
@ -48,13 +58,15 @@ auto search(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::search_impl("Kokkos::search_view_api_default", ex, return Impl::search_exespace_impl("Kokkos::search_view_api_default", ex,
KE::begin(view), KE::end(view), KE::begin(s_view), KE::begin(view), KE::end(view),
KE::end(s_view)); KE::begin(s_view), KE::end(s_view));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto search(const std::string& label, const ExecutionSpace& ex, auto search(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view) { const ::Kokkos::View<DataType2, Properties2...>& s_view) {
@ -62,31 +74,38 @@ auto search(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::search_impl(label, ex, KE::begin(view), KE::end(view), return Impl::search_exespace_impl(label, ex, KE::begin(view), KE::end(view),
KE::begin(s_view), KE::end(s_view)); KE::begin(s_view), KE::end(s_view));
} }
// overload set 2: binary predicate passed // overload set 2: binary predicate passed
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class BinaryPredicateType> typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 search(const ExecutionSpace& ex, IteratorType1 first, IteratorType1 search(const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first, IteratorType1 last, IteratorType2 s_first,
IteratorType2 s_last, const BinaryPredicateType& pred) { IteratorType2 s_last, const BinaryPredicateType& pred) {
return Impl::search_impl("Kokkos::search_iterator_api_default", ex, first, return Impl::search_exespace_impl("Kokkos::search_iterator_api_default", ex,
last, s_first, s_last, pred); first, last, s_first, s_last, pred);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class BinaryPredicateType> typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType1 search(const std::string& label, const ExecutionSpace& ex, IteratorType1 search(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first, IteratorType1 last, IteratorType1 first, IteratorType1 last,
IteratorType2 s_first, IteratorType2 s_last, IteratorType2 s_first, IteratorType2 s_last,
const BinaryPredicateType& pred) { const BinaryPredicateType& pred) {
return Impl::search_impl(label, ex, first, last, s_first, s_last, pred); return Impl::search_exespace_impl(label, ex, first, last, s_first, s_last,
pred);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto search(const ExecutionSpace& ex, auto search(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view, const ::Kokkos::View<DataType2, Properties2...>& s_view,
@ -95,13 +114,15 @@ auto search(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::search_impl("Kokkos::search_view_api_default", ex, return Impl::search_exespace_impl("Kokkos::search_view_api_default", ex,
KE::begin(view), KE::end(view), KE::begin(s_view), KE::begin(view), KE::end(view),
KE::end(s_view), pred); KE::begin(s_view), KE::end(s_view), pred);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicateType> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto search(const std::string& label, const ExecutionSpace& ex, auto search(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view, const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view, const ::Kokkos::View<DataType2, Properties2...>& s_view,
@ -110,8 +131,70 @@ auto search(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::search_impl(label, ex, KE::begin(view), KE::end(view), return Impl::search_exespace_impl(label, ex, KE::begin(view), KE::end(view),
KE::begin(s_view), KE::end(s_view), pred); KE::begin(s_view), KE::end(s_view), pred);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
// overload set 1: no binary predicate passed
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType1 search(const TeamHandleType& teamHandle,
IteratorType1 first, IteratorType1 last,
IteratorType2 s_first,
IteratorType2 s_last) {
return Impl::search_team_impl(teamHandle, first, last, s_first, s_last);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto search(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental;
return Impl::search_team_impl(teamHandle, KE::begin(view), KE::end(view),
KE::begin(s_view), KE::end(s_view));
}
// overload set 2: binary predicate passed
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2, typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType1 search(const TeamHandleType& teamHandle,
IteratorType1 first, IteratorType1 last,
IteratorType2 s_first,
IteratorType2 s_last,
const BinaryPredicateType& pred) {
return Impl::search_team_impl(teamHandle, first, last, s_first, s_last, pred);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
typename BinaryPredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto search(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view,
const ::Kokkos::View<DataType2, Properties2...>& s_view,
const BinaryPredicateType& pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(s_view);
namespace KE = ::Kokkos::Experimental;
return Impl::search_team_impl(teamHandle, KE::begin(view), KE::end(view),
KE::begin(s_view), KE::end(s_view), pred);
} }
} // namespace Experimental } // namespace Experimental

131
lib/kokkos/algorithms/src/std_algorithms/Kokkos_SearchN.hpp
View File

@ -23,68 +23,86 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// overload set 1: no binary predicate passed // overload set 1: no binary predicate passed
template <class ExecutionSpace, class IteratorType, class SizeType, template <
class ValueType> class ExecutionSpace, class IteratorType, class SizeType, class ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType search_n(const ExecutionSpace& ex, IteratorType first, IteratorType search_n(const ExecutionSpace& ex, IteratorType first,
IteratorType last, SizeType count, IteratorType last, SizeType count,
const ValueType& value) { const ValueType& value) {
return Impl::search_n_impl("Kokkos::search_n_iterator_api_default", ex, first, return Impl::search_n_exespace_impl("Kokkos::search_n_iterator_api_default",
last, count, value); ex, first, last, count, value);
} }
template <class ExecutionSpace, class IteratorType, class SizeType, template <
class ValueType> class ExecutionSpace, class IteratorType, class SizeType, class ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType search_n(const std::string& label, const ExecutionSpace& ex, IteratorType search_n(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, SizeType count, IteratorType first, IteratorType last, SizeType count,
const ValueType& value) { const ValueType& value) {
return Impl::search_n_impl(label, ex, first, last, count, value); return Impl::search_n_exespace_impl(label, ex, first, last, count, value);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <class ExecutionSpace, class DataType, class... Properties,
class SizeType, class ValueType> class SizeType, class ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto search_n(const ExecutionSpace& ex, auto search_n(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
SizeType count, const ValueType& value) { SizeType count, const ValueType& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::search_n_impl("Kokkos::search_n_view_api_default", ex, return Impl::search_n_exespace_impl("Kokkos::search_n_view_api_default", ex,
KE::begin(view), KE::end(view), count, value); KE::begin(view), KE::end(view), count,
value);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <class ExecutionSpace, class DataType, class... Properties,
class SizeType, class ValueType> class SizeType, class ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto search_n(const std::string& label, const ExecutionSpace& ex, auto search_n(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
SizeType count, const ValueType& value) { SizeType count, const ValueType& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::search_n_impl(label, ex, KE::begin(view), KE::end(view), count, return Impl::search_n_exespace_impl(label, ex, KE::begin(view), KE::end(view),
value); count, value);
} }
// overload set 2: binary predicate passed // overload set 2: binary predicate passed
template <class ExecutionSpace, class IteratorType, class SizeType, template <
class ValueType, class BinaryPredicateType> class ExecutionSpace, class IteratorType, class SizeType, class ValueType,
class BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType search_n(const ExecutionSpace& ex, IteratorType first, IteratorType search_n(const ExecutionSpace& ex, IteratorType first,
IteratorType last, SizeType count, const ValueType& value, IteratorType last, SizeType count, const ValueType& value,
const BinaryPredicateType& pred) { const BinaryPredicateType& pred) {
return Impl::search_n_impl("Kokkos::search_n_iterator_api_default", ex, first, return Impl::search_n_exespace_impl("Kokkos::search_n_iterator_api_default",
last, count, value, pred); ex, first, last, count, value, pred);
} }
template <class ExecutionSpace, class IteratorType, class SizeType, template <
class ValueType, class BinaryPredicateType> class ExecutionSpace, class IteratorType, class SizeType, class ValueType,
class BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType search_n(const std::string& label, const ExecutionSpace& ex, IteratorType search_n(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, SizeType count, IteratorType first, IteratorType last, SizeType count,
const ValueType& value, const BinaryPredicateType& pred) { const ValueType& value, const BinaryPredicateType& pred) {
return Impl::search_n_impl(label, ex, first, last, count, value, pred); return Impl::search_n_exespace_impl(label, ex, first, last, count, value,
pred);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <class ExecutionSpace, class DataType, class... Properties,
class SizeType, class ValueType, class BinaryPredicateType> class SizeType, class ValueType, class BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto search_n(const ExecutionSpace& ex, auto search_n(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
SizeType count, const ValueType& value, SizeType count, const ValueType& value,
@ -92,13 +110,15 @@ auto search_n(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::search_n_impl("Kokkos::search_n_view_api_default", ex, return Impl::search_n_exespace_impl("Kokkos::search_n_view_api_default", ex,
KE::begin(view), KE::end(view), count, value, KE::begin(view), KE::end(view), count,
pred); value, pred);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <class ExecutionSpace, class DataType, class... Properties,
class SizeType, class ValueType, class BinaryPredicateType> class SizeType, class ValueType, class BinaryPredicateType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
auto search_n(const std::string& label, const ExecutionSpace& ex, auto search_n(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
SizeType count, const ValueType& value, SizeType count, const ValueType& value,
@ -106,8 +126,65 @@ auto search_n(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::search_n_impl(label, ex, KE::begin(view), KE::end(view), count, return Impl::search_n_exespace_impl(label, ex, KE::begin(view), KE::end(view),
value, pred); count, value, pred);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
// overload set 1: no binary predicate passed
template <class TeamHandleType, class IteratorType, class SizeType,
class ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType search_n(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
SizeType count, const ValueType& value) {
return Impl::search_n_team_impl(teamHandle, first, last, count, value);
}
template <
class TeamHandleType, class DataType, class... Properties, class SizeType,
class ValueType,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION auto search_n(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view, SizeType count,
const ValueType& value) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental;
return Impl::search_n_team_impl(teamHandle, KE::begin(view), KE::end(view),
count, value);
}
// overload set 2: binary predicate passed
template <class TeamHandleType, class IteratorType, class SizeType,
class ValueType, class BinaryPredicateType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType search_n(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
SizeType count, const ValueType& value,
const BinaryPredicateType& pred) {
return Impl::search_n_team_impl(teamHandle, first, last, count, value, pred);
}
template <
class TeamHandleType, class DataType, class... Properties, class SizeType,
class ValueType, class BinaryPredicateType,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION auto search_n(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view, SizeType count,
const ValueType& value, const BinaryPredicateType& pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
namespace KE = ::Kokkos::Experimental;
return Impl::search_n_team_impl(teamHandle, KE::begin(view), KE::end(view),
count, value, pred);
} }
} // namespace Experimental } // namespace Experimental

54
lib/kokkos/algorithms/src/std_algorithms/Kokkos_ShiftLeft.hpp
View File

@ -23,36 +23,70 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType shift_left(const ExecutionSpace& ex, IteratorType first, IteratorType shift_left(const ExecutionSpace& ex, IteratorType first,
IteratorType last, IteratorType last,
typename IteratorType::difference_type n) { typename IteratorType::difference_type n) {
return Impl::shift_left_impl("Kokkos::shift_left_iterator_api_default", ex, return Impl::shift_left_exespace_impl(
first, last, n); "Kokkos::shift_left_iterator_api_default", ex, first, last, n);
} }
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType shift_left(const std::string& label, const ExecutionSpace& ex, IteratorType shift_left(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
typename IteratorType::difference_type n) { typename IteratorType::difference_type n) {
return Impl::shift_left_impl(label, ex, first, last, n); return Impl::shift_left_exespace_impl(label, ex, first, last, n);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto shift_left(const ExecutionSpace& ex, auto shift_left(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
typename decltype(begin(view))::difference_type n) { typename decltype(begin(view))::difference_type n) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::shift_left_impl("Kokkos::shift_left_view_api_default", ex, return Impl::shift_left_exespace_impl("Kokkos::shift_left_view_api_default",
begin(view), end(view), n); ex, begin(view), end(view), n);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto shift_left(const std::string& label, const ExecutionSpace& ex, auto shift_left(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
typename decltype(begin(view))::difference_type n) { typename decltype(begin(view))::difference_type n) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::shift_left_impl(label, ex, begin(view), end(view), n); return Impl::shift_left_exespace_impl(label, ex, begin(view), end(view), n);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType
shift_left(const TeamHandleType& teamHandle, IteratorType first,
IteratorType last, typename IteratorType::difference_type n) {
return Impl::shift_left_team_impl(teamHandle, first, last, n);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto shift_left(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view,
typename decltype(begin(view))::difference_type n) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::shift_left_team_impl(teamHandle, begin(view), end(view), n);
} }
} // namespace Experimental } // namespace Experimental

54
lib/kokkos/algorithms/src/std_algorithms/Kokkos_ShiftRight.hpp
View File

@ -23,36 +23,70 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType> //
// overload set accepting execution space
//
template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType shift_right(const ExecutionSpace& ex, IteratorType first, IteratorType shift_right(const ExecutionSpace& ex, IteratorType first,
IteratorType last, IteratorType last,
typename IteratorType::difference_type n) { typename IteratorType::difference_type n) {
return Impl::shift_right_impl("Kokkos::shift_right_iterator_api_default", ex, return Impl::shift_right_exespace_impl(
first, last, n); "Kokkos::shift_right_iterator_api_default", ex, first, last, n);
} }
template <class ExecutionSpace, class IteratorType> template <
typename ExecutionSpace, typename IteratorType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType shift_right(const std::string& label, const ExecutionSpace& ex, IteratorType shift_right(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
typename IteratorType::difference_type n) { typename IteratorType::difference_type n) {
return Impl::shift_right_impl(label, ex, first, last, n); return Impl::shift_right_exespace_impl(label, ex, first, last, n);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto shift_right(const ExecutionSpace& ex, auto shift_right(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
typename decltype(begin(view))::difference_type n) { typename decltype(begin(view))::difference_type n) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::shift_right_impl("Kokkos::shift_right_view_api_default", ex, return Impl::shift_right_exespace_impl("Kokkos::shift_right_view_api_default",
begin(view), end(view), n); ex, begin(view), end(view), n);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <
typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto shift_right(const std::string& label, const ExecutionSpace& ex, auto shift_right(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
typename decltype(begin(view))::difference_type n) { typename decltype(begin(view))::difference_type n) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::shift_right_impl(label, ex, begin(view), end(view), n); return Impl::shift_right_exespace_impl(label, ex, begin(view), end(view), n);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType
shift_right(const TeamHandleType& teamHandle, IteratorType first,
IteratorType last, typename IteratorType::difference_type n) {
return Impl::shift_right_team_impl(teamHandle, first, last, n);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto shift_right(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view,
typename decltype(begin(view))::difference_type n) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::shift_right_team_impl(teamHandle, begin(view), end(view), n);
} }
} // namespace Experimental } // namespace Experimental

66
lib/kokkos/algorithms/src/std_algorithms/Kokkos_SwapRanges.hpp
View File

@ -23,15 +23,21 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class IteratorType1, class IteratorType2> //
// overload set accepting execution space
//
template <typename ExecutionSpace, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType2 swap_ranges(const ExecutionSpace& ex, IteratorType1 first1, IteratorType2 swap_ranges(const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 last1, IteratorType2 first2) { IteratorType1 last1, IteratorType2 first2) {
return Impl::swap_ranges_impl("Kokkos::swap_ranges_iterator_api_default", ex, return Impl::swap_ranges_exespace_impl(
first1, last1, first2); "Kokkos::swap_ranges_iterator_api_default", ex, first1, last1, first2);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2> typename DataType2, typename... Properties2,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
auto swap_ranges(const ExecutionSpace& ex, auto swap_ranges(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
@ -39,19 +45,23 @@ auto swap_ranges(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
assert(source.extent(0) == dest.extent(0)); assert(source.extent(0) == dest.extent(0));
return Impl::swap_ranges_impl("Kokkos::swap_ranges_view_api_default", ex, return Impl::swap_ranges_exespace_impl("Kokkos::swap_ranges_view_api_default",
begin(source), end(source), begin(dest)); ex, begin(source), end(source),
begin(dest));
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <typename ExecutionSpace, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
IteratorType2 swap_ranges(const std::string& label, const ExecutionSpace& ex, IteratorType2 swap_ranges(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2) { IteratorType2 first2) {
return Impl::swap_ranges_impl(label, ex, first1, last1, first2); return Impl::swap_ranges_exespace_impl(label, ex, first1, last1, first2);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2> typename DataType2, typename... Properties2,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
auto swap_ranges(const std::string& label, const ExecutionSpace& ex, auto swap_ranges(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) { ::Kokkos::View<DataType2, Properties2...>& dest) {
@ -59,8 +69,38 @@ auto swap_ranges(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
assert(source.extent(0) == dest.extent(0)); assert(source.extent(0) == dest.extent(0));
return Impl::swap_ranges_impl(label, ex, begin(source), end(source), return Impl::swap_ranges_exespace_impl(label, ex, begin(source), end(source),
begin(dest)); begin(dest));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType1,
typename IteratorType2,
std::enable_if_t<is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION IteratorType2 swap_ranges(const TeamHandleType& teamHandle,
IteratorType1 first1,
IteratorType1 last1,
IteratorType2 first2) {
return Impl::swap_ranges_team_impl(teamHandle, first1, last1, first2);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto swap_ranges(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
assert(source.extent(0) == dest.extent(0));
return Impl::swap_ranges_team_impl(teamHandle, begin(source), end(source),
begin(dest));
} }
} // namespace Experimental } // namespace Experimental

208
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Transform.hpp
View File

@ -23,31 +23,39 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIterator, class OutputIterator, //
class UnaryOperation> // overload set accepting execution space
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< //
InputIterator, OutputIterator>::value, template <
OutputIterator> typename ExecutionSpace, typename InputIterator, typename OutputIterator,
transform(const ExecutionSpace& ex, InputIterator first1, InputIterator last1, typename UnaryOperation,
OutputIterator d_first, UnaryOperation unary_op) { std::enable_if_t<Impl::are_iterators_v<InputIterator, OutputIterator> &&
return Impl::transform_impl("Kokkos::transform_iterator_api_default", ex, is_execution_space_v<ExecutionSpace>,
first1, last1, d_first, std::move(unary_op)); int> = 0>
OutputIterator transform(const ExecutionSpace& ex, InputIterator first1,
InputIterator last1, OutputIterator d_first,
UnaryOperation unary_op) {
return Impl::transform_exespace_impl("Kokkos::transform_iterator_api_default",
ex, first1, last1, d_first,
std::move(unary_op));
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator, template <
class UnaryOperation> typename ExecutionSpace, typename InputIterator, typename OutputIterator,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< typename UnaryOperation,
InputIterator, OutputIterator>::value, std::enable_if_t<Impl::are_iterators_v<InputIterator, OutputIterator> &&
OutputIterator> is_execution_space_v<ExecutionSpace>,
transform(const std::string& label, const ExecutionSpace& ex, int> = 0>
InputIterator first1, InputIterator last1, OutputIterator d_first, OutputIterator transform(const std::string& label, const ExecutionSpace& ex,
UnaryOperation unary_op) { InputIterator first1, InputIterator last1,
return Impl::transform_impl(label, ex, first1, last1, d_first, OutputIterator d_first, UnaryOperation unary_op) {
std::move(unary_op)); return Impl::transform_exespace_impl(label, ex, first1, last1, d_first,
std::move(unary_op));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class UnaryOperation> typename DataType2, typename... Properties2, typename UnaryOperation,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform(const ExecutionSpace& ex, auto transform(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest, ::Kokkos::View<DataType2, Properties2...>& dest,
@ -55,13 +63,14 @@ auto transform(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::transform_impl("Kokkos::transform_view_api_default", ex, return Impl::transform_exespace_impl("Kokkos::transform_view_api_default", ex,
begin(source), end(source), begin(dest), begin(source), end(source), begin(dest),
std::move(unary_op)); std::move(unary_op));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class UnaryOperation> typename DataType2, typename... Properties2, typename UnaryOperation,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform(const std::string& label, const ExecutionSpace& ex, auto transform(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest, ::Kokkos::View<DataType2, Properties2...>& dest,
@ -69,38 +78,44 @@ auto transform(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::transform_impl(label, ex, begin(source), end(source), return Impl::transform_exespace_impl(label, ex, begin(source), end(source),
begin(dest), std::move(unary_op)); begin(dest), std::move(unary_op));
} }
template <class ExecutionSpace, class InputIterator1, class InputIterator2, template <
class OutputIterator, class BinaryOperation> typename ExecutionSpace, typename InputIterator1, typename InputIterator2,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< typename OutputIterator, typename BinaryOperation,
InputIterator1, InputIterator2, OutputIterator>::value, std::enable_if_t<
OutputIterator> Impl::are_iterators_v<InputIterator1, InputIterator2, OutputIterator> &&
transform(const ExecutionSpace& ex, InputIterator1 first1, InputIterator1 last1, is_execution_space_v<ExecutionSpace>,
InputIterator2 first2, OutputIterator d_first, int> = 0>
BinaryOperation binary_op) { OutputIterator transform(const ExecutionSpace& ex, InputIterator1 first1,
return Impl::transform_impl("Kokkos::transform_iterator_api_default", ex, InputIterator1 last1, InputIterator2 first2,
first1, last1, first2, d_first, OutputIterator d_first, BinaryOperation binary_op) {
std::move(binary_op)); return Impl::transform_exespace_impl("Kokkos::transform_iterator_api_default",
ex, first1, last1, first2, d_first,
std::move(binary_op));
} }
template <class ExecutionSpace, class InputIterator1, class InputIterator2, template <
class OutputIterator, class BinaryOperation> typename ExecutionSpace, typename InputIterator1, typename InputIterator2,
std::enable_if_t< ::Kokkos::Experimental::Impl::are_iterators< typename OutputIterator, typename BinaryOperation,
InputIterator1, InputIterator2, OutputIterator>::value, std::enable_if_t<
OutputIterator> Impl::are_iterators_v<InputIterator1, InputIterator2, OutputIterator> &&
transform(const std::string& label, const ExecutionSpace& ex, is_execution_space_v<ExecutionSpace>,
InputIterator1 first1, InputIterator1 last1, InputIterator2 first2, int> = 0>
OutputIterator d_first, BinaryOperation binary_op) { OutputIterator transform(const std::string& label, const ExecutionSpace& ex,
return Impl::transform_impl(label, ex, first1, last1, first2, d_first, InputIterator1 first1, InputIterator1 last1,
std::move(binary_op)); InputIterator2 first2, OutputIterator d_first,
BinaryOperation binary_op) {
return Impl::transform_exespace_impl(label, ex, first1, last1, first2,
d_first, std::move(binary_op));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class DataType3, typename DataType2, typename... Properties2, typename DataType3,
class... Properties3, class BinaryOperation> typename... Properties3, typename BinaryOperation,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform(const ExecutionSpace& ex, auto transform(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source1, const ::Kokkos::View<DataType1, Properties1...>& source1,
const ::Kokkos::View<DataType2, Properties2...>& source2, const ::Kokkos::View<DataType2, Properties2...>& source2,
@ -110,14 +125,15 @@ auto transform(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source2);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::transform_impl("Kokkos::transform_view_api_default", ex, return Impl::transform_exespace_impl(
begin(source1), end(source1), begin(source2), "Kokkos::transform_view_api_default", ex, begin(source1), end(source1),
begin(dest), std::move(binary_op)); begin(source2), begin(dest), std::move(binary_op));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class DataType3, typename DataType2, typename... Properties2, typename DataType3,
class... Properties3, class BinaryOperation> typename... Properties3, typename BinaryOperation,
std::enable_if_t<is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform(const std::string& label, const ExecutionSpace& ex, auto transform(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source1, const ::Kokkos::View<DataType1, Properties1...>& source1,
const ::Kokkos::View<DataType2, Properties2...>& source2, const ::Kokkos::View<DataType2, Properties2...>& source2,
@ -127,9 +143,79 @@ auto transform(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source2); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source2);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::transform_impl(label, ex, begin(source1), end(source1), return Impl::transform_exespace_impl(label, ex, begin(source1), end(source1),
begin(source2), begin(dest), begin(source2), begin(dest),
std::move(binary_op)); std::move(binary_op));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <
typename TeamHandleType, typename InputIterator, typename OutputIterator,
typename UnaryOperation,
std::enable_if_t<Impl::are_iterators_v<InputIterator, OutputIterator> &&
is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIterator transform(const TeamHandleType& teamHandle,
InputIterator first1,
InputIterator last1,
OutputIterator d_first,
UnaryOperation unary_op) {
return Impl::transform_team_impl(teamHandle, first1, last1, d_first,
std::move(unary_op));
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename UnaryOperation,
std::enable_if_t<is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto transform(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
::Kokkos::View<DataType2, Properties2...>& dest, UnaryOperation unary_op) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::transform_team_impl(teamHandle, begin(source), end(source),
begin(dest), std::move(unary_op));
}
template <
typename TeamHandleType, typename InputIterator1, typename InputIterator2,
typename OutputIterator, typename BinaryOperation,
std::enable_if_t<
Impl::are_iterators_v<InputIterator1, InputIterator2, OutputIterator> &&
is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIterator transform(const TeamHandleType& teamHandle,
InputIterator1 first1,
InputIterator1 last1,
InputIterator2 first2,
OutputIterator d_first,
BinaryOperation binary_op) {
return Impl::transform_team_impl(teamHandle, first1, last1, first2, d_first,
std::move(binary_op));
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename DataType3,
typename... Properties3, typename BinaryOperation,
std::enable_if_t<is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto transform(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source1,
const ::Kokkos::View<DataType2, Properties2...>& source2,
::Kokkos::View<DataType3, Properties3...>& dest,
BinaryOperation binary_op) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source1);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source2);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::transform_team_impl(teamHandle, begin(source1), end(source1),
begin(source2), begin(dest),
std::move(binary_op));
} }
} // namespace Experimental } // namespace Experimental

130
lib/kokkos/algorithms/src/std_algorithms/Kokkos_TransformExclusiveScan.hpp
View File

@ -23,44 +23,52 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
template <class ExecutionSpace, class InputIteratorType, //
class OutputIteratorType, class ValueType, class BinaryOpType, // overload set accepting execution space
class UnaryOpType> //
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< template <typename ExecutionSpace, typename InputIteratorType,
InputIteratorType, OutputIteratorType>::value, typename OutputIteratorType, typename ValueType,
OutputIteratorType> typename BinaryOpType, typename UnaryOpType,
transform_exclusive_scan(const ExecutionSpace& ex, InputIteratorType first, std::enable_if_t<
InputIteratorType last, OutputIteratorType first_dest, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
ValueType init_value, BinaryOpType binary_op, Kokkos::is_execution_space_v<ExecutionSpace>,
UnaryOpType unary_op) { int> = 0>
OutputIteratorType transform_exclusive_scan(
const ExecutionSpace& ex, InputIteratorType first, InputIteratorType last,
OutputIteratorType first_dest, ValueType init_value, BinaryOpType binary_op,
UnaryOpType unary_op) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::transform_exclusive_scan_impl( return Impl::transform_exclusive_scan_exespace_impl(
"Kokkos::transform_exclusive_scan_custom_functors_iterator_api", ex, "Kokkos::transform_exclusive_scan_custom_functors_iterator_api", ex,
first, last, first_dest, init_value, binary_op, unary_op); first, last, first_dest, std::move(init_value), binary_op, unary_op);
} }
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class ValueType, class BinaryOpType, typename OutputIteratorType, typename ValueType,
class UnaryOpType> typename BinaryOpType, typename UnaryOpType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< std::enable_if_t<
InputIteratorType, OutputIteratorType>::value, Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
OutputIteratorType> Kokkos::is_execution_space_v<ExecutionSpace>,
transform_exclusive_scan(const std::string& label, const ExecutionSpace& ex, int> = 0>
InputIteratorType first, InputIteratorType last, OutputIteratorType transform_exclusive_scan(
OutputIteratorType first_dest, ValueType init_value, const std::string& label, const ExecutionSpace& ex, InputIteratorType first,
BinaryOpType binary_op, UnaryOpType unary_op) { InputIteratorType last, OutputIteratorType first_dest, ValueType init_value,
BinaryOpType binary_op, UnaryOpType unary_op) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::transform_exclusive_scan_impl(label, ex, first, last, first_dest, return Impl::transform_exclusive_scan_exespace_impl(
init_value, binary_op, unary_op); label, ex, first, last, first_dest, std::move(init_value), binary_op,
unary_op);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class BinaryOpType, class UnaryOpType> typename DataType2, typename... Properties2, typename ValueType,
typename BinaryOpType, typename UnaryOpType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform_exclusive_scan( auto transform_exclusive_scan(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -69,18 +77,20 @@ auto transform_exclusive_scan(
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::transform_exclusive_scan_impl( return Impl::transform_exclusive_scan_exespace_impl(
"Kokkos::transform_exclusive_scan_custom_functors_view_api", ex, "Kokkos::transform_exclusive_scan_custom_functors_view_api", ex,
KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest), KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest),
init_value, binary_op, unary_op); std::move(init_value), binary_op, unary_op);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class ValueType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class BinaryOpType, class UnaryOpType> typename DataType2, typename... Properties2, typename ValueType,
typename BinaryOpType, typename UnaryOpType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform_exclusive_scan( auto transform_exclusive_scan(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -89,12 +99,56 @@ auto transform_exclusive_scan(
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible."); "ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::transform_exclusive_scan_impl( return Impl::transform_exclusive_scan_exespace_impl(
label, ex, KE::cbegin(view_from), KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), init_value, binary_op, unary_op); KE::begin(view_dest), std::move(init_value), binary_op, unary_op);
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType, typename ValueType,
typename BinaryOpType, typename UnaryOpType,
std::enable_if_t<
Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
Kokkos::is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType transform_exclusive_scan(
const TeamHandleType& teamHandle, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest, ValueType init_value,
BinaryOpType binary_op, UnaryOpType unary_op) {
Impl::static_assert_is_not_openmptarget(teamHandle);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
return Impl::transform_exclusive_scan_team_impl(
teamHandle, first, last, first_dest, std::move(init_value), binary_op,
unary_op);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
typename BinaryOpType, typename UnaryOpType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto transform_exclusive_scan(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
ValueType init_value, BinaryOpType binary_op, UnaryOpType unary_op) {
Impl::static_assert_is_not_openmptarget(teamHandle);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental;
return Impl::transform_exclusive_scan_team_impl(
teamHandle, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), std::move(init_value), binary_op, unary_op);
} }
} // namespace Experimental } // namespace Experimental

244
lib/kokkos/algorithms/src/std_algorithms/Kokkos_TransformInclusiveScan.hpp
View File

@ -23,40 +23,53 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// overload set 1 (no init value) // overload set 1 (no init value)
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class BinaryOpType, class UnaryOpType> typename OutputIteratorType, typename BinaryOpType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< typename UnaryOpType,
InputIteratorType, OutputIteratorType>::value, std::enable_if_t<
OutputIteratorType> Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
transform_inclusive_scan(const ExecutionSpace& ex, InputIteratorType first, Kokkos::is_execution_space_v<ExecutionSpace>,
InputIteratorType last, OutputIteratorType first_dest, int> = 0>
BinaryOpType binary_op, UnaryOpType unary_op) { OutputIteratorType transform_inclusive_scan(const ExecutionSpace& ex,
InputIteratorType first,
InputIteratorType last,
OutputIteratorType first_dest,
BinaryOpType binary_op,
UnaryOpType unary_op) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::transform_inclusive_scan_impl( return Impl::transform_inclusive_scan_exespace_impl(
"Kokkos::transform_inclusive_scan_custom_functors_iterator_api", ex, "Kokkos::transform_inclusive_scan_custom_functors_iterator_api", ex,
first, last, first_dest, binary_op, unary_op); first, last, first_dest, binary_op, unary_op);
} }
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class BinaryOpType, class UnaryOpType> typename OutputIteratorType, typename BinaryOpType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators< typename UnaryOpType,
InputIteratorType, OutputIteratorType>::value, std::enable_if_t<
OutputIteratorType> Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
transform_inclusive_scan(const std::string& label, const ExecutionSpace& ex, Kokkos::is_execution_space_v<ExecutionSpace>,
InputIteratorType first, InputIteratorType last, int> = 0>
OutputIteratorType first_dest, BinaryOpType binary_op, OutputIteratorType transform_inclusive_scan(
UnaryOpType unary_op) { const std::string& label, const ExecutionSpace& ex, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest,
BinaryOpType binary_op, UnaryOpType unary_op) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::transform_inclusive_scan_impl(label, ex, first, last, first_dest, return Impl::transform_inclusive_scan_exespace_impl(
binary_op, unary_op); label, ex, first, last, first_dest, binary_op, unary_op);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryOpType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class UnaryOpType> typename DataType2, typename... Properties2, typename BinaryOpType,
typename UnaryOpType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform_inclusive_scan( auto transform_inclusive_scan(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -66,15 +79,17 @@ auto transform_inclusive_scan(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::transform_inclusive_scan_impl( return Impl::transform_inclusive_scan_exespace_impl(
"Kokkos::transform_inclusive_scan_custom_functors_view_api", ex, "Kokkos::transform_inclusive_scan_custom_functors_view_api", ex,
KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest), KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest),
binary_op, unary_op); binary_op, unary_op);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryOpType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class UnaryOpType> typename DataType2, typename... Properties2, typename BinaryOpType,
typename UnaryOpType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform_inclusive_scan( auto transform_inclusive_scan(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -84,46 +99,59 @@ auto transform_inclusive_scan(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::transform_inclusive_scan_impl( return Impl::transform_inclusive_scan_exespace_impl(
label, ex, KE::cbegin(view_from), KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), binary_op, unary_op); KE::begin(view_dest), binary_op, unary_op);
} }
// overload set 2 (init value) // overload set 2 (init value)
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class BinaryOpType, class UnaryOpType, typename OutputIteratorType, typename BinaryOpType,
class ValueType> typename UnaryOpType, typename ValueType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators<
InputIteratorType, OutputIteratorType>::value, std::enable_if_t<
OutputIteratorType> Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
transform_inclusive_scan(const ExecutionSpace& ex, InputIteratorType first, Kokkos::is_execution_space_v<ExecutionSpace>,
InputIteratorType last, OutputIteratorType first_dest, int> = 0>
BinaryOpType binary_op, UnaryOpType unary_op, OutputIteratorType transform_inclusive_scan(
ValueType init_value) { const ExecutionSpace& ex, InputIteratorType first, InputIteratorType last,
OutputIteratorType first_dest, BinaryOpType binary_op, UnaryOpType unary_op,
ValueType init_value) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::transform_inclusive_scan_impl( static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
return Impl::transform_inclusive_scan_exespace_impl(
"Kokkos::transform_inclusive_scan_custom_functors_iterator_api", ex, "Kokkos::transform_inclusive_scan_custom_functors_iterator_api", ex,
first, last, first_dest, binary_op, unary_op, init_value); first, last, first_dest, binary_op, unary_op, std::move(init_value));
} }
template <class ExecutionSpace, class InputIteratorType, template <typename ExecutionSpace, typename InputIteratorType,
class OutputIteratorType, class BinaryOpType, class UnaryOpType, typename OutputIteratorType, typename BinaryOpType,
class ValueType> typename UnaryOpType, typename ValueType,
std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators<
InputIteratorType, OutputIteratorType>::value, std::enable_if_t<
OutputIteratorType> Impl::are_iterators_v<InputIteratorType, OutputIteratorType>&& ::
transform_inclusive_scan(const std::string& label, const ExecutionSpace& ex, Kokkos::is_execution_space_v<ExecutionSpace>,
InputIteratorType first, InputIteratorType last, int> = 0>
OutputIteratorType first_dest, BinaryOpType binary_op, OutputIteratorType transform_inclusive_scan(
UnaryOpType unary_op, ValueType init_value) { const std::string& label, const ExecutionSpace& ex, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest,
BinaryOpType binary_op, UnaryOpType unary_op, ValueType init_value) {
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
return Impl::transform_inclusive_scan_impl(label, ex, first, last, first_dest, static_assert(std::is_move_constructible_v<ValueType>,
binary_op, unary_op, init_value); "ValueType must be move constructible.");
return Impl::transform_inclusive_scan_exespace_impl(
label, ex, first, last, first_dest, binary_op, unary_op,
std::move(init_value));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryOpType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class UnaryOpType, class ValueType> typename DataType2, typename... Properties2, typename BinaryOpType,
typename UnaryOpType, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform_inclusive_scan( auto transform_inclusive_scan(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -132,16 +160,21 @@ auto transform_inclusive_scan(
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::transform_inclusive_scan_impl( return Impl::transform_inclusive_scan_exespace_impl(
"Kokkos::transform_inclusive_scan_custom_functors_view_api", ex, "Kokkos::transform_inclusive_scan_custom_functors_view_api", ex,
KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest), KE::cbegin(view_from), KE::cend(view_from), KE::begin(view_dest),
binary_op, unary_op, init_value); binary_op, unary_op, std::move(init_value));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryOpType, typename ExecutionSpace, typename DataType1, typename... Properties1,
class UnaryOpType, class ValueType> typename DataType2, typename... Properties2, typename BinaryOpType,
typename UnaryOpType, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto transform_inclusive_scan( auto transform_inclusive_scan(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& view_from, const ::Kokkos::View<DataType1, Properties1...>& view_from,
@ -150,10 +183,97 @@ auto transform_inclusive_scan(
Impl::static_assert_is_not_openmptarget(ex); Impl::static_assert_is_not_openmptarget(ex);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental; namespace KE = ::Kokkos::Experimental;
return Impl::transform_inclusive_scan_impl( return Impl::transform_inclusive_scan_exespace_impl(
label, ex, KE::cbegin(view_from), KE::cend(view_from), label, ex, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), binary_op, unary_op, init_value); KE::begin(view_dest), binary_op, unary_op, std::move(init_value));
}
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
// overload set 1 (no init value)
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType, typename BinaryOpType,
typename UnaryOpType,
std::enable_if_t<
Impl::are_iterators_v<InputIteratorType, OutputIteratorType> &&
Kokkos::is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType transform_inclusive_scan(
const TeamHandleType& teamHandle, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest,
BinaryOpType binary_op, UnaryOpType unary_op) {
Impl::static_assert_is_not_openmptarget(teamHandle);
return Impl::transform_inclusive_scan_team_impl(
teamHandle, first, last, first_dest, binary_op, unary_op);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryOpType,
typename UnaryOpType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto transform_inclusive_scan(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
BinaryOpType binary_op, UnaryOpType unary_op) {
Impl::static_assert_is_not_openmptarget(teamHandle);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
namespace KE = ::Kokkos::Experimental;
return Impl::transform_inclusive_scan_team_impl(
teamHandle, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), binary_op, unary_op);
}
// overload set 2 (init value)
template <typename TeamHandleType, typename InputIteratorType,
typename OutputIteratorType, typename BinaryOpType,
typename UnaryOpType, typename ValueType,
std::enable_if_t<
Impl::are_iterators_v<InputIteratorType, OutputIteratorType> &&
Kokkos::is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIteratorType transform_inclusive_scan(
const TeamHandleType& teamHandle, InputIteratorType first,
InputIteratorType last, OutputIteratorType first_dest,
BinaryOpType binary_op, UnaryOpType unary_op, ValueType init_value) {
Impl::static_assert_is_not_openmptarget(teamHandle);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
return Impl::transform_inclusive_scan_team_impl(
teamHandle, first, last, first_dest, binary_op, unary_op,
std::move(init_value));
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryOpType,
typename UnaryOpType, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto transform_inclusive_scan(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& view_from,
const ::Kokkos::View<DataType2, Properties2...>& view_dest,
BinaryOpType binary_op, UnaryOpType unary_op, ValueType init_value) {
Impl::static_assert_is_not_openmptarget(teamHandle);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_from);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view_dest);
static_assert(std::is_move_constructible_v<ValueType>,
"ValueType must be move constructible.");
namespace KE = ::Kokkos::Experimental;
return Impl::transform_inclusive_scan_team_impl(
teamHandle, KE::cbegin(view_from), KE::cend(view_from),
KE::begin(view_dest), binary_op, unary_op, std::move(init_value));
} }
} // namespace Experimental } // namespace Experimental

276
lib/kokkos/algorithms/src/std_algorithms/Kokkos_TransformReduce.hpp
View File

@ -23,34 +23,44 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
//
// overload set accepting execution space
//
// ---------------------------- // ----------------------------
// overload set1: // overload set1:
// no custom functors passed, so equivalent to // no custom functors passed, so equivalent to
// transform_reduce(first1, last1, first2, init, plus<>(), multiplies<>()); // transform_reduce(first1, last1, first2, init, plus<>(), multiplies<>());
// ---------------------------- // ----------------------------
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <typename ExecutionSpace, typename IteratorType1,
class ValueType> typename IteratorType2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType transform_reduce(const ExecutionSpace& ex, IteratorType1 first1, ValueType transform_reduce(const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 last1, IteratorType2 first2, IteratorType1 last1, IteratorType2 first2,
ValueType init_reduction_value) { ValueType init_reduction_value) {
return Impl::transform_reduce_default_functors_impl( return Impl::transform_reduce_default_functors_exespace_impl(
"Kokkos::transform_reduce_default_functors_iterator_api", ex, first1, "Kokkos::transform_reduce_default_functors_iterator_api", ex, first1,
last1, first2, std::move(init_reduction_value)); last1, first2, std::move(init_reduction_value));
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <typename ExecutionSpace, typename IteratorType1,
class ValueType> typename IteratorType2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType transform_reduce(const std::string& label, const ExecutionSpace& ex, ValueType transform_reduce(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 first2,
ValueType init_reduction_value) { ValueType init_reduction_value) {
return Impl::transform_reduce_default_functors_impl( return Impl::transform_reduce_default_functors_exespace_impl(
label, ex, first1, last1, first2, std::move(init_reduction_value)); label, ex, first1, last1, first2, std::move(init_reduction_value));
} }
// overload1 accepting views // overload1 accepting views
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class ValueType> typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType transform_reduce( ValueType transform_reduce(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& first_view, const ::Kokkos::View<DataType1, Properties1...>& first_view,
@ -60,14 +70,16 @@ ValueType transform_reduce(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view);
return Impl::transform_reduce_default_functors_impl( return Impl::transform_reduce_default_functors_exespace_impl(
"Kokkos::transform_reduce_default_functors_iterator_api", ex, "Kokkos::transform_reduce_default_functors_iterator_api", ex,
KE::cbegin(first_view), KE::cend(first_view), KE::cbegin(second_view), KE::cbegin(first_view), KE::cend(first_view), KE::cbegin(second_view),
std::move(init_reduction_value)); std::move(init_reduction_value));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class ValueType> typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType transform_reduce( ValueType transform_reduce(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& first_view, const ::Kokkos::View<DataType1, Properties1...>& first_view,
@ -77,7 +89,7 @@ ValueType transform_reduce(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view);
return Impl::transform_reduce_default_functors_impl( return Impl::transform_reduce_default_functors_exespace_impl(
label, ex, KE::cbegin(first_view), KE::cend(first_view), label, ex, KE::cbegin(first_view), KE::cend(first_view),
KE::cbegin(second_view), std::move(init_reduction_value)); KE::cbegin(second_view), std::move(init_reduction_value));
} }
@ -95,8 +107,11 @@ ValueType transform_reduce(
// https://en.cppreference.com/w/cpp/algorithm/transform_reduce // https://en.cppreference.com/w/cpp/algorithm/transform_reduce
// api accepting iterators // api accepting iterators
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class ValueType, class BinaryJoinerType, class BinaryTransform> typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
typename ValueType, typename BinaryJoinerType, typename BinaryTransform,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value, int> =
0>
ValueType transform_reduce(const ExecutionSpace& ex, IteratorType1 first1, ValueType transform_reduce(const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 last1, IteratorType2 first2, IteratorType1 last1, IteratorType2 first2,
ValueType init_reduction_value, ValueType init_reduction_value,
@ -105,14 +120,17 @@ ValueType transform_reduce(const ExecutionSpace& ex, IteratorType1 first1,
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::transform_reduce_custom_functors_impl( return Impl::transform_reduce_custom_functors_exespace_impl(
"Kokkos::transform_reduce_custom_functors_iterator_api", ex, first1, "Kokkos::transform_reduce_custom_functors_iterator_api", ex, first1,
last1, first2, std::move(init_reduction_value), std::move(joiner), last1, first2, std::move(init_reduction_value), std::move(joiner),
std::move(transformer)); std::move(transformer));
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <
class ValueType, class BinaryJoinerType, class BinaryTransform> typename ExecutionSpace, typename IteratorType1, typename IteratorType2,
typename ValueType, typename BinaryJoinerType, typename BinaryTransform,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value, int> =
0>
ValueType transform_reduce(const std::string& label, const ExecutionSpace& ex, ValueType transform_reduce(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, ValueType init_reduction_value, IteratorType2 first2, ValueType init_reduction_value,
@ -121,15 +139,17 @@ ValueType transform_reduce(const std::string& label, const ExecutionSpace& ex,
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::transform_reduce_custom_functors_impl( return Impl::transform_reduce_custom_functors_exespace_impl(
label, ex, first1, last1, first2, std::move(init_reduction_value), label, ex, first1, last1, first2, std::move(init_reduction_value),
std::move(joiner), std::move(transformer)); std::move(joiner), std::move(transformer));
} }
// accepting views // accepting views
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class ValueType, typename DataType2, typename... Properties2, typename ValueType,
class BinaryJoinerType, class BinaryTransform> typename BinaryJoinerType, typename BinaryTransform,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType transform_reduce( ValueType transform_reduce(
const ExecutionSpace& ex, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& first_view, const ::Kokkos::View<DataType1, Properties1...>& first_view,
@ -143,16 +163,18 @@ ValueType transform_reduce(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view);
return Impl::transform_reduce_custom_functors_impl( return Impl::transform_reduce_custom_functors_exespace_impl(
"Kokkos::transform_reduce_custom_functors_view_api", ex, "Kokkos::transform_reduce_custom_functors_view_api", ex,
KE::cbegin(first_view), KE::cend(first_view), KE::cbegin(second_view), KE::cbegin(first_view), KE::cend(first_view), KE::cbegin(second_view),
std::move(init_reduction_value), std::move(joiner), std::move(init_reduction_value), std::move(joiner),
std::move(transformer)); std::move(transformer));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <typename ExecutionSpace, typename DataType1, typename... Properties1,
class DataType2, class... Properties2, class ValueType, typename DataType2, typename... Properties2, typename ValueType,
class BinaryJoinerType, class BinaryTransform> typename BinaryJoinerType, typename BinaryTransform,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value,
int> = 0>
ValueType transform_reduce( ValueType transform_reduce(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& first_view, const ::Kokkos::View<DataType1, Properties1...>& first_view,
@ -166,7 +188,7 @@ ValueType transform_reduce(
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view);
return Impl::transform_reduce_custom_functors_impl( return Impl::transform_reduce_custom_functors_exespace_impl(
label, ex, KE::cbegin(first_view), KE::cend(first_view), label, ex, KE::cbegin(first_view), KE::cend(first_view),
KE::cbegin(second_view), std::move(init_reduction_value), KE::cbegin(second_view), std::move(init_reduction_value),
std::move(joiner), std::move(transformer)); std::move(joiner), std::move(transformer));
@ -176,43 +198,50 @@ ValueType transform_reduce(
// overload set3: // overload set3:
// //
// accepting iterators // accepting iterators
template <class ExecutionSpace, class IteratorType, class ValueType, template <typename ExecutionSpace, typename IteratorType, typename ValueType,
class BinaryJoinerType, class UnaryTransform> typename BinaryJoinerType, typename UnaryTransform,
// need this to avoid ambiguous call std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators<
std::enable_if_t< IteratorType>::value &&
::Kokkos::Experimental::Impl::are_iterators<IteratorType>::value, ValueType> is_execution_space<ExecutionSpace>::value,
transform_reduce(const ExecutionSpace& ex, IteratorType first1, int> = 0>
IteratorType last1, ValueType init_reduction_value, ValueType transform_reduce(const ExecutionSpace& ex, IteratorType first1,
BinaryJoinerType joiner, UnaryTransform transformer) { IteratorType last1, ValueType init_reduction_value,
BinaryJoinerType joiner,
UnaryTransform transformer) {
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::transform_reduce_custom_functors_impl( return Impl::transform_reduce_custom_functors_exespace_impl(
"Kokkos::transform_reduce_custom_functors_iterator_api", ex, first1, "Kokkos::transform_reduce_custom_functors_iterator_api", ex, first1,
last1, std::move(init_reduction_value), std::move(joiner), last1, std::move(init_reduction_value), std::move(joiner),
std::move(transformer)); std::move(transformer));
} }
template <class ExecutionSpace, class IteratorType, class ValueType, template <typename ExecutionSpace, typename IteratorType, typename ValueType,
class BinaryJoinerType, class UnaryTransform> typename BinaryJoinerType, typename UnaryTransform,
// need this to avoid ambiguous call std::enable_if_t<::Kokkos::Experimental::Impl::are_iterators<
std::enable_if_t< IteratorType>::value &&
::Kokkos::Experimental::Impl::are_iterators<IteratorType>::value, ValueType> is_execution_space<ExecutionSpace>::value,
transform_reduce(const std::string& label, const ExecutionSpace& ex, int> = 0>
IteratorType first1, IteratorType last1, ValueType transform_reduce(const std::string& label, const ExecutionSpace& ex,
ValueType init_reduction_value, BinaryJoinerType joiner, IteratorType first1, IteratorType last1,
UnaryTransform transformer) { ValueType init_reduction_value,
BinaryJoinerType joiner,
UnaryTransform transformer) {
static_assert(std::is_move_constructible<ValueType>::value, static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible."); "ValueType must be move constructible.");
return Impl::transform_reduce_custom_functors_impl( return Impl::transform_reduce_custom_functors_exespace_impl(
label, ex, first1, last1, std::move(init_reduction_value), label, ex, first1, last1, std::move(init_reduction_value),
std::move(joiner), std::move(transformer)); std::move(joiner), std::move(transformer));
} }
// accepting views // accepting views
template <class ExecutionSpace, class DataType, class... Properties, template <
class ValueType, class BinaryJoinerType, class UnaryTransform> typename ExecutionSpace, typename DataType, typename... Properties,
typename ValueType, typename BinaryJoinerType, typename UnaryTransform,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value, int> =
0>
ValueType transform_reduce(const ExecutionSpace& ex, ValueType transform_reduce(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ValueType init_reduction_value, ValueType init_reduction_value,
@ -224,14 +253,17 @@ ValueType transform_reduce(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::transform_reduce_custom_functors_impl( return Impl::transform_reduce_custom_functors_exespace_impl(
"Kokkos::transform_reduce_custom_functors_view_api", ex, KE::cbegin(view), "Kokkos::transform_reduce_custom_functors_view_api", ex, KE::cbegin(view),
KE::cend(view), std::move(init_reduction_value), std::move(joiner), KE::cend(view), std::move(init_reduction_value), std::move(joiner),
std::move(transformer)); std::move(transformer));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <
class ValueType, class BinaryJoinerType, class UnaryTransform> typename ExecutionSpace, typename DataType, typename... Properties,
typename ValueType, typename BinaryJoinerType, typename UnaryTransform,
std::enable_if_t<::Kokkos::is_execution_space<ExecutionSpace>::value, int> =
0>
ValueType transform_reduce(const std::string& label, const ExecutionSpace& ex, ValueType transform_reduce(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
ValueType init_reduction_value, ValueType init_reduction_value,
@ -243,12 +275,154 @@ ValueType transform_reduce(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::transform_reduce_custom_functors_impl( return Impl::transform_reduce_custom_functors_exespace_impl(
label, ex, KE::cbegin(view), KE::cend(view), label, ex, KE::cbegin(view), KE::cend(view),
std::move(init_reduction_value), std::move(joiner), std::move(init_reduction_value), std::move(joiner),
std::move(transformer)); std::move(transformer));
} }
//
// overload set accepting a team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
// ----------------------------
// overload set1:
// no custom functors passed, so equivalent to
// transform_reduce(first1, last1, first2, init, plus<>(), multiplies<>());
// ----------------------------
template <
typename TeamHandleType, typename IteratorType1, typename IteratorType2,
typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION ValueType transform_reduce(const TeamHandleType& teamHandle,
IteratorType1 first1,
IteratorType1 last1,
IteratorType2 first2,
ValueType init_reduction_value) {
return Impl::transform_reduce_default_functors_team_impl(
teamHandle, first1, last1, first2, std::move(init_reduction_value));
}
// overload1 accepting views
template <
typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION ValueType
transform_reduce(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& first_view,
const ::Kokkos::View<DataType2, Properties2...>& second_view,
ValueType init_reduction_value) {
namespace KE = ::Kokkos::Experimental;
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view);
return Impl::transform_reduce_default_functors_team_impl(
teamHandle, KE::cbegin(first_view), KE::cend(first_view),
KE::cbegin(second_view), std::move(init_reduction_value));
}
//
// overload set2:
// accepts a custom transform and joiner functor
//
// Note the std refers to the arg BinaryReductionOp
// but in the Kokkos naming convention, it corresponds
// to a "joiner" that knows how to join two values
// NOTE: "joiner/transformer" need to be commutative.
// https://en.cppreference.com/w/cpp/algorithm/transform_reduce
// api accepting iterators
template <
typename TeamHandleType, typename IteratorType1, typename IteratorType2,
typename ValueType, typename BinaryJoinerType, typename BinaryTransform,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION ValueType transform_reduce(
const TeamHandleType& teamHandle, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, ValueType init_reduction_value,
BinaryJoinerType joiner, BinaryTransform transformer) {
static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible.");
return Impl::transform_reduce_custom_functors_team_impl(
teamHandle, first1, last1, first2, std::move(init_reduction_value),
std::move(joiner), std::move(transformer));
}
// accepting views
template <
typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename ValueType,
typename BinaryJoinerType, typename BinaryTransform,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION ValueType
transform_reduce(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& first_view,
const ::Kokkos::View<DataType2, Properties2...>& second_view,
ValueType init_reduction_value, BinaryJoinerType joiner,
BinaryTransform transformer) {
namespace KE = ::Kokkos::Experimental;
static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible.");
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(first_view);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(second_view);
return Impl::transform_reduce_custom_functors_team_impl(
teamHandle, KE::cbegin(first_view), KE::cend(first_view),
KE::cbegin(second_view), std::move(init_reduction_value),
std::move(joiner), std::move(transformer));
}
//
// overload set3:
//
// accepting iterators
template <typename TeamHandleType, typename IteratorType, typename ValueType,
typename BinaryJoinerType, typename UnaryTransform,
std::enable_if_t<Impl::are_iterators<IteratorType>::value &&
is_team_handle<TeamHandleType>::value,
int> = 0>
KOKKOS_FUNCTION ValueType transform_reduce(const TeamHandleType& teamHandle,
IteratorType first1,
IteratorType last1,
ValueType init_reduction_value,
BinaryJoinerType joiner,
UnaryTransform transformer) {
static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible.");
return Impl::transform_reduce_custom_functors_team_impl(
teamHandle, first1, last1, std::move(init_reduction_value),
std::move(joiner), std::move(transformer));
}
// accepting views
template <
typename TeamHandleType, typename DataType, typename... Properties,
typename ValueType, typename BinaryJoinerType, typename UnaryTransform,
std::enable_if_t<::Kokkos::is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION ValueType
transform_reduce(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view,
ValueType init_reduction_value, BinaryJoinerType joiner,
UnaryTransform transformer) {
namespace KE = ::Kokkos::Experimental;
static_assert(std::is_move_constructible<ValueType>::value,
"ValueType must be move constructible.");
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::transform_reduce_custom_functors_team_impl(
teamHandle, KE::cbegin(view), KE::cend(view),
std::move(init_reduction_value), std::move(joiner),
std::move(transformer));
}
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

125
lib/kokkos/algorithms/src/std_algorithms/Kokkos_Unique.hpp
View File

@ -23,71 +23,132 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
// note: the enable_if below is to avoid "call to ... is ambiguous" //
// for example in the unit test when using a variadic function // overload set1: default predicate, accepting execution space
//
// overload set1 template <typename ExecutionSpace, typename IteratorType,
template <class ExecutionSpace, class IteratorType> std::enable_if_t<Impl::is_iterator_v<IteratorType> &&
std::enable_if_t<!::Kokkos::is_view<IteratorType>::value, IteratorType> unique( is_execution_space<ExecutionSpace>::value,
const ExecutionSpace& ex, IteratorType first, IteratorType last) { int> = 0>
return Impl::unique_impl("Kokkos::unique_iterator_api_default", ex, first, IteratorType unique(const ExecutionSpace& ex, IteratorType first,
last); IteratorType last) {
return Impl::unique_exespace_impl("Kokkos::unique_iterator_api_default", ex,
first, last);
} }
template <class ExecutionSpace, class IteratorType> template <typename ExecutionSpace, typename IteratorType,
std::enable_if_t<!::Kokkos::is_view<IteratorType>::value, IteratorType> unique( std::enable_if_t<Impl::is_iterator_v<IteratorType> &&
const std::string& label, const ExecutionSpace& ex, IteratorType first, is_execution_space<ExecutionSpace>::value,
IteratorType last) { int> = 0>
return Impl::unique_impl(label, ex, first, last); IteratorType unique(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last) {
return Impl::unique_exespace_impl(label, ex, first, last);
} }
template <class ExecutionSpace, class DataType, class... Properties> template <typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<is_execution_space<ExecutionSpace>::value, int> = 0>
auto unique(const ExecutionSpace& ex, auto unique(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return ::Kokkos::Experimental::unique("Kokkos::unique_view_api_default", ex, return Impl::unique_exespace_impl("Kokkos::unique_view_api_default", ex,
begin(view), end(view)); begin(view), end(view));
} }
template <class ExecutionSpace, class DataType, class... Properties> template <typename ExecutionSpace, typename DataType, typename... Properties,
std::enable_if_t<is_execution_space<ExecutionSpace>::value, int> = 0>
auto unique(const std::string& label, const ExecutionSpace& ex, auto unique(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view) { const ::Kokkos::View<DataType, Properties...>& view) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return ::Kokkos::Experimental::unique(label, ex, begin(view), end(view)); return Impl::unique_exespace_impl(label, ex, begin(view), end(view));
} }
// overload set2 //
template <class ExecutionSpace, class IteratorType, class BinaryPredicate> // overload set2: custom predicate, accepting execution space
//
template <typename ExecutionSpace, typename IteratorType,
typename BinaryPredicate,
std::enable_if_t<is_execution_space<ExecutionSpace>::value, int> = 0>
IteratorType unique(const ExecutionSpace& ex, IteratorType first, IteratorType unique(const ExecutionSpace& ex, IteratorType first,
IteratorType last, BinaryPredicate pred) { IteratorType last, BinaryPredicate pred) {
return Impl::unique_impl("Kokkos::unique_iterator_api_default", ex, first, return Impl::unique_exespace_impl("Kokkos::unique_iterator_api_default", ex,
last, pred); first, last, pred);
} }
template <class ExecutionSpace, class IteratorType, class BinaryPredicate> template <typename ExecutionSpace, typename IteratorType,
typename BinaryPredicate,
std::enable_if_t<is_execution_space<ExecutionSpace>::value, int> = 0>
IteratorType unique(const std::string& label, const ExecutionSpace& ex, IteratorType unique(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
BinaryPredicate pred) { BinaryPredicate pred) {
return Impl::unique_impl(label, ex, first, last, pred); return Impl::unique_exespace_impl(label, ex, first, last, pred);
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class BinaryPredicate> typename BinaryPredicate,
std::enable_if_t<is_execution_space<ExecutionSpace>::value, int> = 0>
auto unique(const ExecutionSpace& ex, auto unique(const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
BinaryPredicate pred) { BinaryPredicate pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::unique_impl("Kokkos::unique_view_api_default", ex, begin(view), return Impl::unique_exespace_impl("Kokkos::unique_view_api_default", ex,
end(view), std::move(pred)); begin(view), end(view), std::move(pred));
} }
template <class ExecutionSpace, class DataType, class... Properties, template <typename ExecutionSpace, typename DataType, typename... Properties,
class BinaryPredicate> typename BinaryPredicate,
std::enable_if_t<is_execution_space<ExecutionSpace>::value, int> = 0>
auto unique(const std::string& label, const ExecutionSpace& ex, auto unique(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType, Properties...>& view, const ::Kokkos::View<DataType, Properties...>& view,
BinaryPredicate pred) { BinaryPredicate pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(view);
return Impl::unique_impl(label, ex, begin(view), end(view), std::move(pred)); return Impl::unique_exespace_impl(label, ex, begin(view), end(view),
std::move(pred));
}
//
// overload set3: default predicate, accepting team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
std::enable_if_t<Impl::is_iterator_v<IteratorType> &&
is_team_handle<TeamHandleType>::value,
int> = 0>
KOKKOS_FUNCTION IteratorType unique(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last) {
return Impl::unique_team_impl(teamHandle, first, last);
}
template <typename TeamHandleType, typename DataType, typename... Properties,
std::enable_if_t<is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION auto unique(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view) {
return Impl::unique_team_impl(teamHandle, begin(view), end(view));
}
//
// overload set4: custom predicate, accepting team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename IteratorType,
typename BinaryPredicate,
std::enable_if_t<is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION IteratorType unique(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
BinaryPredicate pred) {
return Impl::unique_team_impl(teamHandle, first, last, std::move(pred));
}
template <typename TeamHandleType, typename DataType, typename... Properties,
typename BinaryPredicate,
std::enable_if_t<is_team_handle<TeamHandleType>::value, int> = 0>
KOKKOS_FUNCTION auto unique(const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType, Properties...>& view,
BinaryPredicate pred) {
return Impl::unique_team_impl(teamHandle, begin(view), end(view),
std::move(pred));
} }
} // namespace Experimental } // namespace Experimental

163
lib/kokkos/algorithms/src/std_algorithms/Kokkos_UniqueCopy.hpp
View File

@ -23,67 +23,90 @@
namespace Kokkos { namespace Kokkos {
namespace Experimental { namespace Experimental {
// overload set1 //
template <class ExecutionSpace, class InputIterator, class OutputIterator> // overload set1: default predicate, accepting execution space
std::enable_if_t<!::Kokkos::is_view<InputIterator>::value, OutputIterator> //
unique_copy(const ExecutionSpace& ex, InputIterator first, InputIterator last, template <
OutputIterator d_first) { typename ExecutionSpace, typename InputIterator, typename OutputIterator,
return Impl::unique_copy_impl("Kokkos::unique_copy_iterator_api_default", ex, std::enable_if_t<Impl::are_iterators_v<InputIterator, OutputIterator> &&
first, last, d_first); is_execution_space_v<ExecutionSpace>,
int> = 0>
OutputIterator unique_copy(const ExecutionSpace& ex, InputIterator first,
InputIterator last, OutputIterator d_first) {
return Impl::unique_copy_exespace_impl(
"Kokkos::unique_copy_iterator_api_default", ex, first, last, d_first);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator> template <
std::enable_if_t<!::Kokkos::is_view<InputIterator>::value, OutputIterator> typename ExecutionSpace, typename InputIterator, typename OutputIterator,
unique_copy(const std::string& label, const ExecutionSpace& ex, std::enable_if_t<Impl::are_iterators_v<InputIterator, OutputIterator> &&
InputIterator first, InputIterator last, OutputIterator d_first) { is_execution_space_v<ExecutionSpace>,
return Impl::unique_copy_impl(label, ex, first, last, d_first); int> = 0>
OutputIterator unique_copy(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last,
OutputIterator d_first) {
return Impl::unique_copy_exespace_impl(label, ex, first, last, d_first);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto unique_copy(const ExecutionSpace& ex, auto unique_copy(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
const ::Kokkos::View<DataType2, Properties2...>& dest) { const ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return ::Kokkos::Experimental::unique_copy( return Impl::unique_copy_exespace_impl("Kokkos::unique_copy_view_api_default",
"Kokkos::unique_copy_view_api_default", ex, cbegin(source), cend(source), ex, cbegin(source), cend(source),
begin(dest)); begin(dest));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto unique_copy(const std::string& label, const ExecutionSpace& ex, auto unique_copy(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
const ::Kokkos::View<DataType2, Properties2...>& dest) { const ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return ::Kokkos::Experimental::unique_copy(label, ex, cbegin(source), return Impl::unique_copy_exespace_impl(label, ex, cbegin(source),
cend(source), begin(dest)); cend(source), begin(dest));
} }
// overload set2 //
template <class ExecutionSpace, class InputIterator, class OutputIterator, // overload set2: custom predicate, accepting execution space
class BinaryPredicate> //
template <
typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename BinaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator unique_copy(const ExecutionSpace& ex, InputIterator first, OutputIterator unique_copy(const ExecutionSpace& ex, InputIterator first,
InputIterator last, OutputIterator d_first, InputIterator last, OutputIterator d_first,
BinaryPredicate pred) { BinaryPredicate pred) {
return Impl::unique_copy_impl("Kokkos::unique_copy_iterator_api_default", ex, return Impl::unique_copy_exespace_impl(
first, last, d_first, pred); "Kokkos::unique_copy_iterator_api_default", ex, first, last, d_first,
pred);
} }
template <class ExecutionSpace, class InputIterator, class OutputIterator, template <
class BinaryPredicate> typename ExecutionSpace, typename InputIterator, typename OutputIterator,
typename BinaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
OutputIterator unique_copy(const std::string& label, const ExecutionSpace& ex, OutputIterator unique_copy(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, InputIterator first, InputIterator last,
OutputIterator d_first, BinaryPredicate pred) { OutputIterator d_first, BinaryPredicate pred) {
return Impl::unique_copy_impl(label, ex, first, last, d_first, pred); return Impl::unique_copy_exespace_impl(label, ex, first, last, d_first, pred);
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicate> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto unique_copy(const ExecutionSpace& ex, auto unique_copy(const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
const ::Kokkos::View<DataType2, Properties2...>& dest, const ::Kokkos::View<DataType2, Properties2...>& dest,
@ -91,13 +114,15 @@ auto unique_copy(const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::unique_copy_impl("Kokkos::unique_copy_view_api_default", ex, return Impl::unique_copy_exespace_impl("Kokkos::unique_copy_view_api_default",
cbegin(source), cend(source), begin(dest), ex, cbegin(source), cend(source),
std::move(pred)); begin(dest), std::move(pred));
} }
template <class ExecutionSpace, class DataType1, class... Properties1, template <
class DataType2, class... Properties2, class BinaryPredicate> typename ExecutionSpace, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicate,
std::enable_if_t<::Kokkos::is_execution_space_v<ExecutionSpace>, int> = 0>
auto unique_copy(const std::string& label, const ExecutionSpace& ex, auto unique_copy(const std::string& label, const ExecutionSpace& ex,
const ::Kokkos::View<DataType1, Properties1...>& source, const ::Kokkos::View<DataType1, Properties1...>& source,
const ::Kokkos::View<DataType2, Properties2...>& dest, const ::Kokkos::View<DataType2, Properties2...>& dest,
@ -105,8 +130,70 @@ auto unique_copy(const std::string& label, const ExecutionSpace& ex,
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest); Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::unique_copy_impl(label, ex, cbegin(source), cend(source), return Impl::unique_copy_exespace_impl(
begin(dest), std::move(pred)); label, ex, cbegin(source), cend(source), begin(dest), std::move(pred));
}
//
// overload set3: default predicate, accepting team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <
typename TeamHandleType, typename InputIterator, typename OutputIterator,
std::enable_if_t<Impl::are_iterators_v<InputIterator, OutputIterator> &&
Kokkos::is_team_handle_v<TeamHandleType>,
int> = 0>
KOKKOS_FUNCTION OutputIterator unique_copy(const TeamHandleType& teamHandle,
InputIterator first,
InputIterator last,
OutputIterator d_first) {
return Impl::unique_copy_team_impl(teamHandle, first, last, d_first);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto unique_copy(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
const ::Kokkos::View<DataType2, Properties2...>& dest) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::unique_copy_team_impl(teamHandle, cbegin(source), cend(source),
begin(dest));
}
//
// overload set4: custom predicate, accepting team handle
// Note: for now omit the overloads accepting a label
// since they cause issues on device because of the string allocation.
//
template <typename TeamHandleType, typename InputIterator,
typename OutputIterator, typename BinaryPredicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION OutputIterator unique_copy(const TeamHandleType& teamHandle,
InputIterator first,
InputIterator last,
OutputIterator d_first,
BinaryPredicate pred) {
return Impl::unique_copy_team_impl(teamHandle, first, last, d_first, pred);
}
template <typename TeamHandleType, typename DataType1, typename... Properties1,
typename DataType2, typename... Properties2, typename BinaryPredicate,
std::enable_if_t<::Kokkos::is_team_handle_v<TeamHandleType>, int> = 0>
KOKKOS_FUNCTION auto unique_copy(
const TeamHandleType& teamHandle,
const ::Kokkos::View<DataType1, Properties1...>& source,
const ::Kokkos::View<DataType2, Properties2...>& dest,
BinaryPredicate pred) {
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(source);
Impl::static_assert_is_admissible_to_kokkos_std_algorithms(dest);
return Impl::unique_copy_team_impl(teamHandle, cbegin(source), cend(source),
begin(dest), std::move(pred));
} }
} // namespace Experimental } // namespace Experimental

58
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_AdjacentDifference.hpp
View File

@ -63,14 +63,15 @@ struct StdAdjacentDiffFunctor {
m_op(std::move(op)) {} m_op(std::move(op)) {}
}; };
//
// exespace impl
//
template <class ExecutionSpace, class InputIteratorType, template <class ExecutionSpace, class InputIteratorType,
class OutputIteratorType, class BinaryOp> class OutputIteratorType, class BinaryOp>
OutputIteratorType adjacent_difference_impl(const std::string& label, OutputIteratorType adjacent_difference_exespace_impl(
const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
InputIteratorType first_from, InputIteratorType first_from, InputIteratorType last_from,
InputIteratorType last_from, OutputIteratorType first_dest, BinaryOp bin_op) {
OutputIteratorType first_dest,
BinaryOp bin_op) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first_from, first_dest); Impl::static_assert_random_access_and_accessible(ex, first_from, first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from, Impl::static_assert_iterators_have_matching_difference_type(first_from,
@ -81,20 +82,45 @@ OutputIteratorType adjacent_difference_impl(const std::string& label,
return first_dest; return first_dest;
} }
// aliases // run
using value_type = typename OutputIteratorType::value_type; const auto num_elements =
using aux_view_type = ::Kokkos::View<value_type*, ExecutionSpace>; Kokkos::Experimental::distance(first_from, last_from);
using functor_t = ::Kokkos::parallel_for(
StdAdjacentDiffFunctor<InputIteratorType, OutputIteratorType, BinaryOp>; label, RangePolicy<ExecutionSpace>(ex, 0, num_elements),
StdAdjacentDiffFunctor(first_from, first_dest, bin_op));
ex.fence("Kokkos::adjacent_difference: fence after operation");
// return
return first_dest + num_elements;
}
//
// team impl
//
template <class TeamHandleType, class InputIteratorType,
class OutputIteratorType, class BinaryOp>
KOKKOS_FUNCTION OutputIteratorType adjacent_difference_team_impl(
const TeamHandleType& teamHandle, InputIteratorType first_from,
InputIteratorType last_from, OutputIteratorType first_dest,
BinaryOp bin_op) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first_from,
first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest);
Impl::expect_valid_range(first_from, last_from);
if (first_from == last_from) {
return first_dest;
}
// run // run
const auto num_elements = const auto num_elements =
Kokkos::Experimental::distance(first_from, last_from); Kokkos::Experimental::distance(first_from, last_from);
aux_view_type aux_view("aux_view", num_elements); ::Kokkos::parallel_for(
::Kokkos::parallel_for(label, TeamThreadRange(teamHandle, 0, num_elements),
RangePolicy<ExecutionSpace>(ex, 0, num_elements), StdAdjacentDiffFunctor(first_from, first_dest, bin_op));
functor_t(first_from, first_dest, bin_op)); teamHandle.team_barrier();
ex.fence("Kokkos::adjacent_difference: fence after operation");
// return // return
return first_dest + num_elements; return first_dest + num_elements;

81
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_AdjacentFind.hpp
View File

@ -27,9 +27,9 @@ namespace Kokkos {
namespace Experimental { namespace Experimental {
namespace Impl { namespace Impl {
template <class IndexType, class IteratorType, class ReducerType, template <class IteratorType, class ReducerType, class PredicateType>
class PredicateType>
struct StdAdjacentFindFunctor { struct StdAdjacentFindFunctor {
using index_type = typename IteratorType::difference_type;
using red_value_type = typename ReducerType::value_type; using red_value_type = typename ReducerType::value_type;
IteratorType m_first; IteratorType m_first;
@ -37,13 +37,13 @@ struct StdAdjacentFindFunctor {
PredicateType m_p; PredicateType m_p;
KOKKOS_FUNCTION KOKKOS_FUNCTION
void operator()(const IndexType i, red_value_type& red_value) const { void operator()(const index_type i, red_value_type& red_value) const {
const auto& my_value = m_first[i]; const auto& my_value = m_first[i];
const auto& next_value = m_first[i + 1]; const auto& next_value = m_first[i + 1];
const bool are_equal = m_p(my_value, next_value); const bool are_equal = m_p(my_value, next_value);
// FIXME_NVHPC using a ternary operator causes problems // FIXME_NVHPC using a ternary operator causes problems
red_value_type value = {::Kokkos::reduction_identity<IndexType>::min()}; red_value_type value = {::Kokkos::reduction_identity<index_type>::min()};
if (are_equal) { if (are_equal) {
value.min_loc_true = i; value.min_loc_true = i;
} }
@ -59,10 +59,14 @@ struct StdAdjacentFindFunctor {
m_p(std::move(p)) {} m_p(std::move(p)) {}
}; };
//
// exespace impl
//
template <class ExecutionSpace, class IteratorType, class PredicateType> template <class ExecutionSpace, class IteratorType, class PredicateType>
IteratorType adjacent_find_impl(const std::string& label, IteratorType adjacent_find_exespace_impl(const std::string& label,
const ExecutionSpace& ex, IteratorType first, const ExecutionSpace& ex,
IteratorType last, PredicateType pred) { IteratorType first, IteratorType last,
PredicateType pred) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first); Impl::static_assert_random_access_and_accessible(ex, first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
@ -76,8 +80,6 @@ IteratorType adjacent_find_impl(const std::string& label,
using index_type = typename IteratorType::difference_type; using index_type = typename IteratorType::difference_type;
using reducer_type = FirstLoc<index_type>; using reducer_type = FirstLoc<index_type>;
using reduction_value_type = typename reducer_type::value_type; using reduction_value_type = typename reducer_type::value_type;
using func_t = StdAdjacentFindFunctor<index_type, IteratorType, reducer_type,
PredicateType>;
reduction_value_type red_result; reduction_value_type red_result;
reducer_type reducer(red_result); reducer_type reducer(red_result);
@ -86,7 +88,8 @@ IteratorType adjacent_find_impl(const std::string& label,
// each index i in the reduction checks i and (i+1). // each index i in the reduction checks i and (i+1).
::Kokkos::parallel_reduce( ::Kokkos::parallel_reduce(
label, RangePolicy<ExecutionSpace>(ex, 0, num_elements - 1), label, RangePolicy<ExecutionSpace>(ex, 0, num_elements - 1),
func_t(first, reducer, pred), reducer); // use CTAD
StdAdjacentFindFunctor(first, reducer, pred), reducer);
// fence not needed because reducing into scalar // fence not needed because reducing into scalar
if (red_result.min_loc_true == if (red_result.min_loc_true ==
@ -98,12 +101,62 @@ IteratorType adjacent_find_impl(const std::string& label,
} }
template <class ExecutionSpace, class IteratorType> template <class ExecutionSpace, class IteratorType>
IteratorType adjacent_find_impl(const std::string& label, IteratorType adjacent_find_exespace_impl(const std::string& label,
const ExecutionSpace& ex, IteratorType first, const ExecutionSpace& ex,
IteratorType last) { IteratorType first,
IteratorType last) {
using value_type = typename IteratorType::value_type; using value_type = typename IteratorType::value_type;
using default_pred_t = StdAlgoEqualBinaryPredicate<value_type>; using default_pred_t = StdAlgoEqualBinaryPredicate<value_type>;
return adjacent_find_impl(label, ex, first, last, default_pred_t()); return adjacent_find_exespace_impl(label, ex, first, last, default_pred_t());
}
//
// team impl
//
template <class TeamHandleType, class IteratorType, class PredicateType>
KOKKOS_FUNCTION IteratorType
adjacent_find_team_impl(const TeamHandleType& teamHandle, IteratorType first,
IteratorType last, PredicateType pred) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
const auto num_elements = Kokkos::Experimental::distance(first, last);
if (num_elements <= 1) {
return last;
}
using index_type = typename IteratorType::difference_type;
using reducer_type = FirstLoc<index_type>;
using reduction_value_type = typename reducer_type::value_type;
reduction_value_type red_result;
reducer_type reducer(red_result);
// note that we use below num_elements-1 because
// each index i in the reduction checks i and (i+1).
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, num_elements - 1),
// use CTAD
StdAdjacentFindFunctor(first, reducer, pred),
reducer);
teamHandle.team_barrier();
if (red_result.min_loc_true ==
::Kokkos::reduction_identity<index_type>::min()) {
return last;
} else {
return first + red_result.min_loc_true;
}
}
template <class TeamHandleType, class IteratorType>
KOKKOS_FUNCTION IteratorType adjacent_find_team_impl(
const TeamHandleType& teamHandle, IteratorType first, IteratorType last) {
using value_type = typename IteratorType::value_type;
using default_pred_t = StdAlgoEqualBinaryPredicate<value_type>;
return adjacent_find_team_impl(teamHandle, first, last, default_pred_t());
} }
} // namespace Impl } // namespace Impl

55
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_AllOfAnyOfNoneOf.hpp
View File

@ -23,23 +23,58 @@ namespace Kokkos {
namespace Experimental { namespace Experimental {
namespace Impl { namespace Impl {
//
// exespace impl
//
template <class ExecutionSpace, class InputIterator, class Predicate> template <class ExecutionSpace, class InputIterator, class Predicate>
bool all_of_impl(const std::string& label, const ExecutionSpace& ex, bool all_of_exespace_impl(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, Predicate predicate) { InputIterator first, InputIterator last,
return (find_if_or_not_impl<false>(label, ex, first, last, predicate) == Predicate predicate) {
last); return (find_if_or_not_exespace_impl<false>(label, ex, first, last,
predicate) == last);
} }
template <class ExecutionSpace, class InputIterator, class Predicate> template <class ExecutionSpace, class InputIterator, class Predicate>
bool any_of_impl(const std::string& label, const ExecutionSpace& ex, bool any_of_exespace_impl(const std::string& label, const ExecutionSpace& ex,
InputIterator first, InputIterator last, Predicate predicate) { InputIterator first, InputIterator last,
return (find_if_or_not_impl<true>(label, ex, first, last, predicate) != last); Predicate predicate) {
return (find_if_or_not_exespace_impl<true>(label, ex, first, last,
predicate) != last);
} }
template <class ExecutionSpace, class IteratorType, class Predicate> template <class ExecutionSpace, class IteratorType, class Predicate>
bool none_of_impl(const std::string& label, const ExecutionSpace& ex, bool none_of_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, Predicate predicate) { IteratorType first, IteratorType last,
return (find_if_or_not_impl<true>(label, ex, first, last, predicate) == last); Predicate predicate) {
return (find_if_or_not_exespace_impl<true>(label, ex, first, last,
predicate) == last);
}
//
// team impl
//
template <class TeamHandleType, class InputIterator, class Predicate>
KOKKOS_FUNCTION bool all_of_team_impl(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last,
Predicate predicate) {
return (find_if_or_not_team_impl<false>(teamHandle, first, last, predicate) ==
last);
}
template <class TeamHandleType, class InputIterator, class Predicate>
KOKKOS_FUNCTION bool any_of_team_impl(const TeamHandleType& teamHandle,
InputIterator first, InputIterator last,
Predicate predicate) {
return (find_if_or_not_team_impl<true>(teamHandle, first, last, predicate) !=
last);
}
template <class TeamHandleType, class IteratorType, class Predicate>
KOKKOS_FUNCTION bool none_of_team_impl(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
Predicate predicate) {
return (find_if_or_not_team_impl<true>(teamHandle, first, last, predicate) ==
last);
} }
} // namespace Impl } // namespace Impl

47
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_Constraints.hpp
View File

@ -55,6 +55,9 @@ using iterator_category_t = typename T::iterator_category;
template <class T> template <class T>
using is_iterator = Kokkos::is_detected<iterator_category_t, T>; using is_iterator = Kokkos::is_detected<iterator_category_t, T>;
template <class T>
inline constexpr bool is_iterator_v = is_iterator<T>::value;
// //
// are_iterators // are_iterators
// //
@ -63,15 +66,18 @@ struct are_iterators;
template <class T> template <class T>
struct are_iterators<T> { struct are_iterators<T> {
static constexpr bool value = is_iterator<T>::value; static constexpr bool value = is_iterator_v<T>;
}; };
template <class Head, class... Tail> template <class Head, class... Tail>
struct are_iterators<Head, Tail...> { struct are_iterators<Head, Tail...> {
static constexpr bool value = static constexpr bool value =
are_iterators<Head>::value && are_iterators<Tail...>::value; are_iterators<Head>::value && (are_iterators<Tail>::value && ... && true);
}; };
template <class... Ts>
inline constexpr bool are_iterators_v = are_iterators<Ts...>::value;
// //
// are_random_access_iterators // are_random_access_iterators
// //
@ -81,17 +87,21 @@ struct are_random_access_iterators;
template <class T> template <class T>
struct are_random_access_iterators<T> { struct are_random_access_iterators<T> {
static constexpr bool value = static constexpr bool value =
is_iterator<T>::value && is_iterator_v<T> && std::is_base_of<std::random_access_iterator_tag,
std::is_base_of<std::random_access_iterator_tag, typename T::iterator_category>::value;
typename T::iterator_category>::value;
}; };
template <class Head, class... Tail> template <class Head, class... Tail>
struct are_random_access_iterators<Head, Tail...> { struct are_random_access_iterators<Head, Tail...> {
static constexpr bool value = are_random_access_iterators<Head>::value && static constexpr bool value =
are_random_access_iterators<Tail...>::value; are_random_access_iterators<Head>::value &&
(are_random_access_iterators<Tail>::value && ... && true);
}; };
template <class... Ts>
inline constexpr bool are_random_access_iterators_v =
are_random_access_iterators<Ts...>::value;
// //
// iterators_are_accessible_from // iterators_are_accessible_from
// //
@ -113,16 +123,18 @@ struct iterators_are_accessible_from<ExeSpace, Head, Tail...> {
iterators_are_accessible_from<ExeSpace, Tail...>::value; iterators_are_accessible_from<ExeSpace, Tail...>::value;
}; };
template <class ExecutionSpace, class... IteratorTypes> template <class ExecutionSpaceOrTeamHandleType, class... IteratorTypes>
KOKKOS_INLINE_FUNCTION constexpr void KOKKOS_INLINE_FUNCTION constexpr void
static_assert_random_access_and_accessible(const ExecutionSpace& /* ex */, static_assert_random_access_and_accessible(
IteratorTypes... /* iterators */) { const ExecutionSpaceOrTeamHandleType& /* ex_or_th*/,
IteratorTypes... /* iterators */) {
static_assert( static_assert(
are_random_access_iterators<IteratorTypes...>::value, are_random_access_iterators<IteratorTypes...>::value,
"Currently, Kokkos standard algorithms require random access iterators."); "Currently, Kokkos standard algorithms require random access iterators.");
static_assert( static_assert(iterators_are_accessible_from<
iterators_are_accessible_from<ExecutionSpace, IteratorTypes...>::value, typename ExecutionSpaceOrTeamHandleType::execution_space,
"Incompatible view/iterator and execution space"); IteratorTypes...>::value,
"Incompatible view/iterator and execution space");
} }
// //
@ -182,10 +194,10 @@ struct not_openmptarget {
#endif #endif
}; };
template <class ExecutionSpace> template <class ExecutionSpaceOrTeamHandleType>
KOKKOS_INLINE_FUNCTION constexpr void static_assert_is_not_openmptarget( KOKKOS_INLINE_FUNCTION constexpr void static_assert_is_not_openmptarget(
const ExecutionSpace&) { const ExecutionSpaceOrTeamHandleType& /*ex_or_th*/) {
static_assert(not_openmptarget<ExecutionSpace>::value, static_assert(not_openmptarget<ExecutionSpaceOrTeamHandleType>::value,
"Currently, Kokkos standard algorithms do not support custom " "Currently, Kokkos standard algorithms do not support custom "
"comparators in OpenMPTarget"); "comparators in OpenMPTarget");
} }
@ -194,7 +206,8 @@ KOKKOS_INLINE_FUNCTION constexpr void static_assert_is_not_openmptarget(
// valid range // valid range
// //
template <class IteratorType> template <class IteratorType>
void expect_valid_range(IteratorType first, IteratorType last) { KOKKOS_INLINE_FUNCTION void expect_valid_range(IteratorType first,
IteratorType last) {
// this is a no-op for release // this is a no-op for release
KOKKOS_EXPECTS(last >= first); KOKKOS_EXPECTS(last >= first);
// avoid compiler complaining when KOKKOS_EXPECTS is no-op // avoid compiler complaining when KOKKOS_EXPECTS is no-op

49
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_CopyBackward.hpp
View File

@ -27,16 +27,18 @@ namespace Kokkos {
namespace Experimental { namespace Experimental {
namespace Impl { namespace Impl {
template <class IndexType, class IteratorType1, class IteratorType2> template <class IteratorType1, class IteratorType2>
struct StdCopyBackwardFunctor { struct StdCopyBackwardFunctor {
static_assert(std::is_signed<IndexType>::value, // we can use difference type from IteratorType1 since
"Kokkos: StdCopyBackwardFunctor requires signed index type"); // the calling functions below already static assert that
// the iterators have matching difference type
using index_type = typename IteratorType1::difference_type;
IteratorType1 m_last; IteratorType1 m_last;
IteratorType2 m_dest_last; IteratorType2 m_dest_last;
KOKKOS_FUNCTION KOKKOS_FUNCTION
void operator()(IndexType i) const { m_dest_last[-i - 1] = m_last[-i - 1]; } void operator()(index_type i) const { m_dest_last[-i - 1] = m_last[-i - 1]; }
KOKKOS_FUNCTION KOKKOS_FUNCTION
StdCopyBackwardFunctor(IteratorType1 _last, IteratorType2 _dest_last) StdCopyBackwardFunctor(IteratorType1 _last, IteratorType2 _dest_last)
@ -44,30 +46,51 @@ struct StdCopyBackwardFunctor {
}; };
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <class ExecutionSpace, class IteratorType1, class IteratorType2>
IteratorType2 copy_backward_impl(const std::string& label, IteratorType2 copy_backward_exespace_impl(const std::string& label,
const ExecutionSpace& ex, IteratorType1 first, const ExecutionSpace& ex,
IteratorType1 last, IteratorType2 d_last) { IteratorType1 first,
IteratorType1 last,
IteratorType2 d_last) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first, d_last); Impl::static_assert_random_access_and_accessible(ex, first, d_last);
Impl::static_assert_iterators_have_matching_difference_type(first, d_last); Impl::static_assert_iterators_have_matching_difference_type(first, d_last);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
// aliases
using index_type = typename IteratorType1::difference_type;
using func_t =
StdCopyBackwardFunctor<index_type, IteratorType1, IteratorType2>;
// run // run
const auto num_elements = Kokkos::Experimental::distance(first, last); const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for(label, ::Kokkos::parallel_for(label,
RangePolicy<ExecutionSpace>(ex, 0, num_elements), RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_t(last, d_last)); // use CTAD
StdCopyBackwardFunctor(last, d_last));
ex.fence("Kokkos::copy_backward: fence after operation"); ex.fence("Kokkos::copy_backward: fence after operation");
// return // return
return d_last - num_elements; return d_last - num_elements;
} }
//
// team-level impl
//
template <class TeamHandleType, class IteratorType1, class IteratorType2>
KOKKOS_FUNCTION IteratorType2
copy_backward_team_impl(const TeamHandleType& teamHandle, IteratorType1 first,
IteratorType1 last, IteratorType2 d_last) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first, d_last);
Impl::static_assert_iterators_have_matching_difference_type(first, d_last);
Impl::expect_valid_range(first, last);
// run
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for(TeamThreadRange(teamHandle, 0, num_elements),
// use CTAD
StdCopyBackwardFunctor(last, d_last));
teamHandle.team_barrier();
// return
return d_last - num_elements;
}
} // namespace Impl } // namespace Impl
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

75
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_CopyCopyN.hpp
View File

@ -27,13 +27,18 @@ namespace Kokkos {
namespace Experimental { namespace Experimental {
namespace Impl { namespace Impl {
template <class IndexType, class InputIterator, class OutputIterator> template <class InputIterator, class OutputIterator>
struct StdCopyFunctor { struct StdCopyFunctor {
// we can use difference type from InputIterator since
// the calling functions below already static assert that
// the iterators have matching difference type
using index_type = typename InputIterator::difference_type;
InputIterator m_first; InputIterator m_first;
OutputIterator m_dest_first; OutputIterator m_dest_first;
KOKKOS_FUNCTION KOKKOS_FUNCTION
void operator()(IndexType i) const { m_dest_first[i] = m_first[i]; } void operator()(index_type i) const { m_dest_first[i] = m_first[i]; }
KOKKOS_FUNCTION KOKKOS_FUNCTION
StdCopyFunctor(InputIterator _first, OutputIterator _dest_first) StdCopyFunctor(InputIterator _first, OutputIterator _dest_first)
@ -41,23 +46,20 @@ struct StdCopyFunctor {
}; };
template <class ExecutionSpace, class InputIterator, class OutputIterator> template <class ExecutionSpace, class InputIterator, class OutputIterator>
OutputIterator copy_impl(const std::string& label, const ExecutionSpace& ex, OutputIterator copy_exespace_impl(const std::string& label,
InputIterator first, InputIterator last, const ExecutionSpace& ex, InputIterator first,
OutputIterator d_first) { InputIterator last, OutputIterator d_first) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first, d_first); Impl::static_assert_random_access_and_accessible(ex, first, d_first);
Impl::static_assert_iterators_have_matching_difference_type(first, d_first); Impl::static_assert_iterators_have_matching_difference_type(first, d_first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
// aliases
using index_type = typename InputIterator::difference_type;
using func_t = StdCopyFunctor<index_type, InputIterator, OutputIterator>;
// run // run
const auto num_elements = Kokkos::Experimental::distance(first, last); const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for(label, ::Kokkos::parallel_for(label,
RangePolicy<ExecutionSpace>(ex, 0, num_elements), RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_t(first, d_first)); // use CTAD
StdCopyFunctor(first, d_first));
ex.fence("Kokkos::copy: fence after operation"); ex.fence("Kokkos::copy: fence after operation");
// return // return
@ -66,16 +68,61 @@ OutputIterator copy_impl(const std::string& label, const ExecutionSpace& ex,
template <class ExecutionSpace, class InputIterator, class Size, template <class ExecutionSpace, class InputIterator, class Size,
class OutputIterator> class OutputIterator>
OutputIterator copy_n_impl(const std::string& label, const ExecutionSpace& ex, OutputIterator copy_n_exespace_impl(const std::string& label,
InputIterator first_from, Size count, const ExecutionSpace& ex,
OutputIterator first_dest) { InputIterator first_from, Size count,
OutputIterator first_dest) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first_from, first_dest); Impl::static_assert_random_access_and_accessible(ex, first_from, first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from, Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest); first_dest);
if (count > 0) { if (count > 0) {
return copy_impl(label, ex, first_from, first_from + count, first_dest); return copy_exespace_impl(label, ex, first_from, first_from + count,
first_dest);
} else {
return first_dest;
}
}
//
// team-level impl
//
template <class TeamHandleType, class InputIterator, class OutputIterator>
KOKKOS_FUNCTION OutputIterator copy_team_impl(const TeamHandleType& teamHandle,
InputIterator first,
InputIterator last,
OutputIterator d_first) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first, d_first);
Impl::static_assert_iterators_have_matching_difference_type(first, d_first);
Impl::expect_valid_range(first, last);
// run
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for(TeamThreadRange(teamHandle, 0, num_elements),
// use CTAD
StdCopyFunctor(first, d_first));
teamHandle.team_barrier();
// return
return d_first + num_elements;
}
template <class TeamHandleType, class InputIterator, class Size,
class OutputIterator>
KOKKOS_FUNCTION OutputIterator
copy_n_team_impl(const TeamHandleType& teamHandle, InputIterator first_from,
Size count, OutputIterator first_dest) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first_from,
first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest);
if (count > 0) {
return copy_team_impl(teamHandle, first_from, first_from + count,
first_dest);
} else { } else {
return first_dest; return first_dest;
} }

73
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_CopyIf.hpp
View File

@ -20,6 +20,7 @@
#include <Kokkos_Core.hpp> #include <Kokkos_Core.hpp>
#include "Kokkos_Constraints.hpp" #include "Kokkos_Constraints.hpp"
#include "Kokkos_HelperPredicates.hpp" #include "Kokkos_HelperPredicates.hpp"
#include "Kokkos_MustUseKokkosSingleInTeam.hpp"
#include <std_algorithms/Kokkos_Distance.hpp> #include <std_algorithms/Kokkos_Distance.hpp>
#include <string> #include <string>
@ -27,8 +28,10 @@ namespace Kokkos {
namespace Experimental { namespace Experimental {
namespace Impl { namespace Impl {
template <class IndexType, class FirstFrom, class FirstDest, class PredType> template <class FirstFrom, class FirstDest, class PredType>
struct StdCopyIfFunctor { struct StdCopyIfFunctor {
using index_type = typename FirstFrom::difference_type;
FirstFrom m_first_from; FirstFrom m_first_from;
FirstDest m_first_dest; FirstDest m_first_dest;
PredType m_pred; PredType m_pred;
@ -40,7 +43,7 @@ struct StdCopyIfFunctor {
m_pred(std::move(pred)) {} m_pred(std::move(pred)) {}
KOKKOS_FUNCTION KOKKOS_FUNCTION
void operator()(const IndexType i, IndexType& update, void operator()(const index_type i, index_type& update,
const bool final_pass) const { const bool final_pass) const {
const auto& myval = m_first_from[i]; const auto& myval = m_first_from[i];
if (final_pass) { if (final_pass) {
@ -57,9 +60,11 @@ struct StdCopyIfFunctor {
template <class ExecutionSpace, class InputIterator, class OutputIterator, template <class ExecutionSpace, class InputIterator, class OutputIterator,
class PredicateType> class PredicateType>
OutputIterator copy_if_impl(const std::string& label, const ExecutionSpace& ex, OutputIterator copy_if_exespace_impl(const std::string& label,
InputIterator first, InputIterator last, const ExecutionSpace& ex,
OutputIterator d_first, PredicateType pred) { InputIterator first, InputIterator last,
OutputIterator d_first,
PredicateType pred) {
/* /*
To explain the impl, suppose that our data is: To explain the impl, suppose that our data is:
@ -90,23 +95,67 @@ OutputIterator copy_if_impl(const std::string& label, const ExecutionSpace& ex,
if (first == last) { if (first == last) {
return d_first; return d_first;
} else { } else {
// aliases
using index_type = typename InputIterator::difference_type;
using func_type = StdCopyIfFunctor<index_type, InputIterator,
OutputIterator, PredicateType>;
// run // run
const auto num_elements = Kokkos::Experimental::distance(first, last); const auto num_elements = Kokkos::Experimental::distance(first, last);
index_type count = 0;
typename InputIterator::difference_type count = 0;
::Kokkos::parallel_scan(label, ::Kokkos::parallel_scan(label,
RangePolicy<ExecutionSpace>(ex, 0, num_elements), RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_type(first, d_first, pred), count); // use CTAD
StdCopyIfFunctor(first, d_first, pred), count);
// fence not needed because of the scan accumulating into count // fence not needed because of the scan accumulating into count
return d_first + count; return d_first + count;
} }
} }
template <class TeamHandleType, class InputIterator, class OutputIterator,
class PredicateType>
KOKKOS_FUNCTION OutputIterator copy_if_team_impl(
const TeamHandleType& teamHandle, InputIterator first, InputIterator last,
OutputIterator d_first, PredicateType pred) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first, d_first);
Impl::static_assert_iterators_have_matching_difference_type(first, d_first);
Impl::expect_valid_range(first, last);
if (first == last) {
return d_first;
}
const std::size_t num_elements = Kokkos::Experimental::distance(first, last);
if constexpr (stdalgo_must_use_kokkos_single_for_team_scan_v<
typename TeamHandleType::execution_space>) {
std::size_t count = 0;
Kokkos::single(
Kokkos::PerTeam(teamHandle),
[=](std::size_t& lcount) {
lcount = 0;
for (std::size_t i = 0; i < num_elements; ++i) {
const auto& myval = first[i];
if (pred(myval)) {
d_first[lcount++] = myval;
}
}
},
count);
// no barrier needed since single above broadcasts to all members
return d_first + count;
} else {
typename InputIterator::difference_type count = 0;
::Kokkos::parallel_scan(TeamThreadRange(teamHandle, 0, num_elements),
StdCopyIfFunctor(first, d_first, pred), count);
// no barrier needed because of the scan accumulating into count
return d_first + count;
}
#if defined KOKKOS_COMPILER_INTEL || \
(defined(KOKKOS_COMPILER_NVCC) && KOKKOS_COMPILER_NVCC >= 1130)
__builtin_unreachable();
#endif
}
} // namespace Impl } // namespace Impl
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

52
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_CountCountIf.hpp
View File

@ -46,37 +46,65 @@ struct StdCountIfFunctor {
}; };
template <class ExecutionSpace, class IteratorType, class Predicate> template <class ExecutionSpace, class IteratorType, class Predicate>
typename IteratorType::difference_type count_if_impl(const std::string& label, typename IteratorType::difference_type count_if_exespace_impl(
const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex, IteratorType first,
IteratorType first, IteratorType last, Predicate predicate) {
IteratorType last,
Predicate predicate) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first); Impl::static_assert_random_access_and_accessible(ex, first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
// aliases
using func_t = StdCountIfFunctor<IteratorType, Predicate>;
// run // run
const auto num_elements = Kokkos::Experimental::distance(first, last); const auto num_elements = Kokkos::Experimental::distance(first, last);
typename IteratorType::difference_type count = 0; typename IteratorType::difference_type count = 0;
::Kokkos::parallel_reduce(label, ::Kokkos::parallel_reduce(label,
RangePolicy<ExecutionSpace>(ex, 0, num_elements), RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_t(first, predicate), count); // use CTAD
StdCountIfFunctor(first, predicate), count);
ex.fence("Kokkos::count_if: fence after operation"); ex.fence("Kokkos::count_if: fence after operation");
return count; return count;
} }
template <class ExecutionSpace, class IteratorType, class T> template <class ExecutionSpace, class IteratorType, class T>
auto count_impl(const std::string& label, const ExecutionSpace& ex, auto count_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, const T& value) { IteratorType first, IteratorType last,
return count_if_impl( const T& value) {
return count_if_exespace_impl(
label, ex, first, last, label, ex, first, last,
::Kokkos::Experimental::Impl::StdAlgoEqualsValUnaryPredicate<T>(value)); ::Kokkos::Experimental::Impl::StdAlgoEqualsValUnaryPredicate<T>(value));
} }
//
// team-level impl
//
template <class TeamHandleType, class IteratorType, class Predicate>
KOKKOS_FUNCTION typename IteratorType::difference_type count_if_team_impl(
const TeamHandleType& teamHandle, IteratorType first, IteratorType last,
Predicate predicate) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
// run
const auto num_elements = Kokkos::Experimental::distance(first, last);
typename IteratorType::difference_type count = 0;
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, num_elements),
// use CTAD
StdCountIfFunctor(first, predicate), count);
teamHandle.team_barrier();
return count;
}
template <class TeamHandleType, class IteratorType, class T>
KOKKOS_FUNCTION auto count_team_impl(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
const T& value) {
return count_if_team_impl(
teamHandle, first, last,
::Kokkos::Experimental::Impl::StdAlgoEqualsValUnaryPredicate<T>(value));
}
} // namespace Impl } // namespace Impl
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

116
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_Equal.hpp
View File

@ -27,15 +27,16 @@ namespace Kokkos {
namespace Experimental { namespace Experimental {
namespace Impl { namespace Impl {
template <class IndexType, class IteratorType1, class IteratorType2, template <class IteratorType1, class IteratorType2, class BinaryPredicateType>
class BinaryPredicateType>
struct StdEqualFunctor { struct StdEqualFunctor {
using index_type = typename IteratorType1::difference_type;
IteratorType1 m_first1; IteratorType1 m_first1;
IteratorType2 m_first2; IteratorType2 m_first2;
BinaryPredicateType m_predicate; BinaryPredicateType m_predicate;
KOKKOS_FUNCTION KOKKOS_FUNCTION
void operator()(IndexType i, std::size_t& lsum) const { void operator()(index_type i, std::size_t& lsum) const {
if (!m_predicate(m_first1[i], m_first2[i])) { if (!m_predicate(m_first1[i], m_first2[i])) {
lsum = 1; lsum = 1;
} }
@ -49,67 +50,130 @@ struct StdEqualFunctor {
m_predicate(std::move(_predicate)) {} m_predicate(std::move(_predicate)) {}
}; };
//
// exespace impl
//
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <class ExecutionSpace, class IteratorType1, class IteratorType2,
class BinaryPredicateType> class BinaryPredicateType>
bool equal_impl(const std::string& label, const ExecutionSpace& ex, bool equal_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType2 first2, IteratorType1 first1, IteratorType1 last1,
BinaryPredicateType predicate) { IteratorType2 first2, BinaryPredicateType predicate) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first1, first2); Impl::static_assert_random_access_and_accessible(ex, first1, first2);
Impl::static_assert_iterators_have_matching_difference_type(first1, first2); Impl::static_assert_iterators_have_matching_difference_type(first1, first2);
Impl::expect_valid_range(first1, last1); Impl::expect_valid_range(first1, last1);
// aliases
using index_type = typename IteratorType1::difference_type;
using func_t = StdEqualFunctor<index_type, IteratorType1, IteratorType2,
BinaryPredicateType>;
// run // run
const auto num_elements = Kokkos::Experimental::distance(first1, last1); const auto num_elements = Kokkos::Experimental::distance(first1, last1);
std::size_t different = 0; std::size_t different = 0;
::Kokkos::parallel_reduce(label, ::Kokkos::parallel_reduce(
RangePolicy<ExecutionSpace>(ex, 0, num_elements), label, RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_t(first1, first2, predicate), different); StdEqualFunctor(first1, first2, predicate), different);
ex.fence("Kokkos::equal: fence after operation"); ex.fence("Kokkos::equal: fence after operation");
return !different; return !different;
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <class ExecutionSpace, class IteratorType1, class IteratorType2>
bool equal_impl(const std::string& label, const ExecutionSpace& ex, bool equal_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2) { IteratorType2 first2) {
using value_type1 = typename IteratorType1::value_type; using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type; using value_type2 = typename IteratorType2::value_type;
using pred_t = StdAlgoEqualBinaryPredicate<value_type1, value_type2>; using pred_t = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return equal_impl(label, ex, first1, last1, first2, pred_t()); return equal_exespace_impl(label, ex, first1, last1, first2, pred_t());
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <class ExecutionSpace, class IteratorType1, class IteratorType2,
class BinaryPredicateType> class BinaryPredicateType>
bool equal_impl(const std::string& label, const ExecutionSpace& ex, bool equal_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType2 first2, IteratorType1 first1, IteratorType1 last1,
IteratorType2 last2, BinaryPredicateType predicate) { IteratorType2 first2, IteratorType2 last2,
BinaryPredicateType predicate) {
const auto d1 = ::Kokkos::Experimental::distance(first1, last1); const auto d1 = ::Kokkos::Experimental::distance(first1, last1);
const auto d2 = ::Kokkos::Experimental::distance(first2, last2); const auto d2 = ::Kokkos::Experimental::distance(first2, last2);
if (d1 != d2) { if (d1 != d2) {
return false; return false;
} }
return equal_impl(label, ex, first1, last1, first2, predicate); return equal_exespace_impl(label, ex, first1, last1, first2, predicate);
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <class ExecutionSpace, class IteratorType1, class IteratorType2>
bool equal_impl(const std::string& label, const ExecutionSpace& ex, bool equal_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType1 first1, IteratorType1 last1, IteratorType2 first2, IteratorType1 first1, IteratorType1 last1,
IteratorType2 last2) { IteratorType2 first2, IteratorType2 last2) {
Impl::expect_valid_range(first1, last1); Impl::expect_valid_range(first1, last1);
Impl::expect_valid_range(first2, last2); Impl::expect_valid_range(first2, last2);
using value_type1 = typename IteratorType1::value_type; using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type; using value_type2 = typename IteratorType2::value_type;
using pred_t = StdAlgoEqualBinaryPredicate<value_type1, value_type2>; using pred_t = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return equal_impl(label, ex, first1, last1, first2, last2, pred_t()); return equal_exespace_impl(label, ex, first1, last1, first2, last2, pred_t());
}
//
// team impl
//
template <class TeamHandleType, class IteratorType1, class IteratorType2,
class BinaryPredicateType>
KOKKOS_FUNCTION bool equal_team_impl(const TeamHandleType& teamHandle,
IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2,
BinaryPredicateType predicate) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first1, first2);
Impl::static_assert_iterators_have_matching_difference_type(first1, first2);
Impl::expect_valid_range(first1, last1);
// run
const auto num_elements = Kokkos::Experimental::distance(first1, last1);
std::size_t different = 0;
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, num_elements),
StdEqualFunctor(first1, first2, predicate),
different);
teamHandle.team_barrier();
return !different;
}
template <class TeamHandleType, class IteratorType1, class IteratorType2>
KOKKOS_FUNCTION bool equal_team_impl(const TeamHandleType& teamHandle,
IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2) {
using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type;
using pred_t = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return equal_team_impl(teamHandle, first1, last1, first2, pred_t());
}
template <class TeamHandleType, class IteratorType1, class IteratorType2,
class BinaryPredicateType>
KOKKOS_FUNCTION bool equal_team_impl(const TeamHandleType& teamHandle,
IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2,
BinaryPredicateType predicate) {
const auto d1 = ::Kokkos::Experimental::distance(first1, last1);
const auto d2 = ::Kokkos::Experimental::distance(first2, last2);
if (d1 != d2) {
return false;
}
return equal_team_impl(teamHandle, first1, last1, first2, predicate);
}
template <class TeamHandleType, class IteratorType1, class IteratorType2>
KOKKOS_FUNCTION bool equal_team_impl(const TeamHandleType& teamHandle,
IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2,
IteratorType2 last2) {
Impl::expect_valid_range(first1, last1);
Impl::expect_valid_range(first2, last2);
using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type;
using pred_t = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return equal_team_impl(teamHandle, first1, last1, first2, last2, pred_t());
} }
} // namespace Impl } // namespace Impl

240
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_ExclusiveScan.hpp
View File

@ -22,6 +22,7 @@
#include "Kokkos_HelperPredicates.hpp" #include "Kokkos_HelperPredicates.hpp"
#include "Kokkos_ValueWrapperForNoNeutralElement.hpp" #include "Kokkos_ValueWrapperForNoNeutralElement.hpp"
#include "Kokkos_IdentityReferenceUnaryFunctor.hpp" #include "Kokkos_IdentityReferenceUnaryFunctor.hpp"
#include "Kokkos_FunctorsForExclusiveScan.hpp"
#include <std_algorithms/Kokkos_TransformExclusiveScan.hpp> #include <std_algorithms/Kokkos_TransformExclusiveScan.hpp>
#include <std_algorithms/Kokkos_Distance.hpp> #include <std_algorithms/Kokkos_Distance.hpp>
#include <string> #include <string>
@ -30,127 +31,15 @@ namespace Kokkos {
namespace Experimental { namespace Experimental {
namespace Impl { namespace Impl {
template <class ExeSpace, class IndexType, class ValueType, class FirstFrom, //
class FirstDest> // exespace impl
struct ExclusiveScanDefaultFunctorForKnownNeutralElement { //
using execution_space = ExeSpace;
ValueType m_init_value;
FirstFrom m_first_from;
FirstDest m_first_dest;
KOKKOS_FUNCTION
ExclusiveScanDefaultFunctorForKnownNeutralElement(ValueType init,
FirstFrom first_from,
FirstDest first_dest)
: m_init_value(std::move(init)),
m_first_from(std::move(first_from)),
m_first_dest(std::move(first_dest)) {}
KOKKOS_FUNCTION
void operator()(const IndexType i, ValueType& update,
const bool final_pass) const {
if (final_pass) m_first_dest[i] = update + m_init_value;
update += m_first_from[i];
}
};
template <class ExeSpace, class IndexType, class ValueType, class FirstFrom,
class FirstDest>
struct ExclusiveScanDefaultFunctor {
using execution_space = ExeSpace;
using value_type =
::Kokkos::Experimental::Impl::ValueWrapperForNoNeutralElement<ValueType>;
ValueType m_init_value;
FirstFrom m_first_from;
FirstDest m_first_dest;
KOKKOS_FUNCTION
ExclusiveScanDefaultFunctor(ValueType init, FirstFrom first_from,
FirstDest first_dest)
: m_init_value(std::move(init)),
m_first_from(std::move(first_from)),
m_first_dest(std::move(first_dest)) {}
KOKKOS_FUNCTION
void operator()(const IndexType i, value_type& update,
const bool final_pass) const {
if (final_pass) {
if (i == 0) {
m_first_dest[i] = m_init_value;
} else {
m_first_dest[i] = update.val + m_init_value;
}
}
const auto tmp = value_type{m_first_from[i], false};
this->join(update, tmp);
}
KOKKOS_FUNCTION
void init(value_type& update) const {
update.val = {};
update.is_initial = true;
}
KOKKOS_FUNCTION
void join(value_type& update, const value_type& input) const {
if (input.is_initial) return;
if (update.is_initial) {
update.val = input.val;
update.is_initial = false;
} else {
update.val = update.val + input.val;
}
}
};
template <class ExecutionSpace, class InputIteratorType,
class OutputIteratorType, class ValueType, class BinaryOpType>
OutputIteratorType exclusive_scan_custom_op_impl(
const std::string& label, const ExecutionSpace& ex,
InputIteratorType first_from, InputIteratorType last_from,
OutputIteratorType first_dest, ValueType init_value, BinaryOpType bop) {
// checks
Impl::static_assert_random_access_and_accessible(ex, first_from, first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest);
Impl::expect_valid_range(first_from, last_from);
// aliases
using index_type = typename InputIteratorType::difference_type;
using unary_op_type = StdNumericScanIdentityReferenceUnaryFunctor<ValueType>;
using func_type =
TransformExclusiveScanFunctor<ExecutionSpace, index_type, ValueType,
InputIteratorType, OutputIteratorType,
BinaryOpType, unary_op_type>;
// run
const auto num_elements =
Kokkos::Experimental::distance(first_from, last_from);
::Kokkos::parallel_scan(
label, RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_type(init_value, first_from, first_dest, bop, unary_op_type()));
ex.fence("Kokkos::exclusive_scan_custom_op: fence after operation");
// return
return first_dest + num_elements;
}
template <typename ValueType>
using ex_scan_has_reduction_identity_sum_t =
decltype(Kokkos::reduction_identity<ValueType>::sum());
template <class ExecutionSpace, class InputIteratorType, template <class ExecutionSpace, class InputIteratorType,
class OutputIteratorType, class ValueType> class OutputIteratorType, class ValueType>
OutputIteratorType exclusive_scan_default_op_impl(const std::string& label, OutputIteratorType exclusive_scan_default_op_exespace_impl(
const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
InputIteratorType first_from, InputIteratorType first_from, InputIteratorType last_from,
InputIteratorType last_from, OutputIteratorType first_dest, ValueType init_value) {
OutputIteratorType first_dest,
ValueType init_value) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first_from, first_dest); Impl::static_assert_random_access_and_accessible(ex, first_from, first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from, Impl::static_assert_iterators_have_matching_difference_type(first_from,
@ -184,17 +73,122 @@ OutputIteratorType exclusive_scan_default_op_impl(const std::string& label,
ExclusiveScanDefaultFunctorForKnownNeutralElement< ExclusiveScanDefaultFunctorForKnownNeutralElement<
ExecutionSpace, index_type, ValueType, InputIteratorType, ExecutionSpace, index_type, ValueType, InputIteratorType,
OutputIteratorType>, OutputIteratorType>,
ExclusiveScanDefaultFunctor<ExecutionSpace, index_type, ValueType, ExclusiveScanDefaultFunctorWithValueWrapper<ExecutionSpace, index_type,
InputIteratorType, OutputIteratorType>>; ValueType, InputIteratorType,
OutputIteratorType>>;
// run
const auto num_elements =
Kokkos::Experimental::distance(first_from, last_from);
::Kokkos::parallel_scan(
label, RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_type(std::move(init_value), first_from, first_dest));
ex.fence("Kokkos::exclusive_scan_default_op: fence after operation");
return first_dest + num_elements;
}
template <class ExecutionSpace, class InputIteratorType,
class OutputIteratorType, class ValueType, class BinaryOpType>
OutputIteratorType exclusive_scan_custom_op_exespace_impl(
const std::string& label, const ExecutionSpace& ex,
InputIteratorType first_from, InputIteratorType last_from,
OutputIteratorType first_dest, ValueType init_value, BinaryOpType bop) {
// checks
Impl::static_assert_random_access_and_accessible(ex, first_from, first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest);
Impl::expect_valid_range(first_from, last_from);
// aliases
using index_type = typename InputIteratorType::difference_type;
using unary_op_type = StdNumericScanIdentityReferenceUnaryFunctor<ValueType>;
using func_type = TransformExclusiveScanFunctorWithValueWrapper<
ExecutionSpace, index_type, ValueType, InputIteratorType,
OutputIteratorType, BinaryOpType, unary_op_type>;
// run // run
const auto num_elements = const auto num_elements =
Kokkos::Experimental::distance(first_from, last_from); Kokkos::Experimental::distance(first_from, last_from);
::Kokkos::parallel_scan(label, ::Kokkos::parallel_scan(label,
RangePolicy<ExecutionSpace>(ex, 0, num_elements), RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_type(init_value, first_from, first_dest)); func_type(std::move(init_value), first_from,
first_dest, bop, unary_op_type()));
ex.fence("Kokkos::exclusive_scan_custom_op: fence after operation");
ex.fence("Kokkos::exclusive_scan_default_op: fence after operation"); // return
return first_dest + num_elements;
}
//
// team impl
//
template <class TeamHandleType, class InputIteratorType,
class OutputIteratorType, class ValueType>
KOKKOS_FUNCTION OutputIteratorType exclusive_scan_default_op_team_impl(
const TeamHandleType& teamHandle, InputIteratorType first_from,
InputIteratorType last_from, OutputIteratorType first_dest,
ValueType init_value) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first_from,
first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest);
Impl::expect_valid_range(first_from, last_from);
static_assert(
::Kokkos::is_detected_v<ex_scan_has_reduction_identity_sum_t, ValueType>,
"The team-level impl of Kokkos::Experimental::exclusive_scan currently "
"does not support types without reduction identity");
// aliases
using exe_space = typename TeamHandleType::execution_space;
using index_type = typename InputIteratorType::difference_type;
using func_type = ExclusiveScanDefaultFunctorForKnownNeutralElement<
exe_space, index_type, ValueType, InputIteratorType, OutputIteratorType>;
const auto num_elements =
Kokkos::Experimental::distance(first_from, last_from);
::Kokkos::parallel_scan(
TeamThreadRange(teamHandle, 0, num_elements),
func_type(std::move(init_value), first_from, first_dest));
teamHandle.team_barrier();
return first_dest + num_elements;
}
template <class TeamHandleType, class InputIteratorType,
class OutputIteratorType, class ValueType, class BinaryOpType>
KOKKOS_FUNCTION OutputIteratorType exclusive_scan_custom_op_team_impl(
const TeamHandleType& teamHandle, InputIteratorType first_from,
InputIteratorType last_from, OutputIteratorType first_dest,
ValueType init_value, BinaryOpType bop) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first_from,
first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest);
Impl::expect_valid_range(first_from, last_from);
static_assert(
::Kokkos::is_detected_v<ex_scan_has_reduction_identity_sum_t, ValueType>,
"The team-level impl of Kokkos::Experimental::exclusive_scan currently "
"does not support types without reduction identity");
// aliases
using exe_space = typename TeamHandleType::execution_space;
using unary_op_type = StdNumericScanIdentityReferenceUnaryFunctor<ValueType>;
using index_type = typename InputIteratorType::difference_type;
using func_type = TransformExclusiveScanFunctorWithoutValueWrapper<
exe_space, index_type, ValueType, InputIteratorType, OutputIteratorType,
BinaryOpType, unary_op_type>;
const auto num_elements =
Kokkos::Experimental::distance(first_from, last_from);
::Kokkos::parallel_scan(TeamThreadRange(teamHandle, 0, num_elements),
func_type(std::move(init_value), first_from,
first_dest, bop, unary_op_type()));
teamHandle.team_barrier();
return first_dest + num_elements; return first_dest + num_elements;
} }

49
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_FillFillN.hpp
View File

@ -41,9 +41,12 @@ struct StdFillFunctor {
: m_first(std::move(_first)), m_value(std::move(_value)) {} : m_first(std::move(_first)), m_value(std::move(_value)) {}
}; };
//
// exespace impl
//
template <class ExecutionSpace, class IteratorType, class T> template <class ExecutionSpace, class IteratorType, class T>
void fill_impl(const std::string& label, const ExecutionSpace& ex, void fill_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, const T& value) { IteratorType first, IteratorType last, const T& value) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first); Impl::static_assert_random_access_and_accessible(ex, first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
@ -52,13 +55,14 @@ void fill_impl(const std::string& label, const ExecutionSpace& ex,
const auto num_elements = Kokkos::Experimental::distance(first, last); const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for(label, ::Kokkos::parallel_for(label,
RangePolicy<ExecutionSpace>(ex, 0, num_elements), RangePolicy<ExecutionSpace>(ex, 0, num_elements),
StdFillFunctor<IteratorType, T>(first, value)); StdFillFunctor(first, value));
ex.fence("Kokkos::fill: fence after operation"); ex.fence("Kokkos::fill: fence after operation");
} }
template <class ExecutionSpace, class IteratorType, class SizeType, class T> template <class ExecutionSpace, class IteratorType, class SizeType, class T>
IteratorType fill_n_impl(const std::string& label, const ExecutionSpace& ex, IteratorType fill_n_exespace_impl(const std::string& label,
IteratorType first, SizeType n, const T& value) { const ExecutionSpace& ex, IteratorType first,
SizeType n, const T& value) {
auto last = first + n; auto last = first + n;
Impl::static_assert_random_access_and_accessible(ex, first); Impl::static_assert_random_access_and_accessible(ex, first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
@ -67,7 +71,40 @@ IteratorType fill_n_impl(const std::string& label, const ExecutionSpace& ex,
return first; return first;
} }
fill_impl(label, ex, first, last, value); fill_exespace_impl(label, ex, first, last, value);
return last;
}
//
// team-level impl
//
template <class TeamHandleType, class IteratorType, class T>
KOKKOS_FUNCTION void fill_team_impl(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
const T& value) {
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for(TeamThreadRange(teamHandle, 0, num_elements),
StdFillFunctor(first, value));
teamHandle.team_barrier();
}
template <class TeamHandleType, class IteratorType, class SizeType, class T>
KOKKOS_FUNCTION IteratorType fill_n_team_impl(const TeamHandleType& teamHandle,
IteratorType first, SizeType n,
const T& value) {
auto last = first + n;
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
if (n <= 0) {
return first;
}
fill_team_impl(teamHandle, first, last, value);
return last; return last;
} }

110
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_FindEnd.hpp
View File

@ -80,12 +80,17 @@ struct StdFindEndFunctor {
m_p(std::move(p)) {} m_p(std::move(p)) {}
}; };
//
// exespace impl
//
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <class ExecutionSpace, class IteratorType1, class IteratorType2,
class BinaryPredicateType> class BinaryPredicateType>
IteratorType1 find_end_impl(const std::string& label, const ExecutionSpace& ex, IteratorType1 find_end_exespace_impl(const std::string& label,
IteratorType1 first, IteratorType1 last, const ExecutionSpace& ex,
IteratorType2 s_first, IteratorType2 s_last, IteratorType1 first, IteratorType1 last,
const BinaryPredicateType& pred) { IteratorType2 s_first,
IteratorType2 s_last,
const BinaryPredicateType& pred) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first, s_first); Impl::static_assert_random_access_and_accessible(ex, first, s_first);
Impl::static_assert_iterators_have_matching_difference_type(first, s_first); Impl::static_assert_iterators_have_matching_difference_type(first, s_first);
@ -97,7 +102,6 @@ IteratorType1 find_end_impl(const std::string& label, const ExecutionSpace& ex,
const auto num_elements = KE::distance(first, last); const auto num_elements = KE::distance(first, last);
const auto s_count = KE::distance(s_first, s_last); const auto s_count = KE::distance(s_first, s_last);
KOKKOS_EXPECTS(num_elements >= s_count); KOKKOS_EXPECTS(num_elements >= s_count);
(void)s_count; // needed when macro above is a no-op
if (s_first == s_last) { if (s_first == s_last) {
return last; return last;
@ -109,7 +113,8 @@ IteratorType1 find_end_impl(const std::string& label, const ExecutionSpace& ex,
// special case where the two ranges have equal size // special case where the two ranges have equal size
if (num_elements == s_count) { if (num_elements == s_count) {
const auto equal_result = equal_impl(label, ex, first, last, s_first, pred); const auto equal_result =
equal_exespace_impl(label, ex, first, last, s_first, pred);
return (equal_result) ? first : last; return (equal_result) ? first : last;
} else { } else {
using index_type = typename IteratorType1::difference_type; using index_type = typename IteratorType1::difference_type;
@ -148,14 +153,97 @@ IteratorType1 find_end_impl(const std::string& label, const ExecutionSpace& ex,
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <class ExecutionSpace, class IteratorType1, class IteratorType2>
IteratorType1 find_end_impl(const std::string& label, const ExecutionSpace& ex, IteratorType1 find_end_exespace_impl(const std::string& label,
IteratorType1 first, IteratorType1 last, const ExecutionSpace& ex,
IteratorType2 s_first, IteratorType2 s_last) { IteratorType1 first, IteratorType1 last,
IteratorType2 s_first,
IteratorType2 s_last) {
using value_type1 = typename IteratorType1::value_type; using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type; using value_type2 = typename IteratorType2::value_type;
using predicate_type = StdAlgoEqualBinaryPredicate<value_type1, value_type2>; using predicate_type = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return find_end_impl(label, ex, first, last, s_first, s_last, return find_end_exespace_impl(label, ex, first, last, s_first, s_last,
predicate_type()); predicate_type());
}
//
// team impl
//
template <class TeamHandleType, class IteratorType1, class IteratorType2,
class BinaryPredicateType>
KOKKOS_FUNCTION IteratorType1
find_end_team_impl(const TeamHandleType& teamHandle, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first,
IteratorType2 s_last, const BinaryPredicateType& pred) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first, s_first);
Impl::static_assert_iterators_have_matching_difference_type(first, s_first);
Impl::expect_valid_range(first, last);
Impl::expect_valid_range(s_first, s_last);
// the target sequence should not be larger than the range [first, last)
namespace KE = ::Kokkos::Experimental;
const auto num_elements = KE::distance(first, last);
const auto s_count = KE::distance(s_first, s_last);
KOKKOS_EXPECTS(num_elements >= s_count);
if (s_first == s_last) {
return last;
}
if (first == last) {
return last;
}
// special case where the two ranges have equal size
if (num_elements == s_count) {
const auto equal_result =
equal_team_impl(teamHandle, first, last, s_first, pred);
return (equal_result) ? first : last;
} else {
using index_type = typename IteratorType1::difference_type;
using reducer_type = LastLoc<index_type>;
using reduction_value_type = typename reducer_type::value_type;
using func_t = StdFindEndFunctor<index_type, IteratorType1, IteratorType2,
reducer_type, BinaryPredicateType>;
// run
reduction_value_type red_result;
reducer_type reducer(red_result);
// decide the size of the range policy of the par_red:
// note that the last feasible index to start looking is the index
// whose distance from the "last" is equal to the sequence count.
// the +1 is because we need to include that location too.
const auto range_size = num_elements - s_count + 1;
// run par reduce
::Kokkos::parallel_reduce(
TeamThreadRange(teamHandle, 0, range_size),
func_t(first, last, s_first, s_last, reducer, pred), reducer);
teamHandle.team_barrier();
// decide and return
if (red_result.max_loc_true ==
::Kokkos::reduction_identity<index_type>::max()) {
// if here, a subrange has not been found
return last;
} else {
// a location has been found
return first + red_result.max_loc_true;
}
}
}
template <class TeamHandleType, class IteratorType1, class IteratorType2>
KOKKOS_FUNCTION IteratorType1 find_end_team_impl(
const TeamHandleType& teamHandle, IteratorType1 first, IteratorType1 last,
IteratorType2 s_first, IteratorType2 s_last) {
using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type;
using predicate_type = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return find_end_team_impl(teamHandle, first, last, s_first, s_last,
predicate_type());
} }
} // namespace Impl } // namespace Impl

80
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_FindFirstOf.hpp
View File

@ -71,13 +71,15 @@ struct StdFindFirstOfFunctor {
m_p(std::move(p)) {} m_p(std::move(p)) {}
}; };
//
// exespace impl
//
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <class ExecutionSpace, class IteratorType1, class IteratorType2,
class BinaryPredicateType> class BinaryPredicateType>
IteratorType1 find_first_of_impl(const std::string& label, IteratorType1 find_first_of_exespace_impl(
const ExecutionSpace& ex, IteratorType1 first, const std::string& label, const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first, IteratorType1 last, IteratorType2 s_first, IteratorType2 s_last,
IteratorType2 s_last, const BinaryPredicateType& pred) {
const BinaryPredicateType& pred) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first, s_first); Impl::static_assert_random_access_and_accessible(ex, first, s_first);
Impl::static_assert_iterators_have_matching_difference_type(first, s_first); Impl::static_assert_iterators_have_matching_difference_type(first, s_first);
@ -116,15 +118,71 @@ IteratorType1 find_first_of_impl(const std::string& label,
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <class ExecutionSpace, class IteratorType1, class IteratorType2>
IteratorType1 find_first_of_impl(const std::string& label, IteratorType1 find_first_of_exespace_impl(
const ExecutionSpace& ex, IteratorType1 first, const std::string& label, const ExecutionSpace& ex, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first, IteratorType1 last, IteratorType2 s_first, IteratorType2 s_last) {
IteratorType2 s_last) {
using value_type1 = typename IteratorType1::value_type; using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type; using value_type2 = typename IteratorType2::value_type;
using predicate_type = StdAlgoEqualBinaryPredicate<value_type1, value_type2>; using predicate_type = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return find_first_of_impl(label, ex, first, last, s_first, s_last, return find_first_of_exespace_impl(label, ex, first, last, s_first, s_last,
predicate_type()); predicate_type());
}
//
// team impl
//
template <class TeamHandleType, class IteratorType1, class IteratorType2,
class BinaryPredicateType>
KOKKOS_FUNCTION IteratorType1
find_first_of_team_impl(const TeamHandleType& teamHandle, IteratorType1 first,
IteratorType1 last, IteratorType2 s_first,
IteratorType2 s_last, const BinaryPredicateType& pred) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first, s_first);
Impl::static_assert_iterators_have_matching_difference_type(first, s_first);
Impl::expect_valid_range(first, last);
Impl::expect_valid_range(s_first, s_last);
if ((s_first == s_last) || (first == last)) {
return last;
}
using index_type = typename IteratorType1::difference_type;
using reducer_type = FirstLoc<index_type>;
using reduction_value_type = typename reducer_type::value_type;
using func_t = StdFindFirstOfFunctor<index_type, IteratorType1, IteratorType2,
reducer_type, BinaryPredicateType>;
// run
reduction_value_type red_result;
reducer_type reducer(red_result);
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, num_elements),
func_t(first, s_first, s_last, reducer, pred),
reducer);
teamHandle.team_barrier();
// decide and return
if (red_result.min_loc_true ==
::Kokkos::reduction_identity<index_type>::min()) {
// if here, nothing found
return last;
} else {
// a location has been found
return first + red_result.min_loc_true;
}
}
template <class TeamHandleType, class IteratorType1, class IteratorType2>
KOKKOS_FUNCTION IteratorType1 find_first_of_team_impl(
const TeamHandleType& teamHandle, IteratorType1 first, IteratorType1 last,
IteratorType2 s_first, IteratorType2 s_last) {
using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type;
using predicate_type = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return find_first_of_team_impl(teamHandle, first, last, s_first, s_last,
predicate_type());
} }
} // namespace Impl } // namespace Impl

72
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_FindIfOrNot.hpp
View File

@ -61,11 +61,15 @@ struct StdFindIfOrNotFunctor {
m_p(std::move(p)) {} m_p(std::move(p)) {}
}; };
//
// exespace impl
//
template <bool is_find_if, class ExecutionSpace, class IteratorType, template <bool is_find_if, class ExecutionSpace, class IteratorType,
class PredicateType> class PredicateType>
IteratorType find_if_or_not_impl(const std::string& label, IteratorType find_if_or_not_exespace_impl(const std::string& label,
const ExecutionSpace& ex, IteratorType first, const ExecutionSpace& ex,
IteratorType last, PredicateType pred) { IteratorType first, IteratorType last,
PredicateType pred) {
// checks // checks
Impl::static_assert_random_access_and_accessible( Impl::static_assert_random_access_and_accessible(
ex, first); // only need one It per type ex, first); // only need one It per type
@ -104,14 +108,68 @@ IteratorType find_if_or_not_impl(const std::string& label,
} }
template <class ExecutionSpace, class InputIterator, class T> template <class ExecutionSpace, class InputIterator, class T>
InputIterator find_impl(const std::string& label, ExecutionSpace ex, InputIterator find_exespace_impl(const std::string& label, ExecutionSpace ex,
InputIterator first, InputIterator last, InputIterator first, InputIterator last,
const T& value) { const T& value) {
return find_if_or_not_impl<true>( return find_if_or_not_exespace_impl<true>(
label, ex, first, last, label, ex, first, last,
::Kokkos::Experimental::Impl::StdAlgoEqualsValUnaryPredicate<T>(value)); ::Kokkos::Experimental::Impl::StdAlgoEqualsValUnaryPredicate<T>(value));
} }
//
// team impl
//
template <bool is_find_if, class TeamHandleType, class IteratorType,
class PredicateType>
KOKKOS_FUNCTION IteratorType
find_if_or_not_team_impl(const TeamHandleType& teamHandle, IteratorType first,
IteratorType last, PredicateType pred) {
// checks
Impl::static_assert_random_access_and_accessible(
teamHandle, first); // only need one It per type
Impl::expect_valid_range(first, last);
if (first == last) {
return last;
}
// aliases
using index_type = typename IteratorType::difference_type;
using reducer_type = FirstLoc<index_type>;
using reduction_value_type = typename reducer_type::value_type;
using func_t = StdFindIfOrNotFunctor<is_find_if, index_type, IteratorType,
reducer_type, PredicateType>;
// run
reduction_value_type red_result;
reducer_type reducer(red_result);
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, num_elements),
func_t(first, reducer, pred), reducer);
teamHandle.team_barrier();
// decide and return
if (red_result.min_loc_true ==
::Kokkos::reduction_identity<index_type>::min()) {
// here, it means a valid loc has not been found,
return last;
} else {
// a location has been found
return first + red_result.min_loc_true;
}
}
template <class TeamHandleType, class InputIterator, class T>
KOKKOS_FUNCTION InputIterator find_team_impl(const TeamHandleType& teamHandle,
InputIterator first,
InputIterator last,
const T& value) {
return find_if_or_not_team_impl<true>(
teamHandle, first, last,
::Kokkos::Experimental::Impl::StdAlgoEqualsValUnaryPredicate<T>(value));
}
} // namespace Impl } // namespace Impl
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

64
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_ForEachForEachN.hpp
View File

@ -41,29 +41,31 @@ struct StdForEachFunctor {
: m_first(std::move(_first)), m_functor(std::move(_functor)) {} : m_first(std::move(_first)), m_functor(std::move(_functor)) {}
}; };
template <class ExecutionSpace, class IteratorType, class UnaryFunctorType> template <class HandleType, class IteratorType, class UnaryFunctorType>
UnaryFunctorType for_each_impl(const std::string& label, UnaryFunctorType for_each_exespace_impl(const std::string& label,
const ExecutionSpace& ex, IteratorType first, const HandleType& handle,
IteratorType last, UnaryFunctorType functor) { IteratorType first, IteratorType last,
UnaryFunctorType functor) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first); Impl::static_assert_random_access_and_accessible(handle, first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
// run // run
const auto num_elements = Kokkos::Experimental::distance(first, last); const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for( ::Kokkos::parallel_for(
label, RangePolicy<ExecutionSpace>(ex, 0, num_elements), label, RangePolicy<HandleType>(handle, 0, num_elements),
StdForEachFunctor<IteratorType, UnaryFunctorType>(first, functor)); StdForEachFunctor<IteratorType, UnaryFunctorType>(first, functor));
ex.fence("Kokkos::for_each: fence after operation"); handle.fence("Kokkos::for_each: fence after operation");
return functor; return functor;
} }
template <class ExecutionSpace, class IteratorType, class SizeType, template <class ExecutionSpace, class IteratorType, class SizeType,
class UnaryFunctorType> class UnaryFunctorType>
IteratorType for_each_n_impl(const std::string& label, const ExecutionSpace& ex, IteratorType for_each_n_exespace_impl(const std::string& label,
IteratorType first, SizeType n, const ExecutionSpace& ex,
UnaryFunctorType functor) { IteratorType first, SizeType n,
UnaryFunctorType functor) {
auto last = first + n; auto last = first + n;
Impl::static_assert_random_access_and_accessible(ex, first, last); Impl::static_assert_random_access_and_accessible(ex, first, last);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
@ -72,8 +74,46 @@ IteratorType for_each_n_impl(const std::string& label, const ExecutionSpace& ex,
return first; return first;
} }
for_each_impl(label, ex, first, last, std::move(functor)); for_each_exespace_impl(label, ex, first, last, std::move(functor));
// no neeed to fence since for_each_impl fences already // no neeed to fence since for_each_exespace_impl fences already
return last;
}
//
// team impl
//
template <class TeamHandleType, class IteratorType, class UnaryFunctorType>
KOKKOS_FUNCTION UnaryFunctorType
for_each_team_impl(const TeamHandleType& teamHandle, IteratorType first,
IteratorType last, UnaryFunctorType functor) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
// run
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for(
TeamThreadRange(teamHandle, 0, num_elements),
StdForEachFunctor<IteratorType, UnaryFunctorType>(first, functor));
teamHandle.team_barrier();
return functor;
}
template <class TeamHandleType, class IteratorType, class SizeType,
class UnaryFunctorType>
KOKKOS_FUNCTION IteratorType
for_each_n_team_impl(const TeamHandleType& teamHandle, IteratorType first,
SizeType n, UnaryFunctorType functor) {
auto last = first + n;
Impl::static_assert_random_access_and_accessible(teamHandle, first, last);
Impl::expect_valid_range(first, last);
if (n == 0) {
return first;
}
for_each_team_impl(teamHandle, first, last, std::move(functor));
// no neeed to fence since for_each_team_impl fences already
return last; return last;
} }

220
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_FunctorsForExclusiveScan.hpp Normal file
View File

@ -0,0 +1,220 @@
//@HEADER
// ************************************************************************
//
// Kokkos v. 4.0
// Copyright (2022) National Technology & Engineering
// Solutions of Sandia, LLC (NTESS).
//
// Under the terms of Contract DE-NA0003525 with NTESS,
// the U.S. Government retains certain rights in this software.
//
// Part of Kokkos, under the Apache License v2.0 with LLVM Exceptions.
// See https://kokkos.org/LICENSE for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//@HEADER
#ifndef KOKKOS_STD_ALGORITHMS_FUNCTORS_FOR_EXCLUSIVE_SCAN_IMPL_HPP
#define KOKKOS_STD_ALGORITHMS_FUNCTORS_FOR_EXCLUSIVE_SCAN_IMPL_HPP
#include <Kokkos_Core.hpp>
#include "Kokkos_ValueWrapperForNoNeutralElement.hpp"
namespace Kokkos {
namespace Experimental {
namespace Impl {
template <typename ValueType>
using ex_scan_has_reduction_identity_sum_t =
decltype(Kokkos::reduction_identity<ValueType>::sum());
template <class ExeSpace, class IndexType, class ValueType, class FirstFrom,
class FirstDest>
struct ExclusiveScanDefaultFunctorForKnownNeutralElement {
using execution_space = ExeSpace;
ValueType m_init_value;
FirstFrom m_first_from;
FirstDest m_first_dest;
KOKKOS_FUNCTION
ExclusiveScanDefaultFunctorForKnownNeutralElement(ValueType init,
FirstFrom first_from,
FirstDest first_dest)
: m_init_value(std::move(init)),
m_first_from(std::move(first_from)),
m_first_dest(std::move(first_dest)) {}
KOKKOS_FUNCTION
void operator()(const IndexType i, ValueType& update,
const bool final_pass) const {
if (final_pass) m_first_dest[i] = update + m_init_value;
update += m_first_from[i];
}
};
template <class ExeSpace, class IndexType, class ValueType, class FirstFrom,
class FirstDest>
struct ExclusiveScanDefaultFunctorWithValueWrapper {
using execution_space = ExeSpace;
using value_type =
::Kokkos::Experimental::Impl::ValueWrapperForNoNeutralElement<ValueType>;
ValueType m_init_value;
FirstFrom m_first_from;
FirstDest m_first_dest;
KOKKOS_FUNCTION
ExclusiveScanDefaultFunctorWithValueWrapper(ValueType init,
FirstFrom first_from,
FirstDest first_dest)
: m_init_value(std::move(init)),
m_first_from(std::move(first_from)),
m_first_dest(std::move(first_dest)) {}
KOKKOS_FUNCTION
void operator()(const IndexType i, value_type& update,
const bool final_pass) const {
if (final_pass) {
if (i == 0) {
m_first_dest[i] = m_init_value;
} else {
m_first_dest[i] = update.val + m_init_value;
}
}
const auto tmp = value_type{m_first_from[i], false};
this->join(update, tmp);
}
KOKKOS_FUNCTION
void init(value_type& update) const {
update.val = {};
update.is_initial = true;
}
KOKKOS_FUNCTION
void join(value_type& update, const value_type& input) const {
if (input.is_initial) return;
if (update.is_initial) {
update.val = input.val;
update.is_initial = false;
} else {
update.val = update.val + input.val;
}
}
};
template <class ExeSpace, class IndexType, class ValueType, class FirstFrom,
class FirstDest, class BinaryOpType, class UnaryOpType>
struct TransformExclusiveScanFunctorWithValueWrapper {
using execution_space = ExeSpace;
using value_type =
::Kokkos::Experimental::Impl::ValueWrapperForNoNeutralElement<ValueType>;
ValueType m_init_value;
FirstFrom m_first_from;
FirstDest m_first_dest;
BinaryOpType m_binary_op;
UnaryOpType m_unary_op;
KOKKOS_FUNCTION
TransformExclusiveScanFunctorWithValueWrapper(ValueType init,
FirstFrom first_from,
FirstDest first_dest,
BinaryOpType bop,
UnaryOpType uop)
: m_init_value(std::move(init)),
m_first_from(std::move(first_from)),
m_first_dest(std::move(first_dest)),
m_binary_op(std::move(bop)),
m_unary_op(std::move(uop)) {}
KOKKOS_FUNCTION
void operator()(const IndexType i, value_type& update,
const bool final_pass) const {
if (final_pass) {
if (i == 0) {
// for both ExclusiveScan and TransformExclusiveScan,
// init is unmodified
m_first_dest[i] = m_init_value;
} else {
m_first_dest[i] = m_binary_op(update.val, m_init_value);
}
}
const auto tmp = value_type{m_unary_op(m_first_from[i]), false};
this->join(update, tmp);
}
KOKKOS_FUNCTION void init(value_type& value) const {
value.val = {};
value.is_initial = true;
}
KOKKOS_FUNCTION
void join(value_type& update, const value_type& input) const {
if (input.is_initial) return;
if (update.is_initial) {
update.val = input.val;
} else {
update.val = m_binary_op(update.val, input.val);
}
update.is_initial = false;
}
};
template <class ExeSpace, class IndexType, class ValueType, class FirstFrom,
class FirstDest, class BinaryOpType, class UnaryOpType>
struct TransformExclusiveScanFunctorWithoutValueWrapper {
using execution_space = ExeSpace;
ValueType m_init_value;
FirstFrom m_first_from;
FirstDest m_first_dest;
BinaryOpType m_binary_op;
UnaryOpType m_unary_op;
KOKKOS_FUNCTION
TransformExclusiveScanFunctorWithoutValueWrapper(ValueType init,
FirstFrom first_from,
FirstDest first_dest,
BinaryOpType bop,
UnaryOpType uop)
: m_init_value(std::move(init)),
m_first_from(std::move(first_from)),
m_first_dest(std::move(first_dest)),
m_binary_op(std::move(bop)),
m_unary_op(std::move(uop)) {}
KOKKOS_FUNCTION
void operator()(const IndexType i, ValueType& update,
const bool final_pass) const {
if (final_pass) {
if (i == 0) {
// for both ExclusiveScan and TransformExclusiveScan,
// init is unmodified
m_first_dest[i] = m_init_value;
} else {
m_first_dest[i] = m_binary_op(update, m_init_value);
}
}
const auto tmp = ValueType{m_unary_op(m_first_from[i])};
this->join(update, tmp);
}
KOKKOS_FUNCTION
void init(ValueType& update) const { update = {}; }
KOKKOS_FUNCTION
void join(ValueType& update, const ValueType& input) const {
update = m_binary_op(update, input);
}
};
} // namespace Impl
} // namespace Experimental
} // namespace Kokkos
#endif

51
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_GenerateGenerateN.hpp
View File

@ -41,32 +41,65 @@ struct StdGenerateFunctor {
: m_first(std::move(_first)), m_generator(std::move(_g)) {} : m_first(std::move(_first)), m_generator(std::move(_g)) {}
}; };
//
// generate impl
//
template <class ExecutionSpace, class IteratorType, class Generator> template <class ExecutionSpace, class IteratorType, class Generator>
void generate_impl(const std::string& label, const ExecutionSpace& ex, void generate_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, Generator g) { IteratorType first, IteratorType last,
Generator g) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first); Impl::static_assert_random_access_and_accessible(ex, first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
// aliases
using func_t = StdGenerateFunctor<IteratorType, Generator>;
// run // run
const auto num_elements = Kokkos::Experimental::distance(first, last); const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for(label, ::Kokkos::parallel_for(label,
RangePolicy<ExecutionSpace>(ex, 0, num_elements), RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_t(first, g)); StdGenerateFunctor(first, g));
ex.fence("Kokkos::generate: fence after operation"); ex.fence("Kokkos::generate: fence after operation");
} }
template <class TeamHandleType, class IteratorType, class Generator>
KOKKOS_FUNCTION void generate_team_impl(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
Generator g) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
// run
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_for(TeamThreadRange(teamHandle, 0, num_elements),
StdGenerateFunctor(first, g));
teamHandle.team_barrier();
}
//
// generate_n impl
//
template <class ExecutionSpace, class IteratorType, class Size, class Generator> template <class ExecutionSpace, class IteratorType, class Size, class Generator>
IteratorType generate_n_impl(const std::string& label, const ExecutionSpace& ex, IteratorType generate_n_exespace_impl(const std::string& label,
IteratorType first, Size count, Generator g) { const ExecutionSpace& ex,
IteratorType first, Size count,
Generator g) {
if (count <= 0) { if (count <= 0) {
return first; return first;
} }
generate_impl(label, ex, first, first + count, g); generate_exespace_impl(label, ex, first, first + count, g);
return first + count;
}
template <class TeamHandleType, class IteratorType, class Size, class Generator>
KOKKOS_FUNCTION IteratorType
generate_n_team_impl(const TeamHandleType& teamHandle, IteratorType first,
Size count, Generator g) {
if (count <= 0) {
return first;
}
generate_team_impl(teamHandle, first, first + count, g);
return first + count; return first + count;
} }

144
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_InclusiveScan.hpp
View File

@ -101,9 +101,12 @@ struct InclusiveScanDefaultFunctor {
} }
}; };
//
// exespace impl
//
template <class ExecutionSpace, class InputIteratorType, template <class ExecutionSpace, class InputIteratorType,
class OutputIteratorType> class OutputIteratorType>
OutputIteratorType inclusive_scan_default_op_impl( OutputIteratorType inclusive_scan_default_op_exespace_impl(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
InputIteratorType first_from, InputIteratorType last_from, InputIteratorType first_from, InputIteratorType last_from,
OutputIteratorType first_dest) { OutputIteratorType first_dest) {
@ -143,7 +146,7 @@ OutputIteratorType inclusive_scan_default_op_impl(
// ------------------------------------------------------------- // -------------------------------------------------------------
template <class ExecutionSpace, class InputIteratorType, template <class ExecutionSpace, class InputIteratorType,
class OutputIteratorType, class BinaryOpType> class OutputIteratorType, class BinaryOpType>
OutputIteratorType inclusive_scan_custom_binary_op_impl( OutputIteratorType inclusive_scan_custom_binary_op_exespace_impl(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
InputIteratorType first_from, InputIteratorType last_from, InputIteratorType first_from, InputIteratorType last_from,
OutputIteratorType first_dest, BinaryOpType binary_op) { OutputIteratorType first_dest, BinaryOpType binary_op) {
@ -158,7 +161,7 @@ OutputIteratorType inclusive_scan_custom_binary_op_impl(
using value_type = using value_type =
std::remove_const_t<typename InputIteratorType::value_type>; std::remove_const_t<typename InputIteratorType::value_type>;
using unary_op_type = StdNumericScanIdentityReferenceUnaryFunctor<value_type>; using unary_op_type = StdNumericScanIdentityReferenceUnaryFunctor<value_type>;
using func_type = TransformInclusiveScanNoInitValueFunctor< using func_type = ExeSpaceTransformInclusiveScanNoInitValueFunctor<
ExecutionSpace, index_type, value_type, InputIteratorType, ExecutionSpace, index_type, value_type, InputIteratorType,
OutputIteratorType, BinaryOpType, unary_op_type>; OutputIteratorType, BinaryOpType, unary_op_type>;
@ -179,7 +182,7 @@ OutputIteratorType inclusive_scan_custom_binary_op_impl(
// ------------------------------------------------------------- // -------------------------------------------------------------
template <class ExecutionSpace, class InputIteratorType, template <class ExecutionSpace, class InputIteratorType,
class OutputIteratorType, class BinaryOpType, class ValueType> class OutputIteratorType, class BinaryOpType, class ValueType>
OutputIteratorType inclusive_scan_custom_binary_op_impl( OutputIteratorType inclusive_scan_custom_binary_op_exespace_impl(
const std::string& label, const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex,
InputIteratorType first_from, InputIteratorType last_from, InputIteratorType first_from, InputIteratorType last_from,
OutputIteratorType first_dest, BinaryOpType binary_op, OutputIteratorType first_dest, BinaryOpType binary_op,
@ -193,7 +196,7 @@ OutputIteratorType inclusive_scan_custom_binary_op_impl(
// aliases // aliases
using index_type = typename InputIteratorType::difference_type; using index_type = typename InputIteratorType::difference_type;
using unary_op_type = StdNumericScanIdentityReferenceUnaryFunctor<ValueType>; using unary_op_type = StdNumericScanIdentityReferenceUnaryFunctor<ValueType>;
using func_type = TransformInclusiveScanWithInitValueFunctor< using func_type = ExeSpaceTransformInclusiveScanWithInitValueFunctor<
ExecutionSpace, index_type, ValueType, InputIteratorType, ExecutionSpace, index_type, ValueType, InputIteratorType,
OutputIteratorType, BinaryOpType, unary_op_type>; OutputIteratorType, BinaryOpType, unary_op_type>;
@ -203,13 +206,142 @@ OutputIteratorType inclusive_scan_custom_binary_op_impl(
::Kokkos::parallel_scan(label, ::Kokkos::parallel_scan(label,
RangePolicy<ExecutionSpace>(ex, 0, num_elements), RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_type(first_from, first_dest, binary_op, func_type(first_from, first_dest, binary_op,
unary_op_type(), init_value)); unary_op_type(), std::move(init_value)));
ex.fence("Kokkos::inclusive_scan_custom_binary_op: fence after operation"); ex.fence("Kokkos::inclusive_scan_custom_binary_op: fence after operation");
// return // return
return first_dest + num_elements; return first_dest + num_elements;
} }
//
// team impl
//
template <class TeamHandleType, class InputIteratorType,
class OutputIteratorType>
KOKKOS_FUNCTION OutputIteratorType inclusive_scan_default_op_team_impl(
const TeamHandleType& teamHandle, InputIteratorType first_from,
InputIteratorType last_from, OutputIteratorType first_dest) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first_from,
first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest);
Impl::expect_valid_range(first_from, last_from);
using value_type =
std::remove_const_t<typename InputIteratorType::value_type>;
// #if defined(KOKKOS_ENABLE_CUDA)
using exe_space = typename TeamHandleType::execution_space;
using index_type = typename InputIteratorType::difference_type;
using func_type = std::conditional_t<
::Kokkos::is_detected<in_scan_has_reduction_identity_sum_t,
value_type>::value,
InclusiveScanDefaultFunctorForKnownIdentityElement<
exe_space, index_type, value_type, InputIteratorType,
OutputIteratorType>,
InclusiveScanDefaultFunctor<exe_space, index_type, value_type,
InputIteratorType, OutputIteratorType>>;
// run
const auto num_elements =
Kokkos::Experimental::distance(first_from, last_from);
::Kokkos::parallel_scan(TeamThreadRange(teamHandle, 0, num_elements),
func_type(first_from, first_dest));
teamHandle.team_barrier();
// return
return first_dest + num_elements;
}
// -------------------------------------------------------------
// inclusive_scan_custom_binary_op_impl
// -------------------------------------------------------------
template <class TeamHandleType, class InputIteratorType,
class OutputIteratorType, class BinaryOpType>
KOKKOS_FUNCTION OutputIteratorType inclusive_scan_custom_binary_op_team_impl(
const TeamHandleType& teamHandle, InputIteratorType first_from,
InputIteratorType last_from, OutputIteratorType first_dest,
BinaryOpType binary_op) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first_from,
first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest);
Impl::expect_valid_range(first_from, last_from);
using value_type =
std::remove_const_t<typename InputIteratorType::value_type>;
static_assert(
::Kokkos::is_detected_v<ex_scan_has_reduction_identity_sum_t, value_type>,
"At the moment inclusive_scan doesn't support types without reduction "
"identity");
// #if defined(KOKKOS_ENABLE_CUDA)
// aliases
using exe_space = typename TeamHandleType::execution_space;
using unary_op_type = StdNumericScanIdentityReferenceUnaryFunctor<value_type>;
using func_type = TeamTransformInclusiveScanNoInitValueFunctor<
exe_space, value_type, InputIteratorType, OutputIteratorType,
BinaryOpType, unary_op_type>;
// run
const auto num_elements =
Kokkos::Experimental::distance(first_from, last_from);
::Kokkos::parallel_scan(
TeamThreadRange(teamHandle, 0, num_elements),
func_type(first_from, first_dest, binary_op, unary_op_type()));
teamHandle.team_barrier();
return first_dest + num_elements;
}
// -------------------------------------------------------------
// inclusive_scan_custom_binary_op_impl with init_value
// -------------------------------------------------------------
template <class TeamHandleType, class InputIteratorType,
class OutputIteratorType, class BinaryOpType, class ValueType>
KOKKOS_FUNCTION OutputIteratorType inclusive_scan_custom_binary_op_team_impl(
const TeamHandleType& teamHandle, InputIteratorType first_from,
InputIteratorType last_from, OutputIteratorType first_dest,
BinaryOpType binary_op, ValueType init_value) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first_from,
first_dest);
Impl::static_assert_iterators_have_matching_difference_type(first_from,
first_dest);
Impl::expect_valid_range(first_from, last_from);
static_assert(
::Kokkos::is_detected_v<ex_scan_has_reduction_identity_sum_t, ValueType>,
"At the moment inclusive_scan doesn't support types without reduction "
"identity");
// #if defined(KOKKOS_ENABLE_CUDA)
// aliases
using exe_space = typename TeamHandleType::execution_space;
using unary_op_type = StdNumericScanIdentityReferenceUnaryFunctor<ValueType>;
using func_type = TeamTransformInclusiveScanWithInitValueFunctor<
exe_space, ValueType, InputIteratorType, OutputIteratorType, BinaryOpType,
unary_op_type>;
// run
const auto num_elements =
Kokkos::Experimental::distance(first_from, last_from);
::Kokkos::parallel_scan(TeamThreadRange(teamHandle, 0, num_elements),
func_type(first_from, first_dest, binary_op,
unary_op_type(), std::move(init_value)));
teamHandle.team_barrier();
// return
return first_dest + num_elements;
}
} // namespace Impl } // namespace Impl
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

71
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_IsPartitioned.hpp
View File

@ -62,9 +62,9 @@ struct StdIsPartitionedFunctor {
}; };
template <class ExecutionSpace, class IteratorType, class PredicateType> template <class ExecutionSpace, class IteratorType, class PredicateType>
bool is_partitioned_impl(const std::string& label, const ExecutionSpace& ex, bool is_partitioned_exespace_impl(const std::string& label,
IteratorType first, IteratorType last, const ExecutionSpace& ex, IteratorType first,
PredicateType pred) { IteratorType last, PredicateType pred) {
// true if all elements in the range [first, last) that satisfy // true if all elements in the range [first, last) that satisfy
// the predicate "pred" appear before all elements that don't. // the predicate "pred" appear before all elements that don't.
// Also returns true if [first, last) is empty. // Also returns true if [first, last) is empty.
@ -97,6 +97,7 @@ bool is_partitioned_impl(const std::string& label, const ExecutionSpace& ex,
const auto num_elements = Kokkos::Experimental::distance(first, last); const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_reduce(label, ::Kokkos::parallel_reduce(label,
RangePolicy<ExecutionSpace>(ex, 0, num_elements), RangePolicy<ExecutionSpace>(ex, 0, num_elements),
func_t(first, reducer, pred), reducer); func_t(first, reducer, pred), reducer);
// fence not needed because reducing into scalar // fence not needed because reducing into scalar
@ -109,8 +110,72 @@ bool is_partitioned_impl(const std::string& label, const ExecutionSpace& ex,
if (red_result.max_loc_true != red_id_max && if (red_result.max_loc_true != red_id_max &&
red_result.min_loc_false != red_id_min) { red_result.min_loc_false != red_id_min) {
// this occurs when the reduction yields nontrivial values
return red_result.max_loc_true < red_result.min_loc_false; return red_result.max_loc_true < red_result.min_loc_false;
} else if (red_result.max_loc_true == red_id_max &&
red_result.min_loc_false == 0) {
// this occurs when all values do NOT satisfy
// the predicate, and this corner case should also be true
return true;
} else if (first + red_result.max_loc_true == --last) { } else if (first + red_result.max_loc_true == --last) {
// this occurs when all values satisfy the predicate,
// this corner case should also be true
return true;
} else {
return false;
}
}
template <class TeamHandleType, class IteratorType, class PredicateType>
KOKKOS_FUNCTION bool is_partitioned_team_impl(const TeamHandleType& teamHandle,
IteratorType first,
IteratorType last,
PredicateType pred) {
/* see exespace impl for the description of the impl */
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
// trivial case
if (first == last) {
return true;
}
// aliases
using index_type = typename IteratorType::difference_type;
using reducer_type = StdIsPartitioned<index_type>;
using reduction_value_type = typename reducer_type::value_type;
using func_t =
StdIsPartitionedFunctor<IteratorType, reducer_type, PredicateType>;
// run
reduction_value_type red_result;
reducer_type reducer(red_result);
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, num_elements),
func_t(first, reducer, pred), reducer);
// fence not needed because reducing into scalar
// decide and return
constexpr index_type red_id_min =
::Kokkos::reduction_identity<index_type>::min();
constexpr index_type red_id_max =
::Kokkos::reduction_identity<index_type>::max();
if (red_result.max_loc_true != red_id_max &&
red_result.min_loc_false != red_id_min) {
// this occurs when the reduction yields nontrivial values
return red_result.max_loc_true < red_result.min_loc_false;
} else if (red_result.max_loc_true == red_id_max &&
red_result.min_loc_false == 0) {
// this occurs when all values do NOT satisfy
// the predicate, and this corner case should also be true
return true;
} else if (first + red_result.max_loc_true == --last) {
// this occurs when all values satisfy the predicate,
// this corner case should also be true
return true; return true;
} else { } else {
return false; return false;

53
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_IsSorted.hpp
View File

@ -48,10 +48,13 @@ struct StdIsSortedFunctor {
: m_first(std::move(_first1)), m_comparator(std::move(comparator)) {} : m_first(std::move(_first1)), m_comparator(std::move(comparator)) {}
}; };
//
// exespace impl
//
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <class ExecutionSpace, class IteratorType, class ComparatorType>
bool is_sorted_impl(const std::string& label, const ExecutionSpace& ex, bool is_sorted_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first, IteratorType last,
ComparatorType comp) { ComparatorType comp) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first); Impl::static_assert_random_access_and_accessible(ex, first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
@ -75,11 +78,49 @@ bool is_sorted_impl(const std::string& label, const ExecutionSpace& ex,
} }
template <class ExecutionSpace, class IteratorType> template <class ExecutionSpace, class IteratorType>
bool is_sorted_impl(const std::string& label, const ExecutionSpace& ex, bool is_sorted_exespace_impl(const std::string& label, const ExecutionSpace& ex,
IteratorType first, IteratorType last) { IteratorType first, IteratorType last) {
using value_type = typename IteratorType::value_type; using value_type = typename IteratorType::value_type;
using pred_t = Impl::StdAlgoLessThanBinaryPredicate<value_type>; using pred_t = Impl::StdAlgoLessThanBinaryPredicate<value_type>;
return is_sorted_impl(label, ex, first, last, pred_t()); return is_sorted_exespace_impl(label, ex, first, last, pred_t());
}
//
// team impl
//
template <class TeamHandleType, class IteratorType, class ComparatorType>
KOKKOS_FUNCTION bool is_sorted_team_impl(const TeamHandleType& teamHandle,
IteratorType first, IteratorType last,
ComparatorType comp) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
const auto num_elements = Kokkos::Experimental::distance(first, last);
if (num_elements <= 1) {
return true;
}
// use num_elements-1 because each index handles i and i+1
const auto num_elements_minus_one = num_elements - 1;
// result is incremented by one if sorting breaks at index i
std::size_t result = 0;
::Kokkos::parallel_reduce(
TeamThreadRange(teamHandle, 0, num_elements_minus_one),
// use CTAD here
StdIsSortedFunctor(first, std::move(comp)), result);
return result == 0;
}
template <class TeamHandleType, class IteratorType>
KOKKOS_FUNCTION bool is_sorted_team_impl(const TeamHandleType& teamHandle,
IteratorType first,
IteratorType last) {
using value_type = typename IteratorType::value_type;
using pred_t = Impl::StdAlgoLessThanBinaryPredicate<value_type>;
return is_sorted_team_impl(teamHandle, first, last, pred_t());
} }
} // namespace Impl } // namespace Impl

74
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_IsSortedUntil.hpp
View File

@ -54,10 +54,15 @@ struct StdIsSortedUntilFunctor {
m_reducer(std::move(reducer)) {} m_reducer(std::move(reducer)) {}
}; };
//
// overloads accepting exespace
//
template <class ExecutionSpace, class IteratorType, class ComparatorType> template <class ExecutionSpace, class IteratorType, class ComparatorType>
IteratorType is_sorted_until_impl(const std::string& label, IteratorType is_sorted_until_exespace_impl(const std::string& label,
const ExecutionSpace& ex, IteratorType first, const ExecutionSpace& ex,
IteratorType last, ComparatorType comp) { IteratorType first,
IteratorType last,
ComparatorType comp) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first); Impl::static_assert_random_access_and_accessible(ex, first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
@ -81,7 +86,6 @@ IteratorType is_sorted_until_impl(const std::string& label,
label, label,
// use num_elements-1 because each index handles i and i+1 // use num_elements-1 because each index handles i and i+1
RangePolicy<ExecutionSpace>(ex, 0, num_elements - 1), RangePolicy<ExecutionSpace>(ex, 0, num_elements - 1),
// use CTAD
StdIsSortedUntilFunctor(first, comp, reducer), reducer); StdIsSortedUntilFunctor(first, comp, reducer), reducer);
/* If the reduction result is equal to the initial value, /* If the reduction result is equal to the initial value,
@ -98,12 +102,66 @@ IteratorType is_sorted_until_impl(const std::string& label,
} }
template <class ExecutionSpace, class IteratorType> template <class ExecutionSpace, class IteratorType>
IteratorType is_sorted_until_impl(const std::string& label, IteratorType is_sorted_until_exespace_impl(const std::string& label,
const ExecutionSpace& ex, IteratorType first, const ExecutionSpace& ex,
IteratorType last) { IteratorType first,
IteratorType last) {
using value_type = typename IteratorType::value_type; using value_type = typename IteratorType::value_type;
using pred_t = Impl::StdAlgoLessThanBinaryPredicate<value_type>; using pred_t = Impl::StdAlgoLessThanBinaryPredicate<value_type>;
return is_sorted_until_impl(label, ex, first, last, pred_t()); return is_sorted_until_exespace_impl(label, ex, first, last, pred_t());
}
//
// overloads accepting team handle
//
template <class ExecutionSpace, class IteratorType, class ComparatorType>
KOKKOS_FUNCTION IteratorType
is_sorted_until_team_impl(const ExecutionSpace& teamHandle, IteratorType first,
IteratorType last, ComparatorType comp) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
const auto num_elements = Kokkos::Experimental::distance(first, last);
// trivial case
if (num_elements <= 1) {
return last;
}
/*
Do a par_reduce computing the *min* index that breaks the sorting.
If one such index is found, then the range is sorted until that element,
if no such index is found, then it means the range is sorted until the end.
*/
using index_type = typename IteratorType::difference_type;
index_type red_result;
index_type red_result_init;
::Kokkos::Min<index_type> reducer(red_result);
reducer.init(red_result_init);
::Kokkos::parallel_reduce( // use num_elements-1 because each index handles i
// and i+1
TeamThreadRange(teamHandle, 0, num_elements - 1),
StdIsSortedUntilFunctor(first, comp, reducer), reducer);
teamHandle.team_barrier();
/* If the reduction result is equal to the initial value,
and it means the range is sorted until the end */
if (red_result == red_result_init) {
return last;
} else {
/* If such index is found, then the range is sorted until there and
we need to return an iterator past the element found so do +1 */
return first + (red_result + 1);
}
}
template <class ExecutionSpace, class IteratorType>
KOKKOS_FUNCTION IteratorType is_sorted_until_team_impl(
const ExecutionSpace& teamHandle, IteratorType first, IteratorType last) {
using value_type = typename IteratorType::value_type;
using pred_t = Impl::StdAlgoLessThanBinaryPredicate<value_type>;
return is_sorted_until_team_impl(teamHandle, first, last, pred_t());
} }
} // namespace Impl } // namespace Impl

92
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_LexicographicalCompare.hpp
View File

@ -84,13 +84,15 @@ struct StdLexicographicalCompareFunctor {
m_comparator(std::move(_comp)) {} m_comparator(std::move(_comp)) {}
}; };
//
// exespace impl
//
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <class ExecutionSpace, class IteratorType1, class IteratorType2,
class ComparatorType> class ComparatorType>
bool lexicographical_compare_impl(const std::string& label, bool lexicographical_compare_exespace_impl(
const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 first1, IteratorType1 last1, IteratorType1 last1, IteratorType2 first2, IteratorType2 last2,
IteratorType2 first2, IteratorType2 last2, ComparatorType comp) {
ComparatorType comp) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first1, first2); Impl::static_assert_random_access_and_accessible(ex, first1, first2);
Impl::static_assert_iterators_have_matching_difference_type(first1, first2); Impl::static_assert_iterators_have_matching_difference_type(first1, first2);
@ -139,16 +141,84 @@ bool lexicographical_compare_impl(const std::string& label,
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <class ExecutionSpace, class IteratorType1, class IteratorType2>
bool lexicographical_compare_impl(const std::string& label, bool lexicographical_compare_exespace_impl(
const ExecutionSpace& ex, const std::string& label, const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 first1, IteratorType1 last1, IteratorType1 last1, IteratorType2 first2, IteratorType2 last2) {
IteratorType2 first2, IteratorType2 last2) {
using value_type_1 = typename IteratorType1::value_type; using value_type_1 = typename IteratorType1::value_type;
using value_type_2 = typename IteratorType2::value_type; using value_type_2 = typename IteratorType2::value_type;
using predicate_t = using predicate_t =
Impl::StdAlgoLessThanBinaryPredicate<value_type_1, value_type_2>; Impl::StdAlgoLessThanBinaryPredicate<value_type_1, value_type_2>;
return lexicographical_compare_impl(label, ex, first1, last1, first2, last2, return lexicographical_compare_exespace_impl(label, ex, first1, last1, first2,
predicate_t()); last2, predicate_t());
}
//
// team impl
//
template <class TeamHandleType, class IteratorType1, class IteratorType2,
class ComparatorType>
KOKKOS_FUNCTION bool lexicographical_compare_team_impl(
const TeamHandleType& teamHandle, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2, ComparatorType comp) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first1, first2);
Impl::static_assert_iterators_have_matching_difference_type(first1, first2);
Impl::expect_valid_range(first1, last1);
Impl::expect_valid_range(first2, last2);
// aliases
using index_type = typename IteratorType1::difference_type;
using reducer_type = FirstLoc<index_type>;
using reduction_value_type = typename reducer_type::value_type;
// run
const auto d1 = Kokkos::Experimental::distance(first1, last1);
const auto d2 = Kokkos::Experimental::distance(first2, last2);
const auto range = Kokkos::min(d1, d2);
reduction_value_type red_result;
reducer_type reducer(red_result);
using func1_t =
StdLexicographicalCompareFunctor<index_type, IteratorType1, IteratorType2,
reducer_type, ComparatorType>;
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, range),
func1_t(first1, first2, reducer, comp), reducer);
teamHandle.team_barrier();
// no mismatch
if (red_result.min_loc_true ==
::Kokkos::reduction_identity<index_type>::min()) {
auto new_last1 = first1 + range;
auto new_last2 = first2 + range;
bool is_prefix = (new_last1 == last1) && (new_last2 != last2);
return is_prefix;
}
// check mismatched
int less = 0;
auto it1 = first1 + red_result.min_loc_true;
auto it2 = first2 + red_result.min_loc_true;
using func2_t = StdCompareFunctor<index_type, IteratorType1, IteratorType2,
ComparatorType>;
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, 1),
func2_t(it1, it2, comp), less);
teamHandle.team_barrier();
return static_cast<bool>(less);
}
template <class TeamHandleType, class IteratorType1, class IteratorType2>
KOKKOS_FUNCTION bool lexicographical_compare_team_impl(
const TeamHandleType& teamHandle, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2) {
using value_type_1 = typename IteratorType1::value_type;
using value_type_2 = typename IteratorType2::value_type;
using predicate_t =
Impl::StdAlgoLessThanBinaryPredicate<value_type_1, value_type_2>;
return lexicographical_compare_team_impl(teamHandle, first1, last1, first2,
last2, predicate_t());
} }
} // namespace Impl } // namespace Impl

83
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_MinMaxMinmaxElement.hpp
View File

@ -63,12 +63,16 @@ struct StdMinMaxElemFunctor {
: m_first(std::move(first)), m_reducer(std::move(reducer)) {} : m_first(std::move(first)), m_reducer(std::move(reducer)) {}
}; };
//
// exespace impl
//
template <template <class... Args> class ReducerType, class ExecutionSpace, template <template <class... Args> class ReducerType, class ExecutionSpace,
class IteratorType, class... Args> class IteratorType, class... Args>
IteratorType min_or_max_element_impl(const std::string& label, IteratorType min_or_max_element_exespace_impl(const std::string& label,
const ExecutionSpace& ex, const ExecutionSpace& ex,
IteratorType first, IteratorType last, IteratorType first,
Args&&... args) { IteratorType last,
Args&&... args) {
// checks // checks
Impl::static_assert_random_access_and_accessible(ex, first); Impl::static_assert_random_access_and_accessible(ex, first);
Impl::expect_valid_range(first, last); Impl::expect_valid_range(first, last);
@ -100,7 +104,7 @@ IteratorType min_or_max_element_impl(const std::string& label,
template <template <class... Args> class ReducerType, class ExecutionSpace, template <template <class... Args> class ReducerType, class ExecutionSpace,
class IteratorType, class... Args> class IteratorType, class... Args>
::Kokkos::pair<IteratorType, IteratorType> minmax_element_impl( ::Kokkos::pair<IteratorType, IteratorType> minmax_element_exespace_impl(
const std::string& label, const ExecutionSpace& ex, IteratorType first, const std::string& label, const ExecutionSpace& ex, IteratorType first,
IteratorType last, Args&&... args) { IteratorType last, Args&&... args) {
// checks // checks
@ -132,6 +136,75 @@ template <template <class... Args> class ReducerType, class ExecutionSpace,
return {first + red_result.min_loc, first + red_result.max_loc}; return {first + red_result.min_loc, first + red_result.max_loc};
} }
//
// team level impl
//
template <template <class... Args> class ReducerType, class TeamHandleType,
class IteratorType, class... Args>
KOKKOS_FUNCTION IteratorType min_or_max_element_team_impl(
const TeamHandleType& teamHandle, IteratorType first, IteratorType last,
Args&&... args) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
if (first == last) {
return last;
}
// aliases
using index_type = typename IteratorType::difference_type;
using value_type = typename IteratorType::value_type;
using reducer_type = ReducerType<value_type, index_type, Args...>;
using reduction_value_type = typename reducer_type::value_type;
using func_t = StdMinOrMaxElemFunctor<IteratorType, reducer_type>;
// run
reduction_value_type red_result;
reducer_type reducer(red_result, std::forward<Args>(args)...);
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, num_elements),
func_t(first, reducer), reducer);
teamHandle.team_barrier();
// maybe the barrier is not needed since reducing into scalar?
// return
return first + red_result.loc;
}
template <template <class... Args> class ReducerType, class TeamHandleType,
class IteratorType, class... Args>
KOKKOS_FUNCTION ::Kokkos::pair<IteratorType, IteratorType>
minmax_element_team_impl(const TeamHandleType& teamHandle, IteratorType first,
IteratorType last, Args&&... args) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first);
Impl::expect_valid_range(first, last);
if (first == last) {
return {first, first};
}
// aliases
using index_type = typename IteratorType::difference_type;
using value_type = typename IteratorType::value_type;
using reducer_type = ReducerType<value_type, index_type, Args...>;
using reduction_value_type = typename reducer_type::value_type;
using func_t = StdMinMaxElemFunctor<IteratorType, reducer_type>;
// run
reduction_value_type red_result;
reducer_type reducer(red_result, std::forward<Args>(args)...);
const auto num_elements = Kokkos::Experimental::distance(first, last);
::Kokkos::parallel_reduce(TeamThreadRange(teamHandle, 0, num_elements),
func_t(first, reducer), reducer);
teamHandle.team_barrier();
// maybe the barrier is not needed since reducing into scalar?
// return
return {first + red_result.min_loc, first + red_result.max_loc};
}
} // namespace Impl } // namespace Impl
} // namespace Experimental } // namespace Experimental
} // namespace Kokkos } // namespace Kokkos

95
lib/kokkos/algorithms/src/std_algorithms/impl/Kokkos_Mismatch.hpp
View File

@ -27,9 +27,10 @@ namespace Kokkos {
namespace Experimental { namespace Experimental {
namespace Impl { namespace Impl {
template <class IndexType, class IteratorType1, class IteratorType2, template <class IteratorType1, class IteratorType2, class ReducerType,
class ReducerType, class BinaryPredicateType> class BinaryPredicateType>
struct StdMismatchRedFunctor { struct StdMismatchRedFunctor {
using index_type = typename IteratorType1::difference_type;
using red_value_type = typename ReducerType::value_type; using red_value_type = typename ReducerType::value_type;
IteratorType1 m_first1; IteratorType1 m_first1;
@ -38,14 +39,14 @@ struct StdMismatchRedFunctor {
BinaryPredicateType m_predicate; BinaryPredicateType m_predicate;
KOKKOS_FUNCTION KOKKOS_FUNCTION
void operator()(const IndexType i, red_value_type& red_value) const { void operator()(const index_type i, red_value_type& red_value) const {
const auto& my_value1 = m_first1[i]; const auto& my_value1 = m_first1[i];
const auto& my_value2 = m_first2[i]; const auto& my_value2 = m_first2[i];
// FIXME_NVHPC using a ternary operator causes problems // FIXME_NVHPC using a ternary operator causes problems
red_value_type rv = {i}; red_value_type rv = {i};
if (m_predicate(my_value1, my_value2)) { if (m_predicate(my_value1, my_value2)) {
rv = {::Kokkos::reduction_identity<IndexType>::min()}; rv = {::Kokkos::reduction_identity<index_type>::min()};
} }
m_reducer.join(red_value, rv); m_reducer.join(red_value, rv);
@ -60,9 +61,12 @@ struct StdMismatchRedFunctor {
m_predicate(std::move(predicate)) {} m_predicate(std::move(predicate)) {}
}; };
//
// exespace impl
//
template <class ExecutionSpace, class IteratorType1, class IteratorType2, template <class ExecutionSpace, class IteratorType1, class IteratorType2,
class BinaryPredicateType> class BinaryPredicateType>
::Kokkos::pair<IteratorType1, IteratorType2> mismatch_impl( ::Kokkos::pair<IteratorType1, IteratorType2> mismatch_exespace_impl(
const std::string& label, const ExecutionSpace& ex, IteratorType1 first1, const std::string& label, const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 last1, IteratorType2 first2, IteratorType2 last2, IteratorType1 last1, IteratorType2 first2, IteratorType2 last2,
BinaryPredicateType predicate) { BinaryPredicateType predicate) {
@ -77,9 +81,6 @@ template <class ExecutionSpace, class IteratorType1, class IteratorType2,
using index_type = typename IteratorType1::difference_type; using index_type = typename IteratorType1::difference_type;
using reducer_type = FirstLoc<index_type>; using reducer_type = FirstLoc<index_type>;
using reduction_value_type = typename reducer_type::value_type; using reduction_value_type = typename reducer_type::value_type;
using functor_type =
StdMismatchRedFunctor<index_type, IteratorType1, IteratorType2,
reducer_type, BinaryPredicateType>;
// trivial case: note that this is important, // trivial case: note that this is important,
// for OpenMPTarget, omitting special handling of // for OpenMPTarget, omitting special handling of
@ -96,7 +97,9 @@ template <class ExecutionSpace, class IteratorType1, class IteratorType2,
reducer_type reducer(red_result); reducer_type reducer(red_result);
::Kokkos::parallel_reduce( ::Kokkos::parallel_reduce(
label, RangePolicy<ExecutionSpace>(ex, 0, num_elemen_par_reduce), label, RangePolicy<ExecutionSpace>(ex, 0, num_elemen_par_reduce),
functor_type(first1, first2, reducer, std::move(predicate)), reducer); // use CTAD
StdMismatchRedFunctor(first1, first2, reducer, std::move(predicate)),
reducer);
// fence not needed because reducing into scalar // fence not needed because reducing into scalar
@ -119,13 +122,83 @@ template <class ExecutionSpace, class IteratorType1, class IteratorType2,
} }
template <class ExecutionSpace, class IteratorType1, class IteratorType2> template <class ExecutionSpace, class IteratorType1, class IteratorType2>
::Kokkos::pair<IteratorType1, IteratorType2> mismatch_impl( ::Kokkos::pair<IteratorType1, IteratorType2> mismatch_exespace_impl(
const std::string& label, const ExecutionSpace& ex, IteratorType1 first1, const std::string& label, const ExecutionSpace& ex, IteratorType1 first1,
IteratorType1 last1, IteratorType2 first2, IteratorType2 last2) { IteratorType1 last1, IteratorType2 first2, IteratorType2 last2) {
using value_type1 = typename IteratorType1::value_type; using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type; using value_type2 = typename IteratorType2::value_type;
using pred_t = StdAlgoEqualBinaryPredicate<value_type1, value_type2>; using pred_t = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return mismatch_impl(label, ex, first1, last1, first2, last2, pred_t()); return mismatch_exespace_impl(label, ex, first1, last1, first2, last2,
pred_t());
}
//
// team impl
//
template <class TeamHandleType, class IteratorType1, class IteratorType2,
class BinaryPredicateType>
KOKKOS_FUNCTION ::Kokkos::pair<IteratorType1, IteratorType2> mismatch_team_impl(
const TeamHandleType& teamHandle, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2, BinaryPredicateType predicate) {
// checks
Impl::static_assert_random_access_and_accessible(teamHandle, first1, first2);
Impl::static_assert_iterators_have_matching_difference_type(first1, first2);
Impl::expect_valid_range(first1, last1);
Impl::expect_valid_range(first2, last2);
// aliases
using return_type = ::Kokkos::pair<IteratorType1, IteratorType2>;
using index_type = typename IteratorType1::difference_type;
using reducer_type = FirstLoc<index_type>;
using reduction_value_type = typename reducer_type::value_type;
// trivial case: note that this is important,
// for OpenMPTarget, omitting special handling of
// the trivial case was giving all sorts of strange stuff.
const auto num_e1 = last1 - first1;
const auto num_e2 = last2 - first2;
if (num_e1 == 0 || num_e2 == 0) {
return return_type(first1, first2);
}
// run
const auto num_elemen_par_reduce = (num_e1 <= num_e2) ? num_e1 : num_e2;
reduction_value_type red_result;
reducer_type reducer(red_result);
::Kokkos::parallel_reduce(
TeamThreadRange(teamHandle, 0, num_elemen_par_reduce),
// use CTAD
StdMismatchRedFunctor(first1, first2, reducer, std::move(predicate)),
reducer);
teamHandle.team_barrier();
// decide and return
constexpr auto red_min = ::Kokkos::reduction_identity<index_type>::min();
if (red_result.min_loc_true == red_min) {
// in here means mismatch has not been found
if (num_e1 == num_e2) {
return return_type(last1, last2);
} else if (num_e1 < num_e2) {
return return_type(last1, first2 + num_e1);
} else {
return return_type(first1 + num_e2, last2);
}
} else {
// in here means mismatch has been found
return return_type(first1 + red_result.min_loc_true,
first2 + red_result.min_loc_true);
}
}
template <class TeamHandleType, class IteratorType1, class IteratorType2>
KOKKOS_FUNCTION ::Kokkos::pair<IteratorType1, IteratorType2> mismatch_team_impl(
const TeamHandleType& teamHandle, IteratorType1 first1, IteratorType1 last1,
IteratorType2 first2, IteratorType2 last2) {
using value_type1 = typename IteratorType1::value_type;
using value_type2 = typename IteratorType2::value_type;
using pred_t = StdAlgoEqualBinaryPredicate<value_type1, value_type2>;
return mismatch_team_impl(teamHandle, first1, last1, first2, last2, pred_t());
} }
} // namespace Impl } // namespace Impl

Some files were not shown because too many files have changed in this diff Show More