Remove MPI compile-time dependency (#264)

* use dlsym for MPI functions

to allow compiling without MPI support, convert the usage of MPI functions and symbols to be based on a dlopen/dlsym based mechanism. Turns out this cannot be done entirely vendor neutral, slightly different solutions might be required for Open MPI, MPICH and the new MPI ABI.

* checkpoint

more work to be done.

* checkpoint 2

* checkpoint 3

* checkpoint 4

examples compile and link correctly

* checkpoitn 5 (I think)

* Checkpoitn 6

* dyld-mpi: adapt GDA

* dyldmpi: tests that depend on MPI need to link with it themselves

* do not ../mpi_instance.h

* dyldmpi: make the symetricHeapTestFixture compile

* dyldmpi: Change cmakery, compiles and run gda w/o external MPI

* Make it also compile in external MPI mode

* dyldmpi: ipc unit tests compile but do not link

* dyldmpi: new approach, if external mpi required, link with mpi,
otherwise use ompi5 abi

* C-style comments in cmakelist..

* dyldmpi: examples: do not fail compiling if MPI not found at build time,
instead do not compile the MPI required examples

* more updates to CMake logic

* convert RO backend

and a few other cleanups

* update some unit tests

to work with the dlopen MPI environment correctly.

---------

Co-authored-by: Aurelien Bouteiller <abouteil@amd.com>

src/team.cpp

@@ -29,6 +29,7 @@
 #include "rocshmem/rocshmem.hpp"
 #include "backend_bc.hpp"
 #include "util.hpp"
+#include "mpi_instance.hpp"
 
 namespace rocshmem {
 
@@ -84,7 +85,7 @@ __host__ Team::Team(Backend* handle, TeamInfo* team_info_wrt_parent,
       num_pes(_num_pes),
       my_pe(_my_pe) {
   if (_mpi_comm != MPI_COMM_NULL) {
-    MPI_Comm_dup (_mpi_comm, &mpi_comm);
+    mpilib_ftable_.Comm_dup (_mpi_comm, &mpi_comm);
   }
 }
 
@@ -117,7 +118,7 @@ __host__ __device__ int Team::get_pe_in_my_team(int pe_in_world) {
 
 __host__ Team::~Team() {
   if (mpi_comm != MPI_COMM_NULL)
-    MPI_Comm_free (&mpi_comm);
+    mpilib_ftable_.Comm_free (&mpi_comm);
 }
 
 }  // namespace rocshmem