Scippy

UG

Ubiquity Generator framework

ParaCommMpi Class Reference

Communicator object for MPI communications. More...

#include <paraCommMpi.h>

+ Inheritance diagram for ParaCommMpi:

Public Member Functions

 ParaCommMpi ()
 default constructor of ParaCommMpi More...
 
 ParaCommMpi (MPI_Comm comm)
 constructor of ParaCommMpi with MPI communicator More...
 
virtual ~ParaCommMpi ()
 destructor of this communicator More...
 
MPI_Comm & getMpiComm ()
 getter of MPI_Comm More...
 
virtual void init (int argc, char **argv)
 initializer of this communicator More...
 
double getStartTime ()
 get start time of this communicator More...
 
int getRank ()
 get rank of caller's thread More...
 
int getSize ()
 get size of this communicator, which indicates how many threads in a UG process More...
 
int getNumOfMessagesWaitingToSend (int dest=-1)
 get size of the messageQueueTable More...
 
void lcInit (ParaParamSet *paraParamSet)
 initializer for LoadCoordinator More...
 
void solverInit (ParaParamSet *paraParamSet)
 initializer for Solvers More...
 
void abort ()
 abort. How it works sometimes depends on communicator used More...
 
virtual bool waitTerminatedMessage ()
 function to wait Terminated message (This function is not used currently) More...
 
virtual bool waitToken (int rank)
 wait token when UG runs with deterministic mode More...
 
virtual void passToken (int rank)
 pass token to from the rank to the next More...
 
virtual bool passTermToken (int rank)
 pass termination token from the rank to the next More...
 
virtual void setToken (int rank, int *inToken)
 set received token to this communicator More...
 
virtual void lockApp ()
 lock UG application to synchronize with other threads More...
 
virtual void unlockApp ()
 unlock UG application to synchronize with other threads More...
 
ParaTimercreateParaTimer ()
 create ParaTimer object More...
 
int bcast (void *buffer, int count, const int datatypeId, int root)
 broadcast function for standard ParaData types More...
 
virtual int send (void *bufer, int count, const int datatypeId, int dest, const int tag)
 send function for standard ParaData types More...
 
int iSend (void *bufer, int count, const int datatypeId, int dest, const int tag, MPI_Request *req)
 send function for standard ParaData types More...
 
int receive (void *bufer, int count, const int datatypeId, int source, const int tag)
 receive function for standard ParaData types More...
 
void waitSpecTagFromSpecSource (const int source, const int tag, int *receivedTag)
 wait function for a specific tag from a specific source coming from More...
 
bool probe (int *source, int *tag)
 probe function which waits a new message More...
 
bool iProbe (int *source, int *tag)
 iProbe function which checks if a new message is arrived or not More...
 
int ubcast (void *buffer, int count, MPI_Datatype datatype, int root)
 User type bcast for created data type. More...
 
int usend (void *bufer, int count, MPI_Datatype datatype, int dest, int tag)
 User type send for created data type. More...
 
int iUsend (void *bufer, int count, MPI_Datatype datatype, int dest, int tag, MPI_Request *req)
 User type send for created data type. More...
 
int ureceive (void *bufer, int count, MPI_Datatype datatype, int source, int tag)
 User type receive for created data type. More...
 
int testAllIsends ()
 
void waitAllIsends ()
 
virtual const char * getTagString (int tag)
 get Tag string for debugging More...
 
- Public Member Functions inherited from ParaComm
 ParaComm ()
 default constructor of ParaComm More...
 
virtual ~ParaComm ()
 destructor of ParaComm More...
 
virtual void setLocalRank (int inRank)
 
virtual void lockApp (char const *f, int l)
 lock UG application to synchronize with other threads (for debug) More...
 
virtual void unlockApp (char const *f, int l)
 unlock UG application to synchronize with other threads (for debug) More...
 
virtual ParaCalculationStatecreateParaCalculationState ()=0
 transfer object factory More...
 
virtual ParaRacingRampUpParamSetcreateParaRacingRampUpParamSet ()=0
 create ParaRacingRampUpParamSet object More...
 
virtual ParaTaskcreateParaTask ()=0
 create ParaTask object by default constructor More...
 
virtual ParaParamSetcreateParaParamSet ()=0
 create ParaParamSet object More...
 
virtual ParaSolverStatecreateParaSolverState ()=0
 create ParaSolverState object by default constructor More...
 
virtual ParaSolverTerminationStatecreateParaSolverTerminationState ()=0
 create ParaSolverTerminationState object by default constructor More...
 
virtual ParaInstancecreateParaInstance ()=0
 create ParaInstance object by default constructor More...
 
virtual ParaSolutioncreateParaSolution ()=0
 create ParaSolution object by default constructor More...
 
virtual ParaDiffSubproblemcreateParaDiffSubproblem ()
 create ParaDiffSubproblem object by default constructor More...
 

Public Attributes

std::deque< ParaIsendRequest * > iSendRequestDeque
 

Protected Member Functions

virtual bool tagStringTableIsSetUpCoorectly ()
 check if tag string table (for debugging) set up correctly More...
 

Protected Attributes

MPI_Comm myComm
 MPI communicator. More...
 
int myCommSize
 communicator size : number of processes joined in this system More...
 
int myRank
 rank of this process More...
 
int namelen
 length of this process name More...
 
char procName [MPI_MAX_PROCESSOR_NAME]
 process name More...
 
bool tagTraceFlag
 indicate if tags are traced or not More...
 
std::ofstream ofs
 output file stream for tag trace More...
 
std::ostream * tos
 output file stream for tag trace to change file name More...
 
double startTime
 start time of this communicator More...
 
int token [2]
 index 0: token index 1: token color -1: green > 0: yellow ( termination origin solver number ) -2: red ( means the solver can terminate ) More...
 
pthread_mutex_t tokenAccessLock
 mutex for pthread thread More...
 
std::mutex applicationLockMutex
 mutex for applications More...
 

Static Protected Attributes

static MPI_Datatype datatypes [TYPE_LIST_SIZE]
 data type mapping table to MPI data type More...
 
static const char * tagStringTable []
 table for tag name string More...
 

Detailed Description

Communicator object for MPI communications.

Definition at line 93 of file paraCommMpi.h.

Constructor & Destructor Documentation

◆ ParaCommMpi() [1/2]

default constructor of ParaCommMpi

Definition at line 138 of file paraCommMpi.h.

◆ ParaCommMpi() [2/2]

ParaCommMpi ( MPI_Comm  comm)

constructor of ParaCommMpi with MPI communicator

Parameters
commmy communicator

Definition at line 161 of file paraCommMpi.h.

References ParaCommMpi::~ParaCommMpi().

◆ ~ParaCommMpi()

~ParaCommMpi ( )
virtual

destructor of this communicator

Definition at line 316 of file paraCommMpi.cpp.

Referenced by ParaCommMpi::ParaCommMpi().

Member Function Documentation

◆ abort()

void abort ( )
virtual

abort. How it works sometimes depends on communicator used

Implements ParaComm.

Definition at line 118 of file paraCommMpi.cpp.

Referenced by ParaCommMpi::getNumOfMessagesWaitingToSend(), and main().

◆ bcast()

int bcast ( void *  buffer,
int  count,
const int  datatypeId,
int  root 
)
virtual

broadcast function for standard ParaData types

Returns
always 0 (for future extensions)
Parameters
bufferpoint to the head of sending message
countthe number of data in the message
datatypeIddata type in the message
rootroot rank for broadcasting

Implements ParaComm.

Definition at line 338 of file paraCommMpi.cpp.

References ParaCommMpi::datatypes, MPI_CALL, and ParaCommMpi::myComm.

Referenced by ParaCommMpi::createParaTimer(), and main().

◆ createParaTimer()

◆ getMpiComm()

MPI_Comm& getMpiComm ( )

getter of MPI_Comm

Definition at line 185 of file paraCommMpi.h.

References ParaCommMpi::init(), and ParaCommMpi::myComm.

◆ getNumOfMessagesWaitingToSend()

int getNumOfMessagesWaitingToSend ( int  dest = -1)
virtual

get size of the messageQueueTable

Returns
the size of the messageQueueTable

Implements ParaComm.

Definition at line 233 of file paraCommMpi.h.

References ParaCommMpi::abort(), ParaCommMpi::lcInit(), paraParamSet, and ParaCommMpi::solverInit().

◆ getRank()

int getRank ( )
virtual

get rank of caller's thread

Returns
rank of caller's thread

Implements ParaComm.

Definition at line 213 of file paraCommMpi.h.

References ParaCommMpi::myRank.

Referenced by main(), and ParaCommMpi::passTermToken().

◆ getSize()

int getSize ( )
virtual

get size of this communicator, which indicates how many threads in a UG process

Returns
the number of threads

Implements ParaComm.

Definition at line 223 of file paraCommMpi.h.

References ParaCommMpi::myCommSize.

Referenced by main().

◆ getStartTime()

double getStartTime ( )

get start time of this communicator

Returns
start time

Definition at line 203 of file paraCommMpi.h.

References ParaCommMpi::startTime.

◆ getTagString()

const char * getTagString ( int  tag)
virtual

get Tag string for debugging

Returns
string which shows Tag
Parameters
tagtag to be converted to string

Implements ParaComm.

Reimplemented in BbParaCommMpi, and ScipParaCommMpi.

Definition at line 329 of file paraCommMpi.cpp.

References ParaCommMpi::tagStringTable.

Referenced by ParaCommMpi::createParaTimer(), and BbParaCommMpi::getTagString().

◆ init()

void init ( int  argc,
char **  argv 
)
virtual

initializer of this communicator

MPI call wrappers */.

if you add tag, you should add tagStringTale too */

Data Types */

Parameters
argcthe number of arguments
argvpointers to the arguments

Implements ParaComm.

Reimplemented in ScipParaCommMpi.

Definition at line 235 of file paraCommMpi.cpp.

References ParaCommMpi::datatypes, MPI_CALL, ParaCommMpi::namelen, UG::ParaBOOL, UG::ParaBYTE, UG::ParaCHAR, UG::ParaDOUBLE, UG::ParaFLOAT, UG::ParaINT, UG::ParaLONG, UG::ParaLONG_DOUBLE, UG::ParaLONG_LONG, UG::ParaSHORT, UG::ParaSIGNED_CHAR, UG::ParaUNSIGNED, UG::ParaUNSIGNED_CHAR, UG::ParaUNSIGNED_LONG, UG::ParaUNSIGNED_LONG_LONG, UG::ParaUNSIGNED_SHORT, ParaCommMpi::procName, ParaCommMpi::startTime, and ParaCommMpi::tagStringTableIsSetUpCoorectly().

Referenced by ParaCommMpi::getMpiComm(), and ScipParaCommMpi::init().

◆ iProbe()

bool iProbe ( int *  source,
int *  tag 
)
virtual

iProbe function which checks if a new message is arrived or not

Returns
true when a new message exists
Parameters
sourcesource rank of the message arrived
tagtag of the message arrived

Implements ParaComm.

Definition at line 453 of file paraCommMpi.cpp.

References MPI_CALL, ParaCommMpi::myComm, TAG_TRACE, and UG::TagAny.

Referenced by ParaCommMpi::createParaTimer().

◆ iSend()

int iSend ( void *  bufer,
int  count,
const int  datatypeId,
int  dest,
const int  tag,
MPI_Request *  req 
)

send function for standard ParaData types

Returns
always 0 (for future extensions)
Parameters
buferpoint to the head of sending message
countthe number of data in the message
datatypeIddata type in the message
destdestination to send the message
tagtag of this message
reqpoint to MPI_Request

Definition at line 368 of file paraCommMpi.cpp.

References ParaCommMpi::datatypes, MPI_CALL, ParaCommMpi::myComm, and TAG_TRACE.

Referenced by ParaCommMpi::createParaTimer(), and ParaCommMpi::iUsend().

◆ iUsend()

int iUsend ( void *  bufer,
int  count,
MPI_Datatype  datatype,
int  dest,
int  tag,
MPI_Request *  req 
)

User type send for created data type.

Returns
always 0 (for future extensions)
Parameters
buferpoint to the head of sending message
countthe number of created data type
datatypecreated data type
destdestination rank
tagtag of the message
reqpoint to MPI_Request

Definition at line 514 of file paraCommMpi.cpp.

References ParaCommMpi::iSend(), MPI_CALL, ParaCommMpi::myComm, and TAG_TRACE.

Referenced by ParaCommMpi::createParaTimer().

◆ lcInit()

◆ lockApp()

virtual void lockApp ( )
virtual

lock UG application to synchronize with other threads

Implements ParaComm.

Definition at line 320 of file paraCommMpi.h.

References ParaComm::unlockApp().

◆ passTermToken()

bool passTermToken ( int  rank)
virtual

pass termination token from the rank to the next

Returns
true, when the termination token is passed from this rank, false otherwise

Reimplemented from ParaComm.

Definition at line 197 of file paraCommMpi.cpp.

References ParaCommMpi::getRank(), ParaCommMpi::myCommSize, ParaCommMpi::myRank, UG::ParaINT, ParaCommMpi::send(), UG::TagToken, THROW_LOGICAL_ERROR4, ParaCommMpi::token, and ParaCommMpi::tokenAccessLock.

Referenced by ParaCommMpi::waitTerminatedMessage().

◆ passToken()

void passToken ( int  rank)
virtual

pass token to from the rank to the next

Parameters
rankfrom this rank, the token is passed

Reimplemented from ParaComm.

Definition at line 178 of file paraCommMpi.cpp.

References ParaCommMpi::myCommSize, ParaCommMpi::myRank, UG::ParaINT, ParaCommMpi::send(), UG::TagToken, ParaCommMpi::token, and ParaCommMpi::tokenAccessLock.

Referenced by ParaCommMpi::waitTerminatedMessage().

◆ probe()

bool probe ( int *  source,
int *  tag 
)
virtual

probe function which waits a new message

Returns
always true
Parameters
sourcesource rank of the message arrived
tagtag of the message arrived

Implements ParaComm.

Definition at line 437 of file paraCommMpi.cpp.

References MPI_CALL, ParaCommMpi::myComm, and TAG_TRACE.

Referenced by ParaCommMpi::createParaTimer(), and main().

◆ receive()

int receive ( void *  bufer,
int  count,
const int  datatypeId,
int  source,
const int  tag 
)
virtual

receive function for standard ParaData types

Returns
always 0 (for future extensions)
Parameters
buferpoint to the head of receiving message
countthe number of data in the message
datatypeIddata type in the message
sourcesource of the message coming from
tagtag of the message

Implements ParaComm.

Definition at line 385 of file paraCommMpi.cpp.

References ParaCommMpi::datatypes, MPI_CALL, ParaCommMpi::myComm, and TAG_TRACE.

Referenced by ParaCommMpi::createParaTimer(), main(), and ParaCommMpi::waitToken().

◆ send()

int send ( void *  bufer,
int  count,
const int  datatypeId,
int  dest,
const int  tag 
)
virtual

send function for standard ParaData types

Returns
always 0 (for future extensions)
Parameters
buferpoint to the head of sending message
countthe number of data in the message
datatypeIddata type in the message
destdestination to send the message
tagtag of this message

Implements ParaComm.

Definition at line 352 of file paraCommMpi.cpp.

References ParaCommMpi::datatypes, MPI_CALL, ParaCommMpi::myComm, and TAG_TRACE.

Referenced by ParaCommMpi::createParaTimer(), main(), ParaCommMpi::passTermToken(), and ParaCommMpi::passToken().

◆ setToken()

virtual void setToken ( int  rank,
int *  inToken 
)
virtual

set received token to this communicator

Parameters
rankrank to set the token
inTokentoken to be set

Reimplemented from ParaComm.

Definition at line 297 of file paraCommMpi.h.

References ParaComm::lockApp().

◆ solverInit()

void solverInit ( ParaParamSet paraParamSet)
virtual

◆ tagStringTableIsSetUpCoorectly()

bool tagStringTableIsSetUpCoorectly ( )
protectedvirtual

check if tag string table (for debugging) set up correctly

Returns
true if tag string table is set up correctly, false otherwise

Reimplemented in BbParaCommMpi, and ScipParaCommMpi.

Definition at line 322 of file paraCommMpi.cpp.

References ParaCommMpi::tagStringTable.

Referenced by ParaCommMpi::init().

◆ testAllIsends()

int testAllIsends ( )

◆ ubcast()

int ubcast ( void *  buffer,
int  count,
MPI_Datatype  datatype,
int  root 
)

User type bcast for created data type.

Returns
always 0 (for future extensions)
Parameters
bufferpoint to the head of sending message
countthe number of created data type
datatypeMPI data type
rootroot rank for brodcasting

Definition at line 483 of file paraCommMpi.cpp.

References MPI_CALL, and ParaCommMpi::myComm.

Referenced by ParaCommMpi::createParaTimer().

◆ unlockApp()

virtual void unlockApp ( )
virtual

unlock UG application to synchronize with other threads

Implements ParaComm.

Definition at line 330 of file paraCommMpi.h.

◆ ureceive()

int ureceive ( void *  bufer,
int  count,
MPI_Datatype  datatype,
int  source,
int  tag 
)

User type receive for created data type.

Returns
always 0 (for future extensions)
Parameters
buferpoint to the head of receiving message
countthe number of created data type
datatypecreated data type
sourcesource rank
tagtag of the message

Definition at line 532 of file paraCommMpi.cpp.

References MPI_CALL, ParaCommMpi::myComm, and TAG_TRACE.

Referenced by ParaCommMpi::createParaTimer().

◆ usend()

int usend ( void *  bufer,
int  count,
MPI_Datatype  datatype,
int  dest,
int  tag 
)

User type send for created data type.

Returns
always 0 (for future extensions)
Parameters
buferpoint to the head of sending message
countthe number of created data type
datatypecreated data type
destdestination rank
tagtag of the message

Definition at line 497 of file paraCommMpi.cpp.

References MPI_CALL, ParaCommMpi::myComm, and TAG_TRACE.

Referenced by ParaCommMpi::createParaTimer().

◆ waitAllIsends()

void waitAllIsends ( )

◆ waitSpecTagFromSpecSource()

void waitSpecTagFromSpecSource ( const int  source,
const int  tag,
int *  receivedTag 
)
virtual

wait function for a specific tag from a specific source coming from

Returns
always 0 (for future extensions)
Parameters
sourcesource rank which the message should come from
tagtag which the message should wait
receivedTagtag of the message which is arrived

Implements ParaComm.

Definition at line 402 of file paraCommMpi.cpp.

References MPI_CALL, ParaCommMpi::myComm, TAG_TRACE, and UG::TagAny.

Referenced by ParaCommMpi::createParaTimer().

◆ waitTerminatedMessage()

virtual bool waitTerminatedMessage ( )
virtual

function to wait Terminated message (This function is not used currently)

Returns
true when MPI communication is used, false when thread communication used

Implements ParaComm.

Definition at line 265 of file paraCommMpi.h.

References ParaCommMpi::passTermToken(), ParaCommMpi::passToken(), and ParaCommMpi::waitToken().

◆ waitToken()

bool waitToken ( int  rank)
virtual

wait token when UG runs with deterministic mode

Returns
true, when token is arrived to the rank
Parameters
rankrank to check if token is arrived

Reimplemented from ParaComm.

Definition at line 125 of file paraCommMpi.cpp.

References MPI_CALL, ParaCommMpi::myComm, ParaCommMpi::myCommSize, ParaCommMpi::myRank, UG::ParaINT, ParaCommMpi::receive(), TAG_TRACE, UG::TagToken, ParaCommMpi::token, and ParaCommMpi::tokenAccessLock.

Referenced by ParaCommMpi::waitTerminatedMessage().

Member Data Documentation

◆ applicationLockMutex

std::mutex applicationLockMutex
protected

mutex for applications

Definition at line 118 of file paraCommMpi.h.

◆ datatypes

MPI_Datatype datatypes
staticprotected

data type mapping table to MPI data type

Definition at line 110 of file paraCommMpi.h.

Referenced by ParaCommMpi::bcast(), ParaCommMpi::init(), ParaCommMpi::iSend(), ParaCommMpi::receive(), and ParaCommMpi::send().

◆ iSendRequestDeque

std::deque<ParaIsendRequest *> iSendRequestDeque

Definition at line 133 of file paraCommMpi.h.

Referenced by ParaCommMpi::testAllIsends(), and ParaCommMpi::waitAllIsends().

◆ myComm

◆ myCommSize

int myCommSize
protected

communicator size : number of processes joined in this system

Definition at line 97 of file paraCommMpi.h.

Referenced by ParaCommMpi::getSize(), ScipParaCommMpi::init(), ParaCommMpi::passTermToken(), ParaCommMpi::passToken(), and ParaCommMpi::waitToken().

◆ myRank

◆ namelen

int namelen
protected

length of this process name

Definition at line 99 of file paraCommMpi.h.

Referenced by ParaCommMpi::init().

◆ ofs

std::ofstream ofs
protected

output file stream for tag trace

Definition at line 102 of file paraCommMpi.h.

Referenced by ParaCommMpi::lcInit(), and ParaCommMpi::solverInit().

◆ procName

char procName[MPI_MAX_PROCESSOR_NAME]
protected

process name

Definition at line 100 of file paraCommMpi.h.

Referenced by ParaCommMpi::init().

◆ startTime

double startTime
protected

start time of this communicator

Definition at line 104 of file paraCommMpi.h.

Referenced by ParaCommMpi::getStartTime(), and ParaCommMpi::init().

◆ tagStringTable

◆ tagTraceFlag

bool tagTraceFlag
protected

indicate if tags are traced or not

Definition at line 101 of file paraCommMpi.h.

Referenced by ParaCommMpi::lcInit(), and ParaCommMpi::solverInit().

◆ token

int token[2]
protected

index 0: token index 1: token color -1: green > 0: yellow ( termination origin solver number ) -2: red ( means the solver can terminate )

Definition at line 105 of file paraCommMpi.h.

Referenced by BbParaCommMpi::BbParaCommMpi(), ParaCommMpi::lcInit(), ParaCommMpi::passTermToken(), ParaCommMpi::passToken(), and ParaCommMpi::waitToken().

◆ tokenAccessLock

pthread_mutex_t tokenAccessLock
protected

mutex for pthread thread

Definition at line 116 of file paraCommMpi.h.

Referenced by ParaCommMpi::passTermToken(), ParaCommMpi::passToken(), and ParaCommMpi::waitToken().

◆ tos

std::ostream* tos
protected

output file stream for tag trace to change file name

Definition at line 103 of file paraCommMpi.h.

Referenced by ParaCommMpi::lcInit(), and ParaCommMpi::solverInit().