Class that manages the paircalc sections that each Ortho chare communicates with. More...

#include <pcSectionManager.h>

Public Member Functions
	PCSectionManager ()
	Constructor.

void	init (const CkIndex2D orthoIdx, const pc::pcConfig &pcCfg, CkArrayID pcAID, CkGroupID oMCastGID, CkGroupID oRedGID)
	An initializer method that fills this with data. More...

void	pup (PUP::er &p)
	PUP serializer.

void	setupArraySection (CkCallback cb, bool arePhantomsOn, bool useComlibForOrthoToPC)
	Creates a paircalc array section given the necessary info. Replaces initOneRedSect() More...

void	sendResults (int n, internalType ptr1, internalType ptr2, int orthoX, int orthoY, int actionType, int priority)
	Sends out the results to the paircalc section. Replaces finishPairCalcSection() More...

void	sendMatrix (int n, internalType ptr1, internalType ptr2, int orthoX, int orthoY, int actionType, int priority)
	Used to send OrthoT to the asymm instance. Replaces sendMatrix() More...

CkIndex2D	computePCStateIndices (const int orthoX, const int orthoY)
	Identify the state indices of the Paircalc chares this ortho chare needs to talk to. More...

CkIndex2D	computePCStateIndices ()
	Overloaded version that uses the stored ortho indices to compute the PC state indices.

Private Member Functions
void	createPCsection (const int s1, const int s2)
	Create a paircalc section containing all chares with the specified two state indices. More...

Private Attributes
int	numPlanes
	Number of planes that GSpace is decomposed into.

int	numStates
	Number of states in this simulation.

int	numChunks
	The number of chunks (4th dimension decomposition) of paircalcs.

int	pcGrainSize
	The statewise decomposition grain size for the paircalcs.

int	orthoGrainSize
	The statewise decomposition grain size for the ortho chares.

CkArrayID	pcArrayID
	The array ID of the paircalc instance that I will manage comm with.

bool	isSymmetric
	Is this paircalc array a symmetric or asymmetric instance.

CProxySection_PairCalculator	pcSection
	The section of the array that my owner ortho chare will be talking to.

CkIndex2D	orthoIndex
	The index of the calling Ortho chare.

CkGroupID	orthomCastGrpID
	The multicast and reduction groups that handle comm.

CkGroupID	orthoRedGrpID

int	msgPriority
	The priority to use for messages to PC.

Friends
class	::Ortho

Detailed Description

Class that manages the paircalc sections that each Ortho chare communicates with.

Definition at line 17 of file pcSectionManager.h.

Member Function Documentation

CkIndex2D cp::ortho::PCSectionManager::computePCStateIndices	(	const int	orthoX,
		const int	orthoY
	)

inline

Identify the state indices of the Paircalc chares this ortho chare needs to talk to.

ortho and paircalc grainsizes do not complicate this discussion a whole lot because of the restriction that ortho grainsize = multiple of paircalc grain size.

Because of this equal or exact multiple clause, ortho grains will line up perfectly inside a paircalc grain and, hence, every ortho chare will hold a bunch of states that will all get delivered to the same paircalc section.

paircalcs on the other hand will have to chop up their data along the ortho tile boundaries and contribute to multiple reductions that end up at the respective ortho chares. Refer PairCalculator::contributeSubTiles.

Definition at line 81 of file pcSectionManager.h.

References numStates, orthoGrainSize, and pcGrainSize.

Referenced by Ortho::resume(), and Ortho::start_calc().

void cp::ortho::PCSectionManager::createPCsection	(	const int	s1,
		const int	s2
	)

private

Create a paircalc section containing all chares with the specified two state indices.

Ortho chares are a 2D (nstates x nstates) array.

ortho[sx,sy] talks to all the paircalc chares which handle the ordered pair of states (sx,sy). This will be pc[p,s1,s2,c] where p ranges across all planes and c across all chunks.

Create and save this paircalc section

Definition at line 67 of file pcSectionManager.C.

References numChunks, numPlanes, numStates, orthoGrainSize, orthoIndex, pcArrayID, pcGrainSize, and pcSection.

Referenced by setupArraySection().

void cp::ortho::PCSectionManager::init	(	const CkIndex2D	orthoIdx,
		const pc::pcConfig &	pcCfg,
		CkArrayID	pcAID,
		CkGroupID	oMCastGID,
		CkGroupID	oRedGID
	)

An initializer method that fills this with data.

The section manager now finds most of its init data from the global config class.

If we need support for differently configured PC instances, we should make the section managers init themselves from an instance config object and not a global config object. But first, we need to implement the concept of a config class for an instance :)

Definition at line 42 of file pcSectionManager.C.

References cp::paircalc::pcConfig::grainSize, cp::paircalc::pcConfig::inputMsgPriority, cp::paircalc::pcConfig::isSymmetric, isSymmetric, msgPriority, cp::paircalc::pcConfig::numChunks, numChunks, cp::paircalc::pcConfig::numPlanes, numPlanes, cp::paircalc::pcConfig::numStates, numStates, orthoGrainSize, cp::paircalc::pcConfig::orthoGrainSize, orthoIndex, orthomCastGrpID, pcArrayID, and pcGrainSize.

Referenced by Ortho::makeSections().

void cp::ortho::PCSectionManager::sendMatrix	(	int	n,
		internalType *	ptr1,
		internalType *	ptr2,
		int	orthoX,
		int	orthoY,
		int	actionType,
		int	priority
	)

Used to send OrthoT to the asymm instance. Replaces sendMatrix()

Allocate a msg of the right size

Fill it with results

Trigger the backward path for my paircalc section

Definition at line 229 of file pcSectionManager.C.

References pcSection.

Referenced by Ortho::sendOrthoTtoAsymm().

void cp::ortho::PCSectionManager::sendResults	(	int	n,
		internalType *	ptr1,
		internalType *	ptr2,
		int	orthoX,
		int	orthoY,
		int	actionType,
		int	priority
	)

Sends out the results to the paircalc section. Replaces finishPairCalcSection()

Allocate a msg of the right size

Fill it with results

Trigger the backward path for my paircalc section

Definition at line 193 of file pcSectionManager.C.

References pcSection.

Referenced by Ortho::acceptSectionLambda(), Ortho::gamma_done(), and Ortho::resume().

void cp::ortho::PCSectionManager::setupArraySection	(	CkCallback	cb,
		bool	arePhantomsOn,
		bool	useComlibForOrthoToPC
	)

Creates a paircalc array section given the necessary info. Replaces initOneRedSect()

Initialize the planewise section reduction for Ortho sums across all planes and chunks pass through the the owning Ortho chare so the cookie can be placed in the 2d array (grainSize/orthoGrainSize)^2.

Ortho chares talk to paircalc chares based on their state indices. That is, results for an ordered pair of states (s1,s2) from an ortho will end up at the paircalc(s) responsible for that state pair. As paircalcs are decomposed along two other dimensions (planes & points), this results in a section of paircalcs which need to get data for any given state pair (s1,s2) via a multicast from the ortho responsible for that state pair (s1,s2).

The same logic holds for input data from the paircalcs to ortho. As orthos are 2D and oblivious of any plane-wise or point-wise decomposition, all data pertaining to a state pair (s1,s2) from all the paircalcs mut be collated and delivered to the ortho that handles (s1,s2). This results in a reduction across a paircalc section that spans all planes and chunks of the paircalc array.

An extra twist in these straightforward section rules happens because of the presence of phantom chares in symmetric paircalc instance. Phantom chares do not participate in the forward path and hence do not send any input data. Hence they should not be included in the reductions to ortho. However, they work in the backward path off the results from ortho and are hence included in the multicasts back to the paircalcs.

We compound this a bit further in our greed to avoid extra work when possible. It should be noted that because of the limitations of the underlying matrix multiply libraries, we canNOT perform a triangular multiply on what is essentially symmetric input. Hence orthos orchestrate a square matrix multiply and end up with almost identical results (differing only by a transpose) in mirror chares across the array diagonal. Without going into the underlying math, we can say that the orthos that talk to phantom or on-diagonal pc chares end up with the exact data required, whereas the ortho chares that need to talk to the non-phantom sections, have to perform an additional transpose before they can pack the data off to their pc sections. This is where we get lazy and try to avoid this extra transpose if we can. Basically, when phantoms are turned off we rig the section creation so that the orthos which should have spoken to the phantoms instead talk to their non-phantom mirror sections. As these orthos have the data in the correct form already, no one has to perform any extra transposes.

Hence orthos whose indices correspond to those of phantom paircalc chares, will talk to:

their original phantom section if the user turns on phantoms
a mirror non-phantom section if the user turns off phantoms

Find the states indices of the paircalcs this ortho should be talking to.

When phantoms are off, ortho chares that should talk to a phantom section will instead talk to a mirror section

Create the paircalc section that this ortho chare will actually talk to

Paircalcs end their forward path by sending data to the orthos. Irrespective of their type (symm/asymm) or whether phantoms are turned on or not, they always talk to the orthos corresponding to their state indices. All the mirrors and section switching tricks that happen above only apply to the multicast back from ortho to paircalc. Also, phantom paircalcs do not participate in the forward path and do not have any data to send. Hence, only orthos whose indices correspond to the non-phantoms will register with their pc sections to get data

Delegate the pc section –> ortho reduction to CkMulticast

Register this ortho chare with all the paircalcs in this section

Delegate the ortho –> pc section multicast to the appropriate library

Definition at line 134 of file pcSectionManager.C.

References computePCStateIndices(), createPCsection(), isSymmetric, numChunks, numPlanes, orthoIndex, orthomCastGrpID, and pcSection.

Referenced by Ortho::makeSections().

The documentation for this class was generated from the following files:

Public Member Functions

Private Member Functions

Private Attributes

Friends

Detailed Description

Member Function Documentation