FEM Framework Manual
| Previous: 8 IDXL Communication | Up: CHARM++ Finite Element Method | Next: 10 ParFUM |
| Previous: 8 IDXL Communication | Up: CHARM++ Finite Element Method | Next: 10 ParFUM |
(This section is for backward compatability only. The IDXL routines are the new, more flexible way to perform communication.)
The FEM framework handles the updating of the values of shared nodes- that is, it combines shared nodes' values across all processors. The basic mechanism to do this update is the ``field''- numeric data items associated with each node. We make no assumptions about the meaning of the node data, allow various data types, and allow a mix of communicated and non-communicated data associated with each node. The framework uses IDXL layouts to find the data items associated with each node in memory.
Each field represents a (set of) data records stored in a contiguous array, often indexed by node number. You create a field once, with the IDXL layout routines or FEM_Create_field, then pass the resulting field ID to FEM_Update_field (which does the shared node communication), FEM_Reduce_field (which applies a reduction over node values), or one of the other routines described below.
void FEM_Update_field(int Fid,void *nodes);
SUBROUTINE FEM_Update_field(Fid,nodes)
INTEGER, INTENT(IN) :: Fid
varies, INTENT(INOUT) :: nodes
Combine a field of all shared nodes with the other chunks. Sums the value of the given field across all chunks that share each node. For the example above, once each chunk has computed the net force on each local node, this routine will sum the net force across all shared nodes.
FEM_Update_field can only be called from driver, and to be useful, must be called from every chunk's driver routine.
After this routine returns, the given field of each shared node will be the same across all processors that share the node.
This routine is eqivalent to an IDXL_Comm_Sendsum operation.
void FEM_Read_field(int Fid,void *nodes,char *fName);
SUBROUTINE FEM_Read_field(Fid,nodes,fName)
INTEGER, INTENT(IN) :: Fid
varies, INTENT(OUT) :: nodes
CHARACTER*, INTENT(IN) :: fName
Read a field out of the given serial input file. The serial input file is line-oriented ASCII- each line begins with the global node number (which must match the line order in the file), followed by the data to be read into the node field. The remainder of each line is unread. If called from Fortran, the first line must be numbered 1; if called from C, the first line must be numbered zero. All fields are separated by white space (any number of tabs or spaces).
For example, if we have called Create_field to describe 3 doubles, the input file could begin with
FEM_Read_field must be called from driver at any time, independent of other chunks.
This routine has no IDXL equivalent.
void FEM_Reduce_field(int Fid,const void *nodes,void *out,int op);
SUBROUTINE FEM_Reduce_field(Fid,nodes,outVal,op)
INTEGER, INTENT(IN) :: Fid,op
varies, INTENT(IN) :: nodes
varies, INTENT(OUT) :: outVal
Combine one record per node of this field, according to op, across all chunks. Shared nodes are not double-counted- only one copy will contribute to the reduction. After Reduce_field returns, all chunks will have identical values in outVal, which must be vec_len copies of base_type.
May only be called from driver, and to complete, must be called from every chunk's driver routine.
This routine has no IDXL equivalent.
void FEM_Reduce(int Fid,const void *inVal,void *outVal,int op);
SUBROUTINE FEM_Reduce(Fid,inVal,outVal,op)
INTEGER, INTENT(IN) :: Fid,op
varies, INTENT(IN) :: inVal
varies, INTENT(OUT) :: outVal
Combine one record of this field from each chunk, acoording to op, across all chunks. Fid is only used for the base_type and vec_len- offset and dist are not used. After this call returns, all chunks will have identical values in outVal. Op has the same values and meaning as FEM_Reduce_field.
May only be called from driver, and to complete, must be called from every chunk's driver routine.
This routine has no IDXL equivalent.
It is possible to get values for a chunk's ghost nodes and elements from the neighbors. To do this, use:
void FEM_Update_ghost_field(int Fid, int elTypeOrMinusOne, void *data);
SUBROUTINE FEM_Update_ghost_field(Fid,elTypeOrZero,data)
INTEGER, INTENT(IN) :: Fid,elTypeOrZero
varies, INTENT(INOUT) :: data
This has the same requirements and call sequence as FEM_Update_field, except it applies to ghosts. You specify which type of element to exchange using the elType parameter. Specify -1 (C version) or 0 (fortran version) to exchange node values.
It is possible to exchange sparse lists of ghost elements between FEM chunks.
void FEM_Exchange_ghost_lists(int elemType,int nIdx,const int *localIdx);
SUBROUTINE FEM_Exchange_ghost_lists(elemType,nIdx,localIdx)
INTEGER, INTENT(IN) :: elemType,nIdx
INTEGER, INTENT(IN) :: localIdx[nIdx]
This routine sends the local element indices in localIdx to those neighboring chunks that connect to its ghost elements on the other side. That is, if the element localIdx[i] has a ghost on some chunk c, localIdx[i] will be sent to and show up in the ghost list of chunk c.
int FEM_Get_ghost_list_length(void);
Returns the number of entries in my ghost list--the number of my ghosts that
other chunks passed to their call to FEM_Exchange_ghost_lists.
void FEM_Get_ghost_list(int *retLocalIdx);
SUBROUTINE FEM_Get_ghost_list(retLocalIdx)
INTEGER, INTENT(OUT) :: retLocalIdx[FEM_Get_ghost_list_length()]
These routines access the list of local elements sent by other chunks. The returned indices will all refer to ghost elements in my chunk.
November 23, 2009
FEM Homepage
Charm Homepage