5.1 Ghost Numbering

Ghosts and real entities are stored by the framework in separate lists--to access the ghost entity type, add FEM_GHOST to the real entity's type. For example, FEM_GHOST+FEM_ELEM+1 lists the ghost elements for elType 1. To get the number of ghost nodes, you would call FEM_Mesh_get_length(mesh,FEM_GHOST+FEM_NODE).

Figure 7: Node indices used in the element connectivity array. There are real nodes and ghosts.

For real elements, the element connectivity always consists of real nodes. But for ghost elements, the adjacent nodes may be missing, or may themselves be ghosts. Thus ghost element connectivity lists may include the invalid value -1 (in C) or 0 (in Fortran) to indicate that the corresponding node is not present; or may include values less than this, which indicate the corresponding node is a ghost. In C, ghost node is indicated by the value , while in Fortran, ghost node is indicated by the value . This node indexing system is illustrated in Figure 7, This indexing system is bizarre, but it allows us to keep the real and ghost nodes clearly separate, while still allowing real and ghost nodes to be added in increasing order at both ends.

Since the C tests are complicated, in C we recommend using these macros:

• FEM_Is_ghost_index(i) returns true if represents a ghost node. In Fortran, use the test .lt.

• FEM_From_ghost_index(i) returns the ghost node's index given its connectivity entry. In Fortran, use the expression .

• FEM_To_ghost_index(i) returns the connectivity entry for a given ghost node index. In Fortran, again use the expression .

For example, a quadrilateral ghost element that is adjacent to, respectively, two real nodes 23 and 17, the tenth local ghost node, and one not-present node might have a connectivity entry of 23,17,-11,-1 (in C) or 23,17,-10,0 (in Fortran).

Applications may wish to use some other numbering, such as by storing all the ghost nodes after all the real nodes. The code to extract and renumber the connectivity of some 3-node triangles stored in FEM_ELEM+2 would be:

/* C version */
  int nReal=FEM_Mesh_get_length(mesh,FEM_ELEM+2);
  int nGhost=FEM_Mesh_get_length(mesh,FEM_GHOST+FEM_ELEM+2);
  typedef int intTriplet[3];
  intTriplet *conn=new intTriplet[nReal+nGhost];
  /* Extract real triangles into conn[0..nReal-1] */
  FEM_Mesh_data(mesh,FEM_ELEM+2,FEM_CONN, &conn[0][0], 0,nReal, 3,FEM_INDEX_0);
  /* Extract ghost triangles into conn[nReal..nReal+nGhost-1] */
  FEM_Mesh_data(mesh,FEM_GHOST+FEM_ELEM+2,FEM_CONN, &conn[nReal][0], 0,nGhost, 3,FEM_INDEX_0);
  
  /* Renumber the ghost triangle connectivity */
  for (int t=nReal;t<nReal+nGhost;t++)
    for (int i=0;i<3;i++) {
      int in=conn[t][i]; /* uses FEM ghost node numbering */
      int out; /* uses application's ghost numbering */
      if (in==-1) {
        out=some_value_for_missing_nodes;
      } else if (FEM_Is_ghost_index(in)) {
        out=first_application_ghost+FEM_From_ghost_index(in);
      } else /*regular real node*/ {
        out=in;
      }
      conn[t][i]=out;
    }

! F90 version
  INTEGER, ALLOCATABLE :: conn(3,:)
  INTEGER :: nReal,nGhost,t,i,in,out
  nReal=FEM_Mesh_get_length(mesh,FEM_ELEM+2)
  nGhost=FEM_Mesh_get_length(mesh,FEM_GHOST+FEM_ELEM+2)
  ALLOCATE(conn(3,nReal+nGhost))
  ! Extract real triangles into conn[1..nReal]
  CALL FEM_Mesh_data(mesh,FEM_ELEM+2,FEM_CONN, conn, 1,nReal, 3,FEM_INDEX_1)
  ! Extract ghost triangles into conn[nReal+1..nReal+nGhost]
  CALL FEM_Mesh_data(mesh,FEM_GHOST+FEM_ELEM+2,FEM_CONN, conn(1,nReal+1), 1,nGhost, 3,FEM_INDEX_1)
  
  ! Renumber the ghost triangle connectivity
  DO t=nReal+1,nReal+nGhost
    DO i=1,3
      in=conn(i,t)
      IF (in .EQ. 0) out=some_value_for_missing_nodes
      IF (in .LT. 0) out=first_application_ghost-1+(-in)
      IF (in .GT. 0) out=in
      conn(i,t)=out
    END DO
  END DO