1. Cluster Daemon (CD)
    1. Overview

      2. The cluster daemon controls communication to the cluster manager from the Faucets system. It resides on the same node as the cluster manager and uses the scheduler commands listed in the scheduler section. The cluster daemon communicates with the central server and command client with single line XML. On start up, the cluster daemon gathers cluster information to make an initial guess at the work rate compared to the standard cluster and reads a file that specifies applications that the CD will support. The work rate is used to change bids to make accurate runtime and other values for different cluster configurations.

    2. Components

      3. There are eight main components of the cluster daemon.

      1. Application Properties file

        2. The Application Properties file is an XML file that includes the information on the applications that the Cluster Daemon supports. Add applications in the file appList.xml in the following manner:

        <Application >
        <name> hello </name>
        <appspectorServerClass> null </appspectorServerClass>
        <type> charm </type>
        <path> /home/net/jbmeyer/charm/hello/1darray/hello </path>
        <speedUpFunction> 1+((numNodes-1)*.75) </speedUpFunction>
        </Application>

        The name is the name of the function that will be presented to users of Faucets to represent the application. AppspectorServerClass represents the name of the class that corresponds to the application. If no appspector class can be used with the application, write null. The type of the application can be charm, for non-migratable charm jobs, mcharm, for migratable charm jobs, AMPI, for asynchronous MPI jobs, and MPI for standard MPI jobs. The path is the location of the executable file. The speed-up function is a function that defines the speed-up of a job with a given number of nodes. Most arithmetic operators can be used in the function, but because of how the scheduler takes arguments, white space must be removed. For a detailed list of operators, read the fparser3 documentation at http://www.students.tut.fi/~warp/FunctionParser/fparser.txt.

      2. Application Properties

        3. he ApplicationProperties class parses the application properties file and generates a data structure that is used by the Server Thread.

      3. Cluster Daemon Info

        4. The cdInfo class generates the important physical characteristics and pricing characteristics of the cluster in the file cdInfo.xml. This file is used to register with the central server and by the bid generator to generate a work rate value. The default maximum and minimum price per work are 100.0 and 10.0 Barter Units respectively. These values can be changed in the cdInfo.xml file.

      4. Cluster Daemon

        5. The ClusterDaemon class is the start up class for the CD. It opens the Cluster Daemon Property file and creates a SSL server socket factory for communication. The class creates a new application properties class with the application property file in the faucets directory. Then, ClusterDaemon registers its information with the central server. When the central server receives this message, it assumes that all jobs listed as running on the cluster failed. The central server changes the job's status to failed and updates the prices accordingly. Finally the ClusterDaemon class starts a server thread to handle user requests.

      5. Bid Generator

        6. The BidGenerator class is used to generate bid pricing and alter quality of service contracts. Different clusters will have different amounts of computational power. A node's speed, memory size, processors, and cache size will effect the running time of a job. When a user bids an job, the runtime value is used to determine a price. It is not fair for faster machines and users to bid for the same runtime value as slower machines. Thus, the bid generator changes bid runtimes and potentially other variables to make bidding more accurate.

        There are two methods to create a bid generator. The first is to provide the work rate, minimum price per work, and maximum price per work directly to the BidGenerator. The second method determines the work rate of the cluster from the cdInfo file. The method is crude and will be the subject of future research. Currently, it compares the cluster to the standard cluster node, and calculates a work rate. To change the relative importance of the cluster characteristics, change the property multiplier. Again, this is a rough estimate and users submitting jobs are encouraged to overestimate the job's runtime value.

      6. Cluster Daemon Properties

        7. The cd.properties file contains the SSL certificate information to generate sockets to communicate with the Central Server.

      7. Monitor Thread

        8. The MonitorThread class monitors queued and running jobs for changes in status. It communicates the results to the Central Server.

      8. Server Thead

        9. The ServerThread class handles most communication of the command client, and GUI with the Cluster daemon. Requests and responses are given in single line XML.

        1. Server Thread communication requests

          2. query - Query tests if the scheduler can accept the job given the quality of service requirements presented in the XML input.
          job - Job submits a job to the scheduler.
          get-status - Get status returns the status of a specified job.
          get-file - uploads a specific output file from a job.
          list-files - lists file from the application.
          kill - kills the specified application. This will still charge a user for the bid price.
          get-appspector-url - returns the appspector url address of a running job.
          list-apps - returns the supported applications.

    3. Installation and Use
      1. Installation

        2. To compile the Cluster Daemon type make in the faucets directory. Next import and create certificates as detailed in the following certificates section.

      2. Certificates

        3. Generate or request a private or public RSA keypair. Send the public key to the Faucet Certificate Authority to be signed. Put the signed certificate in your keystore, and put the Faucet Certificate Authority's certificate in your truststore. The Faucet Certificate Authority is discussed in the Central Server chapter (section 3.3).

      3. Use

        4. To run the Cluster Daemon type make runCD in the faucets directory.