TeamTime Ratings

Ratings can be used to evaluate alternatives with respect to a covering objective.  (A covering objective is an objective that has no sub-objectives.)

Participants enter Ratings in TeamTime by using either keypads (see Select  Participants) or by logging in to the Comparion TeamTime Session using an invitation sent by the Project Manager (see Invite  Participants).  

The screen as seen by the Project Manager is shown below.  Participants who log in to the TeamTime session see a very similar screen with their name at the top of the list of participants.

Legend:

1)  Name of the alternative being rated.

Ratings are made by the participants by selecting the rating from the pull-down menu, or by using their keypad.

2)   Name of the covering objective (or the complete objective path if specified in the project settings) for which this alternative is being rated.

3)  Information document for the alternative being rated.  The pencil icons allow the Project Manager to edit or create an information document.

You can show the information document content in a frame or a tool-tip by clicking the information document icon .

4)   Information document for the covering objective for which this alternative is being rated.  The pencil icons  allow the Project Manager to edit or create an information document.

5)  Information document for the alternative being evaluated with respect to the covering objective for which this alternative is being rated.  The pencil icons  allow the Project Manager to edit or create an information document.

6) Names and ratings for participants.  Rating is made by the participants by selecting an intensity from the drop-down menu or clicking the intensities in the legend (7).

The names are shown formatted as follows:

  • Bold – a change made; appears for only a few seconds
  • Blue  – participant is online. 
  • Normal (gray) – offline
  • Red – no roles for making judgments on the current step

If the participant is in view-only mode, you can see a (View only) note right beside the participant.

When there are many participants, the list will be paginated.  By default, the "Smart" option is selected which auto-detects the number of participants to display that will best fit the screen and make the loading faster.  You can select users per page by clicking on the links at the right.   You can then navigate through the pages using the page numbers at the left. 

You can enter comments by clicking

Ratings are entered by participants by selecting an intensity from the drop-down menu or clicking the intensity in the legend (7).

7)  Legend for the rating intensities and the corresponding ratio scale priorities and keypad number.  Participants can enter judgments from the legend by clicking the desired intensity.

If the rating intensity descriptions are defined, they will be displayed below the intensity name.

Information documents for each of the elements being compared as well as information documents for each element being compared as they relate to the parent.  Information documents can be collapsed or expanded. 

8)  The arithmetic mean of the rating priorities. 

9) The arithmetic variance of the rating priorities. 

10)  A Navigation Box at the bottom of the screen contains a number for each step in the evaluation sequence.  

a. Steps - The number of steps is NOT the same as the number of evaluations because (1) there may be additional pages showing results; and (2) some pages may have multiple evaluations. 

You can click the "?" icon to see what the step colors pertain to: 

b. Click the "Current Cluster"  icon to open the Objectives hierarchy, where you can select the parent to jump to the first step "with respect to" the selected node.  

c. Jump to a step by clicking on the ellipses , and then entering the step number.  

d. Click the steps list  and select a step. 

e. Click  to show the participants' evaluation progress.

By default, the Project  Manager's evaluation progress is displayed in the navigation box.  The Project Manager can select another user to display by selecting the radio button beside the user's name. 

The "Next Unassessed" button will also depend on the selected participant. 

When a user other than the Project Manager is selected, a person icon will appear; hovering on it will show the user's name/email that the evaluation progress data was based on.

11)  Check boxes are provided for an anonymous mode so that participants' names are replaced by nominal names and a mode to Hide Judgments.  A  typical and recommended process consists of hiding judgments when entering a step and then showing the judgments for discussion once they have been made.  The time since the last judgment is indicated in (19) as discussed below.

12)  Switch to Distributions of Judgments View.  Click to display the number of participants given a selected intensity.  

Clicking the judgment count will display the users who rated the intensity.

Click to return to evaluation view. 

13)  TeamTime Evaluation option settings:

a) The polling time interval for sending judgments to the server is set to 2 seconds.  This can be increased if desired by clicking on the + sign.

b) Select the sorting of the evaluator list (by name, email or keypad number (if any)).

c) Option to show the variance when judgments are hidden.

d) Option to show the pairwise side when using Keypads and when judgments are hidden.

e) Option to hide the combined results in the results page.

f) Option to hide the participants in "View Only" mode.

g) Option to hide the project manager in the list.

14) The project manager can pause collecting judgments by clicking on the pause icon .

15)  The TeamTime meeting can be ended by clicking on the end meeting .

16)  The project manager can make changes to the project during the TeamTime session, including the objectives, alternatives, information documents, participants' roles, etc.  The reload project icon can be used to reload the updated project into the TeamTime session.  Normally the changes are automatically detected; there will be a prompt that will inform the Project Manager of changes in the evaluation.  Clicking OK will reload the page.

17) Current status of communications with server and time since the last judgment. 

18) Jump to the previous or next or next un-assessed step.