Retention Dashboard

The Retention Dashboard is an optional product where Admin and staff users can configure and visualize a weighted, data-driven Risk Score for students across attendance, academic, and other metrics to identify where dropout-avoidance actions should be focused.

Overview Screen

The Retention Dashboard is accessible to all users who have Dashboard permissions on the CourseKey portal; it is viewable by clicking the Retention link on the Dashboard section’s secondary navigation. In the near future, we will be layering in separate View/Edit permissions.

• Summary section
  • Students are tracked if they have shown any activity within the last 60 days. Note this is specific to Retention Risk Scores (so recently graduated and/or dropped students can be analyzed as well as current students), so based on this definition it may be different from the Active counts reported by an SIS or any other reports.
  • Risk Scores are only calculated for students in at least 1 active course; as such, some students may have activity but not a score. 
  • Students are not able to see their Risk Scores.
  • Students are separated into four Risk Levels: Low, Medium, High, Severe
  • Student Risk Trend shows a time-based view which you can adjust by clicking on the pulldown on the top right of this graph
  • Risk Scores are calculated overnight every day for the previous day’s activity. Should data change for dates prior (for instance, an adjustment to attendance a week ago), prior scores do not change, only the new daily score will update as appropriate.

• Pulldown Filters for Schools, Programs, Instructors, Courses, and Risk Levels will constrain the students shown in the graphics and tables below. You can select multiple parameters to narrow the student population you wish to look at and start over at any time by hitting the reset button on the right.
  
  
• Risk Score by Student - depicts the actual list of students represented within the summary.
  • The Export button will export all student rows and columns in the current state of the list (Computed or Actual) in .csv format.
  • Actual vs Computed views - “Actual” displays the raw statistics for each value as they are recorded in our system, while “Computed” displays the resulting contribution to the Risk Score from each (weighted) Risk Factor as they contribute to the student overall Risk Score.
  • Student Details - click any student name to view risk score details and trend for that specific student
    
    
• Settings Gear - access Configurations, Daily Alert, and Data Sources

Configurations

1. Risk Factors

• There are currently 4 types of Risk Factors available: Attendance, Lateness, Last Activity Recorded, and Academics. We anticipate adding more formulations and types over time (such as metrics for Skills Tracker and Assessments) based on customer demand. Contact your Customer Success Manager if you would like to discuss your Risk Factor needs.
• Data availability from integrated grades data sources is subject to provider/customer retention policies. CourseKey will retain data it retrieves for going-forward purposes, but at launch, some providers (for example, Canvas) may not support a full year of grades data in arrears
• Risk Factors are equally weighted by default for new customers and can be customized at the campus or program level by users of the dashboard from there as long as the sum of the resulting weights to 100%
• “Academics” is disabled by default for non-integrated schools, as grades information by default comes from integrated data sources such as Canvas and Pivot Point.

A Note on Schools vs. Programs

• The list of programs available in the Retention Dashboard is populated by either the list of manually created programs uploaded in CourseKey and associated to students, or directly through an integration. If there are no programs listed, a Retention will default to a single risk score calculation for a school with no distinction by the program.
  
  
• If programs are available, Risk Factor weights & thresholds can be customized individually for each program at the school.

2. Weights & Thresholds

• Weights: For each retention metric per school or program, a percentage is set to determine the contribution of the metric to the risk score of a student in the program. The total of all weights for active metrics has to add up to 100.
  
  
• Thresholds: for each retention metric per program, a threshold is optional to set as the value that triggers a metric to contribute to the risk score. If a threshold is not set, students will accrue points toward their risk score for that factor in proportion to their rank-order within the set of students.
  • Attendance: 80% threshold - a student with 80% attendance and higher would have 0 contributions to risk score
  • LAR: 3 days - a student with the last activity recorded of 3 days and less would have 0 contributions to risk score
  • Lateness: Setting as a percentage of the total time available, and accounting for any grace period configured. For example, an 8:56 am check-in for a 9 am start, or 9:04 check-in for the same start adds the 4 mins in both cases as an adjustment. If there is a 5-minute grace period, in which case a non-zero threshold would accrue no points toward the Risk Score.
  • Academics (requires integration): Based on the grading policy used by the integrated data source (for instance, cumulative grade, or average grade), a threshold determines a numeric value above which the student would accrue 0 points to risk score.

3. Risk Levels

Four levels are currently supported and are configurable by users at the campus or program level.

• The numeric settings describe the minimum score on a scale of 0-100 that qualifies a student for that Risk level.
  
  
• Numeric levels should be set in descending order, such that the Severe level’s minimum Risk Score is a greater number than High, High is greater than Medium, and Medium is greater than Low (which by default is set to 0).

Daily Alerts

• Student Retention Risk Scores are calculated daily. To have staff receive updates by email overnight to the resulting list of students who are newly elevated to the Severe Risk level, toggle the switch to ON, and select the staff who should receive this email.
  
  
• The available email addresses in the list are limited to CourseKey users who have at least one staff or administrative permission set (non-student users).

Data Sources

• This view shows the current customer’s full list of data integrations – as of initial launch, it is a known enhancement to constrain the list by a specific campus selected in the pulldown.