The Retention Dashboard is a CourseKey product that gives school admins, instructors and staff a real-time view of students’ performance across five critical metrics, and combines them into one risk score, with varying levels of severity, to identify students that are at risk of dropping out.
The five student performance metrics are last activity recorded (days to last activity), attendance, lateness, academics, and checklists. The five metrics are last activity recorded (days to last activity), attendance, lateness, academics, and checklists. These metrics serve as indicators of a student's performance in critical areas of their educational experience, and factors to consider when calculating a student’s risk of dropping out.
A risk score is calculated daily for active students in every school location/campus and program, which is to serve as an initial indicator, for a school admin, student advisor/coach, or instructor, of the potential risk for a given student who might be trending on or off-track in their academic journey. This information enables school staff to identify at-risk students, prioritize interventions, and optimize student retention/success activities. (Note that in the current implementation, students cannot see their risk scores, only permissioned staff).
The Retention Dashboard is available to all users who have Dashboard permissions on the CourseKey portal, and can be accessed by clicking on Retention, under Workspaces, in the left navigation menu; or the Retention tile in the Home page.
In the Retention Dashboard you can:
- Use pulldown filters to refine the selection of students shown in the dashboard.
- View a summary of Students per Risk Level, for the counts of students grouped by risk level.
- View the Students Risk Trend, for a time-based distribution of students grouped by risk level.
- View the Risk Score By Student, for a list of students, or an individual student, per the pulldown filters' selection, including their latest risk score and performance metric values.
- Configure the Settings for calculating students' risk scores based on a school or program's criteria, and daily alerts. (Only accessible to permissioned Admin users.)
Note that the Retention Dashboard uses the Overview and Settings pages in the Cortex’ Retention Workspace. Contact your Customer Success Manager today, to explore upgrading to the full Retention Workspace, for a richer, more compelling experience offering deeper Insights and Automations within the suite of Cortex offerings.
When first accessing the Retention Dashboard, you land on the Overview page. Within the Retention Workspace, the Overview page can be accessed any time by clicking on Overview in the top navigation bar.
- The list of programs available in the Retention Dashboard is populated by either the list of manually created programs uploaded in CourseKey (for non-SIS-integrated institutions using the Skills Tracker tool), or directly through a SIS integration. Only programs with active student enrollments/members are included.
- When there are no programs configured for a school, the calculation of students’ risk scores is made for a default school program. Note that the school default program is shown as an option in the Program pulldown filter only when there are no other programs available in a school’s dataset.
- The student's most recent program enrollment (based on the student program enrollment’s start date, or when unknown, the student program enrollment’s creation date) is the one the risk score calculation is based on.
- If programs are available for an institution, weights & thresholds for the risk score calculation can be customized individually for each program at the school. This can be done by permissioned Admin users in Settings.
- Students with an active school enrollment status only (‘Active’, ‘Probation’, or ‘LOA’ for SIS-integrated institutions; and ‘Active’ and ‘LOA’ for non-SIS-integrated institutions using the Skills Tracker tool), are considered for the risk score calculation, and hence, included in tables, graphs, and insights within the Retention Dashboard.
- For non-SIS-integrated institutions using the Skills Tracker tool, students’ program enrollment/member associations can be maintained in the Program Info page, where school admins can update the status, start date and end date of a student’s program enrollment.
- If an active student does not have an enrollment/member association to a program (created from a SIS integration, or manually using the Skills Tracker tool), the student is associated with a default school program.
- When the start date of a student’s program enrollment is not known, the current date minus 365 days is set as the start date for the purpose of calculating the risk score.
- Students’ risk scores are calculated daily, taking into account data from the start date of each student’s program enrollment until the previous day. Should data change for dates prior (for instance, an adjustment to attendance a week ago), prior days’ scores do not change, only the new daily score accounts for new/updated data.
- The Retention Overview Page shows data on students’ performance metrics and risk scores that are calculated daily, in the early morning hours, and which account for any student activity up to the previous day. In the case of a delay on the data pipeline processing on any given day, the Retention Overview Page will show the most recent data that is available for students’ performance metrics and risk scores.
- A student’s risk score is calculated for a given day only if the student is in an active school enrollment status, and has had an activity (per the last activity recorded) within the last 60 days.
- For the risk score to be calculated, in addition to having had an activity within the last 60 days, a student needs to have an active enrollment and attendance recorded in an active course, grades data synced from an integrated school’s LMS, and/or checklist items submissions in a Skills Tracker program.
- Students' performance data is tracked from their program enrollment's start date (or if not known, over the last 365 days).
- If a given performance metric’s value is null (because there is no student’s data for calculating it), it is ignored when calculating the student’s risk score.
- In the Overview page, students are separated into four risk levels – low, medium, high, and severe, for easier viewing and triage. (Numeric ranges for the risk levels can be configured by a permissioned Admin user in Settings.)
A set of pulldown filters can be used to refine the selection of students shown in the dashboard.
These pulldown filters are cascading, with each selection refining options for the next set of filters.
- Schools: A list of all school locations/campuses for an institution.
- The selection for schools constraints the list of options for the Programs, Courses, and Instructors filters.
- Note that in the current implementation, students with records in multiple school locations/campuses are only shown for the school location/campus of their most recent program enrollment.
- Programs: A list of programs offered at the school(s) selected in the Schools filter.
- Note that a school default program is shown as an option when there are no other programs available in a school’s dataset.
- Courses: A list of courses specific to the school(s) selected in the Schools filter.
- Instructors: A list of instructors for courses specific to the school(s) selected in the Schools filter.
- The Courses and Instructors pulldown filters are additive–the selection of students will include those that match any of the selected course(s) or instructor(s).
- Statuses: A list of school enrollment statuses for active students, including Active, Probation, and LOA. (For non-integrated schools, only the Active status is applicable.)
- Risk Levels: A list of the levels (severe, high, medium, low) for grouping students based on their risk score.
You can select multiple parameters and click Apply to narrow the student population you wish to look at. To start over at any time, you can click the reset icon (circular arrows) on the right.
Note that identical name values in pulldown filters (such as programs and instructors) are combined into one listing (even if the underlying IDs vary) for readability.
Students Per Risk Level
The Students Per Risk Level donut chart shows the counts and the corresponding percentages for the selection of students, grouped by risk level: low, medium, high, and severe. The population of students represented in the graph include those who match the criteria per the pulldown filters' selection, and for whom a risk score was calculated on the previous day.
Students Risk Trend
The Student Risk Trend stacked area bar chart shows a time-based distribution of students (counts), grouped by risk level: low, medium, high, and severe.
- The selection of students included in this graph includes those who match the criteria per the pulldown filters' selection, except for Statuses.
- The graph shows data representing students’ risk scores calculated within the date range ending on the previous day.
- The daily counts include students for which a risk score was calculated within the past 365 days, and independently of their current school status.
- You can adjust the graph’s time interval (1 week, 1 month, 3 months, 6 months, 1 year) by clicking on the pulldown on the top right of this graph.
Risk Score By Student
The Risk Score By Student table depicts the actual population of students per the pulldown filters’ selection, including their current risk score and their percentage points for the performance metrics – attendance, lateness, last activity recorded, academics, and checklists.
- The population of students in the table includes those who match the criteria for the pulldown filters’ selection, and for whom a risk score was calculated on the previous day.
- You can use the search bar above the table to look for students based on their first name and/or last name; or sort the list of students in ascending or descending order based on the values (percentage points) of the risk score or any of the performance metrics.
- You can click any student’s Name within the table to view additional details, including the student’s profile information; performance metrics/risk factors and risk score, and risk score trend.
- There are buttons above the table for selecting whether to show the Actual values for performance metrics or the Computed values for risk factors that contribute to the students’ risk scores. Note that when you first access or reload the Retention Overview page, the table shows the actual performance metric values by default.
- There are controls below the table for selecting the number of students to show per page, and to navigate across pages.
- The Export button above the table is to download a list of the students, including all rows and columns, in .csv format.
Showing the Actual Values for Students’ Performance Metrics
When the Actual button above the Risk Score By Student table is selected (which is the default option), the data shown in the visualization is for the actual values for students’ performance metrics. See our article on student performance metric and risk factor definitions to learn more about how performance metrics are calculated.
- If there are no data available for calculating any of the performance metrics for a student, two dashes (‘--’) are shown. This may occur for a student with no data recorded to date (e.g. for synchronous attendance, course grades, or checklists progress) when the corresponding performance metric has an assigned weight – and is accounted for in the calculation of students’ risk scores.
Showing the Computed Values for Students’ Risk Factors
When the Computed button above the Risk Score By Student table is selected, the data shown in the visualization is for the computed values for risk factors that contribute to the students’ overall risk score. See our article on weight and threshold definitions to learn more about how the computed value for a risk score factor is calculated.
- The computed value for any risk factor represents the number of percentage points that are contributed to the student's overall risk score for the day.
- It is possible for the value of any performance metric that is accounted for in the calculation of a student’s risk score (with a weight assigned) to be null – when there is no data recorded to date for calculating it. In such cases, for calculating the student’s risk score, we ignore that particular performance metric and scale up the weights of the remaining metrics so that they add up to a total of 100%. Note that the corresponding risk factor for a performance metric with a null value shows with a value of 0 for the computed score.
- It is also possible for a student's computed scores for the risk factors not to add up to their overall risk score. This is because to simplify the visualization, we round down computed scores that result in a decimal number to the nearest whole number. When this happens for any student in your selection, you will see a message/link: “Why aren't a student's computed scores for the risk factors adding up to the overall risk score?”.
Viewing an Individual Student’s Detailed Data
When you click any student’s Name within the Risk Score By Student table, you are shown a page with a detailed view of the student risk score, along with information from the student’s profile – including the program, email address, student enrollment status, schools in which they are taking courses, start date and graduation date.
- The Student Risk Score visualization displays the student’s actual and computed risk scores for the previous day. There are buttons for selecting whether to show the Actual values for performance metrics or the Computed values for risk factors that contribute to the student’s risk score. See details on considerations for the calculation of computed values for students’ risk factors (details discussed above).
- You may click the information button for a quick overview of settings for weights and thresholds for the program in which the student is enrolled, and which are used when calculating the student’s computed risk factors.
- The Risk Score Trend stacked area bar chart shows a view on how the student’s risk score has changed over time. This graph shows data representing the student’s risk scores calculated within the date range ending on the previous day and up to 365 days in the past. You can adjust the graph’s time interval (1 week, 1 month, 3 months, 6 months, 1 year) by clicking on the pulldown on the top right of this graph.
- You may click the View Courses button on the top right of this page to go to the student’s Courses page, to further review details on their course engagement and performance.
To access the Settings page, click on Settings in the top navigation bar. This page is accessible to permissioned Admin users, to configure the Retention Dashboard such that it more closely matches parameters, weights and thresholds for calculating students’ risk score for their schools/programs.
As part of their custom configuration, each institution has the option of selecting performance metrics/factors to be considered for the calculation of their students’ risk scores. For each individual school, permissioned Admin users can select any of the five performance metrics/risk factors available currently, to calculate daily percentage points for different aspects of a student’s performance. See our article on student performance metric and risk factor definitions to learn more.
- Only selected performance metrics/risk factors show as columns in the Risk Score By Student table (Actual and Computed views). Note that when more than one school is selected in the pulldown filters and you are viewing data for students across schools and programs, you may see columns in the Risk Score By Student table for any performance metric/risk factor for which at least one student has a value for the performance metric that is not null.
- Adjustments to weights and thresholds can only be made to performance metrics/risk factors that are selected in this page. Note that in the current implementation, we are not blocking you from unselecting a performance metric/risk factor when it has an associated weight. Before you proceed with unselecting a performance metric/risk factor, to not account for it in the calculation of students’ risk scores for a school, you want to make sure to set its weight to a value of 0. Not doing so can lead to accounting for that performance metric as a risk factor for calculating a student’s risk score, if there is data available in the system to calculate it.
Weights & Thresholds
The weights and thresholds are available to ensure that the calculation of students’ risk scores are adjusted based on each school or program's use cases and expected standards for student evaluation. Permisssioned Admin users can adjust the performance metrics' weights and thresholds to use for calculating students’ daily risk scores. See our article on weight and threshold definitions to learn more.
- Adjustments to weights and thresholds can only be made to performance metrics/risk factors that are selected in the Risk Factors page.
Risk levels are used to flag and group students into four categories – based on their daily risk scores, for the purpose of prioritizing communications and interventions, and optimizing student retention/success activities. Permissioned Admin users can set the minimum score for each risk level, for each school location/campus or program.
- The numeric settings describe the minimum score on a scale of 0-100 that qualifies a student for that risk level.
- Numeric levels should be set in descending order, such that the severe level’s minimum risk score is a greater number than high, high is greater than medium, and medium is greater than low (which by default is set to start at 0).
Permissioned Admin users can configure daily alerts, to send an email containing any new students at severe risk to selected staff members each morning.
- Students’ performance metrics and the risk score are calculated daily. To have staff receive updates by email overnight, with the resulting list of students who are newly elevated to the severe risk level, toggle the switch to ON, and select the staff who should receive this email.
- The available email addresses in the list are limited to CourseKey users who have at least one staff or administrative permission set (non-student users).