Risk Score Definition

The risk score is a percentage that represents the student’s risk of dropping out of their respective program and not graduating on time. It is calculated based on a set of assumptions that differs from a use case to another, or even from one student to another; its value is to flag an initial indicator for an admin of a potential risk for a given student being off-track of their expected factors of their education journey. CourseKey takes into account five risk factors, each with a set of assumptions based on the different use cases we support, those are:

Attendance

The attendance risk factor is a percentage that represents the student’s attendance progress versus the expected attendance at any given day looking back at the past year. Different types of courses differ in their expected attendance, and those expectations are defined below:

 

Asynchronous Courses

There are 2 types of strategies async courses: Fixed and Uniform, and both have the same formula for expected time to be submitted, that is conversion assignment time awarded for all assignments for each course a student is enrolled in. The actual time differs for each type:

• Fixed: Awarded time as actual based on time submitted per assignment
• Uniform: Awarded time as actual based on time submitted for the first submission per assignment

 

Synchronous Courses

There are several data points taken into account to properly calculate expected and actual attendance for sync courses, those are: maxPerDay value, attendance start and end times, break durations, and time adjustment.
The expected time is the time from start time to end time minus the break durations per session of a course with the maxPerDay value as a cap per day. The actual time is the time from the check in to check out minus the break durations (and the time adjustments) per session of a course. All this data is based on the same logic as CourseKey’s timesheet.

 

Blended Courses

The expected and actual time is calculated based on the sum of both sync and async components of the course.

 

Flex Courses

There are several data points taken into account to properly calculate expected and actual attendance for flex courses, those are: maxPerDay value, attendance start and end times, break durations, and time adjustment.  

The expected time is the minimum time between:

• The time from start time to end time minus the break durations per session of a course 
• The time equals maxPerDay * 60 seconds

The actual time is the time from the check in to check out minus the break durations (and the time adjustments) per session of a course. All this data is based on the same logic as CourseKey’s timesheet.

 

Special Use Case of Sync/Flex Courses

The 'wheel' course approach is a sync or flex course that is used by beauty and wellness clients as a rolling enrollment course that would span a few years (2-3 years). The expected time is the same as the sync or flex courses disregarding all sessions before the enrollment date of the student to said course.

 

Timesheet Makeup

The timesheet makeup is time stacked to the actual time of the student at any given day.

 

Academics

The academics risk factor is a percentage that represents the student’s grades progress versus the expected total grades at any given day looking back at the past year. Different types of courses/programs differ in their expected grades, and those expectations are defined below:

 

Canvas Grades

The expected total grade is based on the sum of Max Grade of all assignments related to a course that a student is enrolled in. The actual total grade is the final grade for a student per assignment based on the grading method, that is “Unique Method” for Canvas grades that allows for instructor to select the final grade in Canvas.

 

Pivot Point Grades

The expected total grade is based on the sum of Max Grade of all assignments related to a program that a student is enrolled in. The actual total grade is the final grade for a student per assignment based on the grading method, those are:

• “First Attempt” that takes the grade of the first submission
• “Last Attempt” that takes the grade of the last submission
• “Highest Score” that takes the highest grade of a submission
• “Average Score” that takes the average of grades of all submissions

 

Assessments

The internal CourseKey Assessments feature powered by Learnosity is not taken into account for the Academics risk factor at this time - please reach out to your assigned CSM to request leveraging Assessments data for your Risk Score.

 

Lateness

The lateness risk factor is a percentage that represents the student’s lateness count versus the all attended sessions at any given day looking back at the past year. Different types of courses differ in their lateness definitions, and those expectations are defined below:

 

Async Courses

The Lateness risk factor for an async course is the ratio of all submissions with a late flag over all submissions.

 

Sync Courses

The Lateness risk factor for a sync course is the ratio of all check-ins (taken into account the grace period) after the start time per session over all sessions scheduled.

 

Blended Courses

The Lateness risk factor for a blended course is the ratio based on the sum of both sync and async components of the course.

 

Flex Courses

The Lateness risk factor for a flex course is the ratio of all check-ins (taken into account the grace period) after the start time per session over all sessions scheduled.

 

LAR

The LAR risk factor is a percentage that represents the student’s last activity at any given day. The risk factor is a step function that adds 10% to the risk factor for each day after threshold. So, if the threshold is at 3 days and the LAR is 10 days, the LAR risk factor is 70%.

 

Checklists

The checklists risk factor is a percentage that represents the student’s checklists submission progress versus the expected checklists submission at any given day baselined by their progress percentage from their start date to their end date. Skills Tracker programs assume the following data points are available to calculate the checklists risk factor: Start Date, End Date, total submitted checklists items, total required checklists items available per program with a SIS ID.