Leadership: Accountability in Teaching & Learning

Leadership: Accountability in Teaching & Learning

IMPORTANT! The School Principal is the single most important stakeholder in the successful implementation of Empower.


Highly Reliable Schools have one thing in common: real time, understandable, comprehensive data systems that provide an early warning to system failure. School principals who diligently harvest the rich data from Empower validate the hard work of the Teachers who bear the burden of data entry. More importantly, they consume the data and make evidence-based decisions that improve learning and deliver results that provide a measure of success to all stakeholders.

Our experience clearly establishes that for every successful/sustained Empower Implementation, there exists a Principal who is deeply engaged in the Empower data systems. For every failure, there exists a Principal who believed Empower was a teacher-only tool. 

Data is not about adding more to your plate, Data is about having the right things on your plate." Principal Principles 

Never before have we had so much data on individual learning. Never before have principals been so empowered to affect meaningful change at the individual learner level.

What follows is a look into data systems that provide insight to reliable implementation of student-centered teaching, learning & assessment: 




Scoring Reliability

Scoring reliability refers to the consistency with which different people who score against the same scoring guide agree. The belief that children learn at different time frames is foundational. In a system where "mastery" is the goal, progress (movement) becomes a pressure as students are provided more time learn. Where the pressure for movement mounts, scoring reliability is often sacrificed. 

Empower's Marzano True Score Estimator was built to provide feedback on the reliability of the summative score. Classroom teachers can use this feedback as validation for their own professional opinion or as a nudge to seek additional evidence when reliability is in doubt.  Principals can use this data as feedback on general reliability as a system, grade level, content area or individual teacher. This data informs staff development needs, PLC Assessment calibration, scoring guide revision etc..

Marzano True Score Estimator Report 

By comparing a student's starting score for the time period against their current score and the algorithm's Best Fit score, leaders can see not only growth (or regression) but also the relationship between the evidence received and factored into the system's Best Fit recommendation versus the score decided upon by the teacher of record. Discrepancies here may represent a lack of underlying data (not enough evidence being entered to justify the score) or assumptions being drawn by the classroom teachers. On the sunny side, it may also confirm that excellent teaching and learning are taking place. 
  1. Go to the Reporting Tab
  2. Select the Report Printer (Sometimes labeled Progress Report Printer)
  3. Marzano True Score Estimator Report
  4. Run Report
  5. Set the filters for the data you want to see

  6. Hit Preview



Course/ Credit Completion Reliability

TVR Admin Report

(TVR) or Teacher Verification Report is the means by which Teachers post credit for course completion to a students permanent record.  In a competency system, seat time is not a vehicle for credit. Instead, course credit is based on mastery of competencies aligned to a traditional course. Administrators require a tight system to manage and ensure accuracy to the student records and the TVR process is this vehicle. The Admin version of this report gives administrators a report to view course scores and the underlying data before validating that data themselves to be sent to the student's permanent record.

Only users with Admin rights will see this report in their tools menu. 
  1. Go to the Empower Tool Menu
  2. Open the TVR Admin Report
  3. Set the filters for the data you want to see.
  4. Press Run Report

For there to be data in this report, courses must be set up and assigned and teachers must be using the TVR themselves to send data 






Graduation Reports

Graduation Tracker

Since time is the variable and learning (competency) is the constant in a student centered system, pacing is the essential measure of progress. This begs the question: Are students on track to graduate on time? the Graduation Tracker shows any given student's current progress through a content area, the credit expected to date and the actual credit earned. This report provides an early warning for counseling, communication, and intervention needs. It can also be used at the elementary/middle school level to track readiness to advance to the next level of learning.
  1. Go to the Reporting Tab
  2. Select the Report Printer
  3. Find the Graduation Tracker Report
  4. Click Run Report
    Set the filters
  5. Press Preview






Eligibility

Eligibility Report

When moving from a time based to competency based system of learning there exists a need for Principals to re-imagine eligibility metrics and reporting. Since the grade is no longer variable in this model, pacing and a positive contribution to the school culture often become the key metrics in assessing minimum progress for eligibility. The interesting nuance with "real time data" is that eligibility becomes a "real time" metric. In other-words, no surprises to students, parents or coaches/activity directors. 
  1. Go to the Reporting Tab
  2. Report Printer
  3. Eligibility Report
  4. Set the filters
  5. Hit Preview


Eligibility Admin Report

Reports are only as reliable as the data we input.  Site Leaders need a reliable method to verify that all required data has been input and gaps identified for correction. Therefore, timeliness of data entry requires an accountability gap analysis which the Eligibility Admin Report provides. 
  1. Go to the Reporting Tab
  2. Report Printer
  3. Select Eligibility Admin Report
  4. Set the filters
  5. Hit Preview




Pacing

Pacing Report

Challenging Goals and Effective Feedback is of the most important school-level factors according to Dr. Robert Marzano. He refers to goals & feedback together as "Pressure to Achieve".

In a competency-based learning environment, there exist two system goals that are important for Principals which include 
  1. Mastery
  2. Adequate Yearly Progress/Pace.

Because pace is variable to allow mastery to happen for learners with different learning timelines, it is one of the most important key performance indicators for Principals to have on their plate. In fact, it is arguably the most important KPI for all stakeholders to be monitoring on a daily basis because it essentially answers the question "How am I doing now".  So, it is incumbent on the Principal to also monitor "real time data" to ensure effective pacing feedback (see Teacher Activity Report as well as frequent/recurring data chats with teachers and departments related to pace and just in time interventions.  




Stakeholder Use 

Teacher Activity Report

The Teacher Activity Report displays a wealth of information including when teachers have logged in, how often, which tools they do and don't use with what frequency, and the kind and amount of scores they are giving. 
This is invaluable in making sure that the prescribed tools  are being used and proper practices are being maintained.

  1. Go to the Reporting Tab
  2. Report Printer
  3. Select Teacher Activity
  4. Report
  5. Set the filters
  6. Hit Preview

     
Similar reports are available to show Student and Guardian usage.





Attendance and Engagement


Student Engagement Report

Traditionally, schools have relied on attendance to demonstrate that their students are participating in classes. We have always known that attendance can tell you that a student has spent time in a seat, but not much else. If schools are going to move away from recording and reporting seat time as a measure of success, we feel it is important that they are able to collect and act upon data that tells them more than attendance ever did.

With the Student Engagement Report, Empower is able to present data that goes beyond superficial attendance reports. Our data helps schools and teachers understand how much their students are engaging with content in our LMS. Once teachers and schools align this quantitative data with the formative and summative data that Empower can also provide, they are able to understand much more about how their students are working, what they understand, and where they are struggling. This is real data that can make a real difference.



Data fields in this report:
School : Name of the school as it is listed in Empower 
Student Name : Name of the student as it shows in Empower 
Student ID : The student’s unique ID that is synced from the SIS 
Grade : Traditional grade level 
Teacher : The name of the teacher who is leading the class or group 
Class/Group : Every class and group that the student is connected to in Empower. Classes and groups will appear once for each teacher that is associated with it. 
Logins : The total number of times a student has logged into Empower. This only counts the times a student actively logs in with their username and password or through google. 
Activities Accessed : The number of activities that a student has opened in relation to the class or group 
Submitted Evidence : The number of digital evidences that have been turned in through a student’s My Locker tab in activities. These could be text, links, google docs, or any other file. 
Submitted Quizzes : The number of quizzes that a student has taken and submitted in relation to the class or group.  
Social Posts : The number of messages that have been sent in relation to this class or group. 
Completed Assignments : The count of activities where a student has clicked a “Completion Steps”. These act as a student’s self reporting of successfully completing the activity. 

Using this Data

The Student Engagement Report (SER) produces a lot of data. One row of data for every set of student, teacher, and class/group. When Westminster Public Schools was working with Empower to develop this report they realized that while the raw data was helpful, there was a further need to create custom pivot tables in Excel to help their school leaders better understand the data. Rather than build these views into Empower, and dictating how this data should be used, it was decided that if Empower provided this raw data, then schools and districts could create and customize their own graphs and tables to help them understand and use their data.

In order to see what can be possible, Westminster Public Schools has offered to share the workbook templates they created. They run this report daily, as well as each Monday for the previous week. Providing the clean, easy to read tables to their schools leaders has helped them increase Empower usage and more importantly, student engagement. 

Raw Data 
As soon as they started pulling data from this new report, Westminster Public Schools found that the raw data from the SER was helpful for classroom teachers and principals as they look into the engagement levels of their classes. 
They also realized it would be helpful to create additional tools to understand what this data meant for their individual students. They created a template that contained a series of pivot tables along with some basic formulas. Once they paste the report’s data into this template, they have an interactive, student-centric view of the engagement data. This allows them to see how active and engaged their individual students are across all their classes and groups. 


Three Categories
To create this student centric view the raw data was displayed in a pivot table and then grouped into three categories which were used to create data points that could be tracked over time:
Any Empower Activity : log ins, accessed activities, submitted evidences or quizzes, messages, and completed activities
Any Academic Activity : accessed activities, submitted evidences or quizzes, messages, and completed activities
Submitted Work or Quizzes : submitted evidences or quizzes 

Interactive Spreadsheet
Once this table and formulas were created, WPS had an easy to use template that could be updated each day and weekly with current data. 
Filters allow them to explore the data and quickly update their tables and graphs to display data for specific schools and or grade levels.
Using the template has provided them with a flexible tool that helped the district leaders, building principals, and classroom teachers understand and take action based on the large amount of data in the SER. 



Watch a video from WPS explaining how they use the template here: https://tinyurl.com/EngagementDataVideo  
Download the WPS template and modify it to fit your needs here:  https://tinyurl.com/WPS-EngagementTemplate  



    • Related Articles

    • Eligibility

      Eligibility is often an indicator of progress and effort that schools need to use to determine if student should be able to participate in additional school activities.  Although it has traditionally been associated with athletics, schools are able ...

    • Admin Dashboard

      The Admin Dashboard The admin dashboard gives building leaders a quick snapshot of how their learners and teachers are doing across content areas and grade levels.  Each of the sections on the dashboard is dynamic and updated in real-time. We will ...

    • Admin User Tools and Protocols (OEC)

      This article is focused on the role and tools of admin users for an Empower site, especially what we call the "Organization's Empower Contact" (OEC). This article is not necessarily for an "admin" as in a principal. For that, look to our Leadership ...

    • Report Viewer Reports

      Assessment Analysis by Question This report shows individual responses by learners on individual assessments (the “quiz” feature in Empower). It can show any assessment (quiz) between a range of dates, but only one assessment at a time. It does not ...

    • Progress Reports

      Progress Reports This article is about progress reports. For information on other reports in Empower, please refer to Report Viewer Reports Leadership This article also does not address calculating or exporting final scores, transcripts, Teacher ...