As our project portfolio shows, we’ve developed many long-term collaborative partnerships with clients that span several years and multiple projects. We gain an in-depth understanding of our clients’ organizations, their key stakeholders, their data/content and systems that allows us to serve as a trusted advisor and partner when new challenges and opportunities arise.

A recent example was with the “City Connects” program at Boston College’s Lynch School of Education. As the program has expanded, the program team recognized a need to monitor “fidelity of implementation,” or the degree to which delivery of the intervention adheres to the program model. City Connects developed a Fidelity Monitoring System in order to compile and report information on fidelity in specific schools and across districts, manually collecting (from surveys and checklists) data and ‘scores’ across a range of indicators that provide evidence of adherence to the City Connects model.

The program team saw an opportunity to streamline the fidelity system and improve efficiency by using the existing performance support system that we had developed (the Student Support Information System – SSIS) for all fidelity system data entry and reporting. We worked closely with the City Connects team to:

• augment SSIS to allow the collection of fidelity system data which had previously been external to SSIS
• provide automated scoring against fidelity system metrics
• enhance reporting at different aggregations (e.g. school, district)
• provide a system for year to year modification and versioning of the fidelity system model and associated surveys that preserved the ability to make year to year comparisons

We developed intuitive interfaces to improve the workflow both for those who implement the system and for School Site Coordinators and Program Managers who complete the fidelity system surveys and checklists. For example, the survey editing interface allows administrators to add, edit and remove survey questions, and sequence them by dragging and dropping them into the desired order. The editing form adjusts dynamically based on user input. Surveys are presented to respondents in a modal “pop-up” dialog to clearly differentiate survey completion from other site tasks, helping to organize their workflow.

These improvements meant that the City Connects research and evaluation team can easily extend and modify the fidelity system model and surveys year to year, preserving archival data and allowing extended year to year comparisons. Also, incorporating the fidelity system data collection and reporting requirements into the SSIS has greatly simplified the completion process for the survey and checklist respondents.

Together, this improves the team’s understanding of the impact of the program across schools and districts through rigorous and continuous evaluation, directly supporting its mission of improving student outcomes.