CAEP Accountability Measures

1.1 Completer impact in contributing to P-12 student-learning

There are several ways in which NDMU endeavors to assess the impact of our program completers on the P-12 community.

What Do Our Completers Think About How They Impacted P-12 Learning?

The first gauge used is at the conclusion of the completer experience as completers reflect upon their experiences in the field and the impact they have been able to make prior to securing full-time employment as a professional educator. Completers are surveyed and asked to identify and provide examples of how they have made this impact on P-12 learning and development.

The quality of narrative comments seems to have improved during the three-term cycle (Spring 2021, Fall 2021 and Spring 2022). This improvement may have resulted from asking completers in Spring 2019 to be more explicit and “offer an example” of how they have done so. The breadth of responses was very insightful as well, ranging from how they documented these improvements on a daily basis with 'exit tickets' to how they did so using formative and summative assessments; some provided insights into how a group of students improved while others offered a single student as a case study. Action research has also helped to provide a meaningful tool by which completers can more readily plan for and assess this growth in student achievement. Finally, albeit not fully implemented across all programs of study yet, the introduction of edTPA as an instrument to assess the impact of learning has provided completers with a news lens through which to view teaching and learning. The following link “Completers’ Assessments Regarding their Impact on P-12 Learning and Development Fall 2021, Spring 2020, and Fall 2020 Semesters” provides a more in-depth look into these qualitative comments.

What Do Our New Educators Think (after a full year of teaching) How They Impacted P-12 Learning?

NDMU, as part of a consortium of Maryland colleges and universities, opted to explore this question with our program completers after they had completed one full year of teaching. We have completed this inquiry for three cycles, consulting with completers from 2018-19, from 2019-20 and 2020-21, after each cohort had taught for one full year. We also surveyed their employers at the same time.

There are several ways in which NDU endeavors to assess the impact of our program completers on the P-12 community.

What Do Our Completers Think About How They Impact P-12 Learning?

The first observation is that completers’ perceptions of preparedness as surveyed in 2022-2023 were higher on some criteria than 2021-2022.  For example, there was an increase in completer perceptions of preparedness in plans for the diverse needs of students and positively impacts student growth. 

There are several additional takeaways gleaned from these data points:

1. Completers were able to articulate multiple ways in which student achievement is improved, e.g., using math fact fluency, reading comprehension assessments, F&P progress, running records,
2. Completers provided specific examples as to how they have (or have not) impacted student learning, e.g., all students moved up Reading levels, Honors Biology students reached higher levels (but standard Biology did not), students improved in social, emotional, intellectual and fine and gross motor skills.
3. In terms of identified needs, completers expressed more often than not a need to improve their classroom management skills. This is the most frequently cited concern by new educators, not only at NDMU but across the nation. Such feedback has been used over recent years to strengthen course content in classroom management and to develop a website where relevant resources for teacher candidates are maintained.

For more detailed information about these survey results, please refer to the following link: New Teachers Results 2018-2023

When Observed as New Teachers, What Kinds of Evidence Might Be Offered as Justification for Positively Impacting P-12 Student Learning and Development?

NDMU began to explore this measure in Fall 2020.  Supervisors of completers from the previous academic year (2019-2020) agreed to initiate contact with those teachers they had supervised as interns in the previous year, and to meet with them to discuss their perceived preparedness to teach, and to observe them in the classroom in order to assess the impact these new teachers are having on the P-12 community.  This had originally been planned for Spring 2020, but had to be postponed due to COVID.  We are continuing with this project in Spring 2020.  Preliminary results are shared  in Interview with and Observation of New Teachers.

Summary results from this assessment included the following:

Teacher impact during these interviews was measured on outside assessments and observations (observations, IEPs, DIBELS, country assessments, MAP scores). Teachers were observed to integrate and pay attention to student interests to enhance instruction (e.g. sports) and utilized technologies to improve instruction such as promethean board and jamboard. Additionally, observations revealed a good deal of information and ideas learned from NDMU. Specifically, strategies related to cultural relevance, dyslexia and technology were evident. Outside measures to gauge success such as principal observations and county assessments were utilized.  . Teachers also  exhibited collaboration with other teachers to achieve goals for students, and used outside measures (observations, SLOs, informal assessments, MAP scores, country assessments and other assessments) to gauge student progress.

Observations also showed that teachers focused on relationships and building positive rapport with students, and cited examples that students were looking forward to coming to class. Teachers specifically cited instruction at NDMU which helped foster these relationships, and cited numerous technologies that they employed such as google meetings, platforms, docs and slides. Teachers were positive with students and felt this positive attitude really helped some emotional students start to like school.

Reviews by supervisors of principal observations were good, SLOs were meet and students did well on F & P assessments.   Teachers utilized demographic data to plan for instruction and used students’ ethnic background when appropriate.  To create a welcoming environment teachers related to students personally, used stuffed animals and maintained a positive attitude. Teachers also used games with reluctant learners. Students utilized I-Pads, zoom and remote learning, and teachers had greatest success comes in helping students learn to read and write.

Teachers also were prepared for the curriculum. Teachers utilized demographic information and were able to utilize ethnicities in the classroom. Teachers has also incorporated diverse international songs into the class (i.e., a music classroom).

Teachers were also observed fostering relationships by getting buy-in from students. Teachers used demographic information to inform teaching- however, demographics are constantly changing so it posed some challenge. In addition, several teachers had some trouble planning for ESOL students.   Teachers utilized benchmarks and quarterly assessments but also more alternative methods such as students artistic ability. Teachers utilized summative and formative assessments, and SLOs were met.

All teacher observations went very well. Teachers utilized positive feedback, tangible rewards and positive teaching strategies. Teachers utilized a wide variety of technology, such Nearpod, jamboard cohoot and performance matters. Teachers built rapport and respect with students by using small group instruction when appropriate.

Overall, this project provided substantive information about new teacher impact in the classroom and will be continued in the coming year.

When Observed as New Teachers, What Kinds of Evidence Might Be Offered as Justification for Positively Impacting P-12 Student Learning and Development?

NDMU hoped to explore this measure during the Spring of 2020. Supervisors of completers from the previous academic year (2018-2019) agreed to initiate contact with those they had supervised as interns the previous year and to meet with them to discuss their perceived preparedness to teach and to observe them in the classroom in order to assess the impact they are having on the P-12 community. These plans to pilot this initiative with approximately 20 new educators unfortunately had to be scuttled due to the onset of COVID-19. It is hoped that supervisors will be able to complete this initial study in Fall 2020 as new educators begin their second full year of teaching. We look forward to sharing these observations with our stakeholders.

1.2 Completer effectiveness in applying professional knowledge, skills, and dispositions

One way the NDMU describes and evaluates the impact of our graduates on students in their PK-12 classrooms is through case study analysis of performance results in partner schools. NDMU partners with 14 Professional Development Schools (PDS) across four Maryland local school systems.  Six PDS sites are in Baltimore County Public Schools (BCPS): Oakleigh Elementary, Hillcrest Elementary, Johnnycake Elementary, Lansdowne Elementary, Kenwood High and Woodlawn High. Three are in Anne Arundel County Public Schools (AACPS): Arnold Elementary, Belvedere Elementary and Broadneck High School. Three PDS schools are in Baltimore City: Medfield Heights Elementary, John Ruhrah Elementary Middle and Western High.  Finally, two are in Harford County Public Schools (HCPS): Church Creek Elementary and Aberdeen High.  An analysis was undertaken by the SOE faculty this year to examine student outcomes at these 14 sites to investigate the impact of NDMU’s PDS partnerships. A summary of one partner school follows below; full analysis may be viewed here.

Lansdowne Elementary serves a population of over 600 students that is highly diverse; in 2020, approximately one-third identify as White, one-quarter Black, one-quarter Latinx, with the remainder identifying as Asian or more than one race/ethnicity. Approximately 73% are eligible for free/reduced-price meals, over 15% receive special education services and 15% are English language learners. This suggests Lansdowne is a relatively high-needs school compared to Baltimore County LEA, where 49% are eligible for FARMS, 13% receive special education and 8% are English learners. Three recent NDM SoE graduates (two completing elementary education MAT programs in 2018 and one in 2019) serve as 5^th grade teachers at Lansdowne Elementary in Baltimore County Public Schools. Altogether, Lansdowne Elementary has four 5^th grade teachers; thus, NDMU graduates serve the majority (~75%) of 5^th graders at this school. This concentration provides the opportunity to examine the potential impact of NDMU’s completers on student outcomes. 

In 2017 and prior to NDMU graduates’ arrival at Lansdowne, approximately 40% of the school’s 5^th graders were meeting expectations in ELA (i.e., scoring 4 or 5 on PARCC). In 2019 – the most recent year for which assessment data are available -- 17.2% met expectations. However, it is notable that the ELL population at Lansdowne more than doubled between 2017 and 2020. Whereas 6.8% received EL services in 2016-17, this share had increased to 15% in 2019. Further, the PARCC participation rate for 5^th grade ELs at Lansdowne was over 95% in 2019, which may help explain the decline in performance. In mathematics performance on the 5^th grade PARCC increased substantially at Lansdowne. While in 2017, just 9.4% met expectations, in 2019 19.3% did so, more than doubling the share of proficient 5^th graders. 

Another way the NDMU gauges completer impact on the PK12 classroom is to examine success on external measures, such as principal observations, standardized tests, IEP goals and SLOs (see interview analysis here). Two cohorts of completers were analyzed using a pilot instrument that was tested on a small sample in 2019-20 and again in 2020-21: those working in 2019-20 and those working in 2020-21.  Tests on validity and reliability will be conducted and the survey revised prior to the 2021-22 administration using standardly accepted statistical methods. Most completers are employed in local school systems such as Baltimore City and County, with smaller percentages throughout the state (see completer employment here).  For 2019-20, 10 former interns completed the initial survey and sat for interviews (n=10). These former interns worked in a variety of school systems across the state of Maryland. These school systems include Anne Arundel County Public Schools, Baltimore County Public Schools and Montgomery County Public Schools. In addition, 80% of respondents taught in grades K-5, 10% taught in grades 6-8 and 10% taught in grades 9-12.  For 2021, 10 former interns completed the initial survey and sat for interviews (n=10). These former interns worked in a variety of school systems across the state of Maryland. These school systems include Anne Arundel County Public Schools (3), Baltimore County Public Schools (4) and Montgomery County Public Schools (3). Seven out of the 10 respondents teach in elementary schools while three respondents teach in a middle school. 

Eight out of 10 respondents noted that they were rated effective on their classroom observations. Most respondents noted how their students were successful on various standardized measurements such as WIDA tests, DIBELS testing, Kindergarten Readiness Exams, County Assessments, quarterly assessments and benchmarks, F & P assessments and MAP tests. Of course, some former interns did note how they or their students struggled. The quantitative data is telling here as well: 90% of respondents noted how they use a range of assessments to measure learner’s progress, 70% believe that they implement a range of effective strategies to engage students and 50% believe they teach the required content.  Results suggest that NDMU interns have a positive impact. All former interns employed strategies and means to connect with their students and to build relationships and rapport with their students. Many of the former interns noted how they built positive relationships, used games and in some cases even stuffed animals. Interns realized the importance of building relationships and creating a welcoming environment. In addition, 70% of respondents noted how they strove to create a welcoming environment. 

Eighty percent of respondents noted that they planned for diverse needs of their students and understood the diverse needs of their students. For instance, a music teacher incorporated diverse songs in her class to cover many cultures, while another former intern noted they incorporate students’ ethnic backgrounds when appropriate. One respondent did note they were caught a little off guard by some students’ cultural beliefs (one student did not celebrate Halloween).  while demographics and cultural diversity can be challenging, it seems our students were prepared and could handle these trends. Diversity frames modern education. Demographics are rapidly changing, and our former interns prepared for this change by employing several strategies. All former interns noted that they consistently understand the diverse learning needs of their students and that they consistently plan for diversity.  Analysis of the data shows that former interns employ a variety of teaching strategies to meet the needs of their diverse learners. Some strategies that our interns mentioned have to do with connecting lessons with experience, evidencing a focus on experiential and Deweyan education. Other former interns employ SEL instruction and learning circles. The pandemic did create some obstacles, but our former interns do not employ the old “sage on the stage” model of teaching but rather use active teaching and learning strategies as well as mastery learning.  Assessments are crucial to any school and classroom. Interns employ several formal and informal assessment strategies. For one, former interns talk to family members and schedule one on one meetings, as well as check in with, monitor and talk to students to gauge their progress. Former interns also use test scores, pre-assessments, checks for understandings, exit tickets, hand symbols for non-verbal learners, conversations, oral comprehension checks and worksheets as well. In addition, all former interns interviewed noted that they plan for a diverse array of assessments to gauge student learning.

Way forward, we will continue to conduct performance analysis of our partner PDS. We are also planning to initiate our employed teacher’s tracer study. In 2023-24 session, we will execute both the tracer study and school’s performance analysis to triangulate our findings.

2.1 Employer Satisfaction

Securing information from employers of NDMU completers has always been and remains a challenge because the state of Maryland has no comprehensive data base which connects individual teacher performance with the University from which they completed their teacher preparation program. Additionally, local school districts in Maryland are loathe to share teacher performance data for reasons of confidentiality, nor do they track the universities from which teachers graduated. Undaunted by this challenge, NDMU has collaborated with other Maryland Universities to design, validate and implement a new survey instrument to document the perceptions of employers about our program completers. We are extremely proud of the results gathered during our first two years of this project to assess employer satisfaction, with empirical as well as anecdotal information. This link provides summary perceptions of completers employers from Spring 2023.

EMPLOYMENT MILESTONES

While follow-up data (e.g., promotions) on advanced program completers is not pursued in the same manner as initial certification candidates, NDMU does take great pride in the leadership roles that our graduates play in the P-12 community.  Our advanced program completers are quite often recognized as outstanding leaders in their respective fields of expertise, earning “teacher of the Year” and “Principal of the Year” accolades, as well as appointments to key leadership positions in private and public schools across the State of Maryland.  The following link provides greater detail about some of these many instructional leaders.  https://www.ndm.edu/colleges-schools/school-education/award-winning-teachers, https://www.ndm.edu/about-us/stories/mdtesol-teacher-year and the milestones they have achieved.  The School of Education also proudly displays in our SOE Hall of Fame the accomplishments of our initial certification and advanced program certification completers.  Over 30 plaques recognize these honored award recipients.  Come visit NDMU to learn more.

Survey Data obtained from initial certification program completers are obtained each semester documenting NDMU’s efforts to comply with CAEP Component 4.4/A.4.2 regarding the satisfaction of program completers.

These data consistently demonstrate that teacher candidate graduates are confident that their NDMU preparation program was rigorous and fully prepared them to pursue a career in the teaching profession. Each of these criteria is aligned to the InTASC standards. Three data reports, which follow, were obtained from the three most recent cycles/semesters of survey data: Fall 2019,  Spring 2019 and Fall 2018. These surveys included 31 Likert-style criteria that are aligned to the new InTASC standards and Danielson Framework for Effective Teaching and which assess the satisfaction of completers. This link provides the most recent cycle of completers satisfaction for Spring 2023.

Post-Program Employment:

Post program employed summary of shows trend of increasing number of program completers and they are mostly working in Maryland. This link provides the summary of post-program employment since 2019-2020 session to 2021-2022. 

Our interns also employed mostly after Fall and Spring semester. During Fall 21 and Spring 22, most of the interns employed in Montgomery county and Baltimore City. The following link shows the list of counties and number of employed interns for Fall 20, Spring 21, Fall 21 and Spring 22.

NDMU conducts employed intern’s perception survey regularly and uses the findings to improve program performance. During Spring 2023 session, our interns felt most effective at having positive relationships with students, parents, and colleagues.

The interns were also surveyed to understand their perception about the NDMU Teacher Preparedness Program. Most respondents felt that they consistently demonstrated the behaviors listed in the classroom in Spring 2022, with three-fourths of participants indicating as such, and in Spring 2023, with two-thirds of participants indicating as such. The following link will show the Intern Perception of Teacher Preparedness Program Spring 2022, Fall 22 and Spring 2023.

FINDINGS AND RECOMMENDATIONS

Notwithstanding a small ‘N’ in fall 2021, the data across the three cycles were, for the most part, comparable. The entire range of median scores in spring 2023 was within a higher range, from a low of 3.75 to a high of 4.00, showing increased perceptions of completer preparedness across all criteria. The quality of narrative comments continued to show more specifics, especially with respect to completers noting how they have impacted student achievement as the survey questions continued the past practice of asking respondents to provide specific examples. Responses continued to be insightful as well, ranging from how completers documented their impact on a daily basis to how they did so using formative and summative assessments; some respondents provided insights into how a group of students improved while others offered a single student as an example. Action research, for which students were prepared through their portfolio submissions, continued to be evident in the open-ended responses—highlighting completer skill in assessing their impact and student learning in the classroom.

2.2 Stakeholder Involvement

Notre Dame of Maryland University's School of Education Quality Assurance System was developed to ensure program accountability, continuous improvement, and stakeholder (internal and external) involvement.  QAS continues to evolve as national and state requirements change; available information from community partners (stakeholders), professional development schools (PDS), educator and educational leader partners, and completers are gathered and analyzed; and the EPP further grows, reflects, and evolves. The QAS comprises multiple measures that monitor candidate progress, completer achievement, and operational effectiveness.

An essential aspect of this process is the consideration of the functions of and purposes for gathering, reviewing, disseminating, reflecting, and making SoE-based and program-based decisions involving internal and external stakeholders. Characteristic of these decisions are efforts to improve operational effectiveness, candidate progress, and the impact of completers on student learning and development. Several systems facilitate SoE's data collection and analysis, and databased decision making for assessment and continuous improvement and innovation and entail using multiple measures and sources. Sources include Colleague, Anthology, SmartEval in Brightspace, LiveAlumni, and MSDE's Report Card Data.  To elicit outside stakeholder involvement, input, and guidance, the EPP utilizes an Advisory Council and workgroups to improve operational effectiveness, candidate progress, and the impact of completers on student learning and development. The Council is charged with providing feedback on these programs as well as furthering support and improving the EPP's capacity to meet the leadership needs of its partner local school systems and professional development schools.

3.1 Graduation and completion rates

Graduation and Completion Rates for Initial and Advanced Programs of Study are routinely assessed. NDMU seeks to obtain a goal of 80% for both those who are seeking a degree and for those who are seeking a non-degree certification. With the exception of some advanced programs, where there is a greater degree of flexibility, NDMU continually exceeds this goal. Please read a more detailed analysis about our graduation and completer rates HERE

NDMU also analyzes Completers data by their Certification area to understand the trend and uses them for future planning. Please read the details about the completers by their certification area here.

3.2 Licensure Rates and State Requirements

Praxis Core and Praxis Content Knowledge Summary Results

The State of Maryland maintains very rigorous requirements for certification. For example, an elementary teacher candidate in Maryland must pass three basic Core Praxis assessments in Reading, Writing and Mathematics; they must pass four content tests in English, Mathematics, Science, and Social Studies; they must pass Praxis II in their content area; and as of July 2025 they will be required to pass the edTPA or PPAT assessment. These national assessments are in addition to the many rigorous course requirements, GPA requirements, clinical experience, action research, and supervised field internship. No small challenge, indeed.

The link below provides an analysis of the Core Praxis and the Praxis Content Knowledge assessments, all of which NDMU teacher candidates have performed exceedingly well on, as reported to MSCHE annually.
Praxis core and praxis content summary 2020-2021

Praxis II Exam Summary Results

Teacher preparation programs are required by federal law to submit a report annually to the state that describes its goals, pass rates on certain exams, and certain assurances. Each state submits to the U.S. Department of Education a report that contains state-level and institutional information. These reports are available to the general public. NDMU candidates for both initial and advanced levels of certification in Maryland always meet or exceed the pass rates for the state of Maryland, usually achieving near 100% pass rate. We are extremely proud of these results; it is easy to see why local school systems are so very eager to hire our program completers. For more information, see the links below to view the full annual reports for NDMU and for the entire state of Maryland.

For more detailed information about national Praxis Exams and how students at NDMU perform, please refer to the attached report for details about our performance in 2019-2020, and 2020-2021, and 2021-2022.

edTPA Results

“Stanford University faculty and staff at the Stanford Center for Assessment, Learning, and Equity (SCALE) developed edTPA. They received substantive advice and feedback from teachers and teacher educators and drew from experience gained from over 25 years of developing performance-based assessments of teaching (including the National Board for Professional Teaching Standards (NBPTS), the Interstate Teacher Assessment and Support Consortium (InTASC) Standards portfolio, and the Performance Assessment for California Teachers.

The design and review teams have included hundreds of university faculty, national subject-matter organization representatives (e.g., NCTM, NCTE, NSTA, etc.), and K–12 teachers. Stanford University is the exclusive author and owner of edTPA.

edTPA is a performance-based, subject-specific assessment and support system used by teacher preparation programs throughout the United States to emphasize, measure and support the skills and knowledge that all teachers need from Day 1 in the classroom. For each handbook field, the placement is a Pre-Kindergarten to 12th grade classroom. edTPA is a subject-specific assessment that includes versions for 27 teaching fields. The assessment features a common architecture focused on three tasks: Planning, Instruction, and Assessment.

Aspiring teachers must prepare a portfolio of materials during their student teaching clinical experience. edTPA requires aspiring teachers to demonstrate readiness to teach through lesson plans designed to support their students' strengths and needs; engage real students in ambitious learning; analyze whether their students are learning, and adjust their instruction to become more effective. Teacher candidates submit unedited video recordings of themselves at work in a real classroom as part of a portfolio that is scored by highly trained educators. edTPA builds on decades of teacher performance assessment development and research regarding teaching skills and practices that improve student learning. Learn more at https://www.edtpa.com/PageView.aspx?f=GEN_AboutEdTPA.html

With this background information, NDMU has embraced the use of edTPA as one of many tools used to assess the preparedness of our program completers. Preliminary test results from five cohorts of candidates can be found at the following link: Analyses of edTPA Results Over Time.

NDMU uses the edTPA as their Performance Assessment for the EPP (Educators Preparation Program). The following link  performance assessment from 2020-2021 cohort data

INITIAL

Based on direct reporting from 2018-2019, 2019-2020, and 2020-2021 completers in the School of Education who were deemed eligible for their initial teaching certification, the following chart depicts the employment status of these candidates as compared to data gathered in 2013-2014. These data document a significant first employment milestone for our program completers in compliance in terms of their being able to secure positions for which they were prepared.

*These data are updated as new information regarding former completers becomes available. Employment information must be secured on a case-by-case basis as there is no central repository of employment information.

More specific information regarding employment of initial certification completers, by school district and state, can be examined in the following report on “Employment Status of Initial Certification Completers”.

ADVANCED

While follow-up data (e.g., promotions) on advanced program completers is not pursued in the same manner as initial certification candidates, NDMU does take great pride in the leadership roles that our graduates play in the P-12 community. Our advanced program completers are quite often recognized as outstanding leaders in the respective fields of expertise, earning “teacher of the Year” and “Principal of the Year” accolades, as well as appointments to key leadership positions in private and public schools across the State of Maryland. The following link provides greater detail about some of these many instructional leaders. https://www.ndm.edu/colleges-schools/school-education/award-winning-teachers The School of Education also proudly displays in our SOE Hall of Fame the accomplishments of our initial certification and advanced program certification completers. Over 30 plaques recognize these honored award recipients. Come visit NDMU to learn more.