Research Shows LML Impact on Test Results
Nancy Ward, Superintendent of grades Pre-K to 8 in the Runnemede Public School District in New Jersey, recently completed her doctoral dissertation on elementary student writing achievement. To her surprise, the results of her research on state writing scores indicated that one school’s score far exceeded those of three other schools in the same district. Read on to find out what made the difference.
Q. What did your dissertation deal with and to what extent did it address the Let Me Learn Process®?
A. My dissertation was entitled “The Influence of a Metacognitive Learning System on the Writing Achievement of Elementary School Students.” I examined the writing performance of third-grade students in one New Jersey school district. In this district, one school had implemented the LML Process® for five years and the other three elementary schools did not use LML. I looked at the writing achievement using two measures, the Language Arts Literacy portion of the NJ Assessment of Skills and Knowledge (NJASK3), which is the state assessment, and the local writing assessment. The study first measured whether LML was implemented with fidelity. Once that was determined, a statistical analysis was conducted using the 2008 test data. The 2009 test data was also examined using a simple comparison of NJASK3 test results by school. I also examined the writing performance of two subgroups, economically disadvantaged (free and reduced lunch) and male students.
Q. What are some of the highlights of your research and findings?
A. The findings clearly demonstrated that implementation of the LML Process® with fidelity influences the writing achievement of third-grade students as a whole. This was shown through a statistical analysis of both assessments in 2008 and the comparison of 2009 data using what is called “the eyeball test.” The results for economically disadvantaged students and males was important because both of these subgroups typically demonstrate test scores in writing that are below that of their peers who are not economically disadvantaged or females, and are lower than the general population as a whole.
Q. Was there anything in your research that was particularly surprising?
A. I did not conduct an analysis by ethnicity. It was the results of the two subgroup population as stated above that showed the influence of the LML Process® with statistical significance. The most surprising results were those of the 2009 NJASK3. In 2009 the state raised the bar for third-grade students. It was the first year in which students had to score 50% correct or better in order to pass the test. In previous years 40-45% correct represented a passing score. Most school districts showed a considerable drop in their percentage of passing students. While that was the case for the three schools that did not use LML, it was not true for the LML School. Ninety percent of the students in the LML School passed the Language Arts section of the NJASK3. That was 33% higher than the next highest scoring school and 53% higher than the lowest scoring school. When calculated as groups, Group One being the LML School and Group Two being the average of all three other schools, there was a 31% difference between the LML School and the other three for male scores, with Group One having 31% more passing scores. When calculating the scores of the economically disadvantaged students, comparing the two groups the LML group scored with 44% more passing scores than those of Group Two.
Q. To what extent do you think some of your research results can be used by other educators with their students?
A. The literature demonstrated that writing achievement is critical to student success through high school, college, and in the workforce. Educators are consistently exploring new methods to enhance writing skills. The literature also showed that the two subgroups, males and economically disadvantaged students have not achieved writing proficiency based on state and national standards (NAEP). Effective implementation of the LML Process® clearly influences these results. As an educator, a research-based key to success cannot be ignored and is worthy of pursuit, including the allocation of time and monetary resources. The results I found, because they stood the test of statistical significance, can be generalized, thereby leading me to believe that implementing LML holds great promise for student achievement in other subjects.
Q. Have you used the LML Process®?
A. Yes. I was instrumental in facilitating implementation of LML in my former school district. There is one elementary school of about 430 students and LML is being used throughout the building, with plans for implementation at the middle school and high school as well. My plan is to begin its implementation next year in my present school district. I cannot know what I now know about LML and not facilitate this opportunity for student success in my school district.
Q. What do you find to be the most useful aspects of the LML Process®?
A. The most important aspect of LML is that it focuses instruction on the learner and provides a toolbox for each learner that s/he internalizes and uses through life. It promotes internal understanding for each learner and understanding between teacher and learner. Through the Process®, teacher becomes learner and learner becomes teacher. It is an authentic exchange between the two creating a natural environment for true learning. In a world that is packed with facts and accountability, LML is a process that helps teacher and learner to navigate the vast information highway to its ultimate destination where learners are well prepared to contribute to society.
What did you find particularly noteworthy about Dr. Ward’s dissertation?
In reviewing the data analysis, the LML students scored significantly higher than their non-LML counterparts by an average of nearly 8 points with a 226.28 on the ASK3 LAL test versus a non-LML average of 218.46. More striking is the fact when compared to the state norm/average of 200, the LML students scored on average more than 18 points higher. In statistical terms, this created a t-value of over 15, indicating statistical significance 13 standard units beyond what is typically expected (normally, any value above 1.96 is statistically significant).
To what extent were you surprised by the results?
Clearly, the LML Process® is working in the school district Nancy Ward studied. Beyond the effectiveness of the Process®, there is apparent buy-in into the LML by all involved: staff, faculty, and administration. Getting statistically significant improvement is not an easy feat in these times of multiple reforms. Many in the public assume such improvements can be made quickly. The facts reveal that improvements come over time (at least two years) with sustained, competent caring and inculcating a sense of belonging (to the school).
What is your assessment of the role the Let Me Learn Process® played in the school with significantly higher scores?
The LML Process® provides a platform and change mechanism for improving student achievement. The Process® is not a means in itself. There must be a dedicated staff, faculty, and administration behind the scenes. Dr. Ward’s results further confirm this as the non-LML students were also statistically significant and higher than the state average. Our assessment is that LML had favorable results even on the non-LML students. The conclusion is that over time, the LML and non-LML faculty were collaborating on best practices/lessons and having positive effects across the board.
Tags: doctoral dissertation, elementary, literacy, males, minorities, nancy ward, NJ ASK, student achievement, test-prep, test-taking, writing