WriterCoach Connection Gets Results!

king photo WriterCoach Connection uses demonstrable measures to assess its programs and refine its approaches and techniques.

2008-2009 Program Assessment Results

Summary and Highlights

In spring 2009, students from seven Albany, Berkeley, and Oakland secondary schools completed a survey that asked them to rate their experience with 11 aspects of the coaching process on a four-point scale. The scale ranged from “Not Helpful” to “Very Helpful.” Mean scores for each aspect on the survey were analyzed.

The following are highlights taken from the combined responses.

The majority of students reported that overall their coaches were helpful, with
95% of all responses on all items indicating some level of helpfulness, and 75% of all
combined responses scoring 3.0 or above.
The mean score for helpfulness on all eight aspects of the writing process was
3.10, up from 2.88 in the previous year.
Students rated as the most helpful aspects of coaching “Helping me understand the
next steps I need to take to improve my writing” (3.2); “Showing me what’s working in my
writing” (3.1); “Giving me suggestions for organizing my ideas” (3.1); and “Showing me
how to express my ideas more clearly” (3.1). As noted in the introduction, students overall rated “Reading my work out loud” – a hallmark of WCC interaction – as “Helpful” (3.0).
All elements of coaching received student mean scores of “Helpful” (3.0 or
above), except the prompt “Motivating me to turn in my work.”

Go to Executive Summary of 2008-2009 Program Assessment Results in PDF format.

Go to entire 2008-2009 Program Assessment Results in PDF format. Includes results from individual schools.

Past WCC assessments are available for viewing below:

2007-2008 Results

An analysis of the Berkeley Unified School District (BUSD) 2007-2008 student writing assessments showed demonstrable gains among WCC students. (Go to more details farther down this page.)

WCC also conducted a mid-year program assessment survey in January and February 2008, polling all its students and teachers in Berkeley and Albany. The overwhelming majority of students and all the teachers rated coaching as helpful.(Go to more details farther down this page.)

WCC commissioned an independent analysis of Berkeley Unified School District writing assessment data for 2007-2008. (Return to top of page.)

It demonstrates the impact of WriterCoach Connection. While it is impossible to isolate WCC from other factors, the findings suggest that WCC is having a positive effect on students' skills. Some highlights:

Students who had the benefit of writer coaching in 7th grade performed significantly higher than 7th graders who did not participate.
BUSD writing scores for 8th grade, where WCC coached all students for that school year, were statistically significantly higher in the spring than in the fall.
At Longfellow, where 8th graders had individual attention from writer coaches
for two consecutive years, student scores increased by almost a full point on the 5-point scale.

Download the entire 10-page analysis of BUSD writing assessment.

WCC conducted a mid-year program assessment survey in January and February 2008, of all participating students and teachers in Berkeley and Albany. (Return to top of page.)

Some highlights:

Overall, students rated writer coaching as "Helpful." The most common examples of help they cited were: "showing me what’s working in my writing," "giving me suggestions for organizing my ideas," "checking that I understand the assignment completely," "showing me
how to express my ideas more clearly," "reading my work out loud," and
"making sure I understand the next steps I need to take to revise my work."
All teachers said their writer coaches were helpful with improving the quality of student writing. Teachers rated writer coaching as "Helpful" to "Very helpful" in the following areas: improving structure and organization; helping low-skilled or struggling writers; helping English language learner students; improving grades on written assignments; and increasing the number of students turning in written assignments.

Download the 13-page Executive Summary of the program assessment survey.

Download the entire program assessment survey report.

2006-2007 Results

In Spring 2007, WriterCoach Connection surveyed all its students and teachers for the purpose of program assessment. The results were remarkable. The overwhelming majority of students and all the teachers rated coaching as helpful. To read or download the results of our 2006-2007 midyear assessments in Albany and Berkeley, click on either our Executive Report, a summary, or the full Integrated Report:

Executive Report: Assessment 2006-2007

(102 KB)

Integrated Report: Assessment 2006-2007

(784KB)

2005-2006 Results

During the 2005-2006 school year, three rounds of assessment were conducted: mid-year surveys of students and teachers in Berkeley, year-end surveys in Albany, and holistic scoring of essays written by Longfellow Middle School students who had worked with WCC coaches. Results from all three assessments have been impressive.

For the essay assessment, Longfellow eighth-grade teachers assigned their students a timed in-class essay in September 2005 and one on a similar topic, with identical instructions, the following May. Eight trained readers scored both sets holistically, according to a professionally developed scoring guide, and the September and May scores of each student were compared. On average, students improved by one whole score between September and May.

Average September 2005 Score	Average May 2006 Score	Average Change
6.24 out of 12	7.05 out of 12	+.82

In addition, 23 fewer students scored in the lower half of the of range of possible scores, and 23 more scored in the upper half.

Possible Scores	September 2005	May 2006	Change
1	0	0	0
2	2	1	-1
3	5	2	13
4	5	4	-1
5	19	15	-4
6	25	11	-14
7	13	16	+3
8	16	28	+12
9	2	8	+6
10	3	3	0
11	1	2	+1
12	1	2	+1

Excellent teaching and one-on-one writing help get results.

For the mid-year surveys in Berkeley, in January 2006, we asked 8th- and 9th- grade students and teachers at Berkeley High and all three middle schools to rate the effectiveness of various aspects of coaching, and of the coaching experience overall, on a four-point scale, ranging from "Not helpful" to "Very helpful." The results were:

Overall, students rated working with coaches as "Helpful" (3.07 out of 4)
Students rated all 15 aspects of coaching as "Helpful" or "Very Helpful"
The overall average score for all questions was "Helpful" (3.12)
The most helpful aspects of coaching were: reading the work aloud to the coach (3.35), having the coach point out what's working well in the writing (3.35), coach suggestions about essay structure (3.34), and having the coach fill out a worksheet with suggestions for revision (3.26)

The following charts are samples of the results WCC gets according to BUSD students.

To see survey results from all Berkeley schools in better resolution click on: Berkeley High, King Middle School, Longfellow Middle School, and Willard Middle School.

In a separate survey, teachers rated coaching, overall, as "Very Helpful" (3.5)
They rated all 13 aspects of coaching listed on their survey, on average as "Helpful" or "Very Helpful"
Aspects of coaching that teachers rated "Very Helpful" were: increasing student confidence in writing (3.7), helping less prepared students gain skills (3.7), and helping students improve the structure of their written assignments (3.5)
Aspects of coaching that teachers rated "Helpful" were: motivating students, improving students' rate of completing and turning in assignments, and helping more proficient writers. See bar graph

Albany High's 2005-2006 Year-End Survey results were similarly encouraging.

In May, 394 AHS students who had been coached 2-6 times responded to a slightly modified version of the survey given to Berkeley students in January. Overall, 93 % of the AHS respondents rated coaching as helpful to some degree. They identified the most helpful aspects of coaching as having the coach: point out what's working well in their writing (3.18), make suggestions about essay structure (3.17), fill out a worksheet with suggestions for revision, (3.09), and help them clarify their own ideas.

The six participating AHS teachers rated coaching, overall, as "Very Helpful" (3.5).

This was not the first year WriterCoach Connection conducted program assessment:

Spring 2003 Assessment Results

At Berkeley High, student writing was assessed. Using a six-point scale, the assessment showed:

1. On average, students went from unsatisfactory to satisfactory writing proficiency with their coaching session. In the process, they were exposed to skills of revision that can be applied to writing assignments in classes across the curriculum.

2. Data also showed that

89 percent of students who worked with coaches used the coaching session to help revise their essays
all students who revised their essays improved their scores by at least one point, and
36 percent improved two or more points.

3. The average essay score for all student rough drafts was 2.5 (not just unsatisfactory, but showing serious weaknesses).

4. In contrast, the average score among students submitting a final essay after coaching and revision was 4 (a score that reflects writing at a satisfactory level).

The Middle Schools Report, Spring 2003, involved a sampling of pre- and post-coaching essays written by students at two sites, King and Longfellow. Because the assessment results were aggregated, differences between the two schools are masked in the results. Those differences stem from the fact that King operated a fully realized WriterCoach Connection program, while Longfellow was in its first year, piloting the project. Conditions for the project at the two schools differed greatly in terms of the volume of classroom writing samples taken, the number of coaching sessions for each student, teacher preparation and assignments scored. With these disparities, the combined scoring results give only a generalized picture of the impact of coaching on student performance. Still, the results show great promise.

Using a four-point scale, with each paper read twice and the scores added, the assessment showed:

1. While half the students earned a passing score on rough drafts, the number passing jumped to three-fourths after coaching on the final essays.

2. The percentage of students receiving a failing score was cut in half with coaching, from 49.5 percent on the rough drafts to 25.5 percent on final drafts.

3. The average essay score for all student rough drafts was 5.76, where the possible scores ranged from a low of 2 to a high of 8.

4. In contrast, the average score on final essays submitted after coaching was 7. Teachers confirm that the improvements obtained with coaching go well beyond what students achieve when asked to "revise" essays on their own.

King Middle School teacher, Kristin Collins, speaking in support of the Writers’ Room (now WriterCoach Connection) in May 2002, told the Berkeley Board of Education:
"…Coaching supports classroom instruction, but is more effective because it is a 1:1 learning experience. Students are supported in the revision process... They learn that revision is more than recopying the rough draft. ...ALL students now have the support that only SOME of our students have at home. This is one factor in our achievement gap. [The writing coach]…program…chips away at that reality."

Through the WriterCoach Connection, students practice writing at a higher proficiency level than students ordinarily achieve when they are not coached. With step-by-step coaching support during the eighth and ninth grades, this project assumes that students will gradually master the processes for effective writing, applying these skills to writing assignments across the curriculum and beyond school.