Pay-for-performance in Philadelphia is coming and going
By by Bill Hangley, Jr. on Dec 1, 2010 01:21 PM
This fall, Philadelphia's Mastery Charter Schools won $7.4 million from the U.S. Department of Education to expand its system of performance-based teacher salaries. The grant will help Mastery test a policy that's being pushed hard by the Obama administration and widely debated in education circles: tying teacher compensation to student achievement.
But even as Mastery's pay-for-performance system expands, in another group of Philadelphia charter schools, a similar experiment is about to end.
At its peak, the Philadelphia School District's PhillyTAP program ran pay-for-performance programs in 11 charter schools, involving 6,000 students and 376 teachers. PhillyTAP, short for Teacher Advancement Program, links teacher bonuses with a complex system of evaluation and support. The program was launched in 2006 with $12 million from the Teacher Incentive Fund (TIF), the same federal program that now funds Mastery.
In September 2011, PhillyTAP's TIF grant will expire, leaving the future of those projects uncertain at best.
PhillyTAP officials say TAP students have consistently matched or outperformed their counterparts in other schools, sometimes by significant margins. They say TAP's results show that pay-for-performance can work when it's part of a schoolwide effort to monitor and improve teaching practices.
"We're not a pay-for-performance program," said Sue Ostrich, a District employee who runs PhillyTAP. "We are a professional development program that happens to give performance awards."
PhillyTAP's original TIF grant was intended to test pay-for-performance in District schools. But when the District and the Philadelphia Federation of Teachers couldn't agree on implementation, the project and grant funds were shifted to charter schools. All participants adopt the same TAP system, developed by the National Institute for Excellence in Teaching (NIET) and used nationwide.
TAP evaluates student performance using a value-added data system that projects students' expected performance based on their past work and the performance of comparable students. Teacher bonuses are calculated by the degree to which students exceed expected performance, by the teacher's ratings in classroom evaluations and professional development sessions, and by the school's overall performance. Bonuses can range from $500 to $3,900.
Claudia Perez became a TAP teacher when she left a District school for the Pan American Academy Charter School in Kensington. Her job was to observe and assess other teachers using TAP's detailed rubric.
"I really had no idea how much work this was going to entail," Perez said with a laugh. "I still remember my first training – I looked back at my boss, and said, what did you get me into?"
But the TAP assessment rubric made sense to her, she said. It asked her to focus not on the orderliness of the class, or the number of textbook chapters covered, but on the quality of student-teacher interaction.
Not every teacher liked being part of this kind of system. "There was resistance," Perez said. "Some teachers have chosen not to return." But she thinks the assessments helped her and her colleagues improve their classroom technique.
"When I worked for the School District, I had two observations a year, and it was 'satisfactory' or 'unsatisfactory' - that was it," she said. "There was no conversation about what I could do better."
An independent evaluator, Mathematica, is currently conducting a comprehensive assessment of TAP programs nationwide. For now, NIET's initial data show that students at all 11 PhillyTAP schools, including Pan American, have met or exceeded expected student gains. Jonathan Eckert, who assessed PhillyTAP for the NIET, said the program's impact was "significant."
"They still have a long way to go," Eckert said. "But they're definitely moving in the right direction, and it seems to be happening year after year."
Robin Chait, an education specialist with the Center for American Progress in Washington, D.C., said the lessons of TAP seem to fall in line with the body of pay-for-performance research. "The programs that are most successful are those that connect performance to a meaningful evaluation system [and] professional development," she said.
Chait cited a recent controlled study by Vanderbilt University, which concluded that rewarding teachers with bonus pay, in the absence of other support programs, does not boost student test scores.
"If you only pay teachers more, it won't make any difference," Chait said.
The emphasis on teacher supports is a philosophy that Mastery will lean on as it expands its own pay-for-performance system. Mastery awards bonuses when schoolwide goals are met, and also moves individual teachers up the pay scale based not on seniority, but on a combined assessment of classroom performance, student progress, and other contributions to the school's environment.
Mastery's CEO, Scott Gordon, said teacher assessments are so important that he dedicates staff time to evaluating the evaluation process and correcting it when necessary.
"The problem with performance-based pay systems is that people need to be evaluated, they need to get feedback, and that is a very difficult thing to do well," said Gordon. "People can quickly feel like that was an unfair process, so you have to train your managers and monitor it if there's a problem." With its TIF grant, Mastery is moving to improve its value-added measurements for the "student progress" portion of teacher evaluations.
Both Gordon and Ostrich stress the fact that raises or bonuses help encourage everyone to take part in what can be a challenging process of evaluation and improvement.
"Everybody likes a bonus!" Ostrich said. "If they grow as individuals, if they grow their students, why shouldn't teachers have an opportunity to get a bonus?"
As PhillyTAP's sunset approaches, the number of participating schools is down to five. Ostrich said most of the others couldn't afford to meet the TIF grant's escalating matching-fund requirements.
That was the case with the West Philadelphia Achievement Charter School, where principal Rena Morrow is sorry to leave TAP.
"Our students did increase in reading and math skills," Morrow said. "And I noticed our teachers started having better conversations about students and their academic progress – fewer conversations about trivial things, and more focused conversations about students' needs."
PhillyTAP officials are helping former and current TAP schools raise money to continue their pay-for-performance projects. "I'm trying to keep this going," said Ostrich. "But when this is over, the funding for myself and my staff is gone."
In the District itself, plans are underway for a modest system of performance bonuses tied to improved PSSA scores. Made possible by an agreement in the latest union contract, District officials hope to have it running next year but don't know how the bonuses will be funded or how many schools will be involved.
They do know that bonuses wouldn't be tied to assessments of individual teachers; instead, all teachers in a school that showed improvement would get the same bonus.
"The specifics of the TAP program are quite different from what we're going to be doing," said Joe Antonio, an assistant to Superintendent Arlene Ackerman. "The basic message we got from TAP is that you can get widespread support from teachers for this program if they think it's fair."