Am I a geek or a nerd?


Taking Tests in High School Helps Students Succeed in College or the Workplace (AKA WTF?)
March 17, 2012, 9:46 am
Filed under: assessment, evaluation | Tags: , , ,

Heavens, this Race To The Top/Common Core/Teacher Eval/Assessment path meanders…

I’m reading more about how NYS is planning to implement Common Core and teacher evaluations as part of their Race To The Top plan. The rabbit trails of the internet mean lots of unrelated reading but I’m finding much of what I’m looking for. The Partnership for the Assessment of Readiness for College and Careers (PARCC) is the brainchild of the US Dept of Ed to assist states with RTTT assessments and teacher evaluations. Achieve, Inc is a managing partner for PARCC (and after bouncing around on parcconline.org and achieve.org I’m not entirely sure that I know precisely who they are or what they do). PARRC will be creating assessments to be used four times a year in English and Math classes to assess student progress and, ultimately, evaluate teachers (see more about how NYS is doing that here).

On achieve.org  I came across the statement that serves as my title; taking tests in high school helps students succeed in college or the workplace. Pardon my dunce cap for a moment, but how can that be? I’ve never, in the 24 years since I left high school and the 20 years since I left college, been placed in a workplace circumstance that required me to take a test.  Certainly there are things I’ve committed to memory and/or have such a deep understanding of that I do not need to look them up, consult others, or conduct research on in order to respond to my boss/co-workers/clients–I suppose that’s like a test, maybe. But I’m not penalized when I say to someone “let me look in to that” or “I’ll need to do some research” or “give me 10 minutes”. The skills we want students to leave school with are those-the ones that allow them to determine what’s important when reading, how to find information, and how to communicate to people. Again, there are things we need to know off the tops of our heads, but I’m not sold that taking tests in high school is the preparation for that.

So with that in my head I clicked on the citation with the above statement to read the study. It’s from 2005 and indicates that about 40% of college students surveyed felt unprepared by high school for college. Seems quite a leap from that statistic to the title of this post.  They didn’t say “gee, if I’d taken more tests I’d be more prepared” although I can see how more tests would have let them know what they didn’t know.  What these students did say is that they would have worked harder in high school knowing what they know now (who hasn’t thought that; I promise you that I’ve had that thought while running nearly every one of my marathons…if only I’d done one more 20 mile training run, or hadn’t skipped that 18 miler three weeks ago…everyone who wants to do well has these thoughts). I interpret that to mean that perhaps high school was preparing students just fine, but that these 40% didn’t take that preparation seriously.

Just to be clear, I’m all for Common Core standards, I’m all for a more rigorous teacher evaluation system, and I’m all for being clear about what our students know and do not know, but I’m not sold on enigmatic national organizations that use buzzwords and have fuzzy agendas as the best way to support those things.



Everything I could say about RTTT in NY has been said…almost.
January 26, 2012, 9:41 pm
Filed under: assessment, education law | Tags: , , , ,

Here

God bless the New York Times, this article sums up all of my questions and concerns better than I ever could. That said, I have something to add. The author questions the role of State Ed in determining how well 700 school districts are assessing students for growth in non-state tested subjects given the budget cuts at the state level and the number of students and teachers and assessments that would need independent review.

 

I think I know what’s going to happen. The assessment/evaluation police aren’t going to show up. In fact, its likely that no one will ever look closely at how districts meet the mandate. Experience tells me this. While working in Michigan as a Social Studies Consultant the state adopted k-12 Social Studies expectations where there had previously been very little required, but the state also had just one Social Studies Supervisor for the entire state and when she retired she was not replaced. State Ed funding and personnel were dramatically reduced. No one checked what you were teaching, or if you had created a curriculum that was freely available to parents, or if you were assessing students on the new expectations. Some schools even refused to align their grade level subjects to those required by the state. Because, really, what was the limbless state ed going to do about it?

 

I suspect RTTT in NYS will result in similar behavior in schools; do enough to make it look like you’re doing what they want, create a paper trail, but don’t really change much of anything in the end.

 

 

 

 



What have we done!?
December 2, 2010, 11:44 am
Filed under: assessment

This past Sunday’s New York Times opinion pages included letters children wrote to the first lady, Michelle Obama.  Some of these letters inspired confidence in our schools and our kids; like the 12-year-old from Flint, Michigan who wrote about hydrogen fusion, the 7-year-old that wants to be the first female president, the 9-year-old who wants tobacco and alcohol out of her neighborhood, or the 11-year-old who wants both an easier way to immigrate and a statue of himself!

 Then there are the standard ‘what’s it like to be famous’ and some very sad letters from kids whose parents are divorcing, or are unemployed, or just plain poor.

But the letter that gutted me, that completely convinced me that we are doing are kids a disservice with testing, came from a 2nd grader.  He starts by requesting more scary books for his classroom and a backpack to carry all the books he needs for school. He also wants a fish. But then, this:

I want to be a teacher when I grow up because I want to teach other kids things they like learning, like how to take a test. Next year I will be going to third grade. In the third grade I am going to learn very fast because I will practice the tests very fast, but sometimes I get answer wrong. I know I can do better than that.

Oh my.

First, I’m so glad he loves school and is motivated to do well. However, he thinks teaching and school is about learning to take tests. He clearly has been told that next year he will be taking some big tests; what second grader thinks about tests that are a year away? What second grader should be?

I’m not saying no tests. I’m saying I think we’ve lost the focus. The tests are supposed to show us what we know and don’t know and allow us to work from there, but it sounds like tests have become the goal themselves. What is that kids want to learn, should learn? I’m pretty sure it’s not how to take a test, despite what 7-year-old Juan from San Francisco might have been lead to believe.



How Do I Know I’m Doing My Job Well?
November 12, 2010, 2:26 pm
Filed under: assessment, Geek | Tags: , , ,

I’m a Social Studies Consultant for an intermediate school district in Michigan. That means I provide professional development services to our local school districts. Most people still don’t know what that means. What do I do all day? How do I know I’m doing it well? That I have trouble answering that concerns me.

Wow.

Education Week has recently been working on a series on Professional Development for teachers and its effectiveness, generally finding that not only is effectiveness not often measured, but that there are very few standards for what constitutes PD. EW suggests that PD needs to be targeted toward student weaknesses and how teachers can attend to those weaknesses. I agree, but I think there is more to PD as well.

Some of  my job includes attending to specific requests of districts. I assume when a department head or administrator asks me to assist their teachers with something specific it is because there is a known shortcoming. Mostly this is true, but sometimes it isn’t, and as I don’t work with one building, one department, or one district I’m often at a loss to determine what teachers really need. For example, many administrators called me to assist with Social Studies data analysis of last year’s state exams but clearly were not aware that we should not and could not do this. Should not because the exams were slated to address different standards this years, so knowing how we performed on standards no longer in place wouldn’t assist us in preparing students for the next assessment. Could not because the state no longer releases specific social studies items; we can’t see more than the content strand of each question. Without specific question language the percentages and scores provided by the state are as deep as we can go. Now had they asked me to help teachers create assessments and then analyze that data we would have been on our way to some good PD.

Another portion of my job is creating PD opportunities based on what I know many teachers and districts are struggling with. For example, I know that county-wide our students struggle with supporting their assertions in writing. I’ve put together workshop series to address this idea specifically in Social Studies. Do I know if it works? No. I don’t get to observe teachers and I’m fairly confident that the people doing the observing don’t know what PD teachers have attended. I also don’t know if the teacher who attend are those who most need this PD.

A third part of what I do is awareness. When the state adopted new content standards I did some PD about how to read them and adapt to them. I’ve since been part of a project to create a comprehensive k-12 SS curriculum based on these new standards; there has been a tremendous amount of PD for teachers to learn the progressions of learning, the units of instruction, and the various ways to implement lessons. None of this is specifically targeting weaknesses among our teachers, but it is necessary information for them to do their jobs.

On one hand I would like to know that any PD I conduct results in improved instruction and student performance, I’m also aware that some PD doesn’t manifest itself this way. Learning about changes in state requirements, for example, has less to do with improving student performance and more to do with learning the minimum requirements of your job. If I were locked into provided only services that could be measured by student test results I wouldn’t be providing very broad or rich PD.



How Long Does It Take to Grade a Test?
November 11, 2010, 8:38 am
Filed under: assessment | Tags: , , , ,

In Michigan it takes several months

The bulk of MEAP testing is multiple choice with student responses recorded on bubble sheets. I have never understood why it takes two months to get these scores or why the scores released at two months are not for public consumption. Having scored exams in NYS, I know that a single teacher (me, for example) can run 800 electronic score sheets by hand (we didn’t have a fancy self fed machine…) in an hour or two. At the end, a summary sheet that indicates the percent of students who selected each answer is produced. I understand that Michigan does not allow in-house scoring, that the tests need to be packaged and sent back to the state, and that we are testing ALL students in selected grades, but is the state really working with a single machine? Does it have just one person scoring these? Does it have the very old scoring machines that don’t have a way to transfer data to a computer for sorting?

I’ll live with this. But now the added wait for scoring of written assessments. According to the letter linked above, they must be scored and then the state and SBE will determine cut scores. In my understanding of test development, the writing assessments should have been pilot tested on a variety of students, analyzed for validity and reliabililty and cut scores assigned determined prior to scoring the exams. Again, I’ll compare my NYS experience. Each year’s cut scores were different; I could never say precisely to a student  that he must answer 30 of 50 MC correctly and get a 4/5 on each essay to pass before the test arrived at my building, but once we scored (remember, an hour for bubbles, and then 3 days for 1600 essays each read twice) we knew immediately our students results. No waiting for the whole state to be scored and then figure out passing…

I’m sure there is a rationale to this. Thoughts out there?

As an added irritant, none of the items will be released. Schools get general scores, and percentages of student performances on various strands in the tested subjects, but teachers will never know exactly what each student they teach struggled with. They will never know if their students didn’t know a particular core concept or simply didn’t know some non-content vocabulary. If data analysis is one tool for school improvement, and state tests are the measuring stick for that improvement, how can schools know where to apply their efforts if they never see the actual measure? It’s like running a race without any knowledge of how long the race is…how do you pace yourself, determine what nutrition you need, which shoes to wear, how much water to drink.



What is a test?
September 28, 2010, 12:54 pm
Filed under: assessment, Uncategorized

A funny story about our kids and tests…

A good friend of mine has a 9-year-old who has recently begun wearing glasses. That he picked glasses stunning similar to mine is probably just a coincidence, but I like to think he thinks maybe I’m cool. But that’s neither here nor there. Apparently his eyesight is so poor that someone should have noticed his need for glasses quite some time ago, like maybe a two years ago, or more!

How is that possible you say? Teachers notice squinting kids. Parents notice squinting kids. Schools conduct vision screenings annually. How did our little friend go so long without anyone, even the school nurse, noticing? Because our little friend kept passing his vision screening. Every time, he got the bottom the line of letter right. When his teacher brought him down, when Mom asked the nurse to check, little friend could read the letters.

Or could he?

Turns out, back when he had his first vision screening, he heard the words ‘eye test’ and figured it was a test like another; if you knew the answers you got a passing grade. So when the nurse’s back was turned he looked at the chart and memorized the bottom lines (the ones he couldn’t see from far away) so that he would ‘pass’ his test. He’d been reciting memorized letters for quite sometime it seems and was very proud of himself for ‘passing’ his eye test every time.

I laugh at the that story but I also wince. We’ve made our kids believe that every single thing we ask them to do in school is graded and counted for or against them.  Maybe that’s a little extreme, and it’s just a misunderstanding, but have you ever asked students to do something and found that if a grade wasn’t attached or if there wasn’t a probability it would be on the test they would rather not?



Evaluation Time
June 11, 2010, 8:11 am
Filed under: assessment, Geek | Tags: , ,

My office doesn’t evaluate people. We don’t go into schools and evaluate teachers. It’s a strange position we’re in; schools and teachers require staff development (by law in some cases) and we have the responsibility to provide it, but we do not have the authority to evaluate its impact. That’s not to say we don’t collect numbers, but it’s not specific evaluation of people attending our staff development offerings. We get MEAP numbers, MME numbers, teacher cert numbers, drop out numbers, graduation numbers. One of my plans for next year is to actually try to observe teachers from our networks–not an evaluation, just an observation. 

We also don’t get evaluated ourselves. In some ways this is good. When I left my position as an assistant principal I left behind an a boss who was outstanding at his job, but hard to please as well. His evaluations of me were sometimes confusing, in that he would suggest improvements in things that were simply unmeasurable. One I remember was Visibility. He had trouble articulating what he meant and I’m not sure I ever lived up to what we in his mind, but it was a fuzzy thing to be evaluated on. When I came here, I was accustomed to reporting my work constantly, keeping my boss in the loop so to speak. After about three weeks of my regular emails detailing various projects and proposals, my new boss (also outstanding at her job) replied that I had been hired to do a job and that could just do it, I didn’t need her approval. Wow. So I roll along, creating staff development projects based on district requests and my own anticipation of needs.

Things might be changing. For the first time here, I’ve been asked to evaluate my administrative support. I could write pages on the awesomeness that is Carol–she not only keeps a babillion plates spinning at once, she also cleans up after my figurative messes (like forgetting contracts or mixing up dates or ordering the wrong book).  She tolerates my potty mouth, saves me from sales reps, and lets me borrow her pickup truck.  She came up with the ingenious way to track attendance at our networks. She manages grants in her sleep. And she has excellent penmanship. Okay, great eval on the way. Except when I look at this evaluation document it doesn’t allow me express that we make a great team and she makes up for my shortcomings and I make up for hers. Instead, I have to check boxes ranking her as above average and so on for technical knowledge, accuracy, amount of work, cooperation, flexibility, and punctuality.

Not much of  picture of Carol, is it.

Reminds me of the teacher evaluations. You know, where all teachers are excellent.

Bringing me to my point. I know Carol is awesome, I watch her work everyday. Its only because I’m intimately familiar with her work style and work load that I can say she is great at what she does. I’ve had other secretaries, one was horrible, one was awesome. How do you measure their awesomeness, or lack of? Its so much more than a check list. How do I quantify her creativity (for example, I don’t make my own workshop announcements any more, Carol has a great creative sensibility that allows her to whip these things up and the are always better than what I can do)? The same is true when we look at teachers. Ask any principal who his best teachers are and I’m sure you will get a quick response, but look at the paper trail of observations and you’ll find many more teachers seemingly just as good.  How do these administrators know who is the best and why don’t they record it? They know it because they walk around their buildings all day, hear reports from kids and parents, see who is early to work and late to leave. Principal Walk Throughs were one of my best tools to learn who my go to teachers were–a non-evaluative  and unannounced visit can provide a glimpse of what goes on in our classroom.

Public Impact has created a new website and a report dedicated to the idea that identification and better use of our outstanding teachers can create substantial change for our students. Their assessment of the change possible is inspiring, until you realize that the have jumped right over the GIGANTIC task of IDENTIFYING these amazing people. The report doesn’t clearly state how to determine who the high fliers are, but does give some interesting predictions for what will happen when we use them properly. Okay, so we just ask those principals, right? The ones who are only allowed to use the paper trail of observations as evaluative tools even if they know better. Or maybe test scores? Lets use test scores, cause all kids are exactly the same so it should be a snap to see if the teachers achieved something with their widget children. Oh, I know, lets look at training and credentials. That should tell us. Or not. Seems that credentials aren’t at all a predictor.

The task of evaluating teachers must include bits of all of these things. Just like I with my eval of Carol; its includes my day to day sense, examples of great work, a piece of paper ranking some technical aspects, honesty on my part, openness to being evaluated on hers.  Before we can use our great teachers (or heavens, make staffing and salary decisions) we need to overhaul the current system of check lists for evaluation.




%d bloggers like this: