Give it your best shot. Assess. Repeat (Project Block v. 2.0)

Tracking student progress in Project Block

"It stings. It stings bad, but this is the fight business. I've been on the end of many defeats in my life and I've rose back, so I will not shy away from it. I will not make excuses for it. I will assess it and come back."

-Conor McGregor

It’s easy to get caught in the trap of evaluating your success in terms of actual results.

Don’t get me wrong. I’m all for data and evaluation, but what do you do when the data don’t show the success you want?

Option 1: Despair. You suck. Your job sucks. Your life sucks. Lay on the couch and give up.

Option 2: Roll with it. Regroup. Rebound. Reassess your efforts, and adjust. Hey, this is a fight, right? You are in a battle against entropy and gravity and whatever other frictions and fictions are trying to drag you and your clientele down.

For me, I’ll take Option 2. I’ll let that despair roll over me like water off a duck's back. I’ll acknowledge the failure and pain, but then I'll get right back in there for another round. I never thought this would be easy. I always knew it would be a fight. From they day I struggled through the birth canal, I knew this world was a tough place. So let’s go, shall we?

Case in point: Project Block. Our whole school program was founded on the idea that autonomy and independence are keys to motivation and learning. My first year here, students were given several hours per week to work on whatever they wanted to, but the lack of productivity and success I saw changed my mind on motivation and led me to create a more structured approach called Project Block.

In Project Block, students worked through a series of structured science projects of increasing complexity, each of which focused on different aspects of the project process, and then graduated to fully independent projects about midway through the year.

That was the idea, anyway. 

Well, the results are in. While almost all students (88%) finished the 8 structured projects and earned 0.5 science credits, fewer succeeded in completing their first fully independent project (71%), and even fewer succeeded in completing additional independent projects and/or meeting their own deadlines for key milestones after the first 9 projects. Of 19 additional independent projects begun:

8 completed the research step on time (42%),
6 completed their Trello/Bullet Journal planning assignment on time (32%),
2 completed their projects (11%), and
1 completed it on time (5%).

Not exactly the results we had hoped for. My current hypothesis is that these students, many of whom struggle with ADD and related issues, are simply not ready for independent work like this. 

We could give up. Roll over. Play dead.

But it's worth another shot.

We had a whole-school deliberative discussion at the end of last year to talk about how we could regroup and improve Project Block. The students came up with the following idea: Split the school up into groups by credit needs (science, social studies, English) and maintain structured projects throughout the whole year (unless students demonstrate the ability to work independently). 

We will also build in lots of explicit instruction in project planning and execution, and the whole thing will be guided by daily checklists.

Maybe we'll never have a successful independent project program, but we'll learn in the process, and as Marcus Aurelius wrote: 

"The impediment to action advances action. What stands in the way becomes the way."

The obstacle becomes the way we hadn't looked for--the way to new strengths and new paths.

Give it your best shot. Assess. Repeat. Advance.

It's good to be wrong

Depot students having a blast during fitness block.

"I feel like you treat us like delinquents. I feel like we're always being watched with suspicion."

We were sitting around the table, having a little conference. I had been sensing some negative vibes lately between the students and me, and so I called some of them together. The student who just spoke had been quiet, but he drummed up his courage and got right to the heart of the matter.

I was glad he did, because it's good to be wrong. That is, it's good to know you're wrong, because then you can change.

There had been several recent disciplinary incidents, and I had responded by taking away some privileges. On top of that, this year we've vastly increased accountability at the Depot. We added the CHIPS incentive system, ALEKS math block, competency ratings and progress tracking, and careful credit accounting. All this came together to generate lots of stress and negativity. The kids in the meeting had also complained about the Chips and how some students may not be able to go on the outing to Lake Compounce.

As I listened to what they had to say that day, I knew they were right. I knew the importance of focusing on support--that accountability is a poor driver of improvement, but I had lost focus. We had increased support along with the increased standards, but it wasn't enough, and according to our recent school climate survey, compared to last year at this time, more students felt like we don't care.

Time for a course correction. As Brian Tracy says, an airliner is almost never heading straight for it's destination. It's constantly correcting its course. It's constantly wrong, but the pilots know it, and they constantly adjust.

So I reinstated the privileges I had taken away. I reduced the Chips requirement for the latest reward outing, and finally, I made it easier to get math credit through the ALEKS system.

This last adjustment was the result of me looking more closely, after that little conference, at the sources of stress on the kids. I suspected that a few of the behavioral incidents and the slow accumulation of reward Chips were actually due to kids struggling with their math. They were nearing the end of their courses, and only the most difficult topics were left.

It may be coincidence, but since I adjusted the ALEKS requirements, there has been a notable change in behavior and attitude among a few of the students I had been concerned about, and spirits, in general, have been higher since the changes.

Lesson learned.

Being wrong is good stuff.

Change is tough: The slogging phase

"I'm done with my chores now. It's time. I say my prayer and head out on the hunt. The sun isn't up yet; it's cold; the fields are sopping. Brambles scratch my ankles, branches snap back in my face. The hill is a sonofabitch but what can you do? Set one foot in front of another and keep climbing.

An hour passes. I'm warmer now, the pace has got my blood going. The years have taught me one skill: how to be miserable. I know how to shut up and keep humping. This is a great asset because it's human, the proper role for a mortal. It does not offend the gods, but elicits their intercession. 

My bitching self is receding now. The instincts are taking over. Another hour passes. I turn the corner of a thicket and there he is: the nice fat hare I knew would show up if I just kept plugging."

-Steven Pressfield, The War of Art

Not sure exactly where I am in the hype cycle right now, but it's somewhere after the "Peak of Inflated Expectations." I'm in that phase when blogging about all the cool things that are happening, like our awesome Gallery Night and progress on our progress tracking system would seem empty. It would violate Ryan Holiday's rule of not talking about projects until they are almost entirely done. He says it's too easy to trade doing for talking about doing--to blog, post, tweet and brag about all the cool things we are up to and in the end produce nothing. Talk, Holiday says, is the enemy of work, and I'm in the work phase.

It's that phase when the brambles grab at your legs, your boots are wet and your legs are jelly. It's that phase when you wonder if it's all for nothing, and are tempted to let everything just slide back to where it was.

But it's also that phase when you can't stop--when you know you have to keep going, because you know in your heart and mind you are on the right track.

Last year was my first year in administration--my first year at the Depot, and I started with Barry Jentz and Jerome Murphy's Entry Plan approach. I rode over the top of the hype cycle right about then, disillusioned with the autonomy-based approach to education, and landed in the "Trough of Disillusionment." We did a program evaluation and immediately started up the slope with a set of changes we thought were needed.

It's tough not knowing where you are on that slope. I hope we're headed into the "Plateau of Productivity." There's a bunch of stuff in place, and data is coming in, but it's too early to tell for sure. So right now, we keep plugging, waiting for that rabbit to show up.

And I think it's right around the corner.

The procrastination equation: Putting research into practice at the Depot

They were discussing different forms of government, and the conversation had degenerated into a frustrating chorus of "There is no right answer. It's all just a matter of opinion."

"That's not true," I said, trying to keep the frustration out of my voice. "There are benchmarks. Some forms of government work better than others."

I'm not sure they got my point, but that kind of relativism is one of my pet peeves. Unfortunately, it infects many parts of society, including education. People in education often think that there are no real answers--no reference points by which to judge between the myriad of models and practices that promise to improve student success. But that's not true. Research has shown what works. The same science that brought us modern medicine and discovered global warming can help us educate our kids, and if we're not using it, we're fighting this battle blindfolded. Here's the story of how we're putting research into practice at the Depot, and how it's impacting our kids.

Last year was my first year at this little non-traditional school, and it soon became obvious that motivation was a serious problem. Motivation was low, engagement was low, attendance was low, task completion was low, and procrastination was high. Then I came across this article on procrastination from psychologist, Piers Steel. It was a thorough review of research related to motivation, and his "procrastination equation" became my guide. I took the liberty here to make it a bit more user friendly:

Utility is "how desirable a task or choice is for an individual." Expectancy is whether we think we will be able to complete the task, value is how much we value the outcome, the more impulsive among us will have a greater tendency to procrastinate, and the further away the deadline (the greater the delay), the more we'll put off the work.

Using the equation is simple. If utility is low, we procrastinate. To decrease procrastination, we need to increase expectancy and value and decrease impulsivity and delay. At the Depot this year, we're attacking all four factors of the equation.

Expectancy: According to Stanford psychologist Albert Bandura, "efficacy expectancy is somewhat susceptible to verbal persuasion and emotional arousal but is especially influenced by modelling and actual performance accomplishments." With this in mind, we are increasing our use of explicit instruction at the Depot, and our new Project Block is designed to provide students with a string of small successes to build upon. As of last week, about 60% of our students had completed 8 science projects, including hands-on projects, lab reports and a research paper, and had begun their first independent project.

Focused work during Project Block

The ALEKS online math system, which we also started using this year, is another way we're building self-efficacy. It offers a mastery-based approach the kids love, and our students' average progress on their courses is 54%.

And finally, we've really ramped up the 1:1 support this year. With a combination of interns and an instructional assistant, we are able to assign struggling students one-on-one help during ALEKS, Project Block, and any other time they need it.

Value: Dr. Steel points to classical conditioning as one way to increase the value factor. During our summer program, we were having trouble getting kids to do their online math work, so I was tossing around the idea of a token-based incentive system with Ellen. I had planned to make the tokens worth credits. "But what if they don't care about credits?," she asked. 

She was right. Many of our students don't seem to be motivated by threats of credit loss or offers of credit. So what did we do? We still offered credit, but we also tied the tokens to a field trip reward, and it worked. They really valued that reward!

We carried that same approach over into this year: Students earn poker chips for on-task behavior, and if they earn enough, they qualify for special field trips. Many students use our new progress tracking system to keep track of their chips and they value them so much that their anxiety about the chips has actually become a bit of a problem. My hope is that this system will build the habits of work that we adults call intrinsic motivation. According to Steel, this is exactly what happens--students associate reward with the work, and the work becomes intrinsically rewarding.

(Of course, they don't always succeed. The other day, I told a student he would not have enough chips to qualify for the field trip the next day. He pushed his few chips away on the table as if to disavow them and said, "I don't want any more chips!" His protest reminded me of my son when he was young. He wouldn't eat his Thanksgiving dinner, and the consequence was not getting any pie. "I don't like pie!," he repeated, as he marched around the table. The fact that chips can tap into their emotion is testimony that we've tapped into the value factor.)

Impulsivity:  Impulsivity is considered a personality trait, and Steel questions whether we can influence it much, but he does suggest two possibilities: stimulus control and automaticity. "Stimulus control," says Steel, "helps to direct behavior by indicating what is appropriate (i.e., rewarding) under any given circumstance," and I think the chips serve this purpose also. If students stay off their phones and on-task for 10, 15, or 20 minutes (we gradually increase the blocks of time), they receive reward chips. Most students earn the full amount of chips available during ALEKS Block and Project Block, and it's surprising to see how easily most of them can focus for 20 solid minutes under these conditions.

To increase automaticity, Steel recommends schedules and routines. This year we've implemented a new blocked schedule at the Depot, but there's more we could do with routines within the blocks themselves. Interestingly, Steel also mentions reducing choice as a means of helping those with high impulsivity. I think the blocked schedule has also helped here, especially the replacement of "self-directed learning time" with Project Block, but I'm still chewing on this one.

We've also added another research-based strategy to the mix to try to address the impulsivity factor--one that Steel didn't mention. Stephen, one of our teachers, has been teaching mindfulness during morning pick-me-up since the beginning of the year. They've gone over several techniques, from focusing on the breath to mindful eating, and they had a great time identifying thoughts and feeling invoked by popular songs the other day. Several students have said they have started to practice meditation on their own. Our hope is that this will help them not only manage stress but also increase focus.

Delay: Our new progress tracking system is designed to give students continuous, rapid feedback about their progress, and the timed chip blocks have provided small chunked goals for the students, but I think we can help students a lot more with handling delays. My goal for the second semester is to explicitly teach the students two methods of time/project management, and then reward them for each box checked off their to-do lists. (Hat tip to Elizabeth Brott Beese, a Ph.D. student from Purdue who spent some time studying our program, for pointing me in this direction.)

That is how we're using science to help our students. That's our goal, anyway, and so far it seems to be working. Engagement is up, attendance is up, and even task completion seems to be up. I was so excited to find Steel's article last year. It has provided a framework--a science-based guide for improving our program. So often in education, we bounce from one model to another, often without much confidence that any of it will work. But there's no need for that. There is a research base in psychology and education that we can use and build on. We're not blind, we don't stumble in the dark, and we don't have to reinvent the wheel. We can't afford to. Time is precious for every one of our students.

Tracking student progress in a mastery-based system

"She won't be happy until her progress report is 100% green." That's what one of our teachers told me about one particularly driven student. She was talking about our new Google Drive-based progress report system.

One of the challenges of mastery-based learning is tracking and reporting student progress. Our new system allows students, parents, and teachers to see it all at a glance, so students can get the support they need, see their growth over time, and get quick feedback.

At The Depot, we measure student progress toward mastery of six competencies: Communication, Information, Media, and Technology, Career Skills, Life Skills, Critical Thinking, and Creativity and Innovation.

They work on these skills in various blocks during the school day: Pick-me-up, Advisory, Math Block, Book Teams, Project Block, and Fitness, and at their internships.

Teachers rate student performance on the six competencies using our Competency Rubric, and then enter the data in a spreadsheet like this:

A separate "progress report" spreadsheet for each student reports the data out to students, teachers, and parents, like this:

They see a graph of their progress toward each competency, plus a color-coded matrix that indicates their progress by subject area and block. It also reports attendance, credit issues and their accumulated "chips" (a new incentive system we are using this year). Red and orange squares indicate areas of concern. Yellow shows emerging growth, and green indicates competency.

The system also shows progress on a standard sequence of required tasks (the "non-negotiables"), their current credit status, and a worksheet that shows where their subject area credit is coming from this year. 

The system allows us to better identify students who need specific supports, and we are seeing increased use of the progress reports by parents and students. Our hope is that they will find it not only useful, but also motivating, as they see their growth over time. Our end goal, after all, is not only competency, but self-confidence.

The clinical classroom

I've noticed something cool since I started keeping track of engagement in my differentiated bio class: I'm able to view the whole thing more objectively, take it less personally when things go wrong, and see more clearly what works, what doesn't and what and how I need to change (I hope).

A few times during each class, I just walk around, a scientist with his clipboard, and note how many students are actively engaged in working on their projects, reading, writing, or discussing content.


As a scientist, I often see the world in terms of variables and data, but it's funny I never looked at my classroom like that until now.

My goal is to start doing controlled experiments, changing one variable at a time to see how it affects engagement. I know this is a tall order: I can't control all of the important variables (home life, events outside of school, in other classes, etc.), but I hope I can get some good info anyway.

On tap for this week: First, a much cooler project than last week. Let's see if it changes engagement. Secondly, I'm going to try getting more data than just counts of students who are working. Instead of just writing a check-mark if they're working at all, I'm going to use a version of my engagement scale:

1. Students watching a video (I give only brief lectures, but I post instructional videos on our Moodle site).

2. Students reading their text.

3. Students researching online, typing an essay, or drawing a poster.

4. Students discussing the project with classmates.

5. Students researching online, typing an essay, or drawing a poster AND discussing with classmates.

(I was discussing the measurement of engagement with a colleague this week, and we wondered how our flipped classrooms would compare with a traditional teacher-led, whole-class instruction classroom if we used a scale like this.)

My idea is to just record all of the numbers and the total score.

The next step is to track it for individual students, but that would take more time, I think. I'm trying to keep this minimal so it doesn't take away significantly from my teaching.

Testing what really matters

Measurement is good, as long as what you're measuring is meaningful.

Thanks to @kburiano

for the link to this great article that beautifully explains the importance of measurement and the importance of non-cognitive skills. There's been a lot of talk about these so-called "soft skills" lately in the business press, and for good reason. But they're generally neglected in schools (especially high schools), where we focus on content knowledge and skills. This is probably  because many educators don't think it's the job of the school to teach these things, but it could also be due in part to the difficulty in measuring them.

We struggled with this when we were working on our new graduations standards at the high school. Some of us wanted something about community and making a global contribution to be part of the standards, but we dropped it in part because of concerns about how we would measure it.

Maybe this is why typical standardized tests focus on easily measurable things like memory of facts, vocabulary, and mathematical techniques. But it's more likely we just haven't really decided, as schools and as a society, to really value the things that really matter.

Measurement is an essential factor in improvement, and measuring the things that matter really is possible. I believe that everything is inherently measurable, and as I've read books like Mindset, Flow, and the Happiness Hypothesis, I've often been excited by clever ways psychologists come up with for quantifying human psychology. This piece by Susan Engel just confirmed it all. She outlines a set of 6 attributes we should and could measure in schools, attributes that are much more important than a set of SAT vocab words or factoring a polynomial:

1. Reading
2. Inquiry
3. Flexible Thinking and the Use of Evidence
4. Conversation
5. Collaboration
6. Engagement

I'd add a more to the list: maybe perseverance, self-control, creativity, compassion, mindfulness, to name a few, but I don't have metrics handy (though I'm sure we could come up with some).

So let's get going. It's time to focus on what's really important, and we can develop the tools.

No grades, but lots of data

I'm really interested in the concept of the "no grades classroom" that people like Starr Sackstein (@mssackstein) and Mark Barnes are advocating. My students and I have been trying out Starr's goal-setting and self-assessment conference approach lately in biology, and though I'm still giving grades, I'd love to get rid of them.

I think replacing grades with formative feedback could go a long way toward causing a paradigm shift in our students (and teachers)--from a focus on grades and points to learning and growth.

Right now, for some students, the coursework is mostly about getting a good grade so they can get into a good college, while for others, it's just about passing. Either way, it's not about learning. It's not about autonomy, mastery, and purpose.

So getting rid of grades altogether should help. But that can't mean getting rid of data. It seems to me we need regular, quantitative performance measurements in order to assess their progress (and our instruction) objectively and improve it.

At first I thought these two goals were incompatible (no grades but lots of data). But that doesn't have to be the case at all. What I'm envisioning these days is something like this:

The students don't get grades, but they get lots of rich feedback, and I record as much data as possible.

But what about report cards and GPAs?

Well, we'll have to tackle that next.

So that's how it is

"Do you know what I like about our kind of work? You can be happy or unhappy; it makes no difference. It doesn't matter if you like what you find or hate it. You look at it and say, 'So that's how it is!'" -Rosalind Franklin in Double Helix

Watching the BBC's classic story of the discovery of the structure of DNA, today with my biology students, two things stood out.

1) They looked bored. I like to show the film because it depicts science as it's really practiced, with all the political and interpersonal realities. I love the brilliant contrast of Watson's focus on the goal of discovery (and the fame) and Franklin's zen-like devotion to the process of discovery itself. And I want to light that same fire in my students. But maybe there's a better way than this film. I recall a tweet from a teacher friend on Twitter: "If they are bored, STOP."

2) Rosalind Franklin, played by Juliet Stevenson, backs this up. For all my good intentions and all the movie's merits, if it is not the best way to light kids fires for science and siscovery, then it's not the best way. Instead of defending my past practices and habits, I need to look at the data, with Franklin, and say, "So that's how it is." Then adjust my strategies accordingly.

And this may mean I've made a mistake. It may mean I've been doing it wrong. So be it. This is just one movie, but the same principle applies to all my teaching, even the new mastery-based teaching methods I'm piloting now. (It may apply to my whole life.) But when I'm worng, I need to take Rosalind's perspective. I love what she said at the end of the movie when veiwing Watson and Crick's model of DNA, which they deduced from the meticulous data she produced: 

"It doesn't matter. THIS (DNA model) is what matters. Life is the shape it is for a purpose. When you see how things really are, all of the hurt and waste fall away. What is left is the beauty."  

And neither does it matter if I end up having been wrong. What matters is what's true. I want to see things as they really are, regardless of the implications for my pride or lifestyle.

As Sagan said, "It is far better to grasp the universe as it really is than to persist in delusion, however satisfying and reassuring."

Next time, gather more data before it's too late

I can't believe how long I have been teaching like this--giving big summative assessments without really knowing if they were ready or not.

When my students turn in a test or formal lab report and it becomes clear they haven't mastered major objectives, I have no one to blame but myself.

Had I known they were missing key pieces of the puzzle, I could have adjusted, re-taught, added more practice, or changed the summative assessment.

Instead, I didn't find out until it was too late.

Next time, Bill, gather more data before the big one.

After all, your job isn't to measure ability, but to increase it.

Why I'm excited about tech in education

I'm excited about tech in education for seven reasons: the power of tech in collaboration, communication, contribution, augmentation, facilitation, automation, and foundation.

Collaboration - With Google Docs, collaboration is amazing, and students LOVE it. The first time they see their classmates' colored cursors moving across the screen, they get excited, and for good reason. This kind of real-time, parallel-writing collaboration has never been possible before.

Communication - We live in amazing times. Since when can I send a quick message to my favorite author and get a reply? And since when can a teenager instantly publish her work to THE WHOLE WORLD? #awesome

Contribution - Did I mention making a global contribution with 140 characters and the click of a mouse? Link that Tweet to an original video, animation, or essay, and the stand back, because you may just start a #revolution.

Augmentation - We may not have robotic exoskeletons or neural implants yet, but the only thing between our brains and the internet's 4 zettabytes of information is a mouse and a screen. And for students with dysgraphia or short term memory problems, or who can't speak, technology levels the playing field.

Facilitation - Differentiation in a traditional classroom is hard. try delivering 4 different mini-lectures to 20 different students, all at the same time. But with online learning management systems like Moodle and Google Classroom, es muy facile. And have you used pivot tables yet? Data analysis is essential in our (finally) increasingly clinical discipline, and now we all have powerful tools at our fingertips.

Automation - One of my favorite things about Moodle is it's quizzing capabilities. There's nothing like giving students instant feedback after a formative assessment. And messaging an entire class? Easy. Checking for plagiarism? Easy with Google. Even easier with Turnitin.com.

Foundation - Here's where it gets really exciting, because every time we develop a new technology, it's like developing a new skill. Each new tech leads to more possibilities. It's like a tree, where each new development is like a bud, forming the foundation for more branches. Each new skill is a foundation for new skills. Just as there wasn't much need for civil engineers in primitive societies, there was no need for genetic engineers before the discovery of DNA, and there's no need for long division today, when many students are taking calculus in high school. Each new technology is a platform. And who knows what new abilities are being built on today's technologies?  Imagine what's being built on the internet as we speak?

Technology is not about new gadgets (or apps) for the sake of new gadgets or apps (as cool as they are). It's about making old things easier and new things possible. It's about the human drive to explore, to improve ourselves and our lives. It's about collaboration, communication, contribution, augmentation, facilitation, automation, and a foundation for whatever's next for us.

Data, teaching, and education

"I love teaching, but I hate education."

-A veteran teacher

I heard a teacher say this today. I think she was referring to the new teacher evaluation system, which requires us to set quantifiable goals for student performance and then rates us according to whether we meet them or not.


While I think the current system is a bit misguided in it's emphasis on accountability as a driver for improving education, I doubt improvement is possible without a data-driven approach. Looking at the numbers, the "brutal facts" as they are called by Collins, is difficult, but it's the only way to know if we need to change. And we do.

This week I looked over the results of my latest chemistry test and realized that 25% of my students failed it. I also realized it was my fault. It was my failure. But we conducted what Collins calls an autopsy on that failure, and that opened the door to improvement. Next time, we'll do things differently.

I understand why many teachers recoil from the new system. But I think their fear (and that's what it is) may come from too narrow a focus. I think a focus on what I get out of teaching is the wrong focus. The right one is a focus on whether kids are learning (and whether they are all learning). That is, after all, my job. And that's education.