Contextual Questions for Computer Science Education Interventions

If you are thinking of applying for funding to expand computing or to provide computer science professional learning to teachers.  The following questions reveal state-level context that could help you strategize.

1. Who can teach CS?

The following applies to high school teachers.  Can those with a business certification teach CS?  Many states put computer science into Career Technical Education (CTE) which means that only business teachers can teach it and still be considered in field.  Some states also allow those with math and science credentials also teach CS, and often math teachers have an easier time making the transition to teaching CS.  For K-8 teaches, is there a special endorsement, add-on certification or other credential?  Are those without the credential prevented from teaching CS?

2. What CS courses are currently recognized and how are they recognized?

In many states, CS counts toward graduation as a math, science, or business education credit.  Does it satisfy a core credit?  If it counts as a math credit, will state universities recognize it or penalize students for not having the full amount of credits.  For example, one state allows CS to count toward graduation as a math credit, but the state university only grants admission to those students who have satisfied all four math credits with math courses.  So, CS may count toward graduation but not toward admission to college.  This may affect the priority that schools give to CS courses.

3. How do school leaders (superintendents, principals, and assistant principals) define CS?

A recent report from Google and Gallup (2015) showed that 63% of principals perceive “Creating documents or presentations on the computer” to be part of computer science.  Fifty-four percent of principals also perceive that “searching the internet” is part of computer science. Gallup poll found that principals perceive that applications courses like Excel and Access count as CS courses.  Computer science faculty largely do not consider these to be aspects of computer science, and this indicates that school-level projects may have to help principals develop a stronger definition of computing.

4. What policy strategies are most likely to succeed with state legislators?

Some states take an equity view and perceive that the state has an obligation to ensure that all students are offered computer science regardless of race/ethnicity, gender, ability, and SES.  They also see CS as a mechanism for advancement out of low-SES and into middle class jobs.  Other states use job-fulfillment as a strategy for passing legislation to expand computing and argue that the state has thousands of jobs that need to be filled by those with computer science experience.

5. How are CS teachers currently prepared?

Many states rely on pockets of professional development to support teachers, and some states offer statewide resources and professional development for teachers.

6. Has the state adopted computer science standards and to what grade levels do those apply?

Some states have written high school computer science standards and have moved on to writing and adopting K-8 standards.  Many states are in the midst of writing CS (sometimes computational thinking) standards.  Some have adopted existing standards (like CSTA’s Computer Science Standards), and this shifts the burden from standards writing to standards implementation.

7. To what extent are parents asking for computer science?

Another report from Google and Gallup (2016) finds that 84% of parents perceive computer science to be more important or just as important as required courses (math, science, history, and English). However, only 28% of parents have expressed support for CS in the classroom to school officials.


Google. (2015). Images of Computer Science: Perceptions Among Students, Parents and Educators in the U.S. Retrieved from

Google. (2016). Trends in the State of Computer Science in U.S. K-12 Schools. Retrieved from

Examining Effect

John Hattie has spent much of his career listing effect sizes correlating school, teacher, and student factors to student achievement.  Here he lists 195 influences of student achievement along with their effect sizes.  For example, he lists creativity programs, providing formative evaluation, teacher clarity, and interventions for students with learning disabilities as factors that influence student achievement with an effect size greater than 0.4.

His work is important for two reasons:

  1. His list of effects forms a normal curve with a mid-point effect size of 0.4 (using Cohen’s d).  Many have been using Cohen’s rule of thumb interpretation of effect sizes (small effect = 0.2; medium effect = 0.5; large effect = 0.8), but this interpretation has been meaningless in educational interventions, and has led many to say that any positive effect on student achievement is “good.”  Hattie sets a bar for effect sizes at 0.4 and concludes that typical teaching provides effects between .15 and .30.  He implores us to go well beyond typical teaching effects.  john-hattie-effect-sizes-on-achievement-9-728
  2. Hattie’s method of pooling effect sizes can be adopted to discuss effects across programs.  The National Science Foundation (and other federal education funders) have searched for mechanisms to broadly explain the merit and worth of a body of programs, and effect sizes give us a broad approach to do this.  Naturally, we must be cautious about using effect sizes because it may be easy to inflate them, the named effect might contain multiple other factors, and the outcome variable (student achievement) may be difficult to accurately measure.  Still, in the absence of mandating the same instruments across programs, effect sizes give us a method for pooling our data and supporting agencies that need to account for their funded programs.

Evaluating Micro-Credentialing

I wonder if others are noticing a shift in teacher professional development.  The shift moves from the kind of K-12 professional learning we often see in schools – 1) a  teacher attends a week-long professional learning session in the summer to fulfill requirement for continuing education or 2) a teacher attends a professional learning session on Saturdays throughout the school year.  These sessions seem to begin with the premise that teachers need to know something they do not currently know, and it should be delivered in a setting similar to a graduate school education class.  The shift I’ve noticed asks teachers to choose from a menu of targeted learning events.  Digital Promise provides some examples of micro-credentials for educators.  Examples could be “Leading a Professional Learning Community” or “Using Wait Time Effectively.”

How then, do we evaluate micro-credentialing initiatives?  We might adopt Guskey’s (2000) variant of the Kirkpatrick (1996) model.  Here’s what it might look like:

  1.  Participation and selection:  Who is choosing what micro-credential?
    • How did the designers arrive at the set of micro-credentials?  Did they conduct a needs analysis?
    • Did the participant self-select into the program?  If not, how was the participant connected to a specific micro-credential?
  2. Participant Reactions:  Did the participant find the professional learning useful, informative, and engaging?
  3. Participant Knowledge: What did the participant hope to learn?
    • What did the participant learn?
    • What is the connection between what the participant hoped to learn, the learning objectives of the micro-credential, what the participant actually learned, and what the participant applies on the job?
  4. Organizational Support and Change
    • What effect has the professional learning had on the school environment?
    • What barriers prevent participants from using what they have learned?
    • What affordances of the environment promote use?
  5. Participant Actions: How does the micro-credential affect practice?
    • How does the participant intend to use what they learn in the micro-credential?
    • What action does the participant actually take as a result of the micro-credential?
  6. Student Success:
    • Before taking the Micro-Credential:  What affect will the micro-credential have on students?  That is, how will students be different as a result of this micro-credential?
    • During the Micro-Credential:  How has the teacher changed his/her perspective of the effect of the micro-credential on students?  That is, now that the teacher is taking the micro-credential, how has his/her thinking changed in regards to the effect it will have on students?
    • After the Micro-Credential:  What effect has the micro-credential actually had on students?  Can we draw theoretical and ultimately causal connections between the micro-credential and student success?

Evaluating Professional Learning Communities (TCAR)

As we begin preparations for summer professional learning, some may be considering using professional learning communities (PLCs).  Woodland recently published a rubric in the American Journal of Evaluation to evaluate PLCs.  It builds off previous PLC rubrics and incorporates improvement science.  This is particularly timely for those considering research-practitioner partnerships (see this solicitation from NSF).  Researchers are often university faculty who may not know the extent to which PLCs are used in public education, and this provides them with both a framework for understanding these potentially robust communities along with an approach to measuring them.  For practitioners, typically K-12 school leaders and teachers, it provides a strategy to return sleepy, mostly complacent PLCs to their intended use – to foster critical and sometimes tense dialogue around student success.  Here is an overview of the rubric (tcar-form).