Remarks about Change

Heraclitus was a Greek philosopher who lived before Socrates in the area of Ephesus (now part of Turkey). Only fragments of a probably unfinished book he wrote are known to exist. From these fragments, Heraclitus is believed to have been an independent and self-educated thinker known for promoting or provoking deeper comprehension of the world. He is probably best known for his flux principle – no, this has nothing to do with the flux capacitor popularized in “Back to the Future.” The flux principle as attributed to Heraclitus in Plato’s Cratylus involves the notion that all things pass and nothing remains the same, similar to the flow of a river; one cannot step into the same river twice due to constant change. Things change. It is the nature of things to change.

Technologies change. Technologies change what people do, what they can do, what they will do, and what they eventually do. Technologies change.

People change. People change opinions, beliefs, preferences and even habits (an interesting and revealing exercise is to extend this list in a public setting).

Education changes people, or that potential at least exists. Education involves learning, of course. Learning is fundamentally about change. Learning has occurred when there is evidence that stable and persisting changes in what a person or group of people know and can do have occurred. Learning involves change. Without some evidence of some kind of change one cannot maintain that learning has occurred. What changes might be involved in learning? Changes in abilities (e.g., learning to ride a bicycle – try a unicycle for a challenge), beliefs (e.g., learning about the impact of higher education on an individual’s earning potential or a country’s economic productivity), knowledge (e.g., learning how to find the maximum value of a function), mental models (e.g., internalizing a problem-solving approach for a class of problems), and so on. Learning is about change. Those who want to assess learning then need to understand what changes were targeted and the extent to which changes occurred.

One can argue more generally that problem solving is about change. Solving a problem in general involves transforming an incomplete or unacceptable situation into a more complete or acceptable situation. Jonassen argued that life is fundamentally about solving problems that come in about a dozen different flavors, with dilemmas being the most challenging. One cannot avoid problems – like it or not, everyone is compelled to engage in problem solving – in trying to bring about desired changes, some large and a great many small, but nearly all unavoidable. Life is about change.

Some things will change regardless of what people do as it is in their nature, as Heraclitus argued. People can influence some changes. Those are the focus of projects and programs, which are generally aimed at transforming a problematic or undesirable or deficient situation into a less problematic or desirable or acceptable situation. Project and program evaluation, then, is aimed at determining the extent to which desired changes occurred and why or why not.

A project or program begins with a problem or problematic situation, typically identified by a needs assessment. That problem can be stated in the form of a difference between an actual state of affairs and a desired state of affairs. For example, the actual retention rate of first year students at an institution might be 55%. For a variety of reasons, some financial and some more altruistic, the desired retention rate is set at 75%. A 20% gap exists and that can then become the focus of a project or a program. A summative evaluation of such a project or program will focus on the extent to which the 20% gap was reduced. Such an outcomes or impact study is not difficult to construct and conduct. A likely outcome is that not all of the 20% gap will be reduced by a particular project or program. The question then becomes: Why did that happen? To answer that kind of question requires a great deal of information about and study of how the project or program was designed, tested, developed, and implemented. Such an investigation is called an implementation study (in contrast to an impact study) and constitutes a formative evaluation as the purpose is to improve the project or program as it evolves so as to improve the likelihood of success in achieving the goals and positive results in an impact study.

So, an evaluation starts with a problem that identifies a difference between what is and what is desired. The structure of the project or program is then based on prior research and theory that suggested things that are likely to be effective in transforming the problematic state of affairs into a desired state of affairs. This part of the project or program plan is called a theory of change. Why would one expect these particular activities or interventions to be effective in bringing about the desired changes? The ability to cite prior research and theory lends credibility to a particular theory of change that is informing or justifying a particular effort.

Theories of change are of course subject to scrutiny, as are most beliefs and perspectives. Those who fund projects and programs are typically quite interested in the theory of change that informs why a project or program is being designed and developed a particular way.

What to make of these random remarks? In the words of Bob Dylan, “may you have a strong foundation, when the winds of changes shift.” And, if Heraclitus is right, a theory of change is subject to change as is everything else. Or, as Bertrand Russell said, you cannot step into the same river even once. But I just stepped into something and it is causing my head to spin.

A Learning and Instruction Hierarchy

Much has been written about learning objects in the literature. A few (e.g., Dave Merrill) have pointed out that learning objects alone do not constitute instruction. To explain why I think that Merrill is correct, I offer these remarks. First some definitions are needed. An educational system, broadly conceived, is intended to develop responsible, thoughtful and productive members of society (see Dewey’s writings for more on this). Instruction is that which is intended to support or facilitate learning. Learning is characterized by stable and persistent changes in what a person or group of people know or can do.

Some regard anything found on the Internet as a knowledge object that could support learning. However, there is misinformation and mistakes in many Internet resources, so the information found should be confirmed or verified before using it to support learning; when verified, the information object might be considered a knowledge object. Learning is a naturally occurring and ongoing process – sometimes intentional (oriented toward a goal) and sometimes not. Self-directed individuals might make use of many information and knowledge objects in pursuit of their interests. Within the context of an educational system (instructional curriculum learning environment, training program, etc.), learning is typically associated with a specific goal, in which case a knowledge object can be considered a learning object. However, many students in an educational system lack adequate preparation, motivation or self-regulation skills needed to make systematic progress towards the intended outcomes. That is why formative feedback (timely, informative, explanatory feedback) is so important. In addition, summative feedback is typically needed to mark important milestones along the way to success in and completion of a program or curriculum. Learning objects which have support for learning in the form of learning guidance, feedback, and assessment can then be considered instructional objects.

A typical MOOC (massive, open online course) lacks sufficient support for learning as do many so called serious games. While there is a place and use for MOOCs and serious games, it is important to keep in mind the intended learning outcomes, which can and probably should be negotiated with learners early and often.

Where Seldom is Heard a Discouraging Word

“Where Seldom is Heard a Discouraging Word”

I suppose it may be true that out on the range where the buffalo used to roam that discouraging words were seldom heard - probably the same for encouraging words. The range where the politicians roam today is certainly quite different with discouraging and disparaging words much more common. The range of educational discourse is more like the latter than the former, unfortunately.

I suppose when there are many people gathered in relatively confined spaces pursuing a variety of goals that there are like to be conflicting views and competing interests. Many a discouraging word has occurred in the educational discourse pertaining to the quality and value of online courses in comparison with face-to-face courses. Views persist that one or the other context is preferable in terms of learning outcomes or costs or retention or almost any other measure. The evidence, however, is not nearly so conclusive. It depends. It depends on the design and implementation of the course, on the instructor, on the topic, on the institutional support, on the students, and so on. It depends … and it varies a great deal.

Interests also vary a great deal. Some students prefer the flexibility of online courses, especially those that are asynchronous. Some students believe (often mistakenly) that online courses are easier. Some teachers believe (sometimes with some justification) that their face-to-face meetings with students are critical to developing their understanding and improving their performance. Some teachers also prefer the flexibility of online courses. Some administrators prefer paying part-time online instructors and optimizing use of limited campus space, while others prefer optimizing the unique experiences that a campus experience can offer.

While the story is neither simple nor clear, there need not be such brow beating and breast thumping as is so often heard in discussions about online and face-to-face teaching and learning. There are in fact many similar concerns to be addressed in either context in pursuit of quality education. Here are four such concerns: engagement, communication, collaboration, and practical experience. While these are critical to both face-to-face and online learning, how they are implemented and supported in each context is quite different. Engagement refers to the notion that learner interest in the topic (e.g., seeing the topic as relevant and valuable to their success) and subsequent active involvement in learning activities are critical to developing understanding. Engagement is often measured in terms of time-on-task, which has historically been a reliable indicator of learning, especially with simpler learning tasks (declarative and simple procedural knowledge). Of course it is a challenge to measure time-on-task in either context – face-to-face as much occurs outside the classroom or laboratory and online as much is hidden from view for different reasons. Asking students to report time-on-task is not a completely reliable method. However, measuring motivation before, during, and after instruction does seem to provide a sense for meaningful engagement as does evidence in student-created portfolios and in follow-on decisions such as selecting a second elective in the topic area.

Developing communication skills is important for long-term success in many discipline areas. How that is accomplished in face-to-face courses is different from how it might be accomplished in an online course, and the type of communication skill to be developed might also be different. However, it is possible to have written and oral skills addressed and practiced in either context, although doing so will be carried out differently. Likewise, learning to be a productive team player and effective collaborator is a skill that is also desirable for long-term success in many areas. Again, this skill can be supported in both contexts albeit differently. The same goes for practical application of knowledge which is so critical to skill development sought after in so many task domains. While students are hardly ever passive at the cognitive level, active learning in the course of solving problems and working on meaningful tasks is necessary for productive learning to occur. Again, this can occur in both face-to-face and online settings.

What, then, is really fundamentally different? First, as already implied, the means of implementing and supporting engagement, communication, collaboration and practical problem solving are quite different. The constraints and costs are likely to differ, so that one or the other context might be preferred for reasons other than optimal learning outcomes. The targeted learners might be significantly different or unreachable by one means or the other.

What we should be doing as educators is to find the most appropriate means and methods for the targeted learning tasks and learners. There is no need to discredit or disparage other means or methods in order to pursue the best quality educational support we can provide given the constraints within which we must operate. Might not one argue that for this situation, these learners and these constraints that a reasonable approach is X, or Y, or perhaps Z? Might we not find encouraging words for all those who are or wish to support learning and instruction to the best of their abilities? Might we not recognize the value of campus life and associated experiences even if we believe that online learning is the best solution in a particular case? Might we not admit the reality that online learning and instruction works well for many even if we prefer campus-based teaching and learning for certain subjects and students? We might. We might even admit that it would be a remarkable coincidence if the world happened to coincide the limits of our imaginations.

Mike Spector

A Framework for Thinking about Critical Thinking

There are many who argue that teaching critical thinking skills is among the most important tasks of a

n educational system (see http://www.criticalthinking.org//). One way to frame critical thinking is around argumentation – that is, discourse aimed at establishing a point. Logicians conceive of arguments as a collection of statements or assertions, some of which (called the premises, or the foundation of the argument) are offered in support of another (the conclusion, or the point to be established). Logicians typically distinguish two fundamentally types of arguments due to the different standards to be applied in assessing their merit. Arguments that are aimed at establishing the conclusion with certainty are called deductive arguments and require the most rigorous standards of adequacy. Such arguments occur in mathematically intensive domains; methods to establish the validity of deductive arguments are called proofs. 

Arguments that are aimed at establishing the conclusion with less than absolute certainty are called inductive arguments and are the type encountered in the social sciences and in everyday life. Inductive arguments have less rigorous standards of adequacy, but it is worth noting that standards do exist even though they are often ignored in informal everyday discourse. Figure 1 depicts the general structure of an argument, and it also provides the basic framework for critical thinking skills.

  

 Figure 1. The general structure of an argument. 

When does critical thinking arise? Critical thinking arises in the context of a problematic situation (Jonassen, 2007, 2011; Jonassen & Kim, 2010; Jonassen & Land, 2012). A problem exists when one perceives that the current situation is somehow deficient and in need of improvement of when one cannot adequately explain why things happen the way they do (see Figure 2). 

 

Figure 2. The general structure of a problem situation. 

If one accepts the basic notion that reasoning is often in the context of problem situations and can be represented in the form of an argument, then there are a number of places where critical reasoning can focus. First, let’s focus on the problem situation itself. One might focus on the so-called problematic situation and question the facts pertaining to the situation. Suppose the problematic situation is that a significant number of persons are not performing as expected in a particular course. One might ask how many or what percentage are performing poorly, or how poor performance is being defined, or whether this situation has persisted for a period of time. Other questions are also possible that address the facts of the situation or that ask for clarification of key aspects of the situation (e.g., performance criteria). One might also ask questions about the desired state of affairs that reflect a level of critical thinking. For example, one might ask how the desired situation was determined or whether and to what extent achieving the desired situation is realistically attainable.

Additionally, one can inquire what kinds of transformations are required to resolve the problematic situation and why they might be expected to result in the non-problematic situation. Such questions go well beyond simply accepting the claim that doing X, Y and Z will lead to the desired outcome. This can be called questioning the underlying theory of change involved in the argument.

 One can also frame critical reasoning skills in terms of the underlying argumentation. First and foremost, one can ask the fundamental question of whether or not adequate evidence has been provided in support of the conclusion – do the premises provide strong support for the conclusion? This is different from asking whether or not one agrees with the premises, as one might agree with all the premises of an argument that still fails to provide adequate support for the conclusion. This question about the adequacy of the evidence addresses the form of the argument rather than the content. First focus on the form of the argumentation.

 Once the form of the argument is deemed acceptable (a step that is often overlooked), one can then focus on the various pieces of evidence and ask whether or not they should be believed or accepted as convincing. This often leads one to ask about what assumptions have been made. Because assumptions are often not stated, an important critical reasoning skill is making the assumptions explicit and then asking whether or not there is sufficient reason to accept those assumptions.

One can perform a similar kind of provisional or hypothetical reasoning with regard to the conclusion of the argument. One can ask what else must be accepted if one accepts the conclusion. It can happen that a conclusion that might first appear to be reasonable will lead one to other statements or situations that are clearly unacceptable but left unstated or not addressed in the argument. Thinking ahead to the implications of accepting the conclusion is one form of critical reasoning. In summary, one can think of critical reasoning skills in terms of those skills required to assess arguments and to determine if what is alleged to resolve a problematic situation is likely to do so. These skills are not trivial yet they are important in nearly every discipline and in many everyday problem solving situations. It is a wonder that so little emphasis is placed on developing these skills, especially at an early age.

References
Jonassen, D. H. (2007). Learning to solve complex, scientific problems. Mahway, NJ: Erlbaum.

Jonassen, D. H. (2011). Learning to solve problems: A handbook for designing problem-solving learning environments. New York: Routledge.

Jonassen, D. H., & Kim, B. (2010). Arguing to learn and learning to argue: Design justifications and guidelines. Educational Technology Research & Development, 58(4), 439-457.

Jonassen, D. H., & Land, S. M. (2012). Theoretical foundations for learning environments. New York: Routledge.