A Learning and Instruction Hierarchy

Much has been written about learning objects in the literature. A few (e.g., Dave Merrill) have pointed out that learning objects alone do not constitute instruction. To explain why I think that Merrill is correct, I offer these remarks. First some definitions are needed. An educational system, broadly conceived, is intended to develop responsible, thoughtful and productive members of society (see Dewey’s writings for more on this). Instruction is that which is intended to support or facilitate learning. Learning is characterized by stable and persistent changes in what a person or group of people know or can do.

Some regard anything found on the Internet as a knowledge object that could support learning. However, there is misinformation and mistakes in many Internet resources, so the information found should be confirmed or verified before using it to support learning; when verified, the information object might be considered a knowledge object. Learning is a naturally occurring and ongoing process – sometimes intentional (oriented toward a goal) and sometimes not. Self-directed individuals might make use of many information and knowledge objects in pursuit of their interests. Within the context of an educational system (instructional curriculum learning environment, training program, etc.), learning is typically associated with a specific goal, in which case a knowledge object can be considered a learning object. However, many students in an educational system lack adequate preparation, motivation or self-regulation skills needed to make systematic progress towards the intended outcomes. That is why formative feedback (timely, informative, explanatory feedback) is so important. In addition, summative feedback is typically needed to mark important milestones along the way to success in and completion of a program or curriculum. Learning objects which have support for learning in the form of learning guidance, feedback, and assessment can then be considered instructional objects.

A typical MOOC (massive, open online course) lacks sufficient support for learning as do many so called serious games. While there is a place and use for MOOCs and serious games, it is important to keep in mind the intended learning outcomes, which can and probably should be negotiated with learners early and often.

Where Seldom is Heard a Discouraging Word

“Where Seldom is Heard a Discouraging Word”

I suppose it may be true that out on the range where the buffalo used to roam that discouraging words were seldom heard - probably the same for encouraging words. The range where the politicians roam today is certainly quite different with discouraging and disparaging words much more common. The range of educational discourse is more like the latter than the former, unfortunately.

I suppose when there are many people gathered in relatively confined spaces pursuing a variety of goals that there are like to be conflicting views and competing interests. Many a discouraging word has occurred in the educational discourse pertaining to the quality and value of online courses in comparison with face-to-face courses. Views persist that one or the other context is preferable in terms of learning outcomes or costs or retention or almost any other measure. The evidence, however, is not nearly so conclusive. It depends. It depends on the design and implementation of the course, on the instructor, on the topic, on the institutional support, on the students, and so on. It depends … and it varies a great deal.

Interests also vary a great deal. Some students prefer the flexibility of online courses, especially those that are asynchronous. Some students believe (often mistakenly) that online courses are easier. Some teachers believe (sometimes with some justification) that their face-to-face meetings with students are critical to developing their understanding and improving their performance. Some teachers also prefer the flexibility of online courses. Some administrators prefer paying part-time online instructors and optimizing use of limited campus space, while others prefer optimizing the unique experiences that a campus experience can offer.

While the story is neither simple nor clear, there need not be such brow beating and breast thumping as is so often heard in discussions about online and face-to-face teaching and learning. There are in fact many similar concerns to be addressed in either context in pursuit of quality education. Here are four such concerns: engagement, communication, collaboration, and practical experience. While these are critical to both face-to-face and online learning, how they are implemented and supported in each context is quite different. Engagement refers to the notion that learner interest in the topic (e.g., seeing the topic as relevant and valuable to their success) and subsequent active involvement in learning activities are critical to developing understanding. Engagement is often measured in terms of time-on-task, which has historically been a reliable indicator of learning, especially with simpler learning tasks (declarative and simple procedural knowledge). Of course it is a challenge to measure time-on-task in either context – face-to-face as much occurs outside the classroom or laboratory and online as much is hidden from view for different reasons. Asking students to report time-on-task is not a completely reliable method. However, measuring motivation before, during, and after instruction does seem to provide a sense for meaningful engagement as does evidence in student-created portfolios and in follow-on decisions such as selecting a second elective in the topic area.

Developing communication skills is important for long-term success in many discipline areas. How that is accomplished in face-to-face courses is different from how it might be accomplished in an online course, and the type of communication skill to be developed might also be different. However, it is possible to have written and oral skills addressed and practiced in either context, although doing so will be carried out differently. Likewise, learning to be a productive team player and effective collaborator is a skill that is also desirable for long-term success in many areas. Again, this skill can be supported in both contexts albeit differently. The same goes for practical application of knowledge which is so critical to skill development sought after in so many task domains. While students are hardly ever passive at the cognitive level, active learning in the course of solving problems and working on meaningful tasks is necessary for productive learning to occur. Again, this can occur in both face-to-face and online settings.

What, then, is really fundamentally different? First, as already implied, the means of implementing and supporting engagement, communication, collaboration and practical problem solving are quite different. The constraints and costs are likely to differ, so that one or the other context might be preferred for reasons other than optimal learning outcomes. The targeted learners might be significantly different or unreachable by one means or the other.

What we should be doing as educators is to find the most appropriate means and methods for the targeted learning tasks and learners. There is no need to discredit or disparage other means or methods in order to pursue the best quality educational support we can provide given the constraints within which we must operate. Might not one argue that for this situation, these learners and these constraints that a reasonable approach is X, or Y, or perhaps Z? Might we not find encouraging words for all those who are or wish to support learning and instruction to the best of their abilities? Might we not recognize the value of campus life and associated experiences even if we believe that online learning is the best solution in a particular case? Might we not admit the reality that online learning and instruction works well for many even if we prefer campus-based teaching and learning for certain subjects and students? We might. We might even admit that it would be a remarkable coincidence if the world happened to coincide the limits of our imaginations.

Mike Spector

A Framework for Thinking about Critical Thinking

There are many who argue that teaching critical thinking skills is among the most important tasks of a

n educational system (see http://www.criticalthinking.org//). One way to frame critical thinking is around argumentation – that is, discourse aimed at establishing a point. Logicians conceive of arguments as a collection of statements or assertions, some of which (called the premises, or the foundation of the argument) are offered in support of another (the conclusion, or the point to be established). Logicians typically distinguish two fundamentally types of arguments due to the different standards to be applied in assessing their merit. Arguments that are aimed at establishing the conclusion with certainty are called deductive arguments and require the most rigorous standards of adequacy. Such arguments occur in mathematically intensive domains; methods to establish the validity of deductive arguments are called proofs. 

Arguments that are aimed at establishing the conclusion with less than absolute certainty are called inductive arguments and are the type encountered in the social sciences and in everyday life. Inductive arguments have less rigorous standards of adequacy, but it is worth noting that standards do exist even though they are often ignored in informal everyday discourse. Figure 1 depicts the general structure of an argument, and it also provides the basic framework for critical thinking skills.

  

 Figure 1. The general structure of an argument. 

When does critical thinking arise? Critical thinking arises in the context of a problematic situation (Jonassen, 2007, 2011; Jonassen & Kim, 2010; Jonassen & Land, 2012). A problem exists when one perceives that the current situation is somehow deficient and in need of improvement of when one cannot adequately explain why things happen the way they do (see Figure 2). 

 

Figure 2. The general structure of a problem situation. 

If one accepts the basic notion that reasoning is often in the context of problem situations and can be represented in the form of an argument, then there are a number of places where critical reasoning can focus. First, let’s focus on the problem situation itself. One might focus on the so-called problematic situation and question the facts pertaining to the situation. Suppose the problematic situation is that a significant number of persons are not performing as expected in a particular course. One might ask how many or what percentage are performing poorly, or how poor performance is being defined, or whether this situation has persisted for a period of time. Other questions are also possible that address the facts of the situation or that ask for clarification of key aspects of the situation (e.g., performance criteria). One might also ask questions about the desired state of affairs that reflect a level of critical thinking. For example, one might ask how the desired situation was determined or whether and to what extent achieving the desired situation is realistically attainable.

Additionally, one can inquire what kinds of transformations are required to resolve the problematic situation and why they might be expected to result in the non-problematic situation. Such questions go well beyond simply accepting the claim that doing X, Y and Z will lead to the desired outcome. This can be called questioning the underlying theory of change involved in the argument.

 One can also frame critical reasoning skills in terms of the underlying argumentation. First and foremost, one can ask the fundamental question of whether or not adequate evidence has been provided in support of the conclusion – do the premises provide strong support for the conclusion? This is different from asking whether or not one agrees with the premises, as one might agree with all the premises of an argument that still fails to provide adequate support for the conclusion. This question about the adequacy of the evidence addresses the form of the argument rather than the content. First focus on the form of the argumentation.

 Once the form of the argument is deemed acceptable (a step that is often overlooked), one can then focus on the various pieces of evidence and ask whether or not they should be believed or accepted as convincing. This often leads one to ask about what assumptions have been made. Because assumptions are often not stated, an important critical reasoning skill is making the assumptions explicit and then asking whether or not there is sufficient reason to accept those assumptions.

One can perform a similar kind of provisional or hypothetical reasoning with regard to the conclusion of the argument. One can ask what else must be accepted if one accepts the conclusion. It can happen that a conclusion that might first appear to be reasonable will lead one to other statements or situations that are clearly unacceptable but left unstated or not addressed in the argument. Thinking ahead to the implications of accepting the conclusion is one form of critical reasoning. In summary, one can think of critical reasoning skills in terms of those skills required to assess arguments and to determine if what is alleged to resolve a problematic situation is likely to do so. These skills are not trivial yet they are important in nearly every discipline and in many everyday problem solving situations. It is a wonder that so little emphasis is placed on developing these skills, especially at an early age.

References
Jonassen, D. H. (2007). Learning to solve complex, scientific problems. Mahway, NJ: Erlbaum.

Jonassen, D. H. (2011). Learning to solve problems: A handbook for designing problem-solving learning environments. New York: Routledge.

Jonassen, D. H., & Kim, B. (2010). Arguing to learn and learning to argue: Design justifications and guidelines. Educational Technology Research & Development, 58(4), 439-457.

Jonassen, D. H., & Land, S. M. (2012). Theoretical foundations for learning environments. New York: Routledge.

A Remark on Open-mindedness

One often hears someone say that he or she cannot imagine any other explanation for X. Even more common is a rhetorical statement (often dribbling out of the mouth of a politician) that there is no conceivable alternative explanation for Y. A pseudo-scientist might claim that the scientific consensus is that there is only one way to explain phenomenon Z. How should one construe such remarks, especially when they come from oneself?

The simple answer is that such dogmatic attitudes should cause one pause – they typically represent the wrongheaded confidence with which a person holds a position. Nearly every complex issue or situation has alternative explanations, even when one is unable to generate them. The logic of complexity is such that alternative explanations nearly always exist. This ability to resist dogmatism is based on a reminder I have adopted from a remark in one of O. K. Bouwsma’s unpublished journals – namely that it would be a remarkable coincidence if the world happened to coincide with the limits of his imagination. From “I cannot imagine it” to “it cannot possibly be otherwise” there is a huge leap, and it is seldom a leap of faith – it is a leap of hubris. Getting others to see the difference in those two statements (“I cannot imagine  it” vs. “it cannot be”) is a challenge, and it is especially challenging to recognize the difference in one’s own thinking.

The way I try to remind myself of this difference is through the following mantra:

              I know less than I am generally inclined to believe.

I have been reminded of that simple fact on all too many occasions.

Mike Spector

5 August 2012

Skepticism and Happiness

The Learning Technologies Department at the University of North Texas is located off campus in a complex known as Discovery Park. The department has a large meeting area call the agora (Greek for meeting place). As a consequence, I find myself drawn back to my academic training in philosophy on occasion. My memory of my early academic training is not all that clear so some of what follows may be wrong or confused. Anyway, I shall attempt to construct a meaningful story out of those memories.

Many associate skepticism with doubting or even a negative attitude towards various things. I find this association odd. Sextus Empiricus, a second century Greek philosopher who wrote a comprehensive account of skepticism called Outlines of Pyrrhonism, argued that skepticism was a kind of mental therapy that resulted in happiness or at least mental quietude. In short, like other ancient Greek philosophical traditions, achieving happiness was of central concern. Skepticism arose in part as a reaction to Stoicism, another Greek school of thought. According to Stoicism, achieving true and lasting happiness was best achieved by avoiding erroneous judgment and minimizing negative emotions by leading a simple life. The third Greek tradition to add to this mix is Epicureanism, which viewed happiness as the absence of pain and mental disturbance.

Roughly speaking there are three competing traditions in ancient Greek philosophy with regard to finding happiness and living the good life. One proposes avoiding pain and enjoying the pleasures that one finds as best one can – epicureanism. Another proposes avoiding negative emotions through a kind of purity of simplified living – stoicism. And a third proposes following a certain mental discipline to achieve happiness – skepticism. While these three traditions seem quite different, they do have several things in common. Notably, all three do not draw a sharp distinction between what we might be inclined to think of as the cognitive domain (thought, reasoning, etc.) and the affective domain (emotions, feelings, etc.). All three connect a certain mental discipline with achieving happiness or at least mental quietude (peace of mind). Hmm. One can think oneself happy – by thinking, achieve happiness. How strange is that?

Perhaps it is not so strange when one takes a deeper look at the principles of skepticism. Skepticism is not fundamentally about doubting – it is about searching for the truth. The word ‘skeptic’ is derived from the Greek word skepsis which roughly translates to inquiry. A skeptic is someone who is engaged in an inquiry process. That process logically begins with doubt – with an admission of not knowing something that one wishes to understand. The process does not end with that doubt, however. One must then be engaged in a search for the truth. A skeptic is a seeker – a searcher – someone engaged in an inquiry process. Most things worthy of such an investigation are complex and do not lend themselves to simple answers. So, a skeptic must not only be humble and begin by admitting to a kind of ignorance or lack of understanding, a skeptic must also be open to alternative explanations and an examination and re-examination of evidence. Finally, when one is exhausted from the search process, one should not simply accept a finding as conclusive – new evidence might become available or one might have erred along the way. The final stage, then, in a skeptical process of inquiry is to realize that the final truth may not have been found. This is precisely where skeptics parted company with the stoics who held dogmatically to many truths that skeptics believed worth further inquiry.

In my own life, I have come to realize that things that I believed with certainty were not so certain after all. For example, prior to my strong interest in philosophy I had a strong (also strange) attraction to mathematics. I remembered being introduced to Fermat’s Last Theorem in high school – the conjecture that no three positive integers can satisfy the equation aⁿ + bⁿ = cⁿ for any integer value of n > 2. Since the challenge to find a proof had existed for more than 300 years, I naively assumed a proof would never be found. I believed with certainty that no such proof existed. In 1995, a proof was found. The certainty with which I held that belief was wrongheaded.

In my mathematical period I became interested in the number π (pi - the ratio of the circumference of a circle to its diameter). I was taught that the expansion of the number π was infinite. While I was able to follow the proof that π is not the quotient of two integers (i.e., it is an irrational number) or not accurately expressed as the ration of two integers, I was not able to follow the proof that π was a transcendental number or that the expansion of π led to a never ending and non-repeating sequence. I simply accepted that stronger claim as true and came to believe it with certainty. I raise this point because it would seem that continuing to search for or understand the proof or seek a sequence in the expansion that repeated or ended would lead to a lot of mental effort and not result in peace of mind – my own efforts to understand π had led to more questions and a state of perplexity. I do recall one conversation with my father, Rabbi Spector, when I was trying to understand π. Remarkably, he know what π was and was familiar with an ancient Hebrew text that had approximated π as the quotient of 355/113. He also quoted Maimonides who said that π could only be approximated and not known with precision or certainty. There ensued a remarkable discussion about infinity with my father – one of the happiest memories of my life, so perhaps indirectly the search did result in some sort of happiness.

Anyway, I believe the kernel of truth in connecting skepticism with happiness is twofold. First, there is the notion that happiness involves a cognitive element. Second, if one holds a belief with wrongheaded confidence and is unwilling to examine evidence or consider alternatives, then one is likely to encounter at some point a disquieting and unsettling cognitive conflict. I have experienced several of these and have come to accept the small kernel of truth that skepticism is generally a healthy state of mind.

Consider this story from Part IV Leo Tolstoy’s A Confession:

There is an Eastern fable, told long ago, of a traveller overtaken on a plain by an enraged beast. Escaping from the beast he gets into a dry well, but sees at the bottom of the well a dragon that has opened its jaws to swallow him. And the unfortunate man, not daring to climb out lest he should be destroyed by the enraged beast, and not daring to leap to the bottom of the well lest he should be eaten by the dragon, seizes s twig growing in a crack in the well and clings to it. His hands are growing weaker and he feels he will soon have to resign himself to the destruction that awaits him above or below, but still he clings on. Then he sees that two mice, a black one and a white one, go regularly round and round the stem of the twig to which he is clinging and gnaw at it. And soon the twig itself will snap and he will fall into the dragon's jaws. The traveller sees this and knows that he will inevitably perish; but while still hanging he looks around, sees some drops of honey on the leaves of the twig, reaches them with his tongue and licks them. So I too clung to the twig of life, knowing that the dragon of death was inevitably awaiting me, ready to tear me to pieces; and I could not understand why I had fallen into such torment. I tried to lick the honey which formerly consoled me, but the honey no longer gave me pleasure, and the white and black mice of day and night gnawed at the branch by which I hung. I saw the dragon clearly and the honey no longer tasted sweet. I only saw the unescapable dragon and the mice, and I could not tear my gaze from them. and this is not a fable but the real unanswerable truth intelligible to all.

This short tale describes how Tolstoy came to abandon the epicurean approach to happiness. Perhaps it will resonate with some. There is also Camus’ Myth of Sisyphus in which Camus retells the story of Sisyphus being condemned by the gods to forever roll a rock to the top of a mountain only to have it roll back down again – an infinite but repeating sequence, recalling our discussion of pi. Sisyphus’ crime was apparently stealing secrets from and/or offending the gods. The end of Camus’ version ends with the remarkable statement that one must imagine Sisyphus happy. How is that possible? Roll that rock to the top again and again, forever and ever? Where is the joy in that? Asking such a question might lead to an interesting inquiry process …

Here is another tidbit from a renowned skeptical philosopher – Friedrich Nietzsche. This quotation comes from Nietzsche’s The Gay Science, #341 (philosophy is the gay science, by the way):

The greatest weight.-- What, if some day or night a demon were to steal after you into your loneliest loneliness and say to you: "This life as you now live it and have lived it, you will have to live once more and innumerable times more; and there will be nothing new in it, but every pain and every joy and every thought and sigh and everything unutterably small or great in your life will have to return to you, all in the same succession and sequence - even this spider and this moonlight between the trees, and even this moment and I myself. The eternal hourglass of existence is turned upside down again and again, and you with it, speck of dust!" Would you not throw yourself down and gnash your teeth and curse the demon who spoke thus?... Or how well disposed would you have to become to yourself and to life to crave nothing more fervently than this ultimate eternal confirmation and seal?

I encountered this passage in my first year of doctoral studies with O. K. Bouwsma. After reading the passage there ensued a discussion of tremendous moments in which several students shared what thought would justify a positive disposition towards eternal recurrence. For example, one offered the birth of a child as such a moment. Someone asked Bouwsma if he had experienced such a moment of happiness. As best I recall, he was about 75 when asked and he said something like the following: “I remember when I was a child of about 7 playing on a snowy hilltop in Nebraska. I was holding onto the lower rail of a wooden fence sliding my feet back and forth creating a slippery spot to create a starting point for sliding down the hill. A young girl I had not seen before came up next to me and smiled. I smiled back and slid down the hill never to see her again.” He said that seeing her smile was a tremendous moment. Add that story to many other things I am not likely to understand anytime soon.

I conclude this short excursion into skepticism and happiness with this simple realization: I know less than I am generally inclined to believe.

Enjoy the happiness that comes with serious and sustained inquiry.

Remarks on the Pace of Technology Change and Learning Expectations

By way of background:

·       I am an editor for a premiere journal in educational technology – Educational Technology Research & Development; we get peer review feedback to authors in 60 days or less and the typical paper that is accepted (about 12% acceptance rate) goes through 3 revision cycles; once accepted, the publisher (Springer) typically has the paper available online with a DOI in Online First within two weeks and the article is formally published in less than a year; this record is not bad for a top tier academic journal, but many authors and readers complain that it still takes too long from original submission to publication (even in Online First), partly because technologies change so fast that the study being reported is more than two years old and the technology involved has since evolved in significant ways.

·       I am an instructional design scholar and researcher; when I started out in this discipline area more than 25 years ago, I was able to keep up with most of the main educational technologies then available; today I am probably only aware of perhaps 25% of the main educational technologies available and able to make productive use of perhaps 10% (optimistic estimates).

·       I was formally trained as a philosopher – my first and only academic love. I regard philosophy as thought in slow motion. I was trained to critically examine seminal writings in philosophy. I often mention some of the short sentences that have captivated my thinking in my classes. For example, there is Wittegenstein’s remark that “we picture facts to ourselves” (“Wir machen uns Bilder der Tatsachen”) in the Tractatus Logico-Philosophicus (2.1). One may stumble across such an innocent sounding sentence and wonder what it means. At first, one might think to oneself “of course we do.” Then a question may bubble up from those thin pages, such as “what is the form of the picture we create internally?” or “what is the relation of the internal picture to that which is pictured?” which is presumably some external reality. One then looks further into the text and into various commentaries searching for clarification. The likely outcome is still more questions, such as how and why did this ability to create internal representations develop or what internal structures and mechanisms are required to support that ability. “We picture facts to ourselves.” Of course. Keep reading. It is all too obvious. But wait. Sometimes I picture to myself things which are not facts – things that are erroneous representations (“I saw Elvis in the grocery store yesterday”) or altogether outrageous (I refuse to elaborate on the grounds that I might insinuate myself) or even impossible … really … is it possible to imagine something altogether impossible, such as a square circle? Well, I do know several very square egg-heads. I can say things that refer to impossibilities, such as “What I am now saying is false” (I used to be a politician) or “Yesterday I suffered a fatal heart attack” or “The day before yesterday I ate the last cannibal” (yes, nonsense can be distasteful) … or, more appropriate for polite company, “There is someone in this room who loves all and only those persons in this room who do not love themselves.” We picture facts to ourselves. Sometimes we also picture to ourselves things that are not facts. How do we distinguish the two? Well, other philosophers have speculated about this, so the journey from this small, innocent sentence takes us perhaps to Descartes’ Mediations on First Philosophy – for many that was their last philosophy course and they never had the pleasure of delving into Wittgenstein. Oh yes, the Meditations – six to be precise. Perhaps those ideas that are clear and distinct are the ones that represent facts. But suppose one has a vivid imagination – that must be you if you managed to read this far. Can one not have a clear and distinct internal representation of something that does not exist or that is erroneous or outrageous? Yes, quite so – that ferocious lion I found in my bedroom yesterday caused my fatal heart attack … that was a very clear and distinct impression … how could it have been misleading? After all, it was the lion’s roar that did me in.

I know … I have tried your patience too much already. You may already have guessed my point. There is occasionally pleasure and insight to be had by going slow and dwelling on one small thing. Just a single sentence, such as “we picture facts to ourselves.” Thought in slow motion. But in today’s fast-paced world of change and innovation, where is there time to pause and contemplate and explore such a small thing? One must keep moving … keep learning new things … keep trying out new technologies … keep on keeping on …

Of course I am not the first one to have such thoughts – not even the first educational technologist to have such thoughts. I am reminded of the following from the opening stanza from T. S. Eliot’s “Choruses from the Rock:”

The endless cycle of idea and action,
Endless invention, endless experiment,
Brings knowledge of motion, but not of stillness;
Knowledge of speech, but not of silence;
…
Where is the wisdom we have lost in knowledge?
Where is the knowledge we have lost in information?

Here is another sentence which lends itself to the thought in slow motion treatment: “I know less than I am generally inclined to believe.” Try that one on for size.

Remarks about the responsibilities of educators and evaluators

In the Introduction to Innovations in instructional technology: Essays in honor of M. David Merrill I introduced what I call the Educratic Oath (Spector, 2005):

(1) do nothing to impair learning and instruction; (2) do what you can to improve learning and instruction; (3) base your actions on evidence that you and others have gathered and analyzed; (4) share the principles of instruction that you have learned with others; and, (5) respect the individual rights of all those with whom you interact. (p, xxxvi)

While this educator’s version of the Hippocratic Oath may seem harmless or simple-minded, I believe it goes to the heart of an educational value system that is all too easily and all too often ignored. Yes, I am admitting that I have on occasion impaired learning and inhibited instruction, and I believe many others have as well. I have done so by expecting my students to see the relevance and value the importance of my subject matter just as I do. That expectation turns out not to be reasonable in all too many cases. Why expect an engineering major to fall in love with philosophy? Is that a reasonable expectation? Probably not in most cases. That unconscious expectation leads to a kind of instructional behavior that is often not conducive to nor supportive of learning. Of course someone who loves to study philosophy will read Descartes’ Meditations on First Philosophy or Wittgenstein’s Tractatus Logico-Philosophicus willingly, identify key points, explore alternative interpretations, elaborate assumptions, explain implications, reflect on central issues, and so on. After all, that is what I recall doing as a student, and so that is what I expect my students to do. As it happens, most of them do not do that, and so I become frustrated and silently regard them as lazy or ignorant or something much worse. I suspect such a pattern exists in how other professors engage with their students. The consequence of this is that I tend to become overly harsh in my feedback and grading, and I unwittingly discourage many students from pursuing further studies in philosophy. Is not this a kind of harm that I am inflicting on students? I believe it is. I believe I am not alone in inflicting such harm. It is quite easy and convenient to regard students as lazy or ignorant simply because they do not care as much about my academic subject as I do. My fundamental error was in having an inappropriate instructional goal. I really wanted my students to fall in love with philosophy when I should have more simply wanted to help them gain some familiarity with a few basic concepts that recur in many philosophical writings and which might prove useful in everyday life. If I had taken that more reasonable approach, I might even have succeeded in attracting a few of them to the dark side, so to speak – the side of philosophy (or thought in slow motion as I often characterize philosophy).

I am curious if others are guilty of such instructional miscues as this. While it is good to be enthusiastic about one’s subject, there is nothing of real value to gain by believing that those who do not share one’s enthusiasm are lazy or stupid. The interim conclusion is that education involves values regardless of the subject with which one is involved. Values are inescapable in education.

This is also true in evaluation. In these brief comments, I want to focus on program evaluation. Most reputable instructional design curricula require a course in program evaluation. Program evaluation in education is aimed at projects (funded or sponsored efforts that have a definite beginning and ending with a specific focus in response to a particular problem situation) as well as at programs (ongoing efforts developed to address problematic situations that are expected to recur or persist). Stakeholders, sponsors, and funding agencies like the concept of an  independent evaluator or evaluation team that will report the extent to which a project or program has or is achieving its intended purpose. This emphasis has led many program evaluators to focus almost exclusively on summative evaluation – the degree to which a project or program achieved its objectives.

While it is reasonable to expect program evaluators to provide that kind of summative report, there is a more fundamental obligation that evaluators have that is all too often and all too easily overlooked – namely the responsibility to inform project and program leaders during an effort when it appears that a bad decision or counterproductive action is about to occur. I am suggesting that the evaluator’s first priority is to help projects and programs succeed. In other words, formative evaluation is the first task of an evaluator.  Simply reporting the outcomes at the end of a project can result in wasted resources, especially when an evaluator is aware of a bad decision or counterproductive action and does nothing at the time to try to improve things. Formative evaluation is aimed at providing feedback that will improve a project or program, and my claim here is that this is an evaluator’s most important obligation. Certainly project and program leaders want such feedback, and certainly sponsors and funding agencies do not want resources to be wasted. Why, then, is there not more emphasis on formative evaluation?

This is analogous to an instructor’s primary responsibility – formative assessment – providing timely, informative constructive feedback to help individual learners improve their understanding and performance. The instructor’s primary task is not to assign grades at the end of the semester (summative assessment). Rather, the instructor’s primary task is to help learners succeed, just as the evaluator’s primary task is to help projects and programs succeed.

While this may all sound obvious, it is not very well aligned with the common practice of many evaluators and instructors. Values are inescapable in education and in evaluation. What we should value are actions that help, that inform, that improve, that are likely to result in success. Otherwise, we can harm learners and contribute to wasteful investments in projects and programs.

References

Spector, J. M. (2005). Innovations in instructional technology: An introduction to this volume. In J. M. Spector, C. Ohrazda, A. Van Schaack, & D. A. Wiley, (Eds.) (2005), Innovations in instructional technology: Essays in honor of M. David Merrill (pp. xxxi-xxxvi). Mahwah, NJ: Erlbaum.

Posted from Sabancı Üniversitesi, Istanbul, Turkey.

What do we really expect from our educational systems?

Three different kinds of things have led me to this question, which I have asked myself many times before. First, there is the ongoing national debate about performance of American students in public schools coupled with a general lack of real concern and support for improving the situation. Second, there are ongoing debates, mostly of a petty and one-sided nature, among faculty in higher education programs concerned with teacher preparation and educational research. Third, there is an ongoing effort by a very forward thinking and dedicated group of people in a Georgia school district with regard to seriously improving learning and instruction throughout that district by focusing on personalized learning and data and research driven approaches. Based on my own work in other school districts, I find this school district surprisingly focused on real and tangible improvements in learning and instruction apart from arbitrary and superficial emphasis on adequate yearly progress and performance on standardized tests, which are used in a punitive manner rather than in a formative manner to improve student learning. With regard to university programs involving teacher training and education, there seems to be ongoing emphasis on advocacy, minimal focus on educational research that is meaningful and generalizable, and an entrenchment of existing programs that makes innovation and progressive change much too difficult. With regard to the national debates about education in America, I can only say that I am embarrassed when I visit other countries that clearly have national support for education and which are far more innovative and progressive in their efforts to educate their children and citizens.

The question I am now asking myself is one I have thought about before, as have many others: What do we want and expect from our educational systems? There are many different kinds of answers to this question, and they vary significantly with regard to implications for teaching and learning. One answer is that we want our children to have basic knowledge and skills in core areas such as reading, writing, and mathematics. That kind of answer tends to lend itself to a focus on testing and to particular kinds of outcomes research aimed at student performance on tests. Other things could be emphasized in such a response, however, including questions about what it means to be literate, what kinds of mathematical knowledge and skills are important, how literacy is connected with critical thinking, and so on. However, those other potential areas of emphasis are considered secondary or even inconsequential in comparison with what so many regard as the bottom line – can our children read, write, and do math (add, subtract, multiply and divide).

A somewhat more sophisticated and nuanced (albeit vague) answer to the question is that we want our children to be effective problem solvers and contributors to a productive society – i.e., we want to prepare students for life in the global economy of the 21^st century. Those who advocate this kind of answer can find support at a national level and in many scholarly publications. There are funding agencies at the national, state, and local level that also support this kind of response, although many still maintain that performance-based evidence on standardized tests provides good evidence of achieving these kinds of goals. I wonder if holding both positions (emphasis on testing and emphasis on 21^st century skills) is fundamentally a sound position. I am not sure if those two kinds of responses are genuinely compatible and consistent, however.

Nonetheless, there is another kind of answer to the question (and probably many more that have not yet occurred to me) – namely, what we really want is to live in a healthy and enriching environment among neighbors and fellow citizens who respect each other and treat others fairly and with dignity. I guess I should say that is what I would like – others certainly may have different visions. In any case, if something like that is how we would like to live, then it would seem reasonable to aim education at such goals. This kind of answer then places values as the primary consideration in educating our children and citizens – above specific knowledge and skills. Of course knowledge and skills are required in order to be a responsible citizen in the sense suggested here, but the ability to solve a system of two simultaneous equations with two unknowns or to write a coherent and grammatically correct thesis statement for an expository document are perhaps not essential to learning how to treat others with dignity and equity.

What knowledge and skills are required to be or become a responsible citizen who respects and deals fairly with others? Certainly beliefs about oneself and one’s place in society are involved in developing such knowledge. If one believes that one’s own desires and needs are more important than anyone else’s desires and needs, then how is such a person likely to develop?

Or, to ask a related but somewhat different question, why is it that so many in America are concerned with their own success and disinterested in others’ lack of success (poverty, unemployment, lack of medical coverage, etc.)? Why do so many American corporations seem focused on their own profits in spite of what happens to their employees and the environment? Why is there so much theft and violence in American society?

We can educate people to deal with violence. We can teach martial arts to our children. Can we educate our children to be non-violent? Can our society do that on a large and sustained basis? Violence in some societies is quite rare compared with America. How are those societies different? Of course some are worse, and it is probably worthwhile to look at differences there as well – we certainly do not want our society to further deteriorate and disintegrate into opposing factions filled with rancor and hatred. We do seem headed in that direction, though, which I suppose is one reason I am wondering what we really want and need in terms of educating Americans.

I do not want everyone or even a majority to agree with me or think like me. I would, however, want an overwhelming majority of my fellow citizens to be able to think clearly, coherently and critically. This involves being able to (a) determine what kinds of evidence are required to support particular conclusions and positions, (b) find and evaluate that evidence, (c) identify and make explicit the assumptions involved in this process, (d) determine the likely consequences of accepting a particular conclusion or position, and (e) reflect on the quality of one’s thinking and confidence in supporting a particular conclusion or position.

In short, I want my fellow citizens to become skeptical inquirers. A skeptic is someone who is not [yet] convinced and who is actively engaged in a search to find out – that is the original meaning of being a skeptic – being a searcher. Inquiry skills can be associated with a skeptical attitude, and it seems to me that we should emphasize those skills in our public schools. Reading and writing will come along as supporting skills rather than as primary skills. A similar argument could be developed with regard to mathematics. Certainly knowledge of statistics and specific statistical skills are needed to evaluate many different kinds of evidence.

Well, I realize this is a rather radical position, although not altogether unlike that proposed by John Dewey in Democracy and Education: An Introduction to the Philosophy of Education (1916). I suppose I am old fashioned in that case. It is radical in the sense that it suggests dropping primary emphasis on basic skills and focusing on something else. It is radical in the sense that it suggests that 21^st century problem solving skills are not primary. Rather this kind of answer to the question says that what matters most are values – shared values – not the values of one or another religious group, but practical, social values to which everyone can relate and which will benefit all.

Lawrence Ferlinghetti is “waiting for a rebirth of wonder and … for someone to really discover America” (From A Coney Island of the Mind, 1958). Perhaps that poem expresses part of what I am trying to say. Meanwhile, I am waiting for the rebirth of skepticism and inquiry. It is my belief that we know much less than we are generally inclined to believe. To be engaged in an inquiry process, one must acknowledge at some level that one does not know but would like to learn more.

Mike Spector

Remarks on human nature and the implications for learning and instruction

As Wittgenstein said in the Tractatus (2.1), we picture facts to ourselves. We create internal representations to understand that which we experience. These internal representations can be considered a kind of model of reality (in some cases, of non-reality, as some people sometimes also picture non-facts to themselves … they are called politicians). We continuously create these internal models, most often without thought and with little conscious effort. It is what we do … it is part of being a person. We are all mental modelers … one and all.

A model is necessarily a simplification of that which is modeled. Our internal representations do not have all of the detail and complexity of the situation or experience that is being modeled. Internal models could not possibly contain all of the detail of the external reality that is being modeled. Another way of saying this is that we simplify reality in order to understand it. This is not an intention or plan or strategy on our part … it is built into the nature of perception, conception, and coming to know and understand our worlds. We are all simplifiers … one and all.

There are many advantages to and even an evolutionary explanation for this ability to create internal representations and simplify external realities (see Sweller, 2003, for more on this). However, a tension arises, particularly for educators, academics, researchers, and those working with very complex, dynamic and ill-structured problem situations. The tension is to help a person avoid oversimiplifying and focus on key aspects of the problem situation in order to respond appropriately. The need to address this tension can be found in Dietrich Dörner’s (1996) The Logic of Failure: Why Things Go Wrong and What We Can Do To Make Them Right. Balancing our natural simplilfication processes (constructing internal mental representations) with the complexity of actual problems is a challenge for educators in many different disciplines.

An instructional design approach that addresses this tension is called model-faciliated learning (Milrad, Spector, & Davidsen, 2003). MFL suggests three stages of learning and developing complex problem solving skill and expertise:

1. problem-orientation (problem confronting and problem solving), in which learners are presented typical problem situations and asked to solve relatively simply problems;
2. inquiry-exploration (hypothesis formulation and experimentation), in which learners are challenged to explore a complex domain and asked to identify and elaborate causal relationships and dominant underlying structures; and,
3. policy-development (decision making rule and global system elaboration), in which learners are immersed in the full complex system and asked to develop rules and heuristics to guide decision making in order to create stability or avoid undesirable situations.

The MFL principle of graduated complexity suggests a learning sequence such as the following:

1. Challenge learners to characterize the expected behavior of a complex system (how the system typically behaves with an indication of how components are interrelated).
2. Challenge learners to identify key variables and leverage points with respect to a desired outcome. 
3. Challenge learners to identify and explain likely causes for observed system behavior, especially in terms of key factors that might be subject to control and manipulation.. 
4. Challenge learners to reflect on dynamic aspects of the system in the context of decision making and policy guides to achieve desired outcomes.
5. Challenge learners to encapsulate learning in terms of a rationale for system structure, decision-making guidelines, and an elaborated strategy for policy formulation.
6. Challenge learners to diversify and generalize to new problem situations. (To assess deep understanding one might ask learners to create a dynamic model relevant to an apparently new problem situation that is likely to have an underlying structure similar to a problem situation already resolved by the learner).

We create internal representations, we simplify, and we occasionally try to understand the complexity of our worlds … it is who we are.

References

Dörner, D. (1996) (Translated by Rita and Robert Kimber). The logic of failure: Why things go wrong and what we can do to make them right. New York: Holt.

Milrad, M., Spector, J. M., & Davidsen, P. I. (2003). Model facilitated learning. In S. Naidu (Ed.), Learning and Teaching with Technology: Principles and Practices (pp. 13-27). London: Kogan Page.

Sweller, J. (2003). Evolution of human cognitive architecture. In B. Ross (Ed.), The psychology of learning and motivation (Vol. 43, pp. 215-266). San Diego: Academic Press.