Wednesday, August 17, 2016
I have been working in the area of assessing progress of learning in complex problem-solving domains for almost 20 years now (not much progress I am embarrassed to admit), and I have also doing evaluations of university programs, grant projects and large European networks of excellence for about 15 years. In both cases, I have come to the conclusion that it is formative assessment (of learners) and formative evaluation (of programs, projects, and products) that matters most. The primary goal is to help students progress and develop and to help programs and projects achieve their intended goals and objectives. Secondary goals include reporting the extent of progress and success.
In addition, I have been speaking at numerous venues on emerging educational technologies and have come to adopt the mantra that it is not about the technology – it is about the learning and instruction. It is not technology that impacts learning and instruction. It is the use of technology that might impact learning and instruction – use by teachers and students, support for effective use by teachers and students, ongoing training for teachers and students in the effective use of technology … it is how a technology is used and integrated into learning activities that makes a difference (when a difference is in fact reported, which is somewhat rare).
One problem that really gets under my thin skin (and skull) is the advocacy for a particular technology as THE solution. If only every student had a laptop … or an iPad … or an iPhone … nonsense. If only every student learned to think critically, to reflect about the problem space and alternative solution approaches, to question assumptions, to monitor progress of learning … if only … then there might be some real impact on learning. Technologies can be used to support those goals (critical thinking, inquiry learning, reflection, hypothesis testing, self-regulation, etc.), but what matters are the processes associated with learning … not specific technologies. Yes, I am a founding member of the national technology geek society … guilty as charged … but I have seen silly uses and implementations of powerful technologies that resulted in no significant difference … so I am now officially a geek drop-out and advocate of the three Rs – reasoning, reflection, and reliability … I am considering founding a new society to be called R3-D3 for the three Rs plus doubting, deliberating and determining … membership is free and open to all.
Tuesday, July 5, 2016
An often repeated statement is that apologizing is a sign of weakness. Another statement is that apologizing does not change what has already happened. Given recent political events, I have begun to wonder if there is any truth to such statements. I begin with one of my favorite poems – Ozymandias by Percy Bysshe Shelley (see http://www.online-literature.com/shelley_percy/672/) in which the following words appear:
‘My name is Ozymandias, King of Kings:
Look on my works, ye might, and despair!’
Nothing beside remains. Round the decay
Of that colossal wreck, boundless and bare,
The lone and level sands stretch far away.
Who is really strong? Hitler might be said to be responsible for the deaths of some 60 million or more people in the period around World War II, two thirds of whom were civilians (see http://worldwar2-database.blogspot.com/2010/10/world-war-ii-casualties.html). In spite of such horrific numbers, there is no Aryan race dominating the world, the Jewish people still survive, and Germany is a peaceful and thriving nation. Perhaps the strong are those who learn from the past, as my older and wiser brother often reminds me. Perhaps there are too few of those people … perhaps due to how history becomes colored by ambition, bias, local interest and a narrow interpretation of facts. Perhaps.
Whom do you consider strong? I consider my parents (may they rest in peace and their tribe increase) as among the strong. My mother was born in a rural log cabin in LA (lower Alabama) and weighed only two and a half pounds at birth. She lived into her seventies in spite of many chronic illnesses, and she managed to raise four children who have become responsible adults (my apologies for the implied self-praise). My father stayed married to her throughout many trials and tribulations for about sixty years (he survived her by only 100 days), and he managed to return to his rabbinical studies and fulfill his lifelong goal of becoming a rabbi late in life in spite of many setbacks. They were strong in so many ways, yet they readily admitted weaknesses and apologized for things they regretted doing on many occasions.
The people I know often apologize for things they have done and later regret (my apologies for stating the obvious). Does that make them weak? I think it reflects a particular strength – namely the strength of knowing their weaknesses and recognizing their missteps. On the other hand, a refusal to never apologize reflects a certain kind of weakness – namely, a reluctance to admit to being human and subject to the imperfections that affect everyone. I occasionally recall some of the statements of O. K. Bouwsma, one of my mentors in graduate school. He noted that the people blamed for polluting a river usually seemed to be those living upstream. Put otherwise, Bouwsma would say that a common problem involved the wrongheaded confidence that one has in one’s own beliefs. It could be the case that one knows less that one is inclined to believe that one knows. It could be the case that while one cannot undo the past that one can do better in the future. It could be the case that we can all become teachers in the sense of Rabbi Spector: the voice that comforts, the ear that listens, the eye that reflects, the hand that guides, the face that does not turn away.
Saturday, May 7, 2016
When Prof. Ren and I wrote a short history of educational technology for The SAGE Encyclopedia of Educational Technology, we decided to look back many centuries rather than a few decades as others have done. In doing so, I noticed that educational institutions and efforts could be categorized around a small number of goals, some of which are interrelated and overlap: (a) develop basic knowledge and skills, (b) develop specific problem solving skills, (c) develop productive workers, (d) develop higher order reasoning skills, (e) develop responsible citizenship attitudes and behaviors, and (f) develop life-long learners. Those goals have been differently emphasized and prioritized in different places at different times. One could make a reasonable argument that over the centuries, there has been increasing emphasis on the last three of those goals. Such an argument might find support in a number of policy statements and documents published in various countries. For example, one might cite the United States 2016 National Educational Plan (see http://tech.ed.gov/netp/), the European Commission’s Strategic Framework – Education and Training (see http://ec.europa.eu/education/policy/strategic-framework/index_en.htm), Taiwan’s Modern Citizen’s Core Competencies Project (see http://english.moe.gov.tw/ct.asp?xItem=15668&ctNode=11449&mp=11), or UNESCO’s Education 2030 – Incheon Declaration and Framework for Action (see http://unesdoc.unesco.org/images/0024/002432/243278e.pdf). There are many more policy documents and strategic education plans that one could find emphasizing higher-order reasoning skills, responsible citizenry, and life-long learning. Caveat lector – reader beware. The truth that some seem to be telling about educational reform and the future of education may only be the truth that’s selling.
Perhaps it is time to take a closer look. I am reminded of Nielsen’s A Game of India (see http://quod.lib.umich.edu/m/mqrarchive/act2080.0011.002/44:7?cite1=game+of+India;cite1restrict=title;g=mqrg;rgn=full+text;view=image;xc=1;q1=game+of+India) in which the main character manages to figure out the rules of a game being played by those who do not speak his language by squatting down and observing the game up close. With regard to education, the focus might then be on primary and secondary school settings. Are children around the world learning to be critical and independent thinkers and responsible social beings? While most adults would want to say ‘yes’ or ‘hope so’, the reality may be quite different. I want to focus on primary and secondary school since not everyone manages to go on to tertiary education and get a college degree, and many of those who do may still be lacking well-developed reasoning skills and the ability to be independent learners and responsible citizens. My thinking is that developing those higher education goals needs to start early in a person’s life to ensure success. That is the reason to focus on primary and secondary education in this case.
In psychology, there has been a distinction between developmental psychology, historically concerned with the physical, emotional and cognitive development of infants and young children, and the psychology of learning, typically focusing on factors (e.g., individual, environmental, social, etc.) and processes (e.g., cognitive, memory, neurological, etc.) that explain and predict persistent changes in behavior and beliefs. As it happens, developmental psychology is focusing more on life-long development and those with special circumstances (e.g., autism, dementia, dyslexia, etc.), and the psychology of learning is focusing increasingly on developmental factors and a holistic perspective of a person, from infancy to old age. If my perception about those two disciplines is somewhat correct, I regard the overlapping research and increased communication between specialists is those areas as promising and healthy … putting Humpty Dumpty back together again.
Unfortunately, such a healthy collaboration is not happening in areas concerned with the planning and implementation of learning environments and instructional systems. There are those who associate themselves with the learning sciences. Others associate themselves with the instructional design community. Still others see themselves as educational technologists or educational computing specialists with only loose connections with either of those two areas. I think such a situation is not likely to result in sustained progress in an applied science domain that can result in large-scale improvements in learning and instruction, and on several occasions I have attempted to improve cross-collaboration with very little success. However, I wish to set aside that concern in the remainder of these remarks and address a somewhat different problem – namely, lack of alignment between education theory, research, practice and policy. The policy statements and documents mentioned earlier can provide one piece of this four-part puzzle – the message being that the four puzzle pieces (theory, research, practice and policy) are not fitting together very well. Lest one accuse me of idle talk, please know that I am also attempting to address that situation through a major online reference work entitled “Learning, Design and Technology: An International Compendium of Theory, Research, Practice and Policy” (see http://www.springer.com/us/book/9783319174600). In the remaining paragraphs, I want to say why I see a lack of alignment in those four education areas.
There are, of course, theories of learning which are often grouped in such categories as behaviorism, cognitivism and social constructivism. There are also theories pertaining to memory mechanisms, instructional design, and more (see http://www.instructionaldesign.org/about.html for a much more comprehensive account of relevant theories). Theories in general are elaborated in terms of principles, which are supported through research and then can explain or predict what might happen and guide developments to improve what has been happening. When there is a reasonable degree of alignment between theory and research, policies can then be formulated to create practices that are systematic and can be sustained with the expectation that desired outcomes will be attained. When practice results in suboptimal or unacceptable results, there is then a need for more research and perhaps refinement of theory and the formulation of new policies. In short, these four areas should influence each other, and there should be an ongoing articulation between them if we really want to talk about the sciences of learning and instruction and make continuing and sustained improvements in education on a large scale. However, I do not see such fluidity among and across those areas.
Here is an example to illustrate the lack of alignment that I see. Recently, there was a three-year effort to develop the next generation science standards (NGSSs) in the USA (see http://www.nextgenscience.org/). When one reads these standards and how they were developed, one is likely to be impressed. There were many so-called stakeholders involved, review cycles, and a framework to integrate engineering into science and a framework that promotes interdisciplinary inquiry and applied practice of core ideas. Consider this 8th grade science standard pertaining to electromagnetism (see http://www.nextgenscience.org/topic-arrangement/msforces-and-interactions):
Ask questions about data to determine the factors that affect the strength of electric and magnetic forces. [Clarification Statement: Examples of devices that use electric and magnetic forces could include electromagnets, electric motors, or generators. Examples of data could include the effect of the number of turns of wire on the strength of an electromagnet, or the effect of increasing the number or strength of magnets on the speed of an electric motor.] [Assessment Boundary: Assessment about questions that require quantitative answers is limited to proportional reasoning and algebraic thinking.]
How might this standard be implemented in an 8th grade classroom? The National Technology Leadership Coalition (see http://ntlcoalition.org/) addressed that question over a three-year period in its annual summit meetings in the years 2013-2015 (NTLS; see http://www.ntls.info/). The group had previously been exploring the potential of transmedia books constructed using 3D printers for support. Because NTLS meets in Washington DC, contacts had been made with the Smithsonian Institute. The first attempt to address the question at the annual summit meeting was based around constructing and testing a motor built loosely according to the electric motor patented by Charles Page and archived at the Smithsonian Institute (see http://3d.si.edu/invention/motor). Four groups of five NTLS participants (all with PhDs in various fields including educational technology, engineering education, science education, etc.) were given access to a 3D printer, general description of the Page motor, and two half-days to build and test a motor following the above NGSS standard. None of the groups succeeded. That caused us to wonder how that standard might be effectively implemented in schools. My observation of the effort was that too much time was spent on building the motor and that did not leave enough time for testing and explaining and doing the science aspects of the standard. As a result, we decided to try implementing the standard in a number of schools in a summer effort using a partially constructed motor, now called an education kit by the Smithsonian Institute. Four middle schools in the area around Washington DC were selected. A summer project with four or five 8th graders involving a technical expert from NTLS, a 3D printer, and the complete patent for and description of the Page motor were conducted. Basically, each group had the task of building and testing the motor in according with the NGSS standard with the understanding that they would be invited to the next NTLS meeting to explain how the motor worked.
At the following NTLS meeting in Washington DC, each 8th grade group was given access to a 3D printer, the Smithsonian education kit, and 2 hours to reconstruct the motor without outside support. All four groups succeeded. Each group then had to demonstrate to the 60 or so PhDs their motors and explain how they worked. This was also successful. NTLS now felt encouraged that with the proper support (e.g., the Smithsonian education kit and a trained teacher) that the effort could succeed in schools. However, we wanted additional verification prior to proceeding with a larger scale effort in schools, so the group again tried repeating what the 8th graders had done with five groups of PhD at a national conference. Each group was given 4 hours to build the motor. Each of the five participants in each of the five groups had a Smithsonian education kit. Is a PhD smarter than an 8th grader? As it happened, none of the groups managed to even construct the motor much less test it. One confounding factor was the fact that each person in every group had an education kit, and that led to a lot of individual work and very little collaborative problem solving with all the focus on building (without success) and no focus on the science of electromagnetism.
As the unofficial evaluator of the effort, I then pointed out the lack of collaboration and the need to address the basic science prior to the construction effort as well as the need to recognize the desire of most persons to be personally involved in building something. The group then proposed the idea of a second education kit – namely one for a solenoid switch which was much simpler and which could be used to help someone understand how an electrical current could be used to create a magnetic field that could then complete a circuit and turn a device on or off. So a second education kit was created and each person could then build a solenoid switch and apply that experience and knowledge of the core ideas to explain how it worked. That activity could then precede a group effort to build and test an electric motor.
The notions of scaffolding (as elaborated in cognitive apprenticeship) and collaborative learning were now being folded into the planning of a large scale implementation effort to support that NGSS standard. The Smithsonian Institute is participating and has since constructed a dozen different education kits in related areas including the solenoid switch, a telegraph system, a generator, an electric motor and more. The National Science Foundation is supporting a large scale study led by the University of Virginia involving different school districts and the ongoing support of the Smithsonian Institute.
That is a long story spanning several years and it is still in process. This effort is just to support one of those NGSS standards. The effort is taking a policy decision (the NGSS standards) and conducting research, building on relevant theories (cognitive apprenticeship and collaborative learning), and involving practical implementations in representative schools (i.e., not just lab schools with close and ongoing support from university researchers). It took three years to develop the NGSS standards. It is taking more than three years to develop adequate support that aligns theory, research and practice for that policy standard. It is still unknown what the impact on learning and career plans will be from schools implementing that and other standards.
I am not trying to criticize the NGSS standards. They are quite impressive and seem to be emphasizing the right things. However, there was clearly not sufficient involvement of representative teachers, although some teachers were involved. It seems to me that there is a tendency for educational scholars to advise policy makers that something about which they feel passionate and about which they believe will transform learning should be implemented on a large scale and as soon as possible. Likewise, there seems to me to be a tendency for educational technologists to believe that a particular technology will transform learning if only it were embraced on a large scale. In short, it seems like educational theorists, researchers, practitioners and policy makers live in worlds apart. They talk to each other and occasionally to some in another area. An occasional visit to a classroom is not sufficient to gain an understanding of what teachers are doing. Conducting a study in a classroom with the support of researchers and technologists does not provide insight into the everyday world of an everyday teacher. It is not a technology that is going to make a difference. It might be the appropriate and effective use of a technology that can make a difference in learning and instruction. It is not a policy that is going to make a difference. It might be the proper support of a policy with teacher training, ongoing professional development and adequate investment in support personnel that might make a difference. It is not a case study nor a randomized controlled research study that will make a difference. However, synthesizing a number of case studies and experiments might be useful in refining theories and informing policy makers.
We need to put Humpty Dumpty back together again. To do so, we need to begin aligning theory, research, practice and policy and keeping the lines of communication and collaboration open in all directions. Given the lack of alignment, it is not surprising that the wealth of educational materials, tools and technologies widely available has had so little impact on learning. We can do better. Not doing better on a large and sustained scale given all the resources now available amounts to a form of education malpractice.