An issue that arises in many tenure and promotion review meetings concerns what should count as credible and substantial contributions in the area of scholarship. Many universities base promotion, tenure and merit pay on three areas: scholarship, teaching and service. Emphasis varies from institution to institution; teaching colleges and universities typically stress quality of teaching over the other two areas; research institutions typically stress scholarship over the other two areas. In all cases, criteria and associated measures are quite important. These remarks focus primarily on the area of scholarship, although there are many important issues to consider in the other two areas.
What is research and what should count as scholarship? Answers vary from program to program and from discipline to discipline. In the performing and fine arts, performances and exhibitions at prominent venues count significantly, as one would expect. In applied programs (e.g., computer and information science, media studies, etc.), the development, evaluation and deployment of innovative programs count significantly. Traditionally, publications in refereed journals and invited presentations at national and international conferences are considered a mark of distinction. In computer and information science programs, ACM and IEEE proceedings are typically considered top-tier publication venues, whereas in educational technology and instructional design research programs, journals such as ETR&D are considered a preferred publication venue.
Issues that frequently arise concern the order of authorship, co-authoring with students, variety of publication venues, whether or not the journal is indexed and refereed, the impact factor of the journal, and so on. In some instances, publishing in online journals arises as a concern. In our discipline, there are reputable indexed, peer-reviewed journals (e.g., Education, Technology and Society).
In addition to online journals, non-traditional publication venues are now available. Examples include blogs, wikis and podcasts. My experience serving on tenure and promotions committees suggests a strong bias for traditional, peer-reviewed publications. Faculty who submit blogs, wikis and podcasts as publications are sometimes viewed as trying to pad their resumes with publications that do not reflect traditional research contributions. Those who write and read blogs and wikis and create and listen to podcasts argue that one often finds legitimate scholarship in those venues that is just as credible as many journal articles. If one accepts that this is sometimes true and likely to increase with time, what is a reasonable approach to recognize new forms of scholarship?
As with traditional forms of scholarship, there ought to be reasonably well-defined criteria and guidance for faculty. I believe that many institutions lack well-defined criteria and guidance for the traditional forms of research and scholarship. New forms of scholarship only make the task of recognizing scholarly contributions more challenging.
What is research? My sense is that research involves a systematic process of investigating a problem and answering questions where existing knowledge falls short or does not exist. Those who conduct research, then, recognize a gap in knowledge – their own knowledge and that of others – and are open to alternative approaches and answers to unresolved questions. This view draws a distinction between advocacy and research, although the boundaries are admittedly fuzzy and advocacy at its best relies on research and evidence. In addition, openness to alternative approaches recognizes the value of peer review. Double blind peer review in journals such as ETR&D is considered a critical part of assessing research significance. When reviewers critique a journal article without knowing the author or the other reviewers, and when authors revise and respond, again not knowing the reviewers, the presumption is that awareness of prior work and alternative approaches and explanations has occurred.
While a blog may allow for comments, anonymity is lost, which means that comments might be biased; in any case, the comments left by readers of blogs are certainly selective. While a journal editor selects the reviewers for a submission, the reviewers are generally obligated to offer a fair and detailed critique. A blog is more akin to comments presented to a class or at a meeting and quite different from a research paper critiqued by peers for a journal. My sense is that blogs should not be considered in the same category as refereed journal publications. This does not imply that blogs should not be counted at all. Rather, for programs that stress new forms of information and communications technologies, blogs, wikis, and podcasts may well count as a form of scholarship. However, blogs are more akin to presentations than to journal articles. Various programs should then decide how much significance to place on presentations and publications and other forms of scholarship (e.g., the creation of innovative instructional programs, learning environments, etc.). If one wants to create a legitimate place for blogs, wikis, and podcasts in the context of scholarly contributions, I believe it does not make sense to argue that they should count just like refereed journal publications, as that tactic is likely to meet serious resistance and create unnecessary disputes.
It does seem reasonable to argue that new technologies are changing the nature of research and scholarship. New ways of conducting research are available. Powerful tools for planning, implementing, disseminating and evaluating research are available. Online journals are becoming increasingly prevalent, and many good papers have been published in online journals. What we should not lose, however, in our rush to integrate technology into research and scholarship is the value of peer review. I suppose this puts me on one side of the debate of what to count as a significant research contribution in this area of networked technologies and social computing. Actually, there are probably many sides to this debate. Regardless of how this debate evolves, I hope that we do not lose sight of the distinction between advocacy and research, that we continue to value peer review and recognize that research essentially involves a willingness to be shown that one is wrong (see Popper’s Conjectures and Refutation), and that we find reasonable ways to encourage faculty and graduate students to engage in a variety of scholarly activities so as to ensure that knowledge is cumulative and relevant. I worry that fascination with new technologies may result in losing sight of the many relevant contributions from prior generations of technology. I worry that fascination with new technologies may result in devaluing the notion of the accumulation of knowledge – this amounts to a devaluation of science and engineering, in my less than humble opinion. I also worry that over-emphasis on traditional forms of scholarship will disincline many bright young scholars from pursuing careers at some of our best research institutions. I worry too much (I have that on the good authority of my better half).
Sunday, February 21, 2010
Sunday, February 7, 2010
School Improvement and the National Will
We have heard for years about the sad state of America’s schools, including disappointment about the poor performance of students on standardized tests and reports of poor teaching and teacher preparation. You’ve heard the stories – Johnny can’t read, Janey can’t add, Joey thinks New Mexico is a country, Johelen thinks cosmology is about make-up, Jimmy thinks that Barak Hussein Obama is a terrorist, and their teachers read at an 8th grade level
Not long ago (October 22, 2009) the U.S. Secretary of Education, Arne Duncan, said that colleges of education need to make dramatic changes to prepare children to compete in the global economy (http://ed.gov/news/pressreleases/2009/10/10222009a.html). He argued that teacher-preparation programs should ensure that new teachers master the content of the subjects they will teach; in addition, teacher education programs should have well-supported field-based experiences. Duncan went on to say that the ultimate goal of teacher preparation programs should be to create a generation of teachers who are focused on improving student achievement and ready to deliver on that goal (as if that is not the current goal). Such comments are just as insensitive as Duncan’s offhand remark that Hurricane Katrina was the best thing to happen to the New Orleans’ education system. Duncan later apologized for the Hurricane Katrina comment, but he does not appear to see that his recent comments about pre-service teacher preparation programs and in-service teachers ignore a great deal of evidence and are offensive to the many dedicated teachers working in schools and university professors devoting so many hours and so much effort to prepare new teachers for careers in schools.
There are problems with American schools, no doubt. The American education system is complex. Surely it can be improved. Surely it will not be improved by insulting those already working hard on school improvement. Many states and college programs already embed field experiences throughout the curriculum and require teachers to be competent in a subject domain. Of course more can be done, but doing the right thing and doing it right will require more than offhand remarks made in a short speech. Remaking the world of American education at large – everywhere and instantly – is not possible nor would it be wise were it possible. Basing change on the biases and beliefs of a few persons who happen to be in positions of influence is likely to generate new problems. It would be a good idea to be clear about the fundamental problem and likely causes prior to fixating on a solution.
So, what is the fundamental problem? Is it that students are not performing well on standardized tests? Is it that teachers do not stay with the profession very long? Is it that some university programs admit and pass almost anyone regardless of skills and knowledge? I believe those things belong in the category of symptoms – and any and all of those symptoms should be carefully examined based on the evidence. Those symptoms (assuming that there is adequate supporting data supports) are like the aches and pains one feels when suffering from an unidentified ailment. I worry that what appears to be happening with the health care reform effort will happen to the various school reform efforts. With regard to health care reform, we seem to have lost track of the problem, and the debate and discussion are now focused on various aspects of symptoms and solutions. The problem with American health care is not that insurance companies charge too much nor is it about whether or not health care should be provided by an employer or the government. The problem is that Americans are getting sick and dying – at alarming rates compared with other developed nations. It may be convenient to consider one’s own health and medical insurance and feel satisfied that all is okay. However, that is not a systemic view appropriate for policy formulation and large-scale planning and decision making.
From a systems perspective, all is not well with American health care, nor is all well with American schooling. As should be the case with health care, the first step is to be clear about the problem. Too many people are getting sick and dying. That can be established empirically and investigated rigorously – although that does not appear to be happening. Nevertheless, we ought to take a similar attitude with regard to improving American schools. The first step is to state the problem(s) clearly. It simply is a mistake to fixate on standardized test scores. At best that is a symptom.
What then is the problem? Is it that America is losing its competitive edge in the global marketplace? That was also suggested in Duncan’s remarks, and it appears in many of the education grants sponsored by the National Science Foundation and the Institute of Education Sciences. I wonder if this is an adequate description of the problem. Perhaps it is just another symptom. Regardless, we need to insist that our leaders provide a clear and defensible statement of the problem prior to going forth with solutions, especially those linked to test scores. If one accepts the loss of competitive edge, one could then look at what other countries have done when confronted with a similar situation. The examples might include Ireland, Finland, South Korea, and Japan. In those cases, there was a change in national policy placing a great deal more emphasis on education at all levels. The will of the people was well aligned with increased emphasis and spending on public education, even though people realized it would cost a great deal and would take many years to realize any noticeable gains. Gains were realized, though. In none of those cases was the focus on test scores, although test scores did improve dramatically. The focus of decision making and policy formulation was on providing proper support for education at every level. That required more funds – increased taxes – and a great deal of patience.
I mention these cases because there seems to be no real interest in increasing support for education in this country. We seem all too complacent with high rates of illiteracy, high drop-out rates from our high schools, and high rates of attrition among our teachers. We do not seem to be willing to provide more support. We seem to believe that by simply changing standards that the problem will be fixed. This seems all too unlikely. Too many American children are getting sick of school and dropping out of the competitive marketplace. Too many teaches are leaving the profession. Things are getting worse, and focusing on test scores does not seem at all related to getting at the underlying problem – the lack of national will to take pride in and properly support education in American.
I would recommend that our leaders talk with AECT’s Future Minds group to gain an appreciation for a systems perspective on school reform and improvement. I would like our leaders to look next at what the National Technology Leadership Coalition (NTLC) has to say about effective integration of technology in supporting education. Most of all, I would like our national leaders to show some leadership and tell us how long it will take and how much effort will be required to keep our children from getting sick of school and dying intellectually. It is also about time that we lifted the ban on talking about tax increases and talked about what we want for all Americans and what we are willing to pay to make that happen.
Not long ago (October 22, 2009) the U.S. Secretary of Education, Arne Duncan, said that colleges of education need to make dramatic changes to prepare children to compete in the global economy (http://ed.gov/news/pressreleases/2009/10/10222009a.html). He argued that teacher-preparation programs should ensure that new teachers master the content of the subjects they will teach; in addition, teacher education programs should have well-supported field-based experiences. Duncan went on to say that the ultimate goal of teacher preparation programs should be to create a generation of teachers who are focused on improving student achievement and ready to deliver on that goal (as if that is not the current goal). Such comments are just as insensitive as Duncan’s offhand remark that Hurricane Katrina was the best thing to happen to the New Orleans’ education system. Duncan later apologized for the Hurricane Katrina comment, but he does not appear to see that his recent comments about pre-service teacher preparation programs and in-service teachers ignore a great deal of evidence and are offensive to the many dedicated teachers working in schools and university professors devoting so many hours and so much effort to prepare new teachers for careers in schools.
There are problems with American schools, no doubt. The American education system is complex. Surely it can be improved. Surely it will not be improved by insulting those already working hard on school improvement. Many states and college programs already embed field experiences throughout the curriculum and require teachers to be competent in a subject domain. Of course more can be done, but doing the right thing and doing it right will require more than offhand remarks made in a short speech. Remaking the world of American education at large – everywhere and instantly – is not possible nor would it be wise were it possible. Basing change on the biases and beliefs of a few persons who happen to be in positions of influence is likely to generate new problems. It would be a good idea to be clear about the fundamental problem and likely causes prior to fixating on a solution.
So, what is the fundamental problem? Is it that students are not performing well on standardized tests? Is it that teachers do not stay with the profession very long? Is it that some university programs admit and pass almost anyone regardless of skills and knowledge? I believe those things belong in the category of symptoms – and any and all of those symptoms should be carefully examined based on the evidence. Those symptoms (assuming that there is adequate supporting data supports) are like the aches and pains one feels when suffering from an unidentified ailment. I worry that what appears to be happening with the health care reform effort will happen to the various school reform efforts. With regard to health care reform, we seem to have lost track of the problem, and the debate and discussion are now focused on various aspects of symptoms and solutions. The problem with American health care is not that insurance companies charge too much nor is it about whether or not health care should be provided by an employer or the government. The problem is that Americans are getting sick and dying – at alarming rates compared with other developed nations. It may be convenient to consider one’s own health and medical insurance and feel satisfied that all is okay. However, that is not a systemic view appropriate for policy formulation and large-scale planning and decision making.
From a systems perspective, all is not well with American health care, nor is all well with American schooling. As should be the case with health care, the first step is to be clear about the problem. Too many people are getting sick and dying. That can be established empirically and investigated rigorously – although that does not appear to be happening. Nevertheless, we ought to take a similar attitude with regard to improving American schools. The first step is to state the problem(s) clearly. It simply is a mistake to fixate on standardized test scores. At best that is a symptom.
What then is the problem? Is it that America is losing its competitive edge in the global marketplace? That was also suggested in Duncan’s remarks, and it appears in many of the education grants sponsored by the National Science Foundation and the Institute of Education Sciences. I wonder if this is an adequate description of the problem. Perhaps it is just another symptom. Regardless, we need to insist that our leaders provide a clear and defensible statement of the problem prior to going forth with solutions, especially those linked to test scores. If one accepts the loss of competitive edge, one could then look at what other countries have done when confronted with a similar situation. The examples might include Ireland, Finland, South Korea, and Japan. In those cases, there was a change in national policy placing a great deal more emphasis on education at all levels. The will of the people was well aligned with increased emphasis and spending on public education, even though people realized it would cost a great deal and would take many years to realize any noticeable gains. Gains were realized, though. In none of those cases was the focus on test scores, although test scores did improve dramatically. The focus of decision making and policy formulation was on providing proper support for education at every level. That required more funds – increased taxes – and a great deal of patience.
I mention these cases because there seems to be no real interest in increasing support for education in this country. We seem all too complacent with high rates of illiteracy, high drop-out rates from our high schools, and high rates of attrition among our teachers. We do not seem to be willing to provide more support. We seem to believe that by simply changing standards that the problem will be fixed. This seems all too unlikely. Too many American children are getting sick of school and dropping out of the competitive marketplace. Too many teaches are leaving the profession. Things are getting worse, and focusing on test scores does not seem at all related to getting at the underlying problem – the lack of national will to take pride in and properly support education in American.
I would recommend that our leaders talk with AECT’s Future Minds group to gain an appreciation for a systems perspective on school reform and improvement. I would like our leaders to look next at what the National Technology Leadership Coalition (NTLC) has to say about effective integration of technology in supporting education. Most of all, I would like our national leaders to show some leadership and tell us how long it will take and how much effort will be required to keep our children from getting sick of school and dying intellectually. It is also about time that we lifted the ban on talking about tax increases and talked about what we want for all Americans and what we are willing to pay to make that happen.
Subscribe to:
Posts (Atom)
