International interest is moving towards developing better, and expecting more of, assessment practices. A new emerging agenda in teacher education is that the primary goal of assessment should be educational. The implication here is that unlike most of past and current practice, assessment should be in partnership with the two other aspects of curriculum, namely, content (syllabus) and pedagogy. Assessment practices should address higher goals of learning and provide advice to the teacher (and learner) about what is known and where instruction might most profitably head. This paper explores the experiences of a group of secondary and primary mathematics and science teachers as they begin to incorporate this challenging future role of assessment into their classrooms. In particular, the focus is on the reactions of three teachers to the year-long development-based assessment project providing a longitudinal perspective on teachersí insights into the way in which their views evolved.
One most challenging feature in teacher education as we move into the 21st century concerns the implementation of more appropriate assessment practices in the classroom (Black & William, 1998). ì(N)ot the kind of assessment used to give grades or to satisfy accountability demands of an external authority, but rather the kind of assessment that can be used as part of instruction to support and enhance learningî (Shepard, 2000, p.4). This emerging agenda challenges much of current (i.e., traditional) assessment practice (Linn, 2000).
Traditional assessment is predominantly quantitative. It concerns applying norm-referenced, summative approaches to testing in order to encapsulate a studentís ability in a single score. The results of this form of assessment practice are best used for certification and selection purposes. This suits some stakeholders, such as Governments and employers, as it provides rank orders of students on ëability.í Nevertheless, the procedures and results provide little insight into what students know, understand and can do. As such, the approach is not particularly helpful for teachers or learners as it provides only superficial information for teachers (and learners) in terms of teaching and learning directions that may best suit students.
This last point highlights the main change between the old and new assessment agendas. The new assessment agenda is about having a ìconstructive alignmentî (Biggs, 1996) of the three arms of curriculum: assessment, curriculum content, and teaching practices.
An important reason for this movement is to avoid the assessment process distorting curriculum aims. It is a truism that what is assessed and how it is assessed provides the clearest indication of what is valued about formal education (Cole, 1990). Hence, we usually find curricula emphasising higher-order goals such as analysis, synthesis and evaluation, and in contradiction ìcommonly used achievement tests stress(ing) simple skills rather than sophisticated tasks, not because such skills are more important, but because they are easier to measureî (National Research Council, 1989, p.69).
Operationalising the new assessment agenda has been a significant part of the research effort carried out within the Centre for Cognition Research in Learning and Teaching at the University of New England. There are two foci to this work. The first is on developing valid and reliable approaches to assessing the quality of student understandings as exhibited in classrooms. Critical to this approach is a meaning for the word quality. The second involves planning and implementing procedures to assist teachers develop ownership of the new assessment practices so that they are better informed of their studentsí understanding, how they might better sequence their teaching, and where instruction might most profitably be directed.
The meaning of quality in student understandings adopted in the research was enunciated by Biggs and Collis (1982, 1991) in the development of the SOLO (Structure of the Observed Learning Outcome) model. They believe that there are natural progressions in the growth of student learning and these have similarities with the stages of Piaget. More importantly, they identified generic cyclic learning patterns, based on the underlying structure evident in student responses. This then offered the basis of a cognitive developmental continuum. A brief discussion of SOLO is provided below.
The professional development project, discussed in this paper, concerns teachers applying SOLO as a framework to underpin new assessment practices in the classroom. The overarching aim of the project was to empower teachers to apply theoretically-based knowledge concerning the structure of studentsí understanding so that their assessment and teaching practices could become better aligned. The initial focus of the program was, however, assessment based, with each teacher establishing the language, understanding and skills necessary to apply SOLO within their classes.
This paper provides an overview of the reactions of teachers to the development-based assessment project. Of interest are their reflections of the professional development process, gathered over the period of a year.
As indicated above, this project concerned teachers learning about and applying the SOLO model of cognitive development (see Biggs & Collis, 1982, 1991) in mathematics and science classrooms. Overall, SOLO is particularly relevant for assessing achievement in any learning setting. It is both hierarchical and structural in nature. It is a model for interpreting the responses of students in terms of structural characteristics and these are then used to determine the level in the hierarchy represented by the particular response. The model is concerned with specifying ëhow wellí (qualitative) something is learned rather than ëhow muchí (quantitative). This distinction is particularly important in school subjects, such as mathematics and science, where demand in recent times has begun to place strong emphasis on understanding applications and developing problem-solving skills.
The SOLO assessment technique is at the opposite end of the continuum to traditional assessment practice. It is qualitative in nature, representing a departure from a focus on testing for skills, facts and learnt algorithms or procedures. It is in the area of measuring the quality of cognitive assimilation in terms of progressive structural complexity that SOLO has its main strength. However, SOLO is not only useful for response categorisations, but is capable of giving a dynamic model of developing conceptualisations (e.g., see Campbell, Watson, & Collis, 1992; Collis, Jones, Sprod, Watson, & Fraser, 1998; Panizzon & Pegg, 1997; Pegg, 1997).
In addition, the SOLO framework has also been adopted internationally as a valuable tool for research into: studentsí understandings in many specific topic contexts (e.g., Coady & Pegg, 1996; Cuthbert & Pegg, 1993; Pegg, 1995; Reading & Pegg, 1996; Watson, Collis, & Campbell, 1994; Watson, Collis, Callingham, & Moritz, 1995); curriculum development initiatives (e.g., Pegg, 1995); and, ways to assist teachers with assessment issues in their classrooms (e.g., Collis & Romberg, 1991, 1992). There are now possibly more than two hundred publications covering a diverse range of areas as the small sample above indicates.
SOLO, development is described in two ways. First, it depends upon the nature or abstractness of the task/response (referred to as the mode) and, second, it is based on a personís ability to handle, with increased sophistication, relevant cues (referred to as the level of response). There are five modes of thinking and these are referred to as sensori motor, ikonic, concrete symbolic, formal and post formal. Responses within each mode become increasingly complex as the cycle of learning develops and are described in terms of levels using the same generic terms for each mode. Prestructural responses represent no use of relevant aspects of the mode in question; unistructural responses represent the use of only one aspect of the mode; multistructural responses represent several disjoint aspects, often in sequence; relational responses represent an integration of the elements identified previously.
The strength of the model is the linking of the cyclical nature of learning and the hierarchical nature of cognitive development. Each level of functioning within a cycle has its own integrity, its own idiosyncratic selection and use of data. Nevertheless, it serves to provide building blocks for the next higher level. A response more advanced than the relational level in a mode marks the transition to a new mode.
The program involving teachers using SOLO in the classroom we refer to as Developmental-based Assessment (DBA). This form of assessment has the means of interpreting studentsí responses within a framework of cognitive developmental growth, thus allowing teachers (and students) to place learners along a developmental continuum and, at the same time, provide advice to the teacher on possible pathways for future teaching endeavours.
This approach has clear appeal for many educators, especially if one takes the philosophical stance that a primary purpose for assessment is to provide support for teaching techniques that can improve learning. This can occur:
In DBA it is the mental structure of the understanding that is important, and the criteria are merely examples which are typical of the types of levels of performance. They represent examples that highlight key underpinning principles of the topic being studied. The exact nature of these examples could vary depending on different learning experiences and activities or on the background experiences students bring with them. This supports the notion of social and contextual constructivism in which studentsí understandings are influenced by their learning environment (e.g., Ernest 1992), as well as language impasses related to learning experiences (Zevenbergen, 2001). From recent research completed in this area (Panizzon, 1999) there is convincing evidence that the SOLO model, because of its reliance on the underlying structure of a response, is able to accommodate these differences.
The project was devised to trial and evaluate teachersí applications of developmental-based assessment techniques in their own classrooms. This focus arose directly from the changing assessment practices required to satisfy the requirements of new syllabus documents introduced into New South Wales for mathematics and science subjects in 2000. Funding for the professional development program was provided by the Federal Government through its Australian Quality Teacher Project (QTP).
Specifically, it was anticipated that the project would:
In developing the design for the two-year project, three aspects were considered as a means of improving the effectiveness of the professional development process for teachers. First, the project represented a collaborative effort between the Centre for Cognition Research in Learning and Teaching (CRiLT) at the University of New England and the Catholic Schools Office in the Diocese of Armidale. This link was strengthened further with the inclusion of a person from the Catholic Schools Office to act as a link between the staff in schools and the university. The inclusion of support from higher levels of administration, as achieved in the project, was deemed by Zuber-Skerritt (1992) to be crucial as a means of enhancing the effectiveness and the long-term outcomes of a professional development process.
Second, the project included a cross-section of teachers from both primary and secondary sectors. The inclusion of both groups of teachers, although unusual in professional development programs in NSW, was a deliberate aspect of the design of the project. The program provided a forum for the two groups of teachers to share insights about assessment of students. This is especially valuable given the overlap evident in understandings of many students in the senior primary years and the early secondary years.
While primary teachers are not specialists in mathematics and science, they are expected to teach both key learning areas. In terms of the project, their contribution was considered invaluable given the insights they provided in relation to assessment practices and learning needs of younger students. Similarly, the inclusion of secondary mathematics and science teachers allowed primary teachers to gain a deeper appreciation of the kinds of content and assessment being undertaken at the secondary level and the changing expectation of teachers in this regard.
Finally, the design of the project ensured that the process of professional development was an active one thereby allowing teachers to negotiate the kinds of activities they preferred to undertake. Subsequently, teachers identified the problems and issues regarding assessment they were keen to explore, giving them a sense of ëownershipí in the project. Furthermore, they were able to work with their students within their normal classroom environment utilising the knowledge acquired from their involvement with the project. The need for ownership has been found to be one of the most crucial components of ensuring the commitment and likely success of educational professional development projects (Clarke, 1994; Ramsden, 1992; Zuber-Skerritt, 1992).
Seventeen teachers, representing five schools, volunteered to be included in the project (see Table 1). Schools 1, 2, 4, and 5 were rural catholic schools with School 3 being a coastal catholic school. All of the schools were under the jurisdiction of the Armidale Diocese. There were at least two teachers working in the same area (e.g., secondary science) within each school. This structure ensured that teachers did not feel isolated when they returned to the school and were able to work with at least one other member of staff. It also allowed teachers in the same discipline area to share resources.
|
|
|
|
|
School 1 |
|
|
|
School 2 |
|
|
|
School 3 |
|
|
|
|
|
|
|
|
School 4 |
|
||
|
School 5 |
|
|
The project funding was for two years. For 2001, the need for a series of two-day workshops was identified. Each of these was conducted at the University of New England thereby enabling participants to be removed from the normal work environment. In addition to these workshops, each school was visited by members of the University providing teachers with the opportunity to demonstrate their work within their own teaching context. Funding for the project provided release time for teachers to be covered during their absence from their school and for opportunities for teachers to have some time to devote to the project during normal working hours. An overview of the program is provided in Table 2.
The first year of the program was designed to be flexible, enabling teachers to make contributions to the workshop sessions. To achieve this but still provide adequate direction, the initial workshop was fully organised and prepared for presentation based around new approaches to assessment and the theoretical framework of the SOLO Model.
|
Month |
Instructional focus |
|
February |
2-day initial workshop at UNE. Introduction to SOLO Model and issues relating to assessment identified |
|
March to May |
Constant access to expertise in order to maintain momentum of the aims of the project and facilitate teacher growth through the concepts covered at the initial 2-day meeting in February |
|
June |
Interim 2-day workshop at UNE. Review of the model. Report back on work undertaken with students in situ |
|
August to September |
On site visits to each of the schools and faculty groups involved to discuss concerns, programming, and assessment, particular to each teaching staff |
|
November |
End of year 2-day workshop at UNE to discuss progress to date. Each of the groups of teachers reported back on the success and failures they had with assessment and their students |
|
Planning for 2002 based on participant experiences of the first year program |
At the culmination of the first workshop, teachers were given the opportunity to provide input in two areas. First, they identified the kinds of activities they would explore with their students during the March-to-May period. Second, they specified the types of input they required for the June workshop. For example, one team from a school, close to the University, asked for a series of monthly two-hour after-school sessions. These needs were clarified and refocused as the teachers became more familiar with the program and SOLO. Similarly, teachers were able to select the best course of action for the August-to-September period by negotiating with the project team. In this manner, teachers were given a high degree of ownership in terms of the direction of the project, session input and individual group focus.
At the final workshop meeting in November, all participants along with the project team decided on individual school plans for 2002. This was deemed to be the best course of action in that it met the particular needs of teachers in specific schools. As a consequence, each of the four schools has a distinct program for 2002 under the direction of the project team.
In addition to the flexibility of the design, as discussed above, ongoing consultative help was available for the teachers in between workshop sessions. This was provided in a number of ways including the help of a part-time project officer, phone calls to teachers in schools at specified times, and ongoing electronic access to the project Web page.
Overall, every attempt was made to have the project sensitive to and reflective of the wishes and needs of the participants. Within the constraints of the budget, every attempt was made to respond to teacherís request for support. Requests were in the form of money for teaching relief, more information and advice through articles and Web materials, discussions with peers from other schools through telephone, emails and chat rooms on the internet, and school visits by members of the University team.
There were two primary sources of data collection. These were: (i) teacher perspectives and experiences with DBA, and (ii) student scripts coded using the SOLO model to inform developmental pathways of knowledge acquisition within topic and content areas. This paper is concerned only with data collected on teachersí perspectives of the professional development process.
Data were gathered during each of the workshops from two different sources. Initially, teachers completed compulsory questionnaires developed for evaluation purposes by the QTP organisers. The Likert Scale questions related to participant satisfaction with the professional development activity; the degree of learning gained from the experience; and, the extent to which participants believed that their behaviour was modified or influenced by the professional development experience. Supplementing these questions was an optional question asking teachers to provide further comment on their experiences. In addition, each of the workshop sessions was taped and later transcribed. This had the approval of all participants. As different component parts of the workshop sessions were highly interactive, this provided a rich pool of qualitative data.
To provide an overview of the reactions of teachers to the development-based assessment project, three case studies are presented to reflect the range of views of teachers participating in the project. The case studies provide a longitudinal perspective allowing insights into the way in which each teacherís view changed or evolved during the first-year of the project.
The three case studies represent a primary teacher and two secondary mathematics teachers. Each of the case studies is presented in three sections. The first provides the teacherís reflection at the end of the first workshop in February 2001. The second represents a perspective before the individual school visits in August and September. The final section gives insight regarding how the teachers felt by the end of the first year of the project, and possible aspects they would like to pursue during the second year. Because of space limitations, only the comments made on the survey documents are quoted below. Also, because of ethics requirements that teachers not be able to be identified, references to the specific gender of each teacher have been removed.
Case Study 1
Teacher A is an experienced primary teacher with a substantial number of years of teaching experience at the same school. The teacher stated that his/her interests lie in the development of basic skills in students. Teacher A saw his/her role as establishing a firm basis that would serve students well for learning throughout their entire school life. The teacher also felt that it was important to build up self-esteem in students. This teacher provided no written comment in the first workshop session transcribed for data analysis.
After six months
"Our biggest problem was time.""It was hard to get together and codeÖand write questions to get a variety of responses. The crowded curriculum is overwhelming, and it makes it hard to cover all the bases; often other things take priority. At the beginning, the terminology of the model was unclear, there is a need for a basic model for beginners."
"Can we have a chart with clear definitions on hand to help write and code? Our afternoon sessions at UNE made it easier. Now I feel more confident"
"It will help as a reporting tool, with the justification aspect."
After one year
"The acquisition of new skills has been an even slower process, and I did not feel, take place until later in the year. [Writing questions] took some time, and I did not get time to do coding on all questions attempted. The broader the question the harder it was to code.""I was 'softerí' on my kids, so it was good to be working with others who could be more objective. In the end we all pretty much agreed on where to code each response. On the few occasions we argued, we all understood because we all used the same language."
"Interaction with peers helped me to feel more comfortable with coding, I struggle on my own. It has changed my perspective on some of my kids; the poor kids can actually do better than I thought. Knowledge of SOLO is a powerful tool."
"Coding responses to a particular unit can change/improve how it is taught the next time. Next year I would like to refresh and renew what I have learnt this year, and develop a more informed understanding. Next year I would like to have more time to discuss the process with other teachers from my school and other schools. Can we have further half/day sessions at UNE next year, and more release time."
Case Study 2
Teacher B is an experienced mathematics teacher at a relatively large secondary school. The teacher is a member of a staff of approximately ten teachers. Teacher A exhibits a passion for the subject and for the well being of students. The teacher shows concern for the mathematics enjoyment level of students in his/her classes, implementing novel and creative questions.
After the first workshop
"I really enjoyed yesterday. It has given many insights into the frustration of teaching a different generation of students to the ones I first started teaching over 20 years ago. The youth of today have far different demands and I am confident that this course will help me to meet these demands. I would benefit form some quiet study of the material presented to lower my ëworking memory.í What are we going to do with this knowledge when we walk into a classroom tomorrow?"
After six months
"I trialled some questions. I used some questions to help with Year 7 grading at the end of Term 1, but didnít get around to coding them properly. I trialled some questions in the Year 8 exam, and some of the other teachers commented on the ëdifferent style.í They were concerned that some of their ëbrightí students might not do so well. In the end, the brightest students did not necessarily give the highest responses.""Can we have a visit from the Project Team to [our school]? Can the [successfully coded questions] be put on the web? How can I use [this knowledge] to grade my students successfully? I am applying SOLO to different subjectsÖeven RE (Religious Education). Now I need to learn how to give the students more time in the classroom to progress through the levels."
After one year
At the final meeting of the project for the year, Teacher B reported that he/she had devised and tried many questions with various classes. The teacher seemed to love sharing successes and failures with other participants in the project.
"gave a question about how many ping-pong balls would fit into my classroom. I gave no other instructions. The students were really motivatedÖ Some didnít even stop for lunch. They went about it in so many different ways. One girl took pages of words to explain her answer. One boy just gave some calculations. They both ended up with the same answer.î"It was helpful having the project team come to [our school] to help code these. SOLO helped me to reconcile both different types of responses. Other teachers are asking, ìWhy are students being asked to explain more?î They are curious; they see a change in my students (attitude). I have a different atmosphere in my classroom. It has changed the way I teach different topics. I now spend ëreal timeí preparing lessons. I want to be kept up to date next year, regular email contact? Can I involve the rest of my staff?î
Case Study 3
Teacher C is a qualified mathematics and physical education teacher and has taught in urban and rural high schools. The teacher has some experience as a head teacher in both of these subject areas. Teacher C sees it as important that all students in his/her mathematics classes experience success, and nurtures their self-esteem from the first time they enter his/her classroom. Currently, the teacher has a young family that takes up most of his/her out-of-school time.
After the first workshop
"Please provide some of the sample questions you mentioned for us to try in our classrooms. At the next meeting can we all share tasks we have developed, and see how SOLO matches Year 10 assessment demands, and how it can help with reporting."
After six months
"My questioning style has improved in order to draw information from my students. I was involved with marking [commercial tests]. They mainly ask students [for facts], not deeper understandings. I tried some text book and published questions, but they were to ambiguous to get SOLO responses in coding responses [at the workshops].""There is too much pressure to get the same coding result as you. I am anxious to apply this more in the classroom. I want it to help grade students. We need deadlines at regular intervals. We need a package of sample questions. I want some questions to try, can we take some home? Can we use school visits to code things?"
After one year
"I tried a question on factorisation because I could see a problem in class. Something wasnít ëclicking.í The results showed a gap in their understanding, so I retaught that section. I have used more guided questioning. Iíve used questions to gain a deeper insight into studentsí understandings. Iíve tried to spend some time on matching exams to the Course Performance Descriptors. Can we have an item bank of questions? Can we liaise with other schools, especially for item bank questions?"
The three teachers reported very different views. These differences reflected their current positions in their school, the influences on their life outside school, and on their perceived role of what it meant professionally to be a teacher. Teacher A was willing to spend time after school because he/she enjoyed the sharing of ideas and the work with colleagues. Nevertheless, Teacher A was most at ease if the program of expectation was developed formally and set by the university team with him/her making appropriate contributions. Teacher B was willing to do additional reading and spend his/her ëowní time in learning the new information and applying it to his/her classes. Teacher B thrived on the new challenge and the new opportunities the program of professional development opened. Teacher C was most insistent that there be direct relevance to his/her classroom practice, and that there should be ongoing materials made available for use with his/her classes.
All teachers felt they needed more time (within normal working hours) from their classes to spend on the project. Nevertheless, they were cognisant of the need for a balance to be maintained between the teacherís workload, the project team workload, funding available, and school restrictions on time away from classes. However, despite this, teachers were still requesting more time out to reflect, to plan, to meet, and to practice new ideas. They cited that teaching related activities was placing increased demands on their time, leading to relaxation time being seen to be even more precious than in the past. Consequently, they were very protective of such time quality time. This sentiment was much stronger and more forcibly communicated in some teachers, e.g., Teacher C, than in others.
Most significantly, the above three examples of teachersí reflections highlight different learning styles of individual teachers that could be involved in professional development programs. While there is often discussion in the literature directed at the importance of meeting the differing needs and expectations of students, teachers are usually seen as a homogeneous group. Therefore, professional development programs are structured so that one approach is expected to suit all. Clearly, the data from this small sub-sample show that such simplistic assertions are not true.
In this program, despite teachers being linked in pairs with a colleague from the same school, particular personalities, such as exhibited by Teacher C and to a lesser extent Teacher B, seem to prefer solitary working environments. Professional development programs need to be flexible to cater for differences in learning style, commitment and expectation of teachers. Strategies are needed that provide support which ensures the continued strong commitment of those who might prefer to work alone, but one that also facilitates effective team functioning.
Finally, it was interesting to observe that by the last session, two of the teachers (Teacher A and B) were beginning to reflect on how they might involve other colleagues in their school. For these teachers, it was an indication that they were becoming more confident knowing about and applying SOLO. They were also able to reflect positively on the value of the program and its relevance to their teaching context. Hence, they were now reaching a stage where they saw potential in other teachers being sensitised to the new assessment agenda and, importantly, that they could play a role in that professional development program.
Analysis and Discussion
To assist with a deeper analysis and discussion, the data provided in the case study transcripts can be analysed further in relation to how the professional development program met the needs of the participants. Important factors relate to the level of attention paid to (i) the appropriate delivery of the content of the professional development program to the target audience, and (ii) the degree to which the teachers incorporated the new perspectives into their long-term teaching practice and beliefs.
To facilitate this analysis the ten principles of professional development outlined by Clarke (1994) are used (Appendix 1). These ten principles are generic in nature as they were derived from a synthesis of the relevant professional development literature. As such, they provide a conceptual framework to use as a basis for exploration and analysis.
Issue 1: Address the issues of concern and interest, largely identified by the teachers themselves, and involves a degree of choice for participants...
The program of professional development did not emerge from the teachers participating in the project. Instead, it arose as a consequence of several separate but also related initiatives. First, as indicated earlier in the paper, ideas concerning new agendas in assessment were becoming more prolific in the research literature. Second, the move towards outcomes-based education by the State Government was creating tensions for many teachers, as it was so different in concept to their current practice. Finally, several research projects had provided sufficient expertise to make the application of SOLO to classrooms a feasible option.
Hence, while the project was not initiated by teachers, the teachers involved had to nominate if they were wished to be involved. All teachers were informed about the project, at least in general terms. They were alerted also to the support that would be available but that this would come at a cost of some commitment and effort on their part. Comments regarding scepticism of the applicability of the content of the program to everyday teaching were limited to the first workshop session. This can be attributed, in part, to care taken by the Project Team to make the theory as practical and relevant as possible to the teachersí current workplace situation. In addition, the decline in concerns expressed can be seen as n indication that as the teachersí knowledge of SOLO and assessment techniques grew, they were better able to see its value to themselves and their students.
Furthermore, workshops allowed areas of interest and concern to be acknowledged, with the teachers encouraged to try different things suitable to their particular subject or school environment. Acknowledgment of the individual needs of the participants and the ability of the Project Team to address those needs also contributed to a growing sense of confidence and purposefulness in the teachers.
Issue 2: Involve groups of teachers rather than individuals from a number of schools...
Both Teacher A and, to a lesser extent, Teacher B commented on the benefits of being able to work with their own staff, and with the staff from other schools. Teacher C exhibited characteristics of wanting to come to conclusions on his/her own and did not appear to use any other staff (either from within or external) to help in coming to terms with the application of the model. This may explain his/her drive to collect hard data throughout the year.
Teacher B was adamant that involvement of the rest of the staff in this project was very important in order to bring about long term change within his/her school. Teacher A also supported this notion. Interestingly, despite supportive comments, Teacher B did much of his/her work alone.
Issue 3: Recognise and address the many impediments to teachersí growth at the individual, school, and district level.
Of note was that all participants commented on the applicability of DBA to everyday practice, especially in helping to assess studentsí understanding. As such, impediments faced by other professional development programs of teachers seeing new knowledge as being impractical (Clarke, 1994, p.41) were not evident in the current project. This view was most evident towards the end of the first year.
A feature of the program was that attempts were made in the initial planning phase to build elements into the design that would limit or eliminate as many obvious impediments to the growth of the participantsí knowledge as possible. However, some inhibiting factors could not be addressed to such a degree as to reduce their impact completely. For example, Teacher A commented on the crowded primary curriculum and time constraints as impacting on his/her ability to take on as much of the information as he/she wished.
One major positive design feature was the availability of additional release time from face-to-face teaching. There were funds within the project to meet this expense. This enabled teachers to attend the two-day workshops and have some additional time for reflection and team activity at school. This represented an ideal, if atypically, rich situation for teachers in the NSW context. Usually relief from teaching or the opportunities to undertake other activities in school time is very expensive and rare. Both Teacher A and Teacher C felt more time was needed. Teacher A was prepared to find some of that time after school. Teacher B was quickest to accrue the new knowledge, but this teacher was also the teacher who read widely in his/her own time about the theory of the content provided.
Issue 4: Using teachers as participants in classroom activities or students in real situations...
An important focus of the project was the active participation of the teachers in the implementation of new ideas in their own classroom. It was expected that teachers would develop and trial, either with other teachers or on their own, specific assessment items. They would then analyse student responses in terms of SOLO levels, and reflect on their findings.
All three teachers developed items and gave them either to their own classes and/or shared them with colleagues. Teacher A appeared to benefit the most from active involvement with other teachers. This teacher worked cooperatively with another teacher from the same school. The two teachers developed items together, tested classes with these items and carried out the analyses together. Teacher B asked continually for relevant readings and showed a dynamic interest in improving his/her assessment techniques and teaching strategies using the DBA method. This teacher generated numerous items for his/her classes. The better items in eliciting a range of student responses were then shared with other teachers, most not involved directly with the professional development program. Although Teacher C participated independently and was worried by time demands, the teacher still trialled a small number of questions in his/her own classes.
Issue 5: Solicit teachersí conscious commitment to participate actively in the professional development sessions...
All participants entered the professional development program voluntarily. They were made aware of the demands they would face over the year and the attempts, through various forms of support, to help them meet those demands while maintaining a full teaching load. However, personal commitment to the project only became generally evident by the teachers after six-months involvement in the program. This supports previous research that had commented on instances of change of attitude towards professional development programs based on achievement within the programs (Huberman & Miles, 1984).
A significant feature of the program that contributed to this involvement was that teachers were expected to share their experiences with other teachers. This was done informally in the school and more formally as timetabled sessions in the two-day workshops. The effect of this requirement meant that teachers had specific targets (tasks to present to their classes and analyses to undertake) and report-back expectations in front of the entire group. While there was a supportive environment created in the workshop sessions for teacher reporting, this requirement meant that active engagement in the project was essential.
Active participation was encouraged in the very first two-day workshop session. This came about by teachers having opportunities to talk about their views on assessment and to undertake codings of student work samples. Transcripts of subsequent workshops and on-site sessions show that the activity initially was superficial and that an increased level of active involvement of all participants grew slowly. Therefore, it can be concluded tentatively that in this particular case, the active involvement of participants increased along with their understanding of the model and their trialling of materials and processes in their own classrooms.
Issue 6: Recognise that changes in teachersí beliefs about teaching and learning are derived largely from classroom practice...
An intrinsic and stated philosophy of the project was not to challenge overtly existing teacher beliefs and conceptions regarding the participantís teaching practice. Nevertheless, the program was designed to elicit change in the participants in terms of utilising a broader view of assessment, one that would affect teaching practices. This was achieved by encouraging teachers to offer suggestions and advice based on their experiences during workshop sessions.
Each of the teachers involved in the project commented positively about the changes in classroom practices that had been initiated by the discussions and work conducted during the workshops. For example, Teacher B recognised that the kinds of questions being asked during mathematics lessons had changed while student participation had improved. These changes were so overt that other mathematics teachers in the same school (not involved in the project) had observed them and were curious to know the origin of the transformations.
Issue 7: Allow time and opportunities for planning, reflection, and feedback in order to report successes and failures to the group...
Participants provided data for the case studies during different sessions, and in particular, sessions designed for reflection. Such sessions were inbuilt into the professional development model. However, both Teachers A and B made additional comments strongly supporting the importance of such opportunities.
In these reflection sessions, discussion was far ranging. Issues of strengths and concerns were raised openly and considered by all participants. These sessions were important for the project team to understand relevant issues for the teachers and to address them. Also, they played an valuable role for the teachers. Teachers B and C both stated that feedback form the project team was important to keep them ìon the right track.î A large planning session for directions in 2002 was held at the end of year workshop. All participants were actively involved in this session.
Issue 8: Enable participating teachers to gain a substantial degree of ownership by their involvement...
The participants provided no comments relating to this issue. This may have been because the importance of ownership was expressed throughout the program. This arose in terms of students needing ëownershipí of the science or mathematics they met. Similarly, it was just as important for teachers to have, over time, ownership of new ideas and practices in their own classrooms.
As indicated previously, ownership of the use of the assessment tool in the teachersí classroom was a central objective of the program. Also important was a deep understanding of the theoretical implications of the SOLO model. Only in this way would teachers be able to use the model as a tool to improve the education they provided. In fact, a major criticism of the current move towards outcomes-based education is that it lacks a theoretical base (Pegg & Panizzon, 2001) and, consequently, deprives teachers of genuine ownership of curriculum development.
Issue 9: Recognise that change is gradual, difficult, and often painful process...
The professional development model was designed to acknowledge and make genuine efforts to address these issues, at least as far as it was practical. For example, it was accepted that for real sustained change to occur teachers need to be given time. Time to think, time to plan and time to reflect. Setting the program over at least one year and providing support for relief from teaching was an attempt to address this. Being involved in changing means that something can be improved or done better. For some people this realisation is difficult to resolve and can be painful. Grouping teachers in pairs, providing support on request, as well as school visits, may have helped some of the teachers.
Teacher A stated that ìit was a slow process.î This referred to both knowledge about the model and the ability to apply it in the classroom. Opportunities for support in the process were given, with important benefit being gained by interaction with colleagues and project team members. Teacher A also stated that he/she would like to ìrefresh and renewî what he/she learnt in the first year, indicating a slow and gradual process. Teacher B wanted continued contact throughout 2002 to maintain and evaluate progress using the model, thus indicating the importance of ongoing support.
Issue 10: Encourage participants to set further goals for their professional growth.
In the final session of the year, teachers were provided with the chance to think about their needs and plans for the coming year. They were encouraged to reflect on their personal journey over the year and to consider where they might wish to see themselves in terms of the project. Each teacher was asked to consider a plan of action with some specific personal outcomes. This plan was worked out initially with their colleague from the same school. Then each teacher was invited to share his/her plan with the group.
Teacher B had set small goals throughout the year, and was one of two participants to do this in an unprompted way. No comments were made by the other participants directly relating to this issue; however, such information can be gained from analysing directions for planning made by each of the participants in the case studies. Teacher A focused on becoming more adept in using the SOLO model, as well as to discuss the model and what it can offer with other staff at his/her school. Teacher B wanted to involve more staff and learn more deeply about the theoretical model. Both these teachers spoke of ways to be involved in the project for the following year. Teacher C would not be available to undertake the project the following year but indicted there were two directions he/she could have taken. These were to make examinations that matched the course performance descriptors, and to develop a bank of sample codings.
The results described in this paper have provided insights into the experiences of primary and secondary teachers involved in a professional development process to explore alternative assessment practices in their classroom environments. In terms of the content addressed during the course of the first-year, participation gave teachers a means for exploring the understanding of their students using a cognitive model and provided a framework upon which to consider their teaching practices so as to build on their students' understandings. This aspect was valued highly because it gave teachers clear direction as to where to start and the best way in which to proceed with their particular students.
In reference to the design of the professional development program used for the project, it has clearly met the ten principles detailed by Clarke (1994). Not only were the teachers positive about the delivery of the content of the program, they were able to apply the ideas into their individual teaching contexts. Involvement in the process enabled the needs of both primary and secondary, mathematics and science teachers to be met within the same workshop sessions. As a consequence of the outcomes of their participation, all of the teachers have indicated their willingness to be involved in the second year of the project.
Biggs, J. (1996). Enhancing teaching through constructive alignment, Higher Education, 32, 347-364.
Biggs, J., & Collis, K. (1982). Evaluating the quality of learning: The SOLO Taxonomy. NY: Academic Press.
Biggs, J., & Collis, K. (1991). Multimodal learning and the quality of intelligent behaviour. In H. Rowe (Ed.), Intelligence: Reconceptualization and measurement (pp. 56-76). Melbourne: ACER.
Black, P. & William, D. (1998). Assessment and classroom learning, Assessment in Education, 5(1), 7-74.
Campbell, K.J., Watson, J.M., Collis, K.F. (1992). Volume measurement and intellectual development. Journal of Structural Learning, 11(3), 279-298.
Clarke, D. (1994). Ten key principles from research for the professional development of mathematics teachers. In D. Aichele & A. Coxford (Eds.). Professional development for teachers of mathematics: NCTM 1994 Yearbook (pp. 37-48). Reston, V.A.: NCTM.
Coady, C. & Pegg, J. (1996). Levels of formal reasoning in high school algebra. In A. GutiÈrrez (Ed.), Proceedings of the 20th International Group for the Psychology of Mathematics Education (2, pp.233-240). Valencia, Spain: University of Valencia.
Cole, N.S. (1990). Conceptions of educational achievement, Educational Researcher, 19(3), 2-7.
Collis, K., & Romberg, T. (1991). Assessment of mathematical performance: An analysis of open-ended test items. In C. Wittrock & E. Baker (Eds.), Testing and cognition. New Jersey: Prentice Hall.
Collis, K. & Romberg, T. (1992). Collis-Romberg Mathematical Problem-solving Profiles. Hawthrorn, Victoria: Australian Council for Educational Research.
Collis, K., Jones, B., Sprod, T., Watson, J., & Fraser, S. (1998). Mapping development in studentís understanding of vision using a cognitive structural model. International Journal of Science Education, 20(1), 44-66.
Cuthbert, R. & Pegg, J. (1993). Strategies used by Year 9 students to solve problems on speed. In B. Atweh et al. (Eds.), Proceedings of the 16th Mathematics Education Research Group of Australasia (pp.297-302). Brisbane.
Ernest, P. (1992). The nature of Mathematics: Towards a social constructivist account. Science and Education, 1, 89-100.
Gipps, C. (1994). Beyond testing: towards a theory of educational assessment London: Falmer.
Huberman, M. A. & Miles M. B. (1984). Innovation up close: How school improvement works. Plenum Publishing Corp: New York.
Linn, R.L. (2000). Assessments and Accountability Educational Researcher, 29(2), 4-16.
National Research Council (1989). Everybody Counts: A Report to the Nation on the Future of Mathematics Education National Academy Press: Washington.
Panizzon, D. (1999). Senior secondary and early tertiary science studentsí developmental understandings of diffusion and osmosis: A neo-piagetian approach. Unpublished doctoral thesis, University of New England, Armidale.
Panizzon, D., & Pegg, J. (1997, December). Investigating students' understandings of diffusion and osmosis: A post-piagetian analysis. Paper presented at the Annual Conference of the Australian Association for Research in Education, Brisbane.
Pegg, J. (1995). Perspectives on Year 9/10 advanced mathematics syllabus for N.S.W. (Keynote Address NEMA Annual State Conference). In New Syllabus Directions ñ Secondary Mathematics: Proceedings of the 4th Annual Conference (pp.15-24). Armidale: New England Mathematical Association.
Pegg, J. (1997). Mathematics Teaching - Creating the Future. In N. Scott & H. Hollingsworth (Eds.), Mathematics: Creating the Future (pp. 20-38). Adelaide: AAMT.
Pegg, J. & Panizzon, D. (2001). Determining levels of development in outcomes-based education: Nice idea, but where is the research-base for the decisions taken? Paper presented at the American Educational Research Association Conference in Seattle, on 10-14th April, 1-5.
Ramsden, P. (1992). Learning to teach in higher education. London: Routledge press.
Reading, C., & Pegg, J. (1996). Exploring understanding of data reduction. In A. GutiÈrrez (ed.), Proceedings of the 20th International Group for the Psychology of Mathematics Education (4, pp.187-195). Valencia, Spain: University of Valencia.
Shepard, L.A. (2000). The role of assessment in a learning culture, Educational Researcher, 29(7), 4-14.
Watson, J., Collis, K., & Campbell, K. (1994). Developmental structure in the understanding of common and decimal fractions. Focus on Learning Problems in Mathematics, 17(1), 2-25.
Watson, J., Collis, K., Callingham, R., & Moritz, J. (1995). A model for assessing higher order thinking in statistics. Educational Research and Evaluation, 1(3), 247-275.
Zevenbergen, R. (2001). Identifying literacy demands of adult numeracy. Literacy and Numeracy Studies, 10(1/2), 39-53.
Zuber-Skerritt, O. (1992). Professional development in higher education: A theoretical framework for action research. London: Kegan Page.
Ten important principles of professional development (Clarke, 1994, p.38).
|