[Archive] Chapter 4: Evaluation

• Basic Concepts of Evaluation
• Method of Evaluating School-based Gifted Development Programmes
• Processing and Application of Evaluation Findings of School-based Gifted Development Programmes

Basic Concepts of Evaluation

The objective of evaluation is not only to assess outcomes and performance, but also to examine, improve and develop educational plans. Programme evaluation is carried out by collecting data and providing evidence records to relevant personnel (including decision-makers and teachers) with the aim of improving existing programmes and making appropriate judgments about their value and effectiveness. Student assessment is a crucial part of programme evaluation.

Evaluation supports the development of programmes in the following ways:

1. Diagnosis – identifying problems and difficulties, and finding ways to handle them appropriately.

2. Adaptation of programmes – evaluation can serve as the basis for the adaptation of programmes, so that the quality of education can be continuously improved.

3. Comparisons – evaluation can be used to compare different plans, teaching methods and other areas of school education.

4. Anticipation of educational needs – evaluation can be used to assess educational needs and build educational objectives. It can also be used as reference for determining the directions of education reform.

5. To determine the degree to which educational objectives are being realised– evaluation can be used to determine the degree to which educational objectives are being realised, and help decide whether such objectives or programmes require modification to make them better suited to the needs of students.

Method of Evaluating School-based Gifted Development Programmes

1. a. Analysis of Antecedents
   Collecting data from different sources to understand the programme's characteristics:
   1. a. Read extensively all printed materials related to the programmes, to gain a comprehensive understanding of their objectives and implementation methods.
   2. b. Design an open-ended questionnaire to be completed by relevant individuals (including students, parents, teachers and administrative staff) to understand their areas of concern about the programmes.
   3. c. Conduct meetings with related groups to understand their ideas.
   4. d. Observe the actual implementation of the programmes, to clarify or verify previously collected data.
2. b. Integration of information and design of assessment tools:
   After drawing conclusions based on findings of the analysis of antecedents, evaluators may identify the key areas for evaluation and determine their choice of assessment tools, and from whom information will be collected. Multiple assessment tools may be used to evaluate gifted education programmes, such as standardised tests, score tables, assessment tables, face-to-face meeting outlines, learning diaries, adaptive behaviour scale, classroom observations and specific assessment tools designed for designated purpose.
3. c. Collection and Analysis of Data:
   After collecting the necessary data, they may be logically classified according to their common characteristics. The use of statistics and reasoning can also be used to describe and analyse the findings. The method of analysis is generally determined by the nature of the data.
4. d. Preparation of Evaluation Report:
   The evaluation report should contain a description of the programme and evaluation method. It should also include a summative analysis of the merits and demerits of the entire programme, and suggestions for improvements.

Processing and Application of Evaluation Findings of School-based Gifted Development Programmes

Input Evaluation

The objective of input evaluation is to analyse the use of resources, strategies, methods and the design of procedures. The evaluation covers the sources of teachers and students, teachers' participation, teaching materials, financial resources, equipment, teachers' qualifications, teacher training and professional support.
(For further information, please refer to the Evaluation Report on "Pilot School-based Programme for Academically Gifted Children": Chapter 5: Input; 1999; pages 32-43)

Learning Assessment

1. Student assessment is a crucial element in programme evaluation. It is also an important part of the cycle of learning and teaching, and it must fit into learning and teaching processes.

2. All teaching professionals should first refer to "Assessment for Learning", which is part II of the Exemplars of Curriculum Development in Schools (Curriculum Development Council, 2001), and "School Policy on Assessment – Changing Assessment Practices", which is booklet 5 of the Basic Education Curriculum Guide (Curriculum Development Council, 2002). They should also carefully read the content about "assessment" in the curriculum guides for each key learning area, and use these as reference for basic principles for assessing student learning who participate in school-based gifted development programmes.

3. Learning content, processes and outcomes can be assessed either formally or informally. Formal assessment means using standardised tools to conduct pre-implementation or post-implementation tests. Informal assessment includes observations, questionnaires, peer assessment, student self-assessment, and various scales.

   1. In the course of assessing learning content, attention should be paid to whether the learning objectives are being attained. The assessment methods include standardised performance tests, teacher self-designed tests, classroom audiotape analysis, observations of student behaviour in classrooms, etc.
   2. While assessing the learning process, attention may be paid to the number of times a student expresses his or her views, the number of different kinds of ideas and the thinking order shown in answers given by students. Appropriate scales may also be used to evaluate the learning progress of a student. (Reference table 1: Higher-order thinking ability assessment tool exemplar)
   3. In assessing the learning outcomes, the assessment criteria can be formulated by students themselves or other assessors; and diversified and qualitative assessment methods may be introduced, such as certain scales, which assess the creativity, uniqueness, degree of elaboration or details, integration ability, organisational power and information-conversion ability shown in students' work. (Reference table 2: Creativity assessment tool exemplar)

       

4. The assessment mode should include formative assessment and summative assessment. Formative assessment aims to obtain feedback about the process of learning and teaching, in order to enhance the quality of the process. Summative assessment is used to measure results. (Reference table: The theoretical framework of the implementation of assessment in school)

    

5. In view of the extraordinary abilities of gifted students, more extensive use may be made of self-assessment, reflective assessment (Reference table 3: Student after-class reflective assessment tool exemplar), inter-peer assessment, inter-group assessment, learning diaries, computer-aided assessment and others, to encourage students to adopt self-directed learning and raise their quality of learning.

6. If possible, assessment tools should be adjusted and modified according to the abilities and characteristics of gifted students, to facilitate their ability for independent learning.

    

7. More use should be made of a learning portfolio to collect evidence of excellent performance demonstrated by gifted students, as well as information and feedback about their learning progress.  

Evaluation of Programme Effectiveness

1. Evaluation of the effectiveness of a programme should make reference to the evaluation of the learning process itself. Such information should be collected annually to facilitate programme modification whenever necessary. Besides continuous evaluation, the anticipation of the long-term effectiveness of the programme should be recorded every year or every few years. Evaluation information should contain data about student learning process; a detailed record of the programme events compiled by teachers, and also comments made by other people about the programme effectiveness. Such people should include members of the Gifted Education Task Group, students, administrative staff, general teachers, parents, mentors, etc.
   Reference table 4: After-class evaluation questionnaire (to be filled in by teachers)
   Reference table 5: After-class evaluation questionnaire (to be filled in by students)
   Reference table 6: After-class evaluation questionnaire (to be filled in by lower-level primary school students)

2. Factors to be Considered in Selecting Evaluation Tables:

   1. The actual situation of a school. For example, the readiness of teachers, the abilities of students, and whether parents are supportive.

   2. The content of evaluation tables may be increased or reduced according to actual needs. Teachers may modify the original evaluation tables, and choose appropriate topics for evaluation.

   3. Besides the above-mentioned evaluation tables, schools may refer to evaluation information provided by other educational organisations or groups, and make modifications according to their actual needs.

   4. A school may choose one or more evaluation tables, and it may also combine two or more tables to facilitate the evaluation process.

   5. Schools may create their own evaluation tables and give the first priority to their needs.

3. Evaluation must be carried out in a regular and systematic manner. One single evaluation should not be regarded as final. Teachers should conduct several rounds of evaluation, analyse the data objectively, and then formulate future plans.