EDUC 7015NA - Measurement, Assessment & Evaluation

Ngee Ann Academy - Quadmester 2 - 2016

This course assumes knowledge of introductory statistics/quantitative methods and is concerned with the major developments that have occurred during the past 30 years to improve the measurement and evidence-informed progress in education and human sciences. There are many models that are derived from Item Response Theory. This course focuses on those models developed by Georg Rasch and scholars working within the framework that he proposed with the use of the logistic function to transform data so that it would possess sound measurement and objective properties. The course adopts a practical approach, through the analyses and interpretation of datasets, and examines applications in assessment, evaluation and measurement, including test development, test scaling, standard setting, differential item/distractor functioning, test equating, item banking, and computer adaptive testing.

  • General Course Information
    Course Details
    Course Code EDUC 7015NA
    Course Measurement, Assessment & Evaluation
    Coordinating Unit School of Education
    Term Quadmester 2
    Level Postgraduate Coursework
    Location/s Ngee Ann Academy
    Units 3
    Contact Up to 3 hours per week
    Available for Study Abroad and Exchange Y
    Prerequisites EDUC 7011/7011NA Introduction to Quantitative Methods
    Assumed Knowledge EDUC 6553/6553NA Assessment and Evaluation in Education
    Restrictions M Ed students only - Singapore
    Course Description This course assumes knowledge of introductory statistics/quantitative methods and is concerned with the major developments that have occurred during the past 30 years to improve the measurement and evidence-informed progress in education and human sciences. There are many models that are derived from Item Response Theory. This course focuses on those models developed by Georg Rasch and scholars working within the framework that he proposed with the use of the logistic function to transform data so that it would possess sound measurement and objective properties. The course adopts a practical approach, through the analyses and interpretation of datasets, and examines applications in assessment, evaluation and measurement, including test development, test scaling, standard setting, differential item/distractor functioning, test equating, item banking, and computer adaptive testing.
    Course Staff

    Course Coordinator: Associate Professor Sivakumar Alagumalai

    Course Timetable

    The full timetable of all activities for this course can be accessed from Course Planner.

  • Learning Outcomes
    Course Learning Outcomes
    The course introduces the broad concepts, theories and practices in assessment and evaluation. Students are also introduced to the theory of objective measurement with insights into the use of Rasch Measurement models and its applications in education and human sciences.

    The course provides theoretical, collaborative-dialogue and hands-on sessions, and seeks to:
    1) Present accessible overview of the basic principles of measurement, especially Rasch Measurement models (dichotomous, rating scale, partial credit, and many-facet data) and procedures;
    2) Demonstrate how Rasch Measurement models and procedures can be applied to a number of common issues (item and person calibration, performance and judged data, DIF – item bias and DDF, problems with raw scores and rubrics, raters and markers) encountered by researchers;
    3) Elucidate theoretical and methodological issues in measurement, assessment and evaluation;
    4) Foster greater awareness in the fields of education and human sciences of the significance of sound measurement and reporting practices;
    5) Create a platform for scholars to share applications of objective measurement principles and procedures for quality research inferences and policy decisions.
    University Graduate Attributes

    This course will provide students with an opportunity to develop the Graduate Attribute(s) specified below:

    University Graduate Attribute Course Learning Outcome(s)
    Deep discipline knowledge
    • informed and infused by cutting edge research, scaffolded throughout their program of studies
    • acquired from personal interaction with research active educators, from year 1
    • accredited or validated against national or international standards (for relevant programs)
    3
    Critical thinking and problem solving
    • steeped in research methods and rigor
    • based on empirical evidence and the scientific approach to knowledge development
    • demonstrated through appropriate and relevant assessment
    1,2
    Career and leadership readiness
    • technology savvy
    • professional and, where relevant, fully accredited
    • forward thinking and well informed
    • tested and validated by work based experiences
    4
    Self-awareness and emotional intelligence
    • a capacity for self-reflection and a willingness to engage in self-appraisal
    • open to objective and constructive feedback from supervisors and peers
    • able to negotiate difficult social situations, defuse conflict and engage positively in purposeful debate
    5
  • Learning Resources
    Required Resources
    1 Alagumalai, S., Curtis, D., & Hungi, N. (2005). Applied Rasch Measurement: A Book of Exemplars (Eds.). The Netherlands: Springer. [Electronic Resource - UofA Library]

    2 Boone, W.J., Staver, J.R., & Yale, M.S. (2013). Rasch Analysis in the Human Sciences. The Netherlands: Springer. [Electronic Resource - UofA Library]

    3 Christensen, K.B., Krenier, S., & Mesbah, M. (2013). Rasch Models in Health (Eds.). Great Britain: ISTE Ltd. [Electronic Resource - UofA Library]

    4 Additional readings and tutorial notes will be distributed in class (and through MyUni).

    Recommended Resources
    1 Bond, T. and Fox, C. (2000) Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Hillside, NJ.: Erlbaum.

    2 Masters, G.N. and Keeves, J.P. (eds) (1999) Advances in Measurement in Educational Research and Assessment. Oxford: Pergamon.

    3 Smith, E.V. and Smith, R.M.(2004). Introduction to Rasch Measurement: Theory, Models and Application. Maple Grove, Minnesota: JAM Press

    4 Thorndike, R. M. (2005), Measurement and Evaluation in Psychology and Education, Columbus, Ohio: Pearson.d in class (and through MyUni). R. M. (2005), Measurement and Evaluation in Psychology and Education, Columbus, Ohio: Pearson.
    Online Learning
    Additional details to:

    CONQUEST software: http://www.acer.edu.au/conquest/notes-tutorials 
    jMetrik application: http://www.itemanalysis.com/ 
  • Learning & Teaching Activities
    Learning & Teaching Modes
    A balance between ‘student centred’ and ‘teacher centred’ approaches to learning with emphasis on fostering an engaging learning pedagogy will be used in this course. Lectures will be supported by discussions and problem-solving practicals using computer programs which will require active participation from students.
    Workload

    No information currently available.

    Learning Activities Summary
    Intensive/ Session Key Concepts / Themes

    1.Fri 6 May 16 [7-10pm]
    Assessment and Evaluation in Education and the Human Sciences:
    Reliability, Validity, and Bias
    Introduction to Test Theory

    2a.Sat 7 May 16 [1-4pm]
    Classical Test Theory
    Item Analysis and Reliability
    Composite scores & classical (weak) true-score model

    2b.Sat 7 May 16 [5-8pm]
    Introduction to Measurement
    Probabilistic models and Item Response Modeling; Rasch Measurement
    Dichotomous & Polychotomous Formats

    3a.Sun 8 May 16 [9-12pm]
    Item characteristic curve, Person characteristic curve, Item Fit, Person Fit
    Implications for test/questionnaire construction

    3b.Sun 8 May 16 [12-4pm]
    Rating Scale Analysis
    Analysis of Partial Credit Model
    Measures, scales and norms
    Assignment 2: Diagnostic Test [20%] – in class

    Thursday, 19 May 2016 Assignment #1 due: (by 4pm S’pore Time) – Upload onto MyUni

    4.Fri 3 Jun 16 [7-10pm]
    DIF (item) and DDF (distractor)

    5a.Sat 4 Jun 16 [1-4pm]
    Equating of Tests
    Assignment #3 Presentation (Poster)

    5b.Sat 4 Jun 16 [4-8pm]
    Rater/Marker Measurement
    Assignment #3 Presentation (Poster)

    6a.Sun 5 Jun 16 [9-12pm]
    Performance Assessment;
    Examining Rubrics & Standards
    Facet Measurement; Saltus Model; Multidimensional Measurement

    6b.Sun 5 Jun 16 [12-4pm]
    Item Banking; Adaptive Testing
    Summary and Conclusion (Next!)

    Saturday, 11 Jun 2016 Summative Test (Assignment #4): (NAAEC - TBA)
    Specific Course Requirements
    This course utilises an extended-evaluation copy of the CONQUEST (ACER) software application. Moreover, students will have access to large-scale datasets available from UNESCO, OECD and IEA research studies and websites. It is important for students to have access to a computer/laptop, and to install the relevant applications and datasets to undertake and complete all tutorial/SGD activities.
  • Assessment

    The University's policy on Assessment for Coursework Programs is based on the following four principles:

    1. Assessment must encourage and reinforce learning.
    2. Assessment must enable robust and fair judgements about student performance.
    3. Assessment practices must be fair and equitable to students and give them the opportunity to demonstrate what they have learned.
    4. Assessment must maintain academic standards.

    Assessment Summary
    ASSESSMENT TASK       TASK TYPE      WEIGHTING               COURSE LEARNING OUTCOME(S)

    Report - Compare and contrast CTT and Rasch Measurement Formative (Individual) 30% 1, 2, 3, 4, 5

    Diagnostic Test 1 [in class] Formative (Individual) – 1 hour 20% 1, 2, 3, 4, 5, 6

    Poster & Presentation – DIF, Raters and Ratings, Rating Scale, Partial Credit Model, Equating and Facet Analysis Summative (SGD - Pair Work) 20% 2, 3, 4, 6

    Summative Test 2 [NAAEC - TBA] Summative (Individual) – 1.5 hours 30% 1, 2, 3, 4, 5, 6
    Assessment Detail

    No information currently available.

    Submission
    1 Students must retain a copy of all assignments submitted.

    2 All individual assignments must be attached to an Assignment Cover Sheet which must be signed and dated by the student before submission.

    3 All group assignments must be attached to a Group Assignment Cover Sheet which must be signed and dated by all group members before submission. All team members are expected to contribute approximately equally to a group assignment.

    4 Markers can refuse to accept assignments which do not have a signed acknowledgement of the University’s policy on plagiarism (refer to policy on plagiarism).

    5 Requests for extensions will be considered only if they are made three days before the due date for which the extension is being sought. Students must apply to the lecturer concerned on the ‘Application for Extension’ form at the back of the Academic Program Handbook.

    6 Late submission (past deadline) penalty applies (10% deduction for each day)
    Course Grading

    Grades for your performance in this course will be awarded in accordance with the following scheme:

    M10 (Coursework Mark Scheme)
    Grade Mark Description
    FNS   Fail No Submission
    F 1-49 Fail
    P 50-64 Pass
    C 65-74 Credit
    D 75-84 Distinction
    HD 85-100 High Distinction
    CN   Continuing
    NFE   No Formal Examination
    RP   Result Pending

    Further details of the grades/results can be obtained from Examinations.

    Grade Descriptors are available which provide a general guide to the standard of work that is expected at each grade level. More information at Assessment for Coursework Programs.

    Final results for this course will be made available through Access Adelaide.

  • Student Feedback

    The University places a high priority on approaches to learning and teaching that enhance the student experience. Feedback is sought from students in a variety of ways including on-going engagement with staff, the use of online discussion boards and the use of Student Experience of Learning and Teaching (SELT) surveys as well as GOS surveys and Program reviews.

    SELTs are an important source of information to inform individual teaching practice, decisions about teaching duties, and course and program curriculum design. They enable the University to assess how effectively its learning environments and teaching practices facilitate student engagement and learning outcomes. Under the current SELT Policy (http://www.adelaide.edu.au/policies/101/) course SELTs are mandated and must be conducted at the conclusion of each term/semester/trimester for every course offering. Feedback on issues raised through course SELT surveys is made available to enrolled students through various resources (e.g. MyUni). In addition aggregated course SELT data is available.

  • Student Support
  • Policies & Guidelines
  • Fraud Awareness

    Students are reminded that in order to maintain the academic integrity of all programs and courses, the university has a zero-tolerance approach to students offering money or significant value goods or services to any staff member who is involved in their teaching or assessment. Students offering lecturers or tutors or professional staff anything more than a small token of appreciation is totally unacceptable, in any circumstances. Staff members are obliged to report all such incidents to their supervisor/manager, who will refer them for action under the university's student’s disciplinary procedures.

The University of Adelaide is committed to regular reviews of the courses and programs it offers to students. The University of Adelaide therefore reserves the right to discontinue or vary programs and courses without notice. Please read the important information contained in the disclaimer.