Evaluation is a central component of professional development, course/program development and the University's quality assurance process. Planning and Performance reporting provides support to Faculties, Schools, course coordinators and individual staff who wish to evaluate student learning and staff teaching in their courses and programs. One of the major evaluation tools used at the University is the Student Experience of Learning and Teaching (SELT) system. We provide assistance to all staff and Schools wishing to evaluate their teaching for professional development or promotion purposes.
Planning and Performance Reporting assists the University of Adelaide in the implementation of the Student Experience of Learning and Teaching Policy.
SELT is only one of the necessary components for making informed decisions about improving student learning outcomes, and staff and Schools should use other methods, such as reflective practice and peer review, student assessment results and teaching portfolios in order to construct an informed view of the learning and teaching being evaluated. Evaluation is a positive process and should be used for the enhancement of staff development and student learning.
Survey Reference Group Approval is required
if a SELT falls outside the area of Learning and Teaching.
www.adelaide.edu.au/learning/staff/surveys/
The SELT Manual provides detailed documentation on the the SELT system and its use at the University of Adelaide.
Please contact Performance Reporting for any further information.
Email: evaluation@adelaide.edu.au
Phone: +61 8 8313 3496
The SELT system enables you to use either the standard template or to create a questionnaire to suit your particular teaching situation and developmental needs. All services related to standard SELT templates are provided free of charge and include:
Staff may download the standard forms (as pdf files):
To use these forms you do not have to place a request with Evaluation.
You should:
The SELT report will be sent to you by email and the survey sheets will be returned in the internal mail. For surveys that require a fast turn around time (e.g. departmental review, application for academic promotion) the envelope should be clearly and prominently labelled as 'urgent'.
The general policy relating to the return of results and original questionnaires is:
As part of the University's desire to improve the student learning experience, it has identified key questions from the Standard Teacher and Course SELTs which will be used as indicators of quality.
For each of these questions a high level of broad agreement is expected as indicated below. The minimum levels of broad agreement required by our staff and courses increases each year to 2015.
Reports indicating the level of achievement of teachers and courses in meeting the expectation standards will be provided to Heads of School and Executive Deans.
| SELT question |
% broad agreement | |||
|---|---|---|---|---|
| 2013 | 2014 | 2015 | ||
| 6 | [Teacher] is an effective university teacher | 77 | 80 | 80 |
| 1 | This course has clearly identified learning outcomes | 70 | 75 | 80 |
| 5 | This course uses appropriate online resources and technologies to help me achieve its learning outcomes | 70 | 75 | 80 |
| 7 | This course helps me to develop my thinking skills (eg problem solving, critical analysis) | 70 | 75 | 80 |
| 9 | My learning in this course is supported by effective feedback | 70 | 75 | 80 |
| 10 | Overall, I am satisfied with the quality of this course | 77 | 80 | 80 |
Legend:
| Standard Teacher SELT | Standard Course SELT |
| SELT question |
% broad agreement | |||
|---|---|---|---|---|
| 2013 | 2014 | 2015 | ||
| 6 | [Teacher] is an effective university teacher | 80 | 80 | 80 |
| 1 | This course has clearly identified learning outcomes | 73 | 76 | 80 |
| 5 | This course uses appropriate online resources and technologies to help me achieve its learning outcomes | 70 | 75 | 80 |
| 7 | This course helps me to develop my thinking skills (eg problem solving, critical analysis) | 70 | 75 | 80 |
| 9 | My learning in this course is supported by effective feedback | 70 | 75 | 80 |
| 10 | Overall, I am satisfied with the quality of this course | 77 | 80 | 80 |
Legend:
| Standard Teacher SELT | Standard Course SELT |
Each SELT is unique and must be viewed in the context of the particular teaching environment. However, there are some general principles that can be applied when interpreting your results from Standard Teacher or Course SELT questionnaires. For each commentary, there is a brief discussion of the question context, some practical suggestions, and some further reading that is readily available through the University Library.
When you receive the report of your SELT surveys, the first thing you will do is examine the students' description of their experience of you and the course. This information is important to you as you determine how to improve student learning outcomes, the curriculum and assessment tasks.
As you read through student comments and look at the histograms presented in your report, you will be formulating ways in which improvements can be made to the current course.
As you plan changes to the teaching program, curriculum design or assessment tasks, you will need to inform students of your thinking so that they understand how their comments are taken into account.
Respond to the students honestly and positively. This is easy when students have responded positively to the course and your teaching, but may require some thought when the responses have been critical. A good first step is to write down your responses to the survey report, set it aside for 24 hours and then revise your responses, if necessary, to ensure that you are responding to the underlying issues and not emotionally. The second step is to ask a colleague to read your responses. If a significant number of students have been critical of specific aspects of the course, acknowledge the issues and explain how it will be dealt with in the future. If the issues are beyond your control, don't blame others but describe to students the issues that you can influence and those you cannot.
There are a number of methods you can use to respond to student comments.
From the students' perspective, they have filled out SELT forms and there will be no returns on their time investment unless you, their lecturer, provide them. If students can see that their opinions are respected and on occasion acted upon, there is an increased chance they will approach the task of filling out SELTs with a more positive attitude, thus leading to more constructive outcomes from these surveys.
In many cases, you will be seeing these students again or your colleagues will, and so the students have a direct interest in seeing that your teaching and their learning work together, as effectively and enjoyably as possible. Students will understand your approach to teaching and your requirements of students if they are aware of previous SELT reports and your response to them.
The following approach is recommended where possible.
Evaluating online learning is, in principle, no different from evaluating learning in other contexts, such as lectures, clinical work or problem-based learning. That is to say, teachers in both contexts are likely to be interested in such matters as learning outcomes, students' preparation, their experience of learning and the learning environment.
Certainly, the technology supporting online learning gives us the potential to evaluate in different ways. For example, we can use online surveys, collect written feedback from students electronically and monitor learning behaviours. Some of these possibilities still have technical and management problems associated with them. These problems are somewhat too complicated to discuss here.
For the moment, we want to describe resources that are available to you online or by using SELT. We emphasise that there are other methods of gathering information, such as discussion groups, observation, interviews and document analysis.
The following material is intended to help you design your evaluation in a framework that emphasises good principles of online learning. Examples of the kinds of questions you can access from the Student Experience of Learning and Teaching Service (SELT) are listed in relation to each principle.
The questions are available in the SELT Manual (Q1 – Q526) or from a more extensive database held by Planning and Performance Reporting (Q 527 – Q3140). New resources, questions and instruments are being designed and added to the SELT service progressively.
The SELT Manual and forms needed for administration and evaluation are downloadable from the SELT system.
The 'Seven principles for good practice in undergraduate education', originally published in 1987, are a popular framework for evaluating teaching in traditional, face-to-face courses. The principles are based on 50 years of higher education research.
A follow-up article is available online Implementing the Seven Principles: Technology as Lever. The article originally appeared in print in the AAHE Bulletin, October 1996, pp. 3-6.
A team of evaluators from Indiana University's Center for Research on Learning and Technology (CRLT) used the seven principles to evaluate four online courses at a large US university. The evaluations were based on analysis of online course materials, student and teacher discussion-forum postings, and interviews.
The following text material on the seven principles is reproduced from an article which was originally published in The technology source. The article is:
The material is reproduced here (with minor editorial changes to adapt North American terminology to that common in Australia) with the permission of the publisher.
Lesson for online teaching: Teachers should provide clear guidelines for interaction with students.
Teachers wanted to be accessible to online students but were apprehensive about being overwhelmed with email messages or bulletin board postings. They feared that if they failed to respond quickly, students would feel ignored. To address this, we recommend that student expectations and staff concerns be mediated by developing guidelines for student–teacher interactions. These guidelines would do the following:
| Code | Question item |
|---|---|
| 309 | There was sufficient access to teaching staff |
| 310 | There was sufficient access to technical staff |
| 415 | There was a sufficient degree of staff-student contact |
Lesson for online teaching: Well-designed discussion assignments facilitate meaningful cooperation among students.
In our research, we found that teachers often required only 'participation' in the weekly class discussion forum. As a result, discussion often had no clear focus. For example, one course required each of four students in a group to summarise a reading chapter individually and to discuss as a group which summary should be submitted. The communication within the group was shallow. Because the postings were summaries of the same reading, there were no substantive differences to debate, so that discussions often focussed on who wrote the most eloquent summary.
At the CRLT, we have developed guidelines for creating effective asynchronous discussions, based on substantial experience with staff teaching online. In the study, we applied these guidelines as recommendations to encourage meaningful participation in asynchronous online discussions. We recommended the following:
Learners should be required to participate (and their grade should depend on participation).
| Code | Question item |
|---|---|
| 150–167 | Check the SELT manual questions 150–167 to select suitable discussion-type questions. |
| 345–351 | Check the SELT manual questions 345–351 to select suitable questions on feedback |
| 414 | I am developing new friendships in this class |
| 422 | I am learning a great deal from working with my fellow students |
Lesson for online teaching: Students should present course projects.
Projects are often an important part of face-to-face courses. Students learn valuable skills from presenting their projects and are often motivated to perform at a higher level. Students also learn a great deal from seeing and discussing their peers' work. While formal synchronous presentations may not be practical online, teachers can still provide opportunities for projects to be shared and discussed asynchronously.
Of the online courses we evaluated, only one required students to present their work to the class. In this course, students presented case study solutions via the class web site. The other students critiqued the solution and made further comments about the case. After all students had responded, the case presenter updated and reposted their solution, including new insights or conclusions gained from classmates. Only at the end of all presentations did the teacher provide an overall reaction to the cases and specifically comment about issues the class identified or failed to identify. In this way, students learned from one another as well as from the teacher.
| Code | Question item |
|---|---|
| 251–259 | Check the SELT manual questions 251–259 to select suitable project-type questions. |
| 419 | I am benefiting from having to give a presentation to the class |
Lesson for online teaching: Teachers need to provide two types of feedback: information feedback and acknowledgment feedback.
We found during the evaluation that there were two kinds of feedback provided by online teachers: 'information feedback' and 'acknowledgment feedback'.
Information feedback provides information or evaluation, such as an answer to a question, or an assignment grade and comments.
Acknowledgment feedback confirms that some event has occurred. For example, the teacher may send an email acknowledging that they have received a question or assignment and will respond soon.
We found that teachers gave prompt information feedback at the beginning of the semester, but as the semester progressed and teachers became busier, the frequency of responses decreased, and the response time increased. In some cases, students got feedback on postings after the discussion had already moved on to other topics.
Clearly, the ideal is for teachers to give detailed personal feedback to each student. However, when time constraints increase during the semester's busiest times, teachers can still give prompt feedback on discussion assignments by responding to the class as a whole instead of to each individual student. In this way, teachers can address patterns and trends in the discussion without being overwhelmed by the amount of feedback to be given.
Similarly, we found that teachers rarely provided acknowledgment feedback, generally doing so only when they were behind and wanted to inform students that assignments would be graded soon. Neglecting acknowledgment feedback in online courses is common, because such feedback involves purposeful effort. In a face-to-face course, acknowledgment feedback is usually implicit. Eye contact, for example, indicates that the teacher has heard a student's comments; seeing a completed assignment in the teacher's hands confirms receipt.
| Code | Question item |
|---|---|
| 345–351 | Check the SELT manual questions 345–351 to select suitable questions on feedback. |
Lesson for online teaching: Online courses need deadlines.
One course we evaluated allowed students to work at their own pace throughout the semester, without intermediate deadlines. The rationale was that many students needed flexibility because of full-time jobs. However, regularly distributed deadlines encourage students to spend time on tasks and help students with busy schedules avoid procrastination. They also provide a context for regular contact with the teacher and peers.
| Code | Question item |
|---|---|
| 296 | The teaching of the course is well organised |
| 460 | I am able to effectively organise my study time for the course |
| 461 | I spend sufficient time studying for the course |
| 465 | I spend time looking up things that interest me, even if they are unlikely to be examined |
Lesson for online teaching: Challenging tasks, sample cases and praise for quality work communicate high expectations.
Communicating high expectations for student performance is essential. One way for teachers to do this is to give challenging assignments. In the study, one teacher assigned tasks requiring students to apply theories to real-world situations rather than remember facts or concepts. This case-based approach involved real-world problems with authentic data gathered from real-world situations.
Another way to communicate high expectations is to provide examples or models for students to follow, along with comments explaining why the examples are good. One teacher provided examples of student work from a previous semester as models for current students and included comments to illustrate how the examples met her expectations. In another course, the teacher provided examples of the types of interactions she expected from the discussion forum. One example was an exemplary posting while the other two were examples of what not to do, highlighting trends from the past that she wanted students to avoid.
Finally, publicly praising exemplary work communicates high expectations. Teachers do this by calling attention to insightful or well-presented student postings.
| Code | Question item |
|---|---|
| 322 | Teacher expectations are made clear |
Lesson for online teaching: Allowing students to choose project topics incorporates diverse views into online courses.
In several of the courses we evaluated, students shaped their own coursework by choosing project topics according to a set of guidelines. One teacher gave a discussion assignment in which students researched, presented, and defended a current policy issue in the field. The teacher allowed students to research their own issue of interest, instead of assigning particular issues. As teachers give students a voice in selecting their own topics for course projects, they encourage students to express their own diverse points of view. Teachers can provide guidelines to help students select topics relevant to the course while still allowing students to share their unique perspectives.
| Code | Question item |
|---|---|
| 158 | The group work increases my ability to learn independently without having to rely on a teacher |
| 435 | My ability to work independently is increased |
THE UNIVERSITY OF ADELAIDE
The following pages provide links to aggregated reports of the SELT (Student Experience of Learning and Teaching) course and teacher surveys conducted by each academic Faculty and their associated Schools and Disciplines for each academic year listed.
Summary aggregate reports are available for the Faculty, School and Discipline (where available and where course SELT surveys were conducted). Summary aggregate reports are also available for the University of Adelaide as a whole.
The University of Adelaide Adelaide, South Australia, 5005 Australia. CRICOS Provider Number 00123M
Telephone: +61 8 8313 4455
Coordinates: -34.920843, 138.604513
| Authorised by: | Planning and Performance Reporting |
| Maintained by: | Planning and Performance Reporting |