Support for Your Research
The conference organising committee wanted to provide information on the Short Papers as early as possible, so that presenters who wanted to submit a research-stream paper had time to work on researching their MELT approach before the submission was due.
Resources for research were provided, as follows:
If you have been using one of the MELT frameworks, such as the RSD, you may want to treat use till now as a pilot and begin to plan research. For example, you may want to apply for ethics approval in the next 3 months, so that you can gather research data in Semester 1 or Summer Semester 2017.
A good starting point is the various research papers under evidence of effectiveness.
We invite you to consider research approaches for determining the effectiveness of your use of MELT. You can email firstname.lastname@example.org and state:
- the ways in which you have already used one or more of the MELT
- aspects you are thinking of researching with ethics approval
- gaps this may address
- preliminary research questions
- ideas you have on research methods
Some possible research methods for the timeframe include:
- Pre- and Post-Questionnaires
- Exist for research skill development and problem solving development
- analysed for statistically and educationally significant changes over, say one semester in Likert Scale questions.
- Open-ended questions may provide student attribution to what caused any changes noted in Likert scales
- Online response rates are typically so low they are not worth conducting unless...
- Ethics/questionnaire committee approval allows surveys to be completed in class time
- need to be part of the educational program, e.g. for student reflective practice, as well as to improving the course from info gathered in the pre-questionnaire.
- need to be administered by someone not involved in the teaching of the course
- We found pre-and post-questionnaires were not useful outside of a specific context, such as a subject
- Individual Interviews
- good to capture student experience
- we relied on individual interviews when determining RSD efficacy across programs by interviewing graduates and students after they completed three years of their degree
- rich data requires complex analysis
- can be used in conjunction with pre-post questionnaires, especially when students make and attribution of what caused any changes
- Focus groups
- good for improvement of approaches
- students may also make attribution of what caused change
- Observation data
- person external to the course/program
- ethics required
- observer needs to be skilled in what to observe
- Mixed methods
- potential 'triangulation' of data
- quantitative and qualitative aspects need to satisfy their own determinant of quality, such as 'reliability' and 'trustworthiness' respectively
Reliability and Trustworthiness
An upfront consideration of 'reliability' (for quantitative research) or 'trustworthiness' (for qualitative research) is worthwhile in the planning stage.
Likewise, specification of 'disconfirming evidence' is worthwhile in analysis stages i.e. data that does not fit into your standard pattern, and that may not complement your main inferences. Specifying disconfirming evidence is particularly encouraged for the research stream.