Learning & Teaching
Excellence Framework
Good Practice Resources & Tools
Related Links
SUBJECT EVALUATION GUIDELINES
Date approved: |
9 May 2008 |
Date Policy will take effect: |
9 May 2008 |
Date of Next Review: |
June 2014 |
Approved by: |
Academic Registrar | ||||
Custodian title & e-mail address: |
Academic Registrar | ||||
Author: |
Policy Analyst, Policy & Governance Unit, Academic Registrar’s Division | ||||
Responsible Faculty/ |
Academic Registrar’s Division | ||||
Supporting documents, procedures & forms of this policy: |
Subject Evaluation Questions FAQ for Subject Coordinators | ||||
References & Legislation: |
UOW Academic Review Policy | ||||
Audience: |
Internal – UOW Staff Only | ||||
Expiry Date of Policy: |
Not applicable | ||||
Contents
- 1 Introduction / Background 3
- 2 Scope / Purpose 3
- 3 Definitions 3
- 4 Preparation for the Survey 3
- 5 Delivery and collection of survey instruments 4
- 6 Compilation and Distribution of Survey Data 4
- 7 Reporting Subject Evaluation Data 5
- 8 Maintaining Security, Storage and Privacy 6
- 9 Roles & Responsibilities 9
- 10 Version Control Table 10
1 Introduction / Background
- 1. In 2005 consultation was undertaken in developing the Guidelines to ensure they addressed the full scope of relevant issues. A Reference Group consisting of representatives from the Policy and Governance Unit, the former COGNOS Team, Student Administration, the Strategic Planning Unit, Chair of the Quality Assurance Subcommittee and Centre for Academic Systems & Resources (CASR), provided significant input into the development of the Guidelines. In addition, advice on the statistical analysis of Subject Evaluation Survey (SES) data was sought from Professor David Steel, Associate Dean Faculty of Informatics. Faculty Deans were also consulted on issues relevant to the Guidelines via discussions at Dean’s meetings and consultation with the Academic Registrar and the Deputy Vice Chancellor (Academic). Since the introduction of the original Guidelines, external advice has also been sought through the University’s Legal Services Unit to ensure the University is complying with the Privacy Act when managing data generated from SES responses.
- 2. During 2005 and 2006 the SES was administered through Student Online Services (SOLS) using an online survey instrument. Surveys were held every session for every subject at all campuses including off-shore operations. This frequency may have caused over-surveying of students, resulting in lower than expected response rates. From September 2006 to August 2007 the Student Survey Working Party revised the online process to improve response rates and avoid over-surveying. In Spring 2007 the SES is introduced as a paper-based survey administered within the subject review cycle of the UOW Academic Review Policy. Additionally, the survey questions were revised and implemented with the changes in Spring 2007.
- 3. These Guidelines are evolving and will therefore be subject to review and change in the future.
2 Scope / Purpose
- 1. These Guidelines are intended to provide clear information as to how the SES process will be managed and clarify the responsibilities of all staff in regard to proper handling of SES information. They provide information and guidance on various aspects of information management by addressing relevant records management, privacy and confidentially issues.
- 2. It is envisaged that further information will be added to these Guidelines including examples of Good Practice demonstrating how student feedback received through SES’s have been utilised by faculties and examples of effective communication strategies with students.
- 3. The University initiated the SES as a quality mechanism. The survey is not linked to the Teacher Evaluation Survey nor to any teaching performance processes.
- 4. These Guidelines apply to all Subject Evaluations administered within Australia including distance students enrolled in a UOW degree.
- 5. These Guidelines do not apply to Subject Evaluations at Dubai campus.
3 Definitions
4 Preparation for the Survey
Subject selection
- 1. Faculties establish a subject review cycle best suited to its course management or other cyclic needs within the Faculty. The Subject Evaluations will be administered according to the subjects nominated for review in a given session.
- 2. The Subject Review Cycle is established within the guidelines of the UOW Academic Review Policy. The process is more detailed within the Subject Evaluation Procedures.
Small Classes
- 3. Subjects which frequently have classes under 10 students may require different means of reviews. Survey reports are only issued with more than 6 responses per subject. Therefore any survey with a response rate of below or equal to six responses can only be surveyed informally. An alternative method for review could be focus groups within class. Confidentiality issues relevant to students are outlined in Clause 8.13 to 8.15.
- 4. The best practice for collating data over a period of time is for the subject to remain exactly the same for the duration of it being surveyed. That is no changes should be made to the objectives, assessment tasks, learning material and resources provided for this subject. Otherwise it is more likely that there are variances in the data due to the changes.
5 Delivery and collection of survey instruments
Delivery weeks and locations
- 1. The SES is administered between week 10 and week 13 for Autumn and Spring session, and the corresponding weeks for an Annual session. In Intakes the surveys are administered from week seven to nine. If alternative weeks are best this needs to be agreed in liaison with CASR.
- 2. Surveys are administered for each location where the subject is offered (except Dubai). For off-shore locations the permission from the relevant institution should be sought.
Delivery Process
- 3. The paper-based Subject Evaluation is coordinated by CASR using the same process as the teacher survey. For a detailed process see the Subject Evaluation Procedure.
Eligibility to complete survey
- 4. All students who are enrolled in the subject selected for review and who are attending class on the given day will be offered the opportunity to complete the survey.
- 5. Students who have withdrawn from the subject are not surveyed using the paper-based form.
Collection Process
- 6. Faculty administrators return the completed forms to the Audio Visual (AV) Store in Building 20.. When it is impractical to return the forms immediately after the class, the sealed survey pack should be stored in a secure location in the Faculty for return to the AV Store the next working day
- 7. The completed Subject Evaluation forms will be collated by CASR and forwarded to the external scanning provider, Education Assessment Australia (EAA), for scanning of the ranked data and the open text responses. The ranked data will be stored in form of a CSV file, which is sent to the UOW Performance Indicator Unit (PIU).
6 Compilation and Distribution of Survey Data
Compilation of survey data
- 1. The PIU uploads the survey data onto the Performance Indicator reporting system. In doing so the student number is separated from the responses to ensure that individual responses are not identifiable.
- 2. The Performance Indicator reporting system is a data interrogation tool which allows the presentation of information from a variety of University databases.
- 3. Survey data will be compiled at University, faculty and subject level by the PIU, in consultation with ARD and Planning Services.
- 4. All reporting of SES data will be based on the calendar year, being 1 January to 31 December.
Distributing Survey Reports
- 5. Results for faculty subjects will be made available through the Performance Indicator reporting system or other means to Deans, relevant Senior Executive, Head of Schools, Subject Coordinators and other approved staff as soon as possible after the release of results. The PIU will prepare the subject reports (summaries) and ARD will distribute them as PDF reports to Deans, Heads of School and Subject Coordinators. Subject Coordinators may discuss the report with relevant lecturers and tutors and Head of School provided the precautions stated in Clause 5.6 is adhered to.
Distributing Free Text Response Reports
- 6. During the survey administration students are asked to refrain from making reference to specific teachers. Nonetheless, the comments made by students in free text responses may contain comment or opinion about individual teachers, which is personal information under privacy laws. As such, the circulation of this feedback should be undertaken with careful consideration. Free text response comments should be distributed at discretion only to:
- a. individuals with a professional need to know as outlined in Clause 8.16 to 8.18; or
- b. the person concerned (being the person commented on in a free text response question); or
- c. if all identification of individuals is removed from the responses, and the responses cannot enable the identification of an individual, to other staff members within the faculty.
- 7. Free text responses will be sent to the Deans and the Heads of Schools after the release of results.
7 Reporting Subject Evaluation Data
Quality Improvement
- 1. Subject Evaluation data will provide an important aspect to the existing level of feedback received with regards to the quality of education at the University. To ensure Subject Evaluation data is effectively used it is important that data is embedded into the University’s planning and quality processes.
- 2. Once data is released, Faculties are responsible for identifying quality issues raised by the data including particular subjects or groups of subjects potentially requiring improvement, outstanding successes and, upward or downward trends (if comparative data across years is available).
- 3. Subject Evaluation results will be used by each faculty to inform its annual review of performance against its learning and teaching objectives, as part of the Faculty Reporting process coordinated by the Strategic Planning & Quality Unit.
- 4. Faculties conducting a subject review, in accordance with the UOW Academic Review Policy will need to use available Subject Evaluation data to inform the review.
Feedback
- 5. Efforts will also be made to communicate to students some of the actions which have been taken to improve individual subjects as a result of feedback provided through the Subject Evaluation. Academic staff will be required to assist and contribute to this communication process through the Subject Outline as stated in the Code of Practice Teaching and Assessment. Suggested examples1 of feedback include:
- a. Recent improvements to the subject include requiring students to write a reflective journal as part of the assessment. Reflective journals are often used in professional learning settings including the health and teaching professions to encourage self-awareness and conscious skill development.
- b. The tutorial activities have also changed to introduce a number of practical, but not directly assessed, tutorial activities in the first half of the subject. These activities will build on and reinforce the lecture material in a practical way, and give students confidence as communicators before formal assessment begins.
- c. In 2009 this subject will be using two different textbooks to those used in recent years. In particular students provided feedback indicating that they would like to use a bioethics text that was written in a language they found more accessible and that more overtly acknowledged the ethical challenges faced by registered nurses engaged in clinical practice.
- 6. Given its role in supporting the interpretation of educational evaluations Faculties can invite CASR to be involved in subject and course reviews.
Publishing Data on the University website
- 7. To comply with Government requirements, results of Subject Evaluations must be displayed on the University website aggregated to faculty level. Planning Services are responsible for determining what Subject Evaluation results are displayed on the website and will work with the PIU and the Policy and Governance Unit, which will be responsible for loading results onto the Learning and Teaching web pages. Results will be updated on the website for Autumn and Spring sessions each year.
8 Maintaining Security, Storage and Privacy
- 1. It is critical that the University maintain strict procedures regarding the use of responses to Subject Evaluations to ensure legislative compliance, including maintaining the privacy of survey respondents and storage of data. Information should not be published, presented or disseminated by staff with full access to the Performance Indicator reporting system or the survey reports in any way that might enable individual students to be identified.
Security of system
- 2. Every year an external body reviews and tests the security of all UOW ITS systems. This is arranged through the UOW Audit Manager.
Security of data
- 3. It is the responsibility of the Academic Registrar or his/her delegated representative, to ensure that a Custodian is appointed for the collection and handling of Subject Evaluation data within their area of responsibility, and that this is documented.
- 4. Data Custodians have delegated administrative responsibility for controlling data collections within their mandates. In undertaking their role, Custodians have responsibility for establishing and maintaining an acceptable level of protection for the data, for managing the disclosure of data, for ensuring that the privacy of personal information is protected, that the data is used only in accordance with the reasons for which it was collected and that the data itself complies with these guidelines.
- 5. The Academic Registrar has authority to approve or restrict access of individuals to specific Subject Evaluation data where required.
Access to SES data
- 6. Selected staff in the Academic Registrar’s Division (ARD) and the PIU will have access to the raw data. Access will be given to selected staff within the PIU for the Performance Indicator reporting system. All staff are bound by strict Codes of Conduct including confidentiality requirements.
- 7. Nominated persons of the Academic Registrar (AR) with access to Subject Evaluation data are:
- a. The Senior Executive of UOW
- b. Assistant Director, Planning Services, Strategic Planning Unit
- c. Senior Manager, Policy and Governance, Academic Registrar’s Division
- d. Faculty Deans and Head of School
- 8. There are 4 distinct tiers for use of Subject Evaluation data, each with a different level of access to the data and different kinds of information provision.
- 9. Tier 1 Publicly available information
- There is the DEEWR mandated need for aggregated data by faculty for public display. This reporting will be completed by the Planning Services Unit (in consultation with the Policy and Governance Unit) as it impacts on University external reporting under the Higher Education Support Act (HESA). Advice has been sought from Prof David Steel regarding statistical analysis of the data. The posting of information on the web will be a manually generated report that will require UOW Executive approval prior to posting to the web.
- 10. Tier 2 Reports
- Detailed survey reports will be generated after each session from the Performance Indicator reporting system and prepared for staff outlined in Clause 6.5 and 6.6. These reports do not contain identifying information about teachers or students. The reports will include information for each subject by faculty, plus comparison against faculty average, population information regarding number of enrolments and withdrawals per subject (definitions of the terms which define the status of enrolments are at Appendix B).
- The survey reports and the free text response reports, in form of scanned student comments, will be distributed and stored in PDF and TIF format.
- 11. Tier 3 Access to Data
- The data should be stored in the Performance Indicator reporting system. The faculty Deans will have access to data for their faculty. They will have a need to review responses and compare results over time and subject, and interrogate the data in detail. This requires high level access, and will be limited to the Deans, Heads of School and a limited number of approved faculty support staff identified as having a professional need to know. Application for access is completing the User Access Form and access approval must be supported by the Dean and the AR.
- This data set will contain aggregated data and will be separated from student numbers to ensure that student privacy is maintained and that individual students are not identified.
- 12. Tier 4 Raw Data
- The Academic Registrar and Senior Executive of UOW will have full access to all available data via the Performance Indicator reporting system. In addition selected staff in ARD and the PIU will have access to the raw data.
Tier 1:
Public information available through the UOW web.
Analysed faculty level aggregated information
Tier 2:
Reports
- Staff in the need to know
Tier 3:
Access to Data.
Deans and Heads and limited approved faculty staff
Tier 4:
Raw Data
Academic Registrar & Senior Executive of UOW, PIU Team
Ensuring student confidentiality
- 13. The production version of the Performance Indicator reporting system designed to report results, is based on processed data to ensure that small samples are appropriately aggregated and that individual students are not identified.
- 14. In instances where there are less than six responses for a particular subject, reports will not be generated by subject in order to maintain confidentiality of student responses. As a further measure to protect student confidentiality, data will be suppressed in instances where the proportion of responses in a cell exceeds 85%. This approach will be adopted because if, for example, 85% of students give a ‘strongly disagree’ response then the lecturer could reasonably conclude the individual student that gave this response with at least 85% accuracy.
- 15. All reports, including the handwritten open text responses, are released after the release of results. Reports (Tier 2) are only sent to staff in the need to know.
Staff with a need to know
- 16. Staff with a professional need to know include individuals connected to the continuous improvement of courses and subjects offered by the University and to assist in planning, course development and delivery.
- 17. The survey reports must be distributed to those staff with a need to know. In the first instance these staff are the Deans, Heads of School, Subject Coordinators. Other staff who may need to know will be determined by Deans based on individuals roles and responsibilities. Such staff may for example include Sub Deans, Lecturers, Tutors and other approved faculty support staff.
- 18. Free text response reports should only be circulated beyond individuals identified as having a professional need to know or to individuals concerned once any comment or opinion about a lecturer or tutor or, any information which would enable the identity of an individual involved in the delivery of a course to be ascertained even in the absence of a name, has been completely removed.
Storage, retention and disposal of survey data
- 19. Retention and disposal of SES data will be in accordance with the NSW State Records Act and the NSW Privacy and Personal Information Protection Act 1998.
- 20. Entries 3.9.9, 25.4.5 and 25.4.6 of the General Retention and Disposal Authority – University Records (GDA23) cover surveys/evaluations undertaken for the purpose of improving course delivery. The identified retention period is ‘until no longer required for reference or administrative use’ after which their destruction is authorised.
- 21. 25.4.5 on Quality Assurance relates to assessment of data and providing feedback on course delivery. Any findings, reporting, recommendations and actions taken should be maintained until the next subject review cycle has been completed. This will ensure continuation of quality on improvements. A record of past improvements and their rationale should be maintained by the Faculty.
- 22. 25.4.6 of Quality Assurance related to the collection of survey data and forms. Raw data stored at UOW should be maintained at least until the next subject review cycle is complete.
- 23. All reports are stored on the University’s records management system, URM for a minimum period of 10 years. The data in the Performance Indicator reporting system will remain until it is no longer required.
- 24. EAA will return the completed hardcopy survey forms to UOW and will maintain the raw data for a minimum of three years before destroying it.
- 25. Stored data will not be used for any purpose other than those outlined in these Guidelines.
Compliance with legislation and standards
- 26. Information intended for publication or presentation should comply with the requirements of:
- a. Relevant legislation, principles and guidelines relating to data management and the protection of privacy and confidentiality
- b. Relevant standards, policies or recognised best practice for the presentation and analysis of statistical data
- c. Requirements imposed by the owners of any data sets used
- d. Any conditions placed on use of the data as a result of ethical or legal advice.
Access to Information
- 27. The University of Wollongong is bound by the Government Information (Public Access) Act 2009 (“GIPA Act”) in providing public access to University information. Any Access to Information applications received by the University relating to SES data will be managed by the University’s Information Compliance Officer, located in the Legal Services Unit.
9 Roles & Responsibilities
All Staff
- 1. All staff employed within the University are personally responsible, within their roles, for maintaining the privacy, integrity and security of data held and managed by their work unit, and for reporting any suspected breach of procedure. This responsibility is formally acknowledged in an undertaking to observe privacy and security procedures. Codes of Conduct also specify the responsibilities of staff regarding confidentiality, the use of information and public comment.
Academic Registrar’s Division
- 2. The Academic Registrar nominates data custodians and gives approval for access to data on the Performance Indicator reporting system and the free text responses.
- 3. The Policy and Governance Unit is responsible for
- a. the UOW Academic Review Policy
- b. assisting the Planning Services with loading the reports onto the UOW website
- c. records management of the survey and free text response reports, and
- d. maintaining the guidelines and procedures.
Planning Services
- 4. Planning Services are responsible for
- a. preparing the aggregate reports for the UOW website and gaining Senior Executive approval, and
- b. enquiries on data interpretation of the reports.
Legal Services Unit (LSU)
- 5. The LSU is responsible for privacy issues/concerns and Freedom of Information.
CASR
- 6. CASR is responsible for:
- a. collating and providing information to faculty staff in regards to past subject selections
- b. Preparation and distribution of evaluation packs
- c. Collation of returned evaluation packs for forwarding to external company for scanning
10 Version Control Table
Last reviewed: 27 July, 2011
