|
|
|
Simple Canon solution reduces manual labour, supports detailed analysis and almost quadruples the volume of survey responses that can be processed
At a glance
Surveying in the past
A new era in evaluation
Finding a way forward
Information on Demand
At a glance
Monash University's Centre for Higher Education Quality
Industry
Tertiary Education
Geographies
Monash Uni's Clayton compus
Canon Products Used
TeleForm; DR9080 document scanners
"Data quality, reliability, survey governance and co-ordination have all been improved, while the costs of survey development and processing have been markedly reduced, along with distribution, collection and processing times"
Chris Wayland, project manager and independent consultant (nLIVEn) to Monash University
Effectively collecting, analysing, sharing and acting on student feedback is crucial for the continued growth and viability of Melbourne's Monash University.
The university's Centre for Higher Education Quality manages Unit Evaluation, collecting student opinions on each unit (subject) taught at Monash University at the end of each teaching period. The centre analyses the information it gathers and creates reports for internal university purposes so that it can improve teaching and learning and also as a condition for attempting to secure additional government funding. Until recently the survey process was time-consuming and labour-intensive, which was not sustainable given that the university has more than 56,000 students distributed across six Australian campuses, two international campuses and numerous international teaching partnerships.
Surveying in the past
The centre's previous survey management system was 12 years old and paper based. It could only support the most generic forms of evaluation. It relied on an extensive amount of manual data entry, which was time and labour intensive. The survey forms had to be specially printed and then read by scanning systems that were at the end of their useful life and required replacement. The centre had begun using online surveys, but they were administered through a separate system from its print surveys rather than being part of an integrated process. By late 2004 it was clear that action had to be taken to improve the survey process.
A new era in evaluation
Like all Australian universities, Monash was seeking to improve the experience of its students through systematically using their feedback to improve its unit offerings. The will to improve was also supported by external pressure, including the wish to fare well in the Australian Universities Quality Agency audit and also to perform well in the Federal Government's $250 million Learning and Teaching Performance Fund allocation.
While encouraging excellence in teaching and learning is the goal of this program, simply achieving excellence is not enough ? universities must go one step further by actually proving they have achieved excellence. They do this by collecting, analysing and making improvements as a consequence of obtaining student feedback through survey data.
Finding a way forward
While the decision to replace the legacy system was made fairly quickly and easily, the decision concerning what to replace it with was not. Independent consultant Chris Wayland (a partner with nLIVEn) was brought in to work with a university-wide advisory steering group lead by CHEQ's Professor Graham Webb and Dr Sid Nair and the university's information technology services division ? with Ian Hall as the internal technology lead. Wayland was no stranger to modernising survey management systems, having successfully completed a similar project at another Australian university.
The first thing the project team did was develop a detailed project plan. The team began by consulting with all stakeholders and documenting the existing business processes under the previous system. From there, they worked with all stakeholders to design the "ideal" business processes that they would like to introduce through the new system before considering the technology they would use to do this. In partnership with Canon's consultants and software suppliers, the project team then designed and constructed the new system and helped the organisation to embrace it with training and other change management strategies.
The project team selected a Canon solution that was based on TeleForm ? a forms-processing application from Verity, a Canon software partner ? as well as Canon DR9080 high-volume document scanners. TeleForm is modular and highly scalable, and Monash University selected the Scan, Reader, Verifier, eForm and Automerge modules
According to Wayland, choosing the new system was the easy part ? it was the implementation and integration with the university's existing systems that had the potential to be difficult because of the "aggressive" implementation schedule of only four months, which left little time for testing.
The pressure was substantial, but the project team supported by Canon's consultants stepped up to the task with the required discipline, skills and communication. They successfully rolled out the new system in time for the first semester of 2005.
Information on Demand
The survey system's first big test was distributing, collecting and analysing evaluations for units offered in the first semester, 2005. Using the new system, the centre successfully distributed roughly 110,000 survey forms to Monash University students by in-class administration and email. This was a massive improvement over the previous system, which reached capacity at about 30,000 surveys.
These changes have reduced costs in two main ways: the survey process no longer requires a large seasonal support team of up to 14 ? including 10 casual data entry operators ? to spend a month manually entering results; and the university can now print surveys on plain paper rather than on special forms sourced from a commercial printer.
"More than 500,000 survey responses can now be processed annually, and each unit can be evaluated at least once every year it is offered instead of once every three to five years," Mr Wayland said. "The new system provides efficiency, productivity and reliability of service."
The response rate on the first use of the new system was 38 percent, which was higher than anticipated. One reason the response rate was so reasonable is that the new system does not distinguish between paper and online surveys ? units can be evaluated using the method most appropriate and the same information automatically flows back to the same database once processed.
The results that flow back are also guaranteed to be captured accurately by the new system, because it has eliminated manual data entry, which always involves the risk of human error.
"Data quality, reliability, survey governance and co-ordination have all been vastly improved, while the costs of survey development and processing have been markedly reduced, along with distribution, collection and processing times," said Wayland
But the most important change is that the new system reliably collects data from all forms (both quantitative and qualitative) and automatically stores it in one location for detailed analysis. The software is completely integrated with Callista, the university's Oracle-based student management system, and it is providing a level and quality of management reporting and benchmarking (between units, departments, campuses, faculties and the university as a whole) that was previously impossible. It also allows Dr. Nair to conduct further detailed analysis of the whole raw data set which again was previously not possible.
This Canon solution has reduced costs, improved reliability, increased response rates and almost quadrupled the volume of survey responses that can be processed. But even more important, Monash University is able to analyse these survey results to continually improve its teaching and learning, the results of which can be seen on the website by anyone, anywhere in the world.
Related Links
|
|
 |
|
