Aberystwyth University
Using the digital experience insights service has sharpened our thinking and given us the opportunity to focus on digital issues. It has provided a platform for thinking about what we want to do next.
Project lead: Kate Wright, e-learning group manager, Aberystwyth University
Project aims: Reliable data with sector benchmarking
The 2017-18 pilot of the digital experience insights service has provided a reliable survey methodology for Aberystwyth University as they explore student perceptions of the digital aspects of their learning experience. The sector-wide use and credibility of the insight service, together with the benchmarking data, provides valuable data to supplement that gained via the university’s own feedback mechanisms.
We knew that the survey was well designed and widely used by the sector. This widespread use was attractive to us and the benchmarking data is important.
Project lead: Kate Wright, e-learning group manager, Aberystwyth University
An added benefit for Aberystwyth is that the survey is available in Welsh, fulfilling the statutory requirement for Welsh institutions to provide bilingual resources.
The target audience for the pilot was undergraduate students who were on campus. A total of 502 students took part, with students completing in both Welsh and English. The survey ran in December 2017 and was carefully timed to ensure it didn’t encroach on other scheduled surveys.
The university has also participated in the pilot of the teaching staff survey and are analysing their results to compare them with those of their students.
Strategies for engaging students
A stakeholder group was formed specifically to support, steer and take ownership of the project. The stakeholder group comprises representatives from Aberystwyth University Students’ Union, academic and professional services department (including staff development unit, library, student support, careers and the Technology Enhanced Learning (TEL) group). The stakeholder group are responsible for implementing the survey, identifying the target audience, deciding on any customisation and additional questions to be asked and are also leading on processing the results, how key messages arising from the survey will be conveyed and implementing any changes.
A key member of the stakeholder group was an intern, Ania, whose role combined working with the TEL group and the library customer services group, the latter are particularly active in communicating with students. Ania had graduated earlier in the year with a psychology degree and had statistical experience that was invaluable to the project.
The engagement strategy included face-to-face student contact and activities in the various social spaces around the university. Ania set up a stand with tablet computers in the library and directly engaged with students, explaining the purpose of the survey and asking them to complete it. Although stands, posters, flyers, links and notifications on the university’s virtual learning environment (VLE) all contributed to the response rate, it was emails sent out by departmental administrators to students in their departments throughout the three-week survey period that were the most successful method of promoting the survey to students.
A prize draw of some Amazon vouchers was also offered as an incentive.
Data analysis process
The university included customised questions designed to identify examples of good practice from different departments and to ascertain any differences in experience between single and joint honours students. Students were also asked if they were willing to be contacted by a member of the information services (IS) team if they had raised issues that the team could address.
An initial review of the free text data was conducted to allow the IS team to address any specific queries as quickly as possible and to clarify unclear responses.
The students respond well to this support – they know that the team is reading and acting upon what they have told us.
Project lead: Kate Wright, e-learning group manager, Aberystwyth University
Responding quickly and directly to students helps to build engagement and trust between students and staff. It also allows the survey team to highlight the varied support options available and engage support teams directly in the feedback and data review processes.
Initial findings from the Insights service have been presented internally and findings are now being reviewed against national sector benchmarking data provided by the service to identify areas for development. Responses are being categorised into broad themes in a similar way to the internal surveys (eg network, library borrowing).
Although the customised questions designed to identify differences in departmental provision did not yield sufficient data to draw any reliable conclusions it has provided some qualitative information which is being explored further.
Key findings from the student survey
- 30% of students who responded to the survey are using smartphones – nearly the same number as the 33.1% who use laptops. Although the TEL team knew that mobile device use had increased, the closeness of the figures illustrate how learning habits have changed
- 90.1% of respondents rated the university’s digital provision as excellent or good. This compares favourably with the 87.57% benchmark
- Students are broadly happy with how digital technologies are being used. 62% of respondents say they would like digital technologies to be used in much the same way as they are now
We were quite surprised at the 62% satisfaction rating for how digital technologies are being used. There is a tendency to think ‘we have achieved this, let’s see how we can push for the next innovation’. Perhaps we need to concentrate on doing more of what we are doing well and improve that further to give a really good experience.
Project lead: Kate Wright, e-learning group manager, Aberystwyth University
- 74% of respondents said that digital learning and teaching on their courses was good or excellent. Although the TEL and library teams felt they had a good understanding of how their provision is received by students it is gratifying to get validation and positive feedback to share with programme teams
Student responses in relation to what they consider to be assistive technology were interesting, prompting reflection that this is possibly a term better understood by staff working with technology and that students may interpret the term more literally as ‘something that helps me’ rather than something that supports a particular need. Some of the things cited by students as being assistive technology included the student app, Google and EndNote.
Key findings from the staff survey
78 staff took part in the staff version of the digital experience insight survey which ran during May and June. It was advertised via Blackboard and email to staff across the institution.
- 80.8% of staff who responded to the survey rated the overall quality of digital provision at Aberystwyth University as good or above
- There was a spread of preferences as to how much digital technology should feature in teaching, learning and assessment practice. 42.9% felt that technology should be used as it is now, 37.9% felt it should be used more than it is now and 19.5% felt it should be used less
- Most staff have access to basic digital services such as wifi, e-books and file storage when needed. An exception was a video-based skills service such as Lynda.com which only 10% said they had access to.
- 70.5% of staff agreed with the statement ‘I am able to create and manage online assessments’, while 28.2% gave a neutral response
Areas where staff reported lower levels of practice or confidence included: regular use of the VLE for student collaboration; ensuring students stay safe online; data copyright and licencing, data security and assistive technologies.
The word cloud below shows the digital tools or apps that teaching staff identified as being useful to their job roles.
There was a wide array of responses to questions relating to time received for support and to innovate when developing digital aspects of teaching staff roles, recognition and reward and about the opportunities they had to be involved about decisions in relation to digital services. The same issues were reflected in free text comments in response to the question ‘what one thing should the organisation do better to support your digital teaching practice.
Acting on the findings
The project team are in the process of sharing the data back with the stakeholder group and with teams departmental teams, cross-university services such as academic liaison librarians and the wider institutional community so that they can have an opportunity to engage with it and contribute to development plans.
The findings will be discussed by the TEL group to ascertain next steps; to identify quick wins that show how the university is listening to and responding to feedback as well as to invest time and thought into things that require a more strategic approach.
The project team found the guidance and templates provided by the Insights service very helpful in preparing to launch the survey, running it and analysing the data.
The poster templates provided by Jisc have been used to feedback findings and responses to students as well as an infographic.
Tips for others and lessons learned
- Running a survey of this nature requires planning and resource if it is to be meaningful
- Establishing a stakeholder group including representation from the student union was recommended in the guidance documents – this has been instrumental in shaping and steering the project and will be important in determining future development plans
- This made a discrete project for an intern with data analysis skills. It is self-contained but is big enough to have institutional impact but not so big that it is overly daunting. It provides valuable experience for interns and has helps to build confidence
- Responding to student comments takes time but has multiple benefits, such as involving more teams in using the data, helped disseminate information, and enabling the students to see that their comments have been read
Next steps
The university is also running the staff version of the digital experience insight service and will analyse the data from this, compare it with the student data.
The results from both staff and student surveys will be presented to staff at the annual learning and teaching conference held each September with the aim of gaining further staff insight and engagement on the data and findings. The intention is to create opportunities for discussion and collaborative development.
As e-learning manager, Kate Wright, the project lead for piloting the Insight Service at Aberystwyth also intends to use the data to inform staff training, drawing on evidence from the survey of what students value and how they are interacting and accessing materials. The survey provides staff with a whole-university view of what students think and enables them to see the bigger picture, beyond interactions with students on their programmes.
The TEL group are also working with student union representatives on communicating the findings to students and working together on responses.
Kate would like to run the survey again, possibly annually, to review progress against any interventions made.
Additional resources
- See Kate and Ania's PechaKucha presentation and recording at the digital experience insights community of practice event in November 2018
- An infographic created by Ania to feedback survey findings and actions to students
