Technology Enhanced Learning at University of Wales Trinity Saint David

Survey Outcomes and Report

1. Overview:

In 2013 the University of Wales Trinity Saint David merged with Swansea Metropolitan University and established a partnership with Coleg Sir Gâr and Coleg Ceredigion to form a new dual-sector University serving the needs of learners in South West Wales and beyond. It recognised the growing importance of technology in the delivery of education and established a Technology Enhanced Learning team led by an Executive Head of TEL.

It was agreed that the formation of an institutional TEL Strategy and development plan needed a picture of current usage as a baseline to inform the planning process. As a result, an online survey of both staff and student was carried out and the outcomes, analysis and conclusions are presented in this report.

Separate surveys (but with the same question sets) were carried out at Swansea Met and at Carmarthen/Lampeter to enable a comparison. Coleg Sir Gâr and Coleg Ceredigion were also invited to participate. The surveys were carried out using SurveyMonkey over a period of two months from late 2013 to early 2014.

184 staff and 360 students responded to the survey. There was a strong response from both staff and students at all the HE campuses at Swansea, Carmarthen and Lampeter. However, although there was also a very strong response from the Coleg Sir Gâr staff, engagement with the remaining FE college staff and students proved problematic. It was felt, however, that the substantial body of information gathered was indicative of general TEL usage by HE staff and students across the institution and provided a good picture of staff usage at one of the FE colleges. In that regard it made a valuable contribution to the TEL benchmarking and planning processes. A follow-up survey is planned to complete the FE part of the exercise and this report will be updated at that time.

2. The Surveys:

There were 10 questions in the staff survey and 9 in the student survey asking them about their use of Moodle, the way they communicated online, their use of social media and their own devices for engaging with their learning online. The questions included multi choice answers as well as free text comments. The multi choice responses enabled a quantitative evaluation across the institution, while the comments provided for a qualitative assessment of TEL usage and usability.

All of the survey outcomes are available on the TEL Survey Wiki and can be viewed at: http://teluwtsd.pbworks.com.

3. Quantitative and Qualitative Survey Outcomes:

The quantitative information was amalgamated for analytical purposes by combining all the staff and student data as an average response to each question drawn from across the institution. This average could then be compared with the responses from the individual campuses/partners and both general and comparative conclusions drawn.

The tables below present the survey quantitative outcomes, question by question, in the left hand column and a selection of key comments derived from the qualitative information received is presented in the right hand column. The comments complement the quantitative information for each question and contribute to an understanding of the profile displayed and allow conclusions to be drawn. The comments have been edited to fit the matrix format, but express the key messages of the original.

3.1 Staff

Questions and Survey Responses Comments

(Sample total: 1648 answers) (Sample total: 686 comments)

0.173333333333334

0.326666666666668

0.23 0.26

Q1: To what extent do you use Moodle in your teaching delivery?

For on-campus students, Moodle supplements face-to-face teaching by including teaching materials, links etc. For distance learners, Moodle is central to the whole teaching plan of the module.

Students are expected to check Moodle regularly as all lecture notes are uploaded plus material for use in tutorials. However, it’s evident that not all students access Moodle on a regular basis.

I have used Moodle as part of my teaching but students often find it difficult to navigate and to use at home or from mobile devices such as tablets and phones.

No Partly Mostly All

0.456666666666667

0.273333333333333

0.170.1

Q2: Do you use Moodle for student assessment?

Students submit assessments through Moodle. They are marked and then re-uploaded. Campus students then have feedback tutorials. Distance learners have feedback by email or telephone.

All submissions of coursework are through Moodle/Turnitin. Feedback is through Moodle, and some modules use MCQ as a way of revising topics.

All student assessments are on Moodle, their assignments are submitted via Turnitin, but as yet results are not posted online.

0.336666666666668 0.27 0.1866666

666666670.193333333333333

Q3: Do you use Moodle for course management?

I use a mixture of Moodle and e mail. DL students require personal e mails. I do not use the calendar. I do use Forums but mostly us Facebook as it is more versatile and effective.

All course announcements are on Moodle. In addition we use it to check students are accessing Moodle for lectures etc., through the Reports function.

I used to post assignment deadlines and event reminders on Moodle but we use the Gar i system now and almost all our students engage and track their schedule and deadlines and results on it.

No Yes

0.593333333333333

0.406666666666667

Q4: Do you use any other online applica-tions to support learners?

We use Dropbox a lot as students like the simplicity and the ease of access/ integration with all their devices. LinkedIn and Facebook are used for communications and Skype for online tutorials.

Facebook is good communication tool for us – all students can see what's going on and will respond within very short timescales - because they live on Fb!

Facebook gan fod y myfyrwyr yn ei ddefnyddio yn aml ac maent yn derbyn y wybodaeth yn syth.

No Some Most All

0.136666666666667

0.236666666666667

0.266666666666667

0.36

Q5: Do you make your teaching materials available on Moodle?

All information is available before the lecture. This includes links to useful websites, reading material, slides, tutorials and YouTube. It's a key part of the plan.

I was told that it is university policy to do so. Also, it saves printing out large numbers of handouts of.

The materials on Moodle are a back up to group sessions and tutorials. The information is always available to non-traditional learners, particularly those in the work place.

0.106666666666667

0.43

0.586666666666667 0.4866666

666666670.123333333333333

Q6: What formats do you use for your online teaching materials?

Main document types are Word, pdf and PowerPoint. I save what I do on the whiteboard as SmartNotes which I convert to pdf and put on Moodle. Useful for me as the starting point for the next class.

Generally in the form of PowerPoints with text, images and sometimes links. Students then have access to the information we looked at in class.

Word docs, PowerPoint, You Tube and web links mostly, as I lack the skills and time to develop other types of materials

Never Occasionally Often As a key teaching resource

0.193333333333333

0.426666666666667

0.303333333333333

0.0733333333333335

Q7: To what extent do you use open educational resources?

I've only got into this recently following a training course provided by the library staff at Lampeter and will be making more use of some of the resources available.

I will often include useful and relevant links to say YouTube videos. However I am not certain of whether this is officially encouraged, frowned upon, discouraged, or is even a categorical no-no!

E-books and e- journals; electronic resources via CSG Library and TSD Athens accounts; TES online resources, Teachers TV, TED, YouTube.

Never Occasionally Frequently All the time

0.316666666666668

0.316666666666668

0.193333333333333 0.1733333

33333334

Q8: To what extent do you communicate with your students through Moodle?

It’s not very attractive or convenient, better to have a university email with chat facilities like gmail.

Yes a planned component of all the programmes; individually, sub-groups and whole cohort. Forums and announcements and email tools. It works well when the system is up and running.

I message students through Moodle but was not aware of the Forum facility until this survey. Students like the ease with which they can communicate with me.

email Text message Facebook Other

0.916666666666667

0.196666666666667

0.266666666666667

0.17

Q9: Do you communicate online with your students in other ways?

Text messaging is popular as it is quick, simple and immediate. Facebook and email are used for more formal communications. Skype chats are helpful to share desktops and explain concepts in tutorial time.

Student email is the main communication method. I do not use Facebook, although the Programme Director does. Students will typically create their own Facebook groups for joint projects and exercises.

Smartphones mean Facebook is the preferred method for learners. Very successful on last minute changes of rooms / times etc.

3.2 Students

The responses reported here relate only to the Institutional HE Students. The responses from the FE students will be added when they are received.

Questions and Survey Responses Comments

(Sample total: 1706 answers) (Sample total: 759 comments)

0.065

0.565

0.710000000000001

0.20.025

Q1: To what extent do you use Moodle as a learning resource?

Moodle is a useful tool. I think it could be simplified even further, sometimes it is difficult to find something quickly. Also perhaps something could be done to stop the crashing of Moodle. It happens a little too often.

Works well for me, most lecturing staff use it, and it's easy to navigate. Problems with finding materials are usually due to staff not clearly naming files, or designing the layout of their sites/dumping everything into one massive list of files and links.

I found the older version of Moodle much easier and much better to use, this year’s version doesn't work well on android devices, so it doesn’t work very well on my phone or tablet.

0.095

0.350.2 0.25

0.1

Q2: Are your course materials available on Moodle?

On Moodle my lecturers upload lecture notes and assignment essay titles/deadlines. Not all lecturers upload as much material as maybe some of us would like but I think it is a great site to use regardless.

Resources on Moodle are very helpful for my course and include links to podcasts, YouTube clips, websites, images, and articles, as well as the lecture notes and reading lists.

Lecture slides, notes, references and links are made available. The only drawback with Moodle is that some of the information can be difficult to find.

0.205

0.725000000000001

0.0850000000000001 0.07

Q3: Do you use Moodle for assessment submission and feedback?

Yes I use Turnitin for the majority of my assignments and receive marks and feedback. A good point is that the feedback is always online and available to be read when you want it.

I use Turnitin for written assignments, but there are difficulties when using iPad, tablet, etc. It doesn't work because you have limited format & file size capabilities on Turnitin.

Have never used Turnitin (wouldn't know how to). I have completed interactive assignments on Moodle, but I've always submitted papers by e-mailing them directly to the faculty member concerned.

Never Occasionally Frequently All the time

0.390000000000001

0.385000000000001

0.165

0.065

Q4: To what extent do you use Moodle for course communications?

I only use Moodle for communication if the lecturer has posted some new information. I have never used Moodle for general communication.

If I have a question, I submit it directly to my tutor via email. There is very little information on the forums for my course; some of the information is years old and no longer relevant.

I communicate on Moodle occasionally with my lecturers, however, I prefer to communicate with classmates through Facebook if we are working together.

No

Yes, f

or inform

ation

Yes, t

o communica

te

Yes, f

or reso

urces

Other

0.125

0.750000000000001 0.6500000000

00001

0.5

0.015

Q5: Do you use any social media applica-tions in your learning?

I find that Google and Wikipedia can be a good starting point for a topic, just to understand what the topic being researched is about, and then do more research into academic sources for references.

I use Google to search for information and it is pretty easy to tell whether the information is useful or correctly sourced. My class uses Facebook to communicate about the course. This is really useful because nearly every student checks Facebook at least once a day.

Facebook proved invaluable when the IT infrastructure went down. We communicate as peers on the course and in special interest or project specific groups.

No Wikis Blogs Google docs

Other

0.335000000000001

0.215 0.23

0.475

0.16

Q6: Do you use any other online tools? Yes we do. We all use shared Dropbox drives and

Google docs to collaborate and we use Facebook / Email to keep in touch and send things to each other.

Open Athens with its various search engines and access to full-text articles and books. This is crucial as a distance-learning student.

Google scholar, Google image, Google books, Athens, Jstor, Prezi.

I use Wikispaces for group projects and blogs for my individual work as e-portfolios.

None

Smart

phone

Tablet

computer

Laptop PC

Other

0.01

0.660000000000001

0.45

0.88

0.5

0.01

Q7: What personal devices do you use to go online?

I use my phone to check Facebook and email for notifications on lessons and/or any changes in timetables etc. I use my laptop for my work and research.

I am multiply disabled (physical and mental health issues). My iPad is essential, I carry around loads of information and it diarises my day so I don't forget stuff!

I use my Smart phone to receive my student emails, but primarily use my laptop for everything learning related.

I use my phone and computers for learning access. I would love it if there were a Moodle app for iPhone.

0.525 0.375000000000001

0.49

0.025

Q8: What do you feel about using social media and personal devices in your learn-

ing?

If privacy settings used correctly on social media I welcome the chance to use it for my learning. It is an accessible way of sharing stuff on multiple devices.

I don't mind mixing work and social activities- my colleagues are also friends so I think this helps. By not keeping them separate, it allows me to quickly find whatever I need on whatever device I want.

I prefer to keep them separated which means I can take a break from my work by using something that isn't related to my work in any way.

I cannot imagine using Facebook, for instance, as a learning medium. That said, I guess I'd be open to it if it were needed.

4. Analysis and Conclusions:

Overall there were 544 responses to the survey which yielded 3354 answers to specific questions about the current use of Technology Enhanced Learning across the institution and 1445 comments relating to that usage. It is felt that this represents a substantial body of evidence that will be valuable in providing a benchmark of existing TEL practice and informing future development planning.

It should be acknowledged that the respondents to the survey were self-selecting and were hence individually motivated to respond. Typically it would be the enthusiasts and sceptics who have such motivation, but that does not invalidate the exercise at all, for it is they who are both promoting innovation and its attendant risk on the one hand and insisting on evidence based justification for the developments proposed on the other. Both are needed.

From a practical point of view, there are certain factors that need to be considered when analysing the outcomes of the survey. These include:

Technology Enhanced Learning largely relates to the use of computing technologies and the Internet in the support of learning. Both continue to develop rapidly and what we see today will not be what we see tomorrow. Therefore, using the survey outcomes to identify the trajectory of TEL usage and plan for that is where the real value lies;

The Educational community is in a transitional position with regard to TEL. Most existing academic staff teach through the conventional classroom model and TEL is seen as a beneficial add-on. The evidence from the survey indicates that it is perhaps only those concerned with distance learning who have actually embraced the affordances of online learner support as an alternative delivery model;

The real issue for learners is the extent to which TEL aids them in attaining their learning goals. Many of the comments made by the students in the survey refer to the barriers that the technology presents in terms of, for example, the accessibility and usability of Moodle, as well as the benefits of flexible any-time, anywhere access to learning resources and support.

Key Messages:

The quantitative data presented in this report averages the responses from all campuses and curriculum areas. A greater level of detail for planning purposes comes from a comparison between the average scores with the raw data from each campus survey. These are available for viewing, along with all the associated comments on the TEL survey website at:

http://teluwtsd.pbworks.com/w/page/75275948/TEL%20Survey%20Data

The key messages from the survey include:

On average 50% of staff report that they use Moodle as a major part of the teaching mix, whilst 50% report limited or no use of Moodle in their teaching. Reasons for not using Moodle include finding the navigation and functionality frustrating, as well as having available (and using) the alternatives of email, Facebook, Google docs and other social media for online communications and document sharing;

Where Moodle is used and course materials are available, 90% of students use it to access course information and teaching materials. The comments indicate that it delivers significant added value and flexibility to the learning experience, despite usability frustrations and lack of engagement by some academic staff;

A third of HE staff and two thirds of FE staff report that they do not use Moodle for student assessment. However, the HE students report that when assignment submission and feedback is available online, 75% engage with it and find it beneficial;

Only 25% of students use Moodle regularly for course communications (mainly email functionality). They rarely engage with the forums as they are seen to be poorly managed and lack purpose. The main non-Moodle course communications medium is email, and Facebook makes a significant contribution to the communications mix;

The question for staff about using Moodle for course management revealed a significant difference between use by HE staff and by FE staff, a difference that was reflected elsewhere in the survey. Over 51% of HE staff reported that Moodle was used for several management functions. Only 11% of FE staff did the same. Similarly, 25% of FE staff reported that they did not make their teaching materials available on Moodle, whilst only 8% of HE staff did the same. This is based only on the staff response from Coleg Sir Gâr and the FE survey needs to be completed before any firm conclusions can be drawn, but it appears to be indicative of the level of staff engagement;

The use of non-Moodle online resources in support of teaching and learning was clearly significant. The use of email communications was close to 100% and Facebook nearly 30%. The students reported that 48% used Google docs for collaborative working and there was widespread use of Google and Wikipedia for information gathering;

The students reported that they used multiple devices to go online. Most had at least one Smartphone and a wireless enabled laptop. A growing number used iPads and Android tablets. The majority used them in their learning: Smartphones for email and Facebook, laptops and tablets to access Moodle and online resources. Laptops also for taking notes in lectures and preparing assessment materials.

Conclusions:

Surveys require considered interpretation when drawing conclusions from the responses received. Important variables need to be taken into account. These include:

The self-selecting sample: whether their responses are representative of the whole institution and what evidence supports that.

The interpretation of the questions by the respondents: are they all answering the same question?

The agenda of the respondents: is it just factual information they provide or are they trying to make a point?

The analysis and conclusions presented here were based on such considerations and are obviously open to debate. All of the survey outcomes have been made available for others to evaluate and contribute to that debate.

The purpose of the survey was to inform TEL management planning for the newly merged institution. It gathered information from the previously separate universities and colleges and hence tells the story of their individual TEL engagement through the eyes of the staff and students.

Conclusions that may be drawn from the survey in this context include:

That TEL engagement across the institution is primarily based on the use of the Moodle VLE. All previously separate institutions use Moodle and the expectation is that a unified management system for the merged institution will be established;

The current engagement of academic staff with Moodle across the institution is only loosely managed. There is no consistent and systematic process that results in all staff making their teaching resources available online. The quality of service provision to students through the VLE is therefore variable;

A number of reasons for the lack of engagement by staff in the use of Moodle can be identified. These include:

o Insufficient familiarity with the functions and options available through lack of usage and training (although quality training opportunities are available);

o A perceived view that it is not user friendly and difficult to navigate. This may in be part be due to the lack of familiarity;

o The use by staff of other online communications and document sharing applications that are seen to be more effective and convenient than Moodle.

Despite the issues to do with usability, the students value Moodle as an important vehicle for their learning journey. The any-time anywhere access to learning resources is regarded as a significant benefit, as is the ability to submit assessments online and receive feedback;

Students are increasingly using the web as part of their learning. They may be directed in this by their tutors and library staff, but the outcomes of the survey indicate that they are doing it anyway. Typically they use Google, Google scholar, Wikipedia, JSTOR, Open Athens and other information and resource applications;

Students are also increasingly using their own wireless enabled devices to engage with their online learning resources. This clearly points to a future where such engagement is the norm and the conventional classroom model will adapt accordingly.

This survey report summarises the outcomes of an exercise designed to inform TEL strategy and planning at UWTSD. It is intended as a briefing document that alerts readers to the key points emerging from the survey and as an aid to planning discussions. An updated document will be made available when the FE survey is completed. This final version will also reflect comments and further information from institutional partners.

Tony Toole

March 2014