Venue:          T2-84, Eagle Tower , University of Bolton

Present:             

• Andy Graham, Director (Chair), Learning Retention & Professional Practice
• Ged Clarke (note taker), Learning and Teaching Fellow, Bolton Business School
• Rob Eggleton, Principal Lecturer – Professional Standards and Practice, Built Environment and Engineering
• Lisa Cove-Burrell (Principal Lecturer – Quality Teaching and Learning, Bolton Business School)
• Maria Rodriguez-Yborra, Reader in Technology-Assisted Learning, Arts, Media and Education
• Shirley Ward , Learning and Teaching Fellow, Liaison and Development Team Leader, Library
• Zubair Hansolt, Principal Lecturer – Operations and International Leader, Games Computing and Creative Technologies
• Gary Burns, E-Learning Systems Officer, Information Systems and Technology
• Paul Bacsich and Dick Hill, HE Academy consultants (BELA team)

  Apologies:           Received from other team members.   

The full agenda and minutes from the Practice Scoring Meeting were distributed. 

The main item of business was to review the trial scores developed in the scoring rehearsal meeting on 18 October 2007 in the light of new evidence and to consolidate slice scores into moderated institutional scores.  

This item was preceded by a brief discussion on final reports. 

The meeting was reminded that it had to process 25 criteria – the 20 core and five supplementaries. The meeting agreed to consider the criteria in four themes: 

1 Strategic/Planning (9 criteria)

2 Quality (3 criteria)

3 Learning and Teaching (7 criteria)

4 Technology (6 criteria) 

Thus the criteria were discussed in these categories, working in numerical order within each category. Adoption (P01 – part of Strategic/Planning) was taken first of all. 

The following points give a flavour of process issues: 

* the way in which for some criteria the institutional score is not an average of the slice scores and for other criteria it is

* the need for a good understanding of which blended learning courses contain a significant enough proportion of e-learning that one would expect this aspect to crop up in evaluation, decision-making, support etc

* how to cope with a slice where it itself is formed out of very different subslices

* suggested minor but useful improvements in wording for criterion scoring statements at some levels in the next release of Pick&Mix (P04 and P19 especially)

* how to deal with “trajectory” situations where the institution is in managed transition from one state to another, which may lead temporarily to lower scores against relevant criteria. 

Due to the excellent work done at the scoring rehearsal there was time in the agenda also to carry out some reflection on the implications of the scores – in particular to focus on examples of effective practice, opportunities and issues/constraints. This will feed forward into the reflection meeting which will be able now to start from a more strategic standpoint. In particular the future internal role of CETIS/IEC and the value of further development of the Ultraversity concept are likely to be of interest. 

The meeting was completed at 13:40.

Ged Clarke circulated a reminder of the Final Scoring Meeting. A spreadsheet was circulated to members of the Core Team, with the initial scores from our Initial Scoring Meeting (18 October 2007), questions to stimulate thought for each criteria, and extracts from some of the evidence gathered.  

Following Shirley’s feedback from the October Cohort meeting our criteria have been put into provisional “themes”. The use of themes should help to reduce the length/complexity of the final report (in January 2008). The final scoring will work from the initial score and use any new evidence and experience to decide if the score is still appropriate.

The meeting will also record special cases of good practice.  Scoring will rely on the representatives from each school to speak with authority (using their experience, and data from online questionnaires, additional questions and focus groups).

• We are obliged to use it – though it could be a useful checklist.
• The “themes” can be used to “lump” together the different criteria.
• The “Executive Summary” should provide senior management with the info that they need.
• Any detail should go in appendices.
• There is emphasis on “moving forward” & sustainability – to include recommendations and an action plan.
• There is a template for scoring sheets in the appendices that includes e.g. theme, definitions, leader, scoring workshop participants, comments & caveats, existing/new evidence, examples of effective practice, issues/constraints, opportunities/vision, recommendations…. in addition to the scoring!

They will be scheduled for the period between Monday 5^th and Tuesday 13^th November. 

Each focus group will respond to a predetermined set of questions. The questions reflected issues identified during the practice scoring meeting and the staff and student online questionnaires. The groups will be managed by the individual school; each group will have a facilitator and secretarial support. They are programmed to last for one hour and will be recorded. Reports will be published. 

Objectives for the staff focus group: 

1        To gather feedback from staff about their understanding of e-learning at the University of Bolton 

2        To record examples of good practice and areas for improvement 

3        To assess the extent to which e-learning is embedded at the University of Bolton 

4        To discuss the staff/student needs and aspirations for e-learning 

5        To encourage the staff to reflect on the development of e-learning and the opportunities to enhance their student’s learning ‘experience’.    

Questions for the staff focus group: 

(Criteria 06, e-Learning Strategy) 

Question 1:  Do you think e-learning (information and communication technologies) have improved the learning (and teaching) experience? Probes:  What do you think has improved? What could be better? Do you feel that there is a clear strategy/plan in your school? Do you feel well informed about the University’s e-learning strategy? Do you receive enough information? How should it be disseminated/shared?  What would you like more information about?  

(Criteria 01, Adoption)           

Question 2:  How would you describe the level of adoption of e-learning in  your school/department? (Ask for examples). Probes: What would encourage greater adoption?   

(Criteria 03: Tools)(Criteria 10: Training) 

Question 3:  Do you feel that you are given sufficient guidance in the use of e-learning tools (software, hardware)?  Probes: How do you become aware of tools (eg. webmail,      Athens, BISSTO, wireless access, social software…)?  Where/who do you ask for help? What training  is available to students in the use of e-learning tools?  Should students be expected to develop basic skills to support  their e-learning?   

(Criteria 08, Pedagogy) 

Question 4:  How has your teaching job/role (the way you teach) changed to   respond to developments in e-learning? Probes: Ask for examples.  Should pedagogy be given a greater emphasis in your school?     

(Criteria 10, Training)           

Question 5:  What training and technical support is available for staff to develop CIT skills?  Probes: Should training and updates be monitored? What                                 additional training do you feel should be offered?    

(Criteria 11, Academic Workload) 

Question 6: What would encourage you to invest more time in e-learning? Probes: Do timetables make any attempt to recognise the differences that e-learning courses have from traditional courses?What would the benefits of a work planning model that recognised the additional demands?     

(Criteria 13, Planning (Annual))(Criteria 19, Decisions) 

Question 7:  What role do you take in the development of local plans relating  to e-learning? Probes: How should the University share good practice? How effective are local decisions with regard to e-learning?                         

Objectives for the student focus group: 

1        To gather feedback from students about their understanding of e-learning at the University of Bolton 

2        To record examples of good practice and areas for improvement 

3        To discuss the student’s needs and aspirations for e-learning 

4        To encourage the students to reflect on the development of e-learning and the opportunities to enhance their learning ‘experience’.  

Questions for the student focus group: 

(Criteria 06, e-Learning Strategy) 

Question 1:  Do you think e-learning (information and communication technologies) have improved the learning experience? Probes:  What do you think has improved? What could be better? Do you feel that there is a clear strategy/plan in your school? 

(Criteria 01, Adoption)           

Question 2:  How would you describe the level of adoption of e-learning in your school/department? (Ask for examples). Probes: What would encourage greater adoption?   

(Criteria 03: Tools)(Criteria 10: Training) 

Question 3:  Do you feel that you are given sufficient guidance in the use of e-learning tools (software, hardware)?  Probes: How do you become aware of tools (eg. webmail, Athens, BISSTO, wireless access, social software…)?                                 Where/who do you ask for help? What training  available to students in the use of e-learning tools?  Should students be expected to develop basic skills to support  their e-learning?    

Broad Definition of e-Learning

Each of the focus groups were encouraged to consider a ‘broad definition’ of e-learning using the following guidance note: We are using a broad definition of e-learning. It can include learning that takes place using technology (PC, electronic smartboard, pda) either off-line (DVD,CDROM etc) or on-line (internet, WebCT)These are examples of published definitions:            

“If someone is learning in a way that uses information and  communication technologies ICTs, they are using e-learning. They could be a pre-school child playing an interactive game; they could be   a group of pupils collaborating on a history project with pupils in another country via the Internet; they could be geography students watching an animated diagram of a volcanic eruption their lecturer has just downloaded; they could be a nurse taking her driving theory test    online with a reading aid to help her dyslexia — it all counts as e-learning.”  DfES (2003) Towards a Unified e-Learning Strategy “Learning facilitated and supported through the use of information and communications technology (ICT).”  E-learning Pedagogy Programme: JISC — ProgrammesThe Government e-learning strategy defines e-learning as “any learning that uses ICT.”
HEFCE (2005) HEFCE strategy for e-learning

School Directors were advised to share the information with all of the PLs in their School. 

The questions are targeted and relate to specific benchmarking criteria. Ged Clarke will e-mail a set of specific/focused questions and a link to
the Criteria.The questions need short responses and supporting evidence(reports,
documents etc) and will be used to inform the decisions of the Final Scoring Meeting. The responses will not be published and the questions are not designed to be invasive. There are five sets of questions:

Set 1: School Director, School Manager, PL(s) with specific
responsibility
Set 2: Head, Information Systems and Technology
Set 3: Director, Learning Retention and Professional Practice
Set 4: Director, Finance
Set 5: Head, Quality Assurance and Enhancement

Feedback will be provided to Ged Clarke on or before Wednesday 14 November.

Final number of respondents: 

Staff: 167

Students: 226 

The Core Team was encouraged by the level of engagement, the quality of the responses and the positive nature of responses. The responses also included constructive criticisms of e-learning at Bolton.  

The Core Team will use the data to inform issues for the Focus Groups, Additional Questions (issued to academic schools and support units) and the Final Scoring Meeting.

Members of the Core Team: 

• Andy Graham (Director, Learning Retention & Professional Practice)
• Ged Clarke (Learning and Teaching Fellow, Bolton Business School)
• Rob Eggleton ( Principal Lecturer – Professional Standards and Practice, Built Environment and Engineering)
• Lisa Cove-Burrell (Principal Lecturer – Quality Teaching and Learning, Bolton Business School)
• Maria Rodriguez-Yborra (Reader in Technology-Assisted Learning, Arts, Media and Education)
• Debbie Abdel Nabi (Senior Lecturer, School of Health and Social Sciences)
• Mark Johnson (Reader in Applied Research in the Education Technology and Systems, The Institute for Educational Cybernetics ,ICE)
• Shirley Ward (Learning and Teaching Fellow, Liaison and Development Team Leader, Library)
• Patrick O’Reilly (Head of Information Systems and Technology)
• Zubair Hansolt (Principal Lecturer – Operations and International Leader, Games Computing and Creative Technologies.)
• Gary Burns (E-Learning Systems Officer, Information Systems and Technology)
• UBSU  President       

Venue:          Conference Room 1 

Present:              Andy Graham (Chair), Lisa Cove-Burrell, Gary Burns (representing Patrick O’Reilly),  Ged Clarke, Zubair Hanslot, Dick Hill (HEA Consultant), Mark Johnson, Maria Rodriguez-Yborra, Shirley Ward.  

Apologies:           Received from other team members. 

1. Review of Progress

Ged Clarke informed the meeting of progress made. Poppy Pickard had attended the Cohort Meeting in Falmouth (10 September 2007). Poppy’s report had been circulated to the Core Team. The Core Team had attended two meetings (Tuesday 2^nd and Thursday 5^th). The meetings discussed the management of the data collection tools.  

2. Data Collection. 

Online Questionnaires

The online questionnaires (staff and students) were launched on 12 October and were due to close at 5pm on Friday 19 October. Ged Clarke reported that the questionnaires were starting to gather data and the data was encouraging, although the participation was low in a couple of areas. The decision was made to extend the deadline to 5pm on Tuesday 23 October. Andy Graham would arrange for a final reminder for staff. The Students’ Union was very supportive and their help with the questionnaire was appreciated. Ged Clarke to advise SU of the revised deadline. The outcomes will be shared by Ged Clarke. He will seek permission (from Paul Birkett) to allow access to the online data; although data can be exported into MS Excel. 

Focus Groups

The focus groups should take place w/c 7 November. Each school will hold two focus groups, one for staff and one for students. The schools need to manage the process. Common documents will be used. Ged Clarke will distribute a draft of the paperwork (guidance notes and questions.  He will schedule a meeting w/c 22 October (Wednesday or Thursday) for the core team and nominated facilitators. The data from the online questionnaires will be used to frame/highlight questions/topics. 

Specific Interviews (Questionnaires)

In addition to the online questionnaire and focus groups a set of specific questions will be used to gather data from the academic schools and key members of staff.   Ged Clarke will distribute the draft questions to the Core Team for feedback (feedback by 26 October). It is expected that the number of questions will be reduced given the outcomes from the Practice Scoring (see below). The data should be collected and returned to Ged Clarke by Friday 9 November. School-based members of the Core Team will manage the completion of their school’s questions.  Andy has arranged admin support for the Focus Groups. 

Documents  

Please forward copies of any key documents (identified as evidence during the Interviews) to Ged Clarke ASAP.           

Blogs 

Ged Clarke will update the external Blog. The internal blog was not established. It will be considered for future data collection benchmarking exercises. 

3. Scoring 

The main purpose of the meeting was to practice score against each of the Pick&Mix criteria (First 20 and 55, 62, 64, 70, 72).  The members of the Core Team were invited to provide a score in turn before any discussion. The Chair then invited observations and comments to guide the meeting towards a single score for each of the 24 criteria. The process identified many scores which were shared/common, areas of significant difference, and some criteria which required a greater focus in the data collection activities (Focus Groups and Interviews/Questionnaires). 

Greater focus: 

09, Learning Materials- Score 1, Need more evidence

11, Academic Workload – Score 1, Further attention

12, Costs – Score 2, Need more evidence

14, Evaluation – Score 2, Further attention

19 , Decisions – Score 2, Further attention

62, Integration – Score 1, Need more evidence

64, Missed off the list !!!!! 

The process should help the Core Team to be better prepared for the Final Scoring Meeting in November (20^th). Core Team members should note the initial scores as a starting point and make use of additional evidence to arrive at their scores for the Final Scoring Meeting. Core Team should seek to attend the meeting with a set of their ‘considered’ scores representing their school/section.  

4. Cohort Meetings 

Shirley Ward is attending the next Cohort Meeting (London, Monday 29 October). The meeting will focus on the process of final scoring. There will be Phase 1 presentations from Derby and Chester.   Final Cohort Meeting will be at Northhumbria on Wednesday 6 December. The meeting will focus on report writing. Ged Clarke will attend, and another person would be useful.  

5. Final Scoring Meeting 

The Final Scoring Meeting will be held on Tuesday 20 November.  Dick Hill indicated that four hours should be sufficient. The meeting will be from 10:00 – 14:00. There will be cake. Gary Burns will be invited to attend. The meeting decided that there was no need to seek any additional attendees.  Final details and secretarial support need to be confirmed. 

6. Reporting Reporting to the HEA

The University is required to provide a two page report to the HEA before 15 January 2008. The report will focus on the benchmarking process and the University’s approach and experience. It will not relate to the actual scores. The template is available on the HEA Benchmarking Blog. 

Internal Report

We are expected to produce an internal report to distribute the outcomes of the benchmarking activity. A copy of Chester’s report had been distributed in September. The University is encouraged to make the report available to external readers (via the public blog?). 

Data for the HEA

The HEA will produce a summary report (anonymous) at a later date. They will probably request access to our Internal Report.  

7. Consultant’s Feedback  

Dick Hill expressed his satisfaction with the conduct of the meeting and that we had made good progress.   

8. Date and Time of Next Meeting 

The next meeting will take place on Tuesday 20 November at 10:00hrs in Room TBA. 

The Chair thanked members for their attendance and the meeting closed at13:50hrs

This is the launch information: 

HEA/HEFCE Benchmarking of e-Learning in UK Higher Education STAFF ONLINE QUESTIONNAIRE 

The University has joined the second phase of a national benchmarking initiative. The activity will allow us to reflect on progress that we have made ‘embedding’ e-learning and will be ‘measured’ against a set of published criteria.  The assumed ‘definition’ of e-learning is broad and far wider than the ‘narrow’ adoption/use of a VLE and includes learning that takes place using technology (PC, electronic whiteboard/smartboard, pda) either off-line (DVD,CDROM etc) or on-line (internet, WebCT).  Here is the link to our staff questionnaire. It should not take any longer than ten minutes to complete.  It includes a specific set of questions aimed at academic tutors.

The survey will open at 12:00 noon on Friday 12 October and close at 5pm on Friday 19 October.   http://www.survey.bris.ac.uk/bolton/elstaff  

I will distribute a separate message with details of the student questionnaire. The link to the student questionnaire will be available via the Current Students homepage and the Library homepage. 

Thank you Ged Clarke, Learning and Teaching Fellow  

Benchmarking of e-Learning in UK Higher Education: Student Questionnaire  

Hello Please can you encourage your students to respond to an online questionnaire? The University has joined the second phase of a national benchmarking initiative. The activity will allow us to reflect on progress that we have made ‘embedding’ e-learning and will be ‘measured’ against a set of published criteria.  The assumed ‘definition’ of e-learning is broad and far wider than the ‘narrow’ adoption/use of a VLE and includes learning that takes place using technology (PC, electronic whiteboard/smartboard, pda) either off-line (DVD,CDROM etc) or on-line (internet, WebCT). This is the link to the student questionnaire. It should not take them any longer than ten minutes to complete. 

The link will also be available on the Current Students homepage and the Library homepage. http://www.survey.bris.ac.uk/bolton/elstudent The survey will open at 12:00 noon on Friday 12 October and close at 5pm on Friday 19 October.  

I have attached a PowerPoint presentation (4/5 slides) that you are welcome to use with your classes and information slips that you might distribute to your students.  If you are using a computer lab you might be able to allocate a few minutes for the students to complete the questionnaire. The link to the staff questionnaire was posted in the Weekly Bulletin (Friday 12 October).  

Thank you 

Ged Clarke, Learning and Teaching Fellow, x3131, M2-31, gc2@bolton.ac.uk

Ged and Debbie met to review the final set of questions for the online questionnaires.

The questions were developed from the examples of good practice distributed by Peter Chatterton, with local questions added to support Ged Clarke’s wider interest in the use of social software by staff and students.

The questionnaires are to go live on Friday 19 October. Ged will transfer the questions from the spreadsheet to the Bristol Online Survey software.