H817 – 20b – Week 24 – Activity 19 – Reflecting on the ROMA framework
In your learning journal, take the problems and barriers you identified and listed in the previous two activities and map them to the steps of the ROMA framework. You could do this in the form of a list, or you could use presentation software, such as PowerPoint or Keynote, to produce visualisations of the links between steps and issues.
Having read the relevant section of the Ferguson et al. (2015) paper I have listed the barriers identified in the previous activities in the table below. I have associated each of the barriers with the adapted stages of the ROMA framework with more detailed notes (shown as such if quoted directly from the paper) where appropriate.
| Identified Barriers | Mapping to ROMA Framework | Notes |
| Is there ongoing funding available for the project ? | 1. Define a clear set of overarching policy objectives | “This approach to systemic policy and strategy development begins with collaborative development of a vision through the formulation of policy objectives.” |
| Will teachers accepts new working methods? | 3. Identify Key Stakeholders | “The people and processes that affect whether this evidence can be effectively introduced into the policy process. |
| Motivation, will staff see the necessity ? | ||
| Will peer support among teaching staff or students be sufficient to support the changes. | ||
| Privacy expectations of students / other participants | 5. Develop a strategy | “Anticipating ethical dilemmas”, “Establishing data policy and governance measures” |
| Would vulnerable students require a guardian to approve these changes? | ||
| What exists in the system at the moment? Could it be leveraged in any way? | 6. Analyse Capacity; Develop Human Resources | The institution will required a great many skilled individuals to implement the changes. |
| Are new teaching methods required or does this innovation support an existing practice? | ||
| What working parties / SWGs exist who could adopt the new practices | ||
| How does the innovation fit with the existing infrastructure (network, server infrastructure, data use policies. | ||
| Are the technical skills available among the staff to effectively run the project? | ||
| How many staff and with what experience and qualifications will be required if not ? | ||
| How will efficiencies discovered by analytics be received? | 7. Develop a monitoring and learning system | “…assess the effectiveness of the approach, learn lessons for the future” |
Make notes of your reflections on this mapping. Did any of the issues you identified in previous exercises map clearly to steps in the framework? Did the framework prompt you to view some of the problems in new ways?
I was surprised to find that many of my identified barriers mapped directly to points in the framework. Most notably those relating to point 6 – Analyse Capacity ; Develop Human Resources were clearly connected. I had identified that the availability of skilled staff would be a major hindrance to the progress of any analytics project.
The practical approaches to implementation are more clearly defined and explained in this paper than in Scanlon et al. (2013). I found the detail of “The Beyond Prototypes model of the TEL Complex”, Ecology of practices and technical context was not helpful in visualising how analytics could be implemented in a real scenario.
The ROMA framework provides a solid, iterative process which is designed to assist with actual implementation in practice. It would be a valuable reference for creating a strategy document.
References:
Ferguson, R., Macfadyen, L.P., Clow, D., Tynan, B., Alexander, S. and Dawson, S. (2015) ‘Setting learning analytics in context: overcoming the barriers to large-scale adoption’, Journal of Learning Analytics, vol. 1, no. 3, pp. 120–44 [Online]. Available at http://oro.open.ac.uk/ 42115/ (Last accessed 21st July 2020)
Scanlon, E., Sharples, M., Fenton-O’Creevy, M., Fleck, J., Cooban, C., Ferguson, R., Cross, S. and Waterhouse, P. (2013) Beyond Prototypes: Enabling Innovation in Technology-Enhanced Learning. Technology-Enhanced Learning Research Programme [Online]. Available at http://beyondprototypes.com/ (Last accessed 21st July 2020)