L4D Learning for Democracy

"                                 EXECUTIVE SUMMARY

With the 2006 European Recommendation on Key Competences (Council of the European Union, 2006), all EU Member States have agreed on a framework of eight Key Competences for Lifelong Learning. Competences are defined as a combination of knowledge, skills and attitudes appropriate to the context and are seen as necessary by all citizens for personal fulfilment and development, active citizenship, social inclusion and employment. The eight Key Competences include: communication in the mother tongue, communication in foreign languages, mathematical competence and basic competences in science and technology, digital competence, learning to learn, social and civic competences, sense of initiative and entrepreneurship, and cultural awareness and expression.
European countries have made significant progress in incorporating these key competences into national curricula and other steering documents (Eurydice, 2012). However, one of the key challenges for education systems in many European Member States is the assessment of these competences (European Commission, 2012a).
Information and Communication Technologies (ICT) offer many opportunities for the provision of assessment formats which comprehensively capture all Key Competences, respecting the importance of the skills and attitudes dimensions and accounting for the less tangible themes underlying all Key Competences, such as critical thinking or creativity.
There is a vast range of formats and approaches which can foster different aspects of key competence development and can be used to address the specifics of each key competence in a targeted way. However, take-up and implementation in school education is still low. To seize the opportunities offered by ICT, targeted measures are needed to encourage the development, deployment and large-scale implementation of innovative assessment formats in school education.
The conceptual landscape
Currently, two conceptually different approaches to assessing Key Competences have been developing in parallel. On the one hand, Computer-Based Assessment (CBA) approaches have been employed for more than two decades and now go beyond simple multiple choice test formats (Generation 1&2 testing). With this new “Generation Re-Invention” or “transformative” testing, questions are increasingly integrated into more complex and authentic problem contexts, so that the full range of Key Competences can be assessed. Additionally, due to technological advances, a wider range of answer formats, including free text and speech, can be automatically scored. These question formats are already used to a certain extent in national and international large-scale tests and there are numerous interesting small-scale experiments, trials and pilots. However, in school education, the potential of transformative testing remains untapped.
On the other hand, technology-enhanced learning environments offer a promising avenue for embedded assessment of the more complex and behavioural dimensions of Key Competences, based on Learning Analytics. Many of the currently available technology-enhanced learning environments, tools and systems recreate learning situations which require complex thinking, problem-solving and collaboration strategies and thus allow for the development of generic skills. Some of these environments allow learners and teachers to assess performance, understand mistakes and learn from them. There is some evidence of how data on student engagement in these environments can be used as a basis for assessment and can also be directly fed back to students. However, on the whole, many of these programmes and environments are still experimental in scope and have not been mainstreamed in education and training. Embedded assessment has not yet matured, nor has it been widely or critically studied.

Using ICT to support the assessment of Key Competences
Different Key Competences require different approaches to assessment. Thus, the strengths of
different technology-enhanced assessment strategies depend on which competence and
competence area is to be addressed. Figure 2 illustrates these focus areas. Currently, the potential
of ICT in the comprehensive assessment of Key Competences, including less tangible and generic
skills, remains largely untapped.
Current strategies for using ICT to foster competence-based assessment focus on Computer-
Based Testing, online quizzes or simple games, and ePortfolios.
Computer-Based Testing is used widely and successfully for the summative and formative
assessment of (basic) literacy and (advanced) reading skills and basic mathematical skills. Literacy
and numeracy skills are being assessed in a range of national and international tests, which are, in
many cases, electronic. Due to the nature of these competences, it has been possible to embed
complex and authentic tasks in the multiple-choice format, so that mathematical competence can
be comprehensively and reliably assessed by computer-based tests.
In general, however, computer-based tests tend to replicate traditional assessment formats, which
focus on knowledge rather than skills and attitudes, and are not usually employed as a means of
supporting more personalised, engaging, collaborative or authentic tasks. The advantage of
computer-based tests over traditional assessment formats is that they provide instant and targeted
feedback and can automatically adapt the difficulty of the test items to learners’ different
performance levels, to support formative assessment.
The internet is a vast resource for free and commercial computer-based quizzes, games and
tests which can be used in the development and assessment of competences in literacy, reading
and text comprehension and mathematics, in primary and secondary education. However, games,
quizzes and engaging test formats are currently scattered, isolated, limited in scope, and ill-suited
to comprehensive use in curricula and teaching.
ePortfolios are ideally suited to the assessment of collections of work produced by students and
are thus particularly powerful tools for communication in the mother tongue, communication in
foreign languages and cultural awareness and expression. ePortfolios are already widely used in
European schools as a means of supporting the formative and summative assessment of students’
creative productions. However, more innovative formats of cultural and artistic expression, such as
blogs, wikis, tweets, audio and video recordings, etc., are seldom included. Educators often do not
realize that ePortfolios can also be powerful tools for encouraging online collaboration and also
self- and peer assessment, which contribute to and at the same time assess students' learning to
learn skills.

Promising trends for the future include the increased use of digital learning environments in
education, the emergence of virtual environments which replicate authentic learning contexts and
educational software which provides detailed and immediate feedback and allows teachers to
adapt the learning process to each individual learner’s pace and level. The key technology, which
makes behaviour in these and other digital learning environments not only trackable, but also
assessable, is Learning Analytics.
Technology-enhanced learning environments, which are often used in higher education and are
starting to be deployed in school education as well, are used by some schools as a means of
creating learning situations which require complex thinking, problem-solving and collaboration
strategies. Some of these environments allow learners and teachers to assess performance,
understand mistakes and learn from them. The use made of these tools depends highly on
individual teachers’ intentions.
Immersive environments and multiplayer games recreate learning situations which require complex thinking, problem-solving and collaboration strategies and thus allow the development of these skills, which are key components of all eight Key Competences. These environments replicate authentic contexts; encourage collaboration, empathy and negotiation; and reward strategic thinking, initiative and experimentation. For competences in science, in particular, computer simulations and virtual laboratories provide opportunities for students to develop and apply skills and knowledge in more realistic contexts and provide feedback in real time. Practical tasks, embedded in electronic environments, using mobile devices or online resources, are a further promising avenue for developing ICT-enabled assessment formats that better capture Key Competences, particularly digital competences. Since learners’ behaviour in these electronic environments is tracked, their individual learning journeys – and also their possession of these skills – can, at least in principle, be automatically assessed.
Online simulations, virtual laboratories and games fostering competences in science are readily available, often for free. However, though they are being successfully used by many teachers and learners, these tools have not yet been mainstreamed. Environments that follow a games-approach are usually employed outside the curriculum, with no assessment of students’ performance or learning gains.
Educational software solutions such as intelligent tutoring systems combine embedded (formative) assessment with instant feedback and targeted support. For example, these systems allow students to investigate mathematical concepts and problems in complex contexts, at their own pace. These systems include a series of tasks, which can be adapted in level of difficulty, and have helpful hints to encourage students to develop adequate solution strategies. Whereas these tools are popular in the US, they are not widely used in Europe.
In general, Learning Analytics is one of the most promising emerging technological trends for the comprehensive assessment of complex competences. Learning Analytics involves the interpretation of a wide range of data produced by and gathered on behalf of students in electronic environments in order to assess progress, and tailor education to individual students more effectively. Learning Analytics could allow assessment to be embedded in immersive environments, multiplayer games and computer simulations.
Challenges and Ways Ahead
On the whole, many of the more promising tools and environments for the assessment of Key Competences are still experimental in scope and have not become mainstream in education and training. In particular, learning analytics and embedded assessment, which are expected to become the most promising technological innovations for the assessment of Key Competences, have not yet matured and have not been widely or critically studied. Since technology is constantly evolving and many of the other more viable and interesting assessment formats for the assessment of Key Competences have only recently become available, it is not surprising (or worrying) that take-up in schools is slow.
Technological research and development should be focused on the most promising emerging techniques for comprehensive competence-based assessment, such as Learning Analytics and dynamic and interactive educational software for self-regulated learning.
Development, deployment and implementation of existing technological solutions should focus on increasing their scope, usability, variability and curricula-fit and also on the integration of complex and authentic assessment tasks and self- and peer-assessment options.
Pedagogical strategies that use ICT for the assessment of Key Competences should choose assessment formats that encourage alternative solutions and promote experimentation; promote self-regulated learning through self- and peer-assessment; and create learning contexts that allow learners to express themselves across a range of media and communication formats.

Policy Recommendations
To encourage the take-up of available tools and applications in schools, more policy support and guidance is needed for teachers, learners and parents.
In particular, the following policy options should be considered:
→ Improve usability and curricula-fit: Policy action is needed to support the development, use and accessibility of ICT environments and tools that take into account curricular needs and are better suited to use by teachers as part of their daily teaching practice.
→ Teacher networking and knowledge exchange: Many teachers are not aware of the possibilities that ICT offer to enhance assessment for learning. Teacher networks can facilitate knowledge exchange and learning and can contribute to upscaling and mainstreaming existing good practice.
→ Research and development should devote more attention to innovative learning and assessment environments, such as educational multiplayer games and simulations, and consider how learning analytics can meaningully be used to foster formative assessment.
→ Encourage discussion and provide guidance: A critical and open discourse among educators, researchers and policy makers is needed on the advantages and drawbacks of ICT-enhanced assessment strategies, in order to identify viable strategies that allow the comprehensive assessment of all Key Competences for Lifelong Learning"

                                    TABLE OF CONTENTS

EXECUTIVE SUMMARY ...................................................................................................................... 2
1. INTRODUCTION .......................................................................................................................... 8
2. ICT-ENHANCED ASSESSMENT STRATEGIES ................................................................... 11
2.1 eAssessment ....................................................................... 14
2.2 Computer-based assessment tools for schools ......................... 21
2.3 ePortfolios ....................................................... 25
2.4 Peer assessment ............................. 28
2.5 Embedded Assessment ................................... 32
2.6 Augmented reality, virtual worlds, immersive games .............. 34
3. THE USE OF ICT FOR KEY COMPETENCE ASSESSMENT IN SCHOOLS .................. 38
3.1 Communication in the mother tongue ............. 42
3.2 Communication in foreign languages ................... 47
3.3 Mathematical competence ..................................... 52
3.4 Basic competences in science and technology ................... 57
3.5 Digital competence ............................................................. 64
3.6 Learning to learn ........................................................ 67
3.7 Social and civic competences ...................................... 71
3.8 Sense of Initiative and Entrepreneurship .............................. 74
3.9 Cultural awareness and expression .................................... 77
4. CHALLENGES ........................................................................................................................... 80
5. CONCLUSIONS AND POLICY RECOMMENDATIONS ...................................................... 83
6. BIBLIOGRAPHY ........................................................................................................................ 86