Valuing technology-enhanced academic conferences for continuing professional development. A systematic literature review

ABSTRACT

This review presents a systematic search for and analysis of the state of the art concerning research (1993–2018) on technology-enhanced conferences for academics’ professional development. Fifty-nine scientific publications were included in the review which analyses them through the lens of the value creation framework. Conference formats are undergoing innovations focussed on amplifying social learning, and the role of technologies to enrich this new landscape is being explored. Initial results indicated that while new practices are emerging, a coherent perspective on technology-enhanced continuing professional development to help understand and inform the transition towards learning conferences was lacking across the literature. For instance, traditional evaluations of conferences, such as satisfaction surveys applied by the end of the conference, are not yet taking into account the full range of possible values created through participation in conferences. In addition, results about the use of social media for community building and enduring professional development remain inconclusive, and a more guided approach towards the application of social media at academic conferences is needed. The Value Creation Framework seems to be an appropriate conceptual framework for understanding the impact of conference attendance for the development of (digital) professional competences of academics.

KEYWORDS:

• Academic conferences
• communities of practice
• continuing professional development
• learning conferences
• social media
• value creation framework

Introduction

Continuing professional development (CPD) for academics is critical in times of the increased speed of innovation and intensification of responsibilities of the academia (Ferman 2002, Collini 2012). New competencies are essential to fulfil the required functions in the field of knowledge and teaching, but also leadership and administration. King (2004) states: ‘Professional development for all elements of the academic role (including teaching and research) needs to be considered as a normal part of professional life for all academic staff’. While the requirement for CPD in (Higher) Education is recognised, the nature of it is viewed through different lenses. Caffarella and Zinn (1999, p. 242) identified three categories of continuing professional development: ‘(1) self-directed learning experiences; (2) formal professional development programs; and (3) organizational development strategies.’ Kennedy (2005) has introduced a framework for teachers´ continuing professional development with nine categories. Besides a classical training or skill-oriented model, the community of practice model of professional development is regarded as one direction of professional development the author reframed in a later publication as CPD in learning communities (Kennedy 2014).

One can distinguish CPD between formal, non-formal and informal professional learning experiences or activities (King 2004). Professional development activities are furthermore placed on scales from passive to more active (Meijs et al. 2016), or are situated, for instance, in communities of practice as networked learning (Li and Krasny 2019). Thus, CPD for academics takes different forms and takes place in different contexts. In the study by King (2004) academics mention ‘networking with colleagues from other institutions’ as one of the top three sources of CPD. This networking happens at academic conferences.

Academic conferences are usually seen as providing social and non-formal learning experiences (Sangrà, González-Sanmamed, & Guitert, 2013) as well as opportunities for the formation of enduring and productive communities of practice (Wenger, 1998b) (CoPs) and social networks (Travers et al. 2008, Thatcher et al. 2011). Therefore, attendance of conferences is part of the ‘academic citizenship’ (Macfarlane 2007). Nonetheless, Jacobs and McFarlane pointed out that ‘little attention has been paid either to developing a theoretically informed understanding of conference practice as knowledge building, or to assessing the extent to which conferences are successful’ (Jacobs and McFarlane 2005, p. 317).

The topic of non-formal or informal CPD (Looi et al. 2010) through conferences has not yet been investigated with a systematic approach although it is gaining importance with the integration of (connective) technologies, such as microblogging, in the practice of conference attendance. To date, no available systematic literature review addresses the potential of emerging connective technologies to amplify attendees’ social learning practices. Technologies have the potential to take the world into the conference room, and to connect the conference to the outside world (Kelly et al. 2005). In this context, Web 2.0 and social media tools play a prominent role. Selwyn (2008) defines Web 2.0 as ‘an umbrella term for a host of … internet applications such as social networking, wikis, folksonomies, virtual societies, blogging, multiplayer online gaming and mash-up’ (p.4) Tools such as Twitter or Facebook enable a multi-directional communication and interaction. Events can be amplified (Kelly 2011, Osborne 2011) beyond physical and temporal boundaries (Bombaci et al. 2016, Udovicich et al. 2016), but also beyond the typical communities of interest (Pitkin and Shabajee 2012, Deardorff 2015, Su et al. 2016).

In line with some authors (Jacobs and McFarlane 2005, Thatcher et al. 2011), we perceive academic conferences as communities, contributing to continuing professional development beyond the event moment (Anderson and Mason 1993).

Conference organizers need strategies to go beyond the traditional conference formats that are often described as ‘back-to-back’ and ‘sage on the stage’ (Anderson and Anderson 2010, p. 13). In the view of Zuber-Skerritt (2017), ‘many academic, scientific and professional conferences do not seem to offer sufficient opportunities for delegates to engage actively in collaborative learning from dialogue, interchange and critical reflection.’ In the same line, Ravn and Elsborg (2011) characterised traditional conferences as using unidirectional communication, and they pointed out the following critical issues of conventional conferences: (a) too much lecturing; (b) too little time for digestion and reflection; (c) often frustrating group work; (d) workshops as misnomer; (e) experts panel as just more one-way communication; and (f) the ‘Network Lunch’ not being one. Solutions for noted restrictions on continuing professional development through conferences have emerged in two different directions: face-to-face (F2F) conferences are being innovated by the introduction of new session formats, and innovative hybrid conference formats are being designed. In hybrid formats, technologies are being introduced to innovate the traditional conference formats. It is imperative to explore new tools and applications concerning the way the attendees can participate and interact (Siemens et al. 2008).

The systematic review was intended to synthesize to what extent and how continuing professional development can be amplified through technology-enhanced academic conferences, taking a socio-cultural perspective on this non-formal way of CPD.

The objectives of this literature review were as follows: (a) identification of research related to the role of technology-enhanced and amplified conferences for academics’ CPD; (b) identification of research gaps, and presenting suggestions for further research on the role of technology-enhanced conference in CPD.

The rest of this article is structured in the following way. We first introduce the Value Creation Framework (VCF) as a conceptual framework and then present the methodology used for the systematic literature review. Subsequently, we present the results based on the VCF, discuss their implications and highlight the strengths/limitations of the review, outlying further research in the field on technology-enhanced conferences for CPD.

The VCF as an evaluative lens for technology-enhanced conference practices

The Value Creation Framework (VCF) (Wenger et al. 2011) is a conceptual framework for promotion and assessment of value creation in CoPs and social networks. As an evaluation framework for non-formal learning at and through conferences it is helpful for researchers, attendees, and event organisers.

The success of conferences is often evaluated with traditional metrics such as participant satisfaction indicators, which mostly represent the organisers’ perspective. Evaluations have so far focused mainly on the value of transferred knowledge but not on the social networking and community aspects. Also, indicators such as ‘intention to act on knowledge gained’ or ‘agenda/policy changes based on conference outputs’ (Neves et al. 2012) are rarely reported. Taking a socio-cultural perspective on continuing professional development through conferences implies that the evaluation metrics must adapt. In addition, the integration of technology (e.g. social media) has consequences. In other words, the evaluation should take into account community aspects and the possibilities provided by technological amplification.

Wenger et al. (2011) have developed a conceptual framework for promotion and assessment of value creation in CoPs and social networks. The framework proposes five cycles in which different kinds of value are created:

1. Immediate value (refers to participation in the networked learning activity);

2. Potential value (refers to the knowledge capital that the network produces);

3. Applied value (refers to the way practice has changed);

4. Realized value (refers to the application of knowledge capital and its impact);

5. Reframing value (refers to the redefinition of strategies, as well as values, at individual and institutional levels).

Table 1 presents the 5 cycles of the VCF accompanied by queries which clarify their focus.

Table 1. The 5 cycles of the VCF, adapted from Wenger et al. (2011).

Cycle	Title	Focus	Questions
1	Immediate value	Activities and interactions	What happened during the conference? What was my experience with the technology-enhanced conference?
2	Potential value	Knowledge capital	What can be seen as outcome from the conference and can be used in the own context in a near future?
3	Applied value	Changes in practice	What difference did the conference attendance make in my context? What do I use now? A new tool and strategy within my technology-enhanced teaching?
4	Realised value	Performance improvement	Which contextual impact did it have? Do I perform better now by using a new strategy based on an insight?
5	Reframing value	Redefining success	What changed in terms of processes and understandings, e.g. in the institutions as consequence of a conference attendance?

We understand value creation in this study in line with Wenger et al. (2011) as ‘the value of the learning enabled by community involvement and networking’. The VCF includes indicators for value creation, like interactions (e.g. levels of engagement, networking and reflection), knowledge capital (e.g. changes in perspective, developing networks), changes in practice (e.g. reuse, implementation, use of social connections) and performance improvement (e.g. personal and organizational performance), resulting from interactions in CoPs. We adapted the framework taking into account indicators of CPD in the context of technology-enhanced academic conferences which is a new context of use for the VCF.

Method

For this study, we followed guidelines for systematic literature reviews (Moher et al. 2009, Jesson et al. 2011) and in the initial phase of the review process adopted the Grounded Theory Literature Review Method by Wolfswinkel et al. (2013) which offered guidance for conducting a rigorous literature review using this method. Starting without pre-supposed ideas, we undertook the five stages of the Grounded Theory reviewing method: Define, Search, Select, Analyse and Present. In the process, the main question for the review was refined. In the analysis phase and when attempting to present results in a meaningful way, the necessity for applying a conceptual framework emerged. The VCF was considered as a suitable framework to this end.

Search strategy

We first conducted a search in six databases in April 2018 (in alphabetical order): BASE; EBSCO Host; ERIC; IEEE Digital Library; Taylor & Francis; and Web of Science. Databases were selected based on the recognized scientific importance in the field of educational research, social and computer sciences.

The process of conducting the literature review started by an ‘exploratory review’ and passed by the establishment of the framework within which we analysed and present the results. Initially, we used ‘learning’ and ‘academic conferences’ as a keyword combination to capture relevant literature without providing a pre-focus. The small number of results when searching in the title or keywords level was fallacious as ‘learning at conferences’ is often implicitly use and put on the level of ‘knowledge building and sharing’ at conferences and learning at CoPs. On the other hand, as we are interested in new ways of amplified participation at conferences, namely through social media, we also used ‘academic conferences’ in combination with ‘social media’ and ‘twitter’ as keywords.

To identify additional studies and possible gaps (e.g. relevant publications not yet indexed in databases), the authors consulted the reference lists of all included final full-text review and created an alert in Google Alerts for ‘academic conferences’ to maintain the search up-to-date.

Selection criteria

Inclusion criteria

Articles included in this review focused on academic conferences, events organized by educational institutions, universities and alike. Articles were classified as relevant if their primary focus was to report about, explore or analyse academic conferences and the use of emerging technologies and Web 2.0 services/tools in context. Articles were eligible for inclusion if they provided reflections or reported research on academic conferences, empirical data on the effectiveness of the use of emerging technologies and applications, the impact on the degree of knowledge transfer and the promotion of communities and networks, or described innovative practices in the field of academic conferences. Documents were eligible if published in peer-reviewed journals or conference proceeding with peer-review.

Exclusion criteria

We excluded articles with a focus on commercial (professionally organized) conferences or for-profit corporations or associations. We also excluded articles that were descriptions of events, reflecting private opinions. Consequently, regarding the methodological quality, we excluded publications with poorly described samples or outcome measurements or conclusions that were not supported by results found.

Flow of the review on academic conferences

Our first search identified 1130 potential publications for review, ranging from 1993–2018. Through Google Alerts, we identified four additional articles. We also identified relevant papers cited by articles included in our review. This process added 15 publications. Furthermore, we examined the references for the studies consulted. Twenty-six additional papers were identified.

The initial screening focused on title, abstract and keywords. We removed the duplicates (n= 102) after the process of screening. Subsequently, the identified articles were scrutinized to ensure that they fulfilled the established criteria. We excluded 879 articles, as they did not match with the criteria. The full text of the 149 articles was screened and 81 were considered to fit the defined inclusion criteria. From the 81 articles, 22 focused on conferences with no special reference to the use of technology, the remaining 59 focussed on technology-enhanced conferences (cf. Figure 1) for CPD.

Figure 1. Flow of the review on academic conferences (based on Moher et al. (2009)).

Data analysis and categorisation

The selected publications were analysed in two steps. The first step consisted in analysing articles and characterizing them as follows: publication title, author(s), publication year, document type, abstract, keywords, study design, data collection and analysis.

Qualitative classification was performed using an open data analysis approach, stage four of the Grounded Theory Literature Review Method (Wolfswinkel et al. 2013). Each publication was reviewed to identify its focus. Based on the articles’ titles, keywords and abstracts, we identified dimensions such as, learning at conferences, knowledge building and sharing at conferences, CoPs, social networks, conferences and online/virtual conferences and the microblogging usage. These dimensions were interpreted as ways of creating value through conferences.

In accordance with the objectives, the 59 articles focusing on technology-enhanced conferences were analysed and organized through the lens of the VCF (see Table 2). The framework allowed us to define indicators for value creation at academic conferences. For instance, the level of social engagement; the quality of interactions and collaboration at conferences can act as indicators for immediate value creation at conferences. Within the categories of the VCF, we placed dimensions such as the described aims for a conference, the evaluation metrics applied in the reviewed studies, as well as qualitative classification of conference tweets as indicators for value creation.

Table 2. Subdimensions of the value creation, adapted from Wenger et al. (2011).

Cycle	Created Value	Focus	Subdimensions
1	Immediate value	Activities and interactions	Level of participation and engagement Quality of interaction
2	Potential value	Knowledge capital	Skills acquired Information received and artefacts created Change in perspective and Inspiration Types and intensity of social relationships Networking and the value of social connections Collaboration
3	Applied value	Changes in practice	Innovation in practice Reuse of products Use of social connections Transferring learning practices
4	Realised value	Performance improvement	Personal performance Organisational performance Organisational reputation
5	Reframing value	Redefining success	Institutional changes New frameworks

Results

In the sections below, we present the findings on each of the five dimensions and sub-dimensions of the VCF.

Immediate value

The immediate value of conference attendance for academics’ CPD is widely accepted as contributing towards career growth. The following sub-sections provide examples of findings for immediate value at conferences in the reviewed literature.

Level of participation and engagement at conferences

The number of attendees at a conference is one obvious indicator for the level of participation and thus the (immediate) value expected by the attendees, providing options for learning interactions. Still, in assessing the level of engagement, there is a great difference between the passive and active attendance at conferences. The use of Twitter at conferences is becoming more widespread; increasingly so for informational activities around conferences (Parra et al. 2016). According to Wilkinson et al. (2015) the use of technology contributes to effective engagement with the conference topics and with other attendees promoting learning, sharing of knowledge and networking.

Since we searched specifically for literature about technologies being used at conferences, our review returned many articles mentioning the popularity of participation through (micro-)blogging (e.g. Mazarakis and Peters 2015). Microblogging can be envisioned as a combination of blogging, instant messaging, social networking and status notification (Boyd et al. 2010, Mahrt et al. 2013).

Due to its popularity, using Twitter became a synonym for microblogging. Twitter, apart from being a communication tool (Java et al. 2007, Weller and Puschmann 2011, Weller et al. 2011) can be framed as a platform, supporting CoPs (Ebner et al. 2010, Chen 2011, Ebner 2013). Microblogging as a digital backchannel to conference participation is gaining ground (McKendrick et al. 2012, Kimmons and Veletsianos 2016). A backchannel (Toledo and Peters 2010) can be used to share and provide access to links, but also as a space where discussion and interaction take place (Grosseck and Holotescu 2010, Gesthuizen and Rablin 2014).

Considering the number of articles on the use of Twitter at conferences, there are indications for an increased level of active participation at technology-augmented conferences; it suggests this participation is valuable for academics. In contrast, some authors (Jacobs and McFarlane 2005, McCarthy and Boyd 2005, Ross et al. 2011) pointed out a digital backchannel can be a source of distraction. Using social media for enhancement of the value of conference attendance is far from optimized. Ross et al. (2011, p. 232) highlighted there ‘is a tendency for a small group of users to produce the majority of tweets’. They suggested it might be intimidating for newcomers to the field to participate in conversations. Some authors (Wen et al. 2014, Atzmueller and Lemmerich 2018) highlight there is homophily, a sociologic ‘principle that a contact between similar people occurs at a higher rate than among dissimilar people’ (McPherson et al. 2001, p. 416), between certain groups. This challenges the hypothesis of, for example, Reinhardt et al. (2009), who argue Twitter is an opportunity for young researchers to become actively engaged with the research community.

Active participation is important for knowledge building, so it is of importance to examine the ways in which attendants participate in the context of hybrid conferences. Some research looks closer into the types of participation that Twitter allows. A study by Reinhardt et al. (2009) demonstrated how the scientific community incorporates Twitter as a communication tool. Correspondingly, Ebner and Reinhardt (2009) identified ways in which Twitter was used during a conference: to exchange resources, document conference activities, provide conference announcements, give feedback or ask questions, to arrange meetings and discuss with participants online. Ross et al. (2010, 2011) and Li and Greenhow (2015) found similar results about the purposeful Twitter usage as a conference backchannel.

Still, despite best endeavours, a qualitative study by Ebner et al. (2010) suggested a limited usefulness of tweets for the ‘listening Internet auditorium’. They argued that for a ‘non-participant’ (in the sense of not registered and/or physically present at the conference or online) even relevant tweets are ‘senseless without any distinct context of the occurrence, only messages containing additional material (such as pictures, videos, or similar) may be of interest’ (Ebner et al. 2010, p. 110). The value of tweeting at conferences could thus be increased by sending tweets with contextualized information and eliciting replies that contribute to reflexion on the topic addressed.

Quality of interaction

Immediate value is related to how conferences are experienced and to feeling motivated and inspired. The quality of interactions taking place during conference participation can impact on these perceptions. Facilitating reflection is suggested to improve the quality of interactions. Ravn and Elsborg (2011), within the scope of their ‘Learning Conference’ concept, focused on the individual-reflexive and social-interactive process to facilitate learning at conferences. Similarly, based on the ‘Learning Conference’ concept, Louw and Zuber-Skerritt (2011) presented a conceptual paper which highlighted that, during the event, time must be reserved to facilitate shared reflection, in groups, facilitating participants ‘to leap to new ideas’ (Haley et al. 2009).

As for technological enhancement, several authors (Shiffman 2012, Sopan et al. 2012, Bert et al. 2016) suggest that it is fundamental to develop strategies to enhance the effective use of Twitter for learning, and networking with a wider online audience. This might increase the immediate value of a conference.

Potential value

The potential value of a conference refers to the knowledge capital produced by social learning (Wenger et al. 2011). An example for the potential value can be found in the article of Harrison (2010). She examined the unique benefits conference attendance offers to professional development for Librarians. These included the opportunity to recognize current trends, to watch demonstrations of products and resources, to study how the different institutions are solving problems, the opportunity for socialization and networking, and, finally, for ‘professional rejuvenation’.

The following sub-sections provide examples on how the literature addressed indicators for the potential value.

Skills acquired

A broad range of skills can be developed during a conference. For example, pre-conference workshops often provide an opportunity to focus on a specific topic. However, although the importance of conferences is recognised to develop skills (Anderson and Christiansen 2004, Anderson and Anderson 2010), there is no research on the development of concrete digital skills at conferences. Aims for conferences and evaluation metrics found in selected articles do not yet include measures for skill development.

Information received and artefacts shared

From the presenters’ perspective, sharing of knowledge and artefacts in the form of articles and presentations can be included in the ‘applied value’ of a conference. From the attendees’ perspective, however, a form of potential value creation relies on the information received at the conference, through participation in keynotes, talks, and the slide-presentations, and through the reading of the proceedings and/or publication of articles in special issues by reputed journals.

Related to the potential value of technology-enhanced conferences, Ross et al. (2011, p. 2015) listed five benefits of backchannels: ‘being able to ask questions, or provide resources and references, changing the dynamics of the lecture room from one-to-many transmission to a many-to-many interactions, without disrupting the main channel communication’. A tweet stream assumes the function of a record and repository of a conference (Altmann 2014, Ebner and Altmann 2014), a sort of ‘informal report’ of a conference (Letierce et al. 2010a, 2010b), enabling what Chen et al. (2015) denominated as ‘Twitter Archaeology’.

Comments and sharing of links in Twitter can reflect the quality of the artefacts: tweets as filters, as a curation tool (Boyd et al. 2010), as a ‘quality seal’ for how informative the artefacts shared during the conference were. If the artefacts are collected by the organisers, the number of downloads is an alternative way to observe the interest in some topics or research articles. Currently, no empirical research is available dealing with the chain of digital artefact creation and (delayed) access and reuse by both the attendees and the broader scientific community.

Change in perspective and inspiration

The motivation for attending conferences is not limited to presenting research results, but can include gaining insights through attendance of inspirational presentations by colleagues (researchers or practitioners).

Presentations based on the submitted conference papers can be of great value for the attendees by contributing to changes in perspectives. Technology use can possibly enhance the number of perspectives, as the world can follow the conference via Twitter (Gesthuizen and Rablin 2014). Twitter enables insight being filtered by the users (attendees participating at the conference and the ‘non-attendees’), and spread out.

Types and intensity of social relationships

Conferences that extensively integrate social media and promote networking contribute to a greater potential value for the attendee. Whether conferences offer benefits for knowledge sharing can depend on an interrelation of factors (e.g. formats chosen, and composition of the audience). de Vries and Pieters (2007) highlighted that even though the conferences might bring professionals of the educational field together; the conference formats often contribute to a separation among the attendees, tied to and/or associated with their roles (presenters/researchers as experts, attendees as practitioners). They furthermore concluded conferences are spaces for strengthening of existing networks, but they often do not contribute to building of new networks.

An often-stated function or outcome of academic conferences is the seeding and nourishing of communities for specific fields, and the consequent impact on learning and professional development. The possibility of online conferences providing a sustained sense of community has been studied by several researchers (Ho et al. 2006, 2011, Thatcher 2006, Kimura and Ho 2008, Travers et al. 2008, Thatcher et al. 2011).

On the other hand, technology is not only an enabler of social relationships and promoter of networking. Technologies such as Social Network Analysis (SNA) can visualize the type the intensity of networking starting at the conference. Hansen et al. (2011), and other authors (Reinhardt et al. 2009, Ross et al. 2011, McKendrick et al. 2012) stated that the use of social media at conferences brings advantages in terms of efficient information sharing and networking. Hansen et al. (2011) presented the concept of EventGraphs, defined as a diagram of conversations, which takes place in social media networks, related to events such as conferences.

Networking and the value of social connections

Immediate value is dependent on the opportunity for networking that is one of the main functions of academic conferences. It is a critical question of what possibilities for networking attendees get, what the outcomes are of networking activities for attendees, and which opportunities they get to create value out of connections made.

The diversity of attendees of conferences (with various levels of expertise and professional experiences) and their perspectives might increase the value of their experiences at the conference. However, attendees are often examined as a homogenous group, not considering, for instance, the influence of gender on conference attendance (Eden 2016, Mair and Frew 2016). Authors such as de Vries and Pieters (2007) argued that instead of filling gaps between attendees with different backgrounds, traditional conferences are deepening divisions. This leads to missing opportunities to network with diverse others and, therefore, narrows the potential value of a conference.

Another interesting point in this context is that networking does not only start on the first day of the event. Nowadays, Conference Management Systems (CMSs) provide networking opportunities before the event starts. These CMSs (Atzmueller et al. 2010, Wongchokprasitti et al. 2010, Kounavis et al. 2011, Scholz et al. 2014, Brusilovsky et al. 2016) afford opportunities for social networking and provide, for instance, background and contact information of the attendees.

When networking becomes a more deliberate focus, research indicates how the analysis of networks can inform conference design (Jussila et al. 2013, 2014). Organizers can react more actively and provide support on issues concerning specific groups; they can identify influential people in the community (as potential organizers or marketers for future conferences); and supply connections between the attendees (providing more sessions for special interest groups).

Collaboration

Another indicator for immediate value creation is participation in collaborative conference projects. Tramontin et al. (2018) state ‘academic events are also aimed at connecting researchers and promoting potential collaborations.’ When, for instance, similar topics are presented in one session, this may lead to collaboration in the future between co-presenters and/or people who come to talk about these topics.

The work presented is often already the product of a collaboration since many conference papers are co-authored. Tools such as Google Docs, Microsoft Word Online, allow the co-writing of articles online in a collaborative way. Additionally, other forms of communication (e.g. Skype) can be used. The use of technology in preparations and during attendance is, nowadays, indispensable (Siemens et al. 2008).

Applied value

Regarding the applied value, the question is: how can the knowledge, tools and social relationships acquired/developed at the conference be used? Some insights resulting from research or practical demonstrations can already be used during the conference. Others are expected to be applied in the future. How to measure this applied value immediately after a conference? A request for answering an online survey some months after the event could be a good alternative to identify the transfer from the conference into the practice of participants. The perceived applied value could then be more accurately defined.

The subsequent sub-sections provide examples on how the reviewed literature addressed the following indicators for the applied value: innovation in practice; reuse of products; use of social connections; and transferring learning practice.

Innovation in practice

The applied value reflects that, from the insights of the conference, the attendees applied new ways of doing things, applied concepts or contextualized experiences, e.g., related to new strategical approaches, contributing to the professional development (Anderson and Anderson 2010). For instance, the presentation of a case study on using Web 2.0 for educational purposes can lead to the use of Web 2.0 tools in the own educational context. Innovation in practice as a result of the attendance of an online conference can be translated in new forms of teaching, fully online or blended.

Reuse of products

The products related to a conference are the accepted articles, but can also be the presentations during the parallel sessions. These can be used, reused, remixed, creating artefacts in new contexts contributing to the development of individuals and continents (Carr 2016). In this sense, these artefacts can be seen as Open Educational Resources (OERs). The applied value can be, therefore, multi-layered.

Use of social connections

Social networking in (online) conferences does not end with the last conference day: ‘The interactions across the boundaries of multiples communities of practice support participants in pursuing conversations with new and well-known colleagues’ (Carr 2016, p. 298). Conferences can contribute to establish and maintain an online community of academics (Thatcher 2006, Thatcher et al. 2011).

Turning to the use of microblogging at conferences, visual network analysis of Twitter data can be applied for co-organizing conferences (Jussila et al. 2013, 2014, Aramo-Immonen et al. 2015, 2016). Jussila et al. (2014) analysed the use of Twitter focusing on the network of conference participants and the conference’s discussion topics of the conference. They identified metrics such as the levels of influence of participants (co-organizers, speaker, or regular participants); the level of interest in certain content; and similarities between interests of the participants.

Transferring learning practices

Applied value of academic conferences can also be expressed through the shifting of learned practices/experiences from one context to another: the presentation at conferences (face-to-face or virtual) can be a good way to improve the lecturing capacity of academics; papers submission for the conference can be a good way to improve the scientific writing for peer-reviewed journals (Li and Greenhow 2015). Online conferences can promote the use of technology and show how online courses can be realized (Carr 2016).

Realised value

The realised value focuses on performance improvement of conference participants. The application of acquired knowledge and the use of newly acquired competences (e.g. communicational competences online, proficiency in a foreign language) in the relevant professional context includes the attendees’ expectation of a performance improvement when integrating knowledge in the own context. The indicators for the realised value which can be taken into consideration are: personal performance and organisational performance.

Right now, we can only speculate on the impact of conference attendance on realised value as we did not identify any studies that addressed this type of outcome of conference attendance.

Reframing value

The reframing value focuses on the redefinition of ‘success’ through the process of social learning. For instance, the aim of adopting new strategies in the own environment and context can be a result of an inspirational talk with an expert at a conference. The indicators for the reframing value, which can be taken into consideration, are: institutional changes and new frameworks.

As in the realised value dimension, also here, we can only conjecture on the impact of conference attendance on the reframing value for the attendees, as no literature was identified addressing this dimension of value created at conferences.

Discussion

This paper presents a systematic literature review on the value creation of academic conferences. It pays special attention to the role of emerging social technologies that contribute to enduring communities and, consequently, to the academics’ CPD.

Nearly all conferences using technologies are being evaluated with traditional metrics and (satisfaction) indicators. This occurs often from the organizers perspective. Evidently, the metrics which are relevant for conference organisers are not always in accordance with, or do not amply cover, the metrics which are relevant for professional development. Additionally, the integration of technology (e.g. social media) has consequences (e.g. the imperative use of SNA). This literature review examined the articles through the VCF, highlighting the range of values (immediate, potential, applied, realised and reframing) which are, up to now, unevenly researched. Studies capturing realised or reframing value of learning at conferences are inexistent. Opportunities to evaluate long-term effects on professional development are underexploited because follow-up questionnaires that can take into account delayed-effects are rarely used. Besides, when capturing the five value creation dimensions, the different functions of academics (teaching, research and management) and the various stages of the career (junior researcher to full professor) are often disregarded.

Primarily the importance of academic conferences is recognised. The review indicates that the literature hints at immediate, potential and applied value of (amplified) conference participation, but they are most often formulated in very general terms (opportunities, potentials, enablers, possibilities and benefits on several functional levels). The reviewed articles provide mostly speculative evidence on these three dimensions of value created.

Moreover, there is a clear lack of a coherent framework to help operationalize the impact of amplification on learning and community building. Often, a technology-deterministic stance is taken (Chandler 1995), meaning that technology as such will lead to better learning and knowledge sharing at and beyond conferences. Nevertheless, this impact was not quantitatively measured. Technologies are being integrated in existing practice and format. Several questions remain unanswered such as if services as Twitter are contributing to an elitism of experts or to an egalitarian function during conferences. In other words, the opportunity for networking (immediate value) at conferences conducting to an intensification of social/professional relationships (potential value) is underexploited or not yet researched.

Nowadays, it is common to use tools like Twitter or Facebook at conferences. The quantity and quality of interactions and networking activities are indicators of immediate value being created. Scarcely, there is facilitation before and/or during the event, no follow-up to seed and feed a community of learning or a particular concern with collecting data on the conference after the event. As a possible consequence, there is no research on the dimensions of realised value (focusing on the performance improvement) and reframing value (focusing on the redefinition of successful scenarios or strategies). Surveys and Twitter analysis, conducted immediately after the conference, are ‘snapshots’ of a specific moment. The dimensions applied, realised and reframed are difficult to measure as they occur usually later in the timeline, after the ending of the conference.

Concerning the realised value, we argue that relevant indicator for the personal performance could be the speed and accuracy in the realization of an activity, for instance, because of an information received at the conference. Likewise, the use of communication technologies in a multi-cultural setting can contribute to the consolidation of such competences and have an impact on the daily interactions with colleagues and students. One indicator of organisational performance can be the improvement of the students’ satisfaction with a specific course or program, which could be evaluated in organisational performance evaluations.

In the context of the reframing value, we were not able to find studies addressing subsequent institutional changes in the workplaces of attendants to conferences. The conference can germinate, for instance, a partner group that defines strategies for a broader field or institutions. New ideas and concepts can be developed and implemented. These may have an impact on the individuals, attendees, but also on the institutions. Additionally, results of ongoing or recently accomplished research are presented at conferences. These can lead to new frameworks for institutional settings, as well as to an evolution in the scientific fields.

Still related with the realised and the reframing values, contextual factors (like the impact of heterogeneous participant groups on the outcomes for learning at conferences) represent another underexplored element in the current research concerning academic conferences in the purpose of learning. While the importance of conferences for CPD is being recognized, studies predominantly feature librarians and health professionals (Travers et al. 2008, Tomaszewski and MacDonald 2009). Little research is being conducted in other domains. Whether there are domain-dependent factors that influence the organization of conferences (e.g., target-audience requiring a more traditional conference versus communities that expect the use of emerging technologies), and how they impact the (immediate) knowledge-gain and learning at academic conferences remains an open question. On the other hand, Pradhan (2014) added another contextual factor. He argues that organizers are not aware of issues connected, for instance, with the cross-culturality of the audience. An international conference is an opportunity to develop and consolidate cross-cultural competencies. These competencies do have an impact on the scholars’ level, but also on the institutional one (reframing value).

To enhance CPD in the context of conferences, design needs to be based on theoretical perspectives. Some of the articles highlight the CoPs perspective (Thatcher 2006). Wenger (1998a) apprehends communities of practice as progressing through five stages: potential, coalescing, active, dispersed, and memorable, with levels of interaction and types of activities varying across the stages. Members’ interaction within the community generally increases through the active level and then declines through the dispersed stage, and disappears at the memorable level, although memories, stories, and artefacts of the community still remain. Academic conferences can be regarded as (part of) communities of practice and different conferences might be seen as different stages of development of the community. Moreover, the potential of technology lies in the opportunity for strengthening connections within a community that can work against the decline of it.

In turn, the chosen framework for CPD in the context of academic conferences should provide clear metrics for determining the impact of amplification on learning and community building. The value creation framework does this by proposing indicators of value creation for the different dimensions and some potential data sources: traditional data sources, such as surveys and interviews); and emergent data sources, gathering data through SNA.

This study has several strengths and limitations. The study applied a methodological and systematic approach to review articles from 1993 to the present. Moreover, it raises awareness about the applicability of an instrument for evaluating non-formal learning at conferences (VCF). In terms of limitations, the general topic can barely be called ‘new’ (Jacobs and McFarlane 2005, de Vries and Pieters 2007, Anderson and Anderson 2010, Letierce et al. 2010a). Still, we have aimed to provide new perspectives and insights in several ways. The use of emerging technologies and Web 2.0 services is recent, presenting a restricted number of published empirical studies addressing its potential. Therefore, we have included a plethora of publications which have only been published in recent years. Additionally, the review contributes a novel application of the VCF framework towards reviewing CPD at conferences. Concerning the sample, some of the included empirical studies can be regarded as being of restricted value, due to small samples size or being confined to a ‘snapshot’ of a specific (edition of an) event. This strongly limits the extent to which the studies’ findings can be applied and generalized.

The importance of this review builds on the fact that it cross-checks the triad academic conferences (continuing), professional development, and use of digital technologies. The main focus of this study is the intersection of multiple specialized domains, ranging from the field of successful event organisation and technological enhancement, to the educational sciences. A suitable evaluation instrument for CPD thus has a range of useful applications. For instance, the VCF can help delineate vocational curricula for academics, guidelines for event organisers, and guide the effort of technological amplification. Third, it can provide both, practitioners and researchers, organizers and attendees with an overview about traditional and innovative formats and strategies for value creation at conferences.

It is imperative to conduct further research on academic conferences using a framework to assess and promote CPD through conferences in their plenitude of formats and functions and the integration of technology, just as the evaluation of formats and strategies of technology-enhanced conferences, pointing out opportunities and challenges in comparison with traditional formats, and taking into special consideration the creation of value (including a perspective on development of an enduring learning community). Concomitantly, the value of a conference is difficult to measure as a scholar can participate in pursuing different roles (e.g. attendee, presenter, reviewer, chat) and objectives. Therefore, further research must also take into consideration the different value layers and how they are weighted by the participators.

Other research paths should include the impact of factors such as the conference size, profile of participants (including gender particularities) and cultural factors on augmented social learning. The differentiation between the diverse attendees, practitioners versus researcher and within the group of academics, considering the different stages on their career, might impact value creation at conferences.

Conclusion

This literature review is based on the most recent research and provides a ground for future research, pointing out knowledge gaps and refining research questions. The included body of the literature on academic conferences for CPD is mainly represented by case studies, analyses of data gathered from traditional metrics (satisfaction surveys) or, more recently, studies based on data gathered from social media such as Twitter. We must for the moment be cautious when claiming effects on CPD based on value indicators.

Since formal professional development programs can be difficult to organise and are also more difficult to schedule for individuals, attendance at academic conferences, which belong already within the job profile of academics, represents an important form professional development. Combined with opportunities to establish networked learning practices and enduring communities of practice with the help of social media, this channel for continuing professional development can be expected to increase in importance.

Academic conferences can seed communities and the enduring nature of these communities can be supported by developing an online place where the community can continue the ‘conversations’ started at these conferences. They provide CPD opportunities for scholars in different stages of their academic career. Continuing professional development through conferences is social and non-formal, so it makes sense to focus on social technologies. Services such as Twitter are well accepted as an extension to the event site or integrated in event management systems. However, they are mainly used for spreading information, not covering its full potential. CPD takes place at conferences, but current research hardly measures how and to what extent technology-enhanced conferences contribute to CPD.

The Value Creation Framework seems to be an appropriate conceptual framework for understanding the impact of conference attendance for the academics’ development of (digital) professional competences. Findings reveal that traditional indicators of successful conferences related to ‘satisfaction’ are focused on potential short-term outcomes of conference attendance, rarely on applied values and do not capture the (long-term or far away in time) realised and reframing value of conferences. Even if the realised and reframing value creation at a conference is difficult to measure, a survey based on the VCF may capture expected outcomes and ‘value creation stories’ (Wenger et al. 2011) expressed by the attendees can permit the collection and interpretation of unexpected outcomes as valid indicators of technology-enhanced learning. Nevertheless, additional methods for assessing the ‘cycle of value creation’ (Wenger et al. 2011) in context of academic conferences are required, and methods like such as social network analysis (Chen et al. 2015) can increase insights into the emerging professional development networks of academics. Therefore, technology could be employed to promote and assess value creation at conferences, contributing to more diverse and larger communities, meaningful for Continuous Professional Development.

Disclosure statement

No potential conflict of interest was reported by the authors.