Effects of Brief Integrated Information Literacy Education Sessions on Undergraduate Engineering Students' Interdisciplinary Research

ABSTRACT

Engineering students often conduct information searches without sufficient consideration of the context of their research topic. This article discusses how development of a new information literacy (IL) mindset through instruction in integrated IL education affects students' understanding of research problems and formulation of information search questions. The course observed is a mechanical engineering seminar for undergraduate engineering students with integrated IL education. A survey-based research method is utilized with surveys carried out at three stages of the course. The results show that as the course progressed, students' interdisciplinary research problem descriptions became more profound and their formulations of information search questions developed more advanced understanding of the principles of information use. Preliminary parallel results have been analyzed from 2013 to 2016 at a university of technology in Finland.

KEYWORDS:

• Information literacy
• collaboration
• university libraries
• academic staff
• library staff

Introduction

A key characteristic of modern higher engineering education, particularly with innovative approaches such as task-based learning and blended learning, is that students undertake research as a part of their subject studies. In connection with their research assignments, students engage in information seeking to find answers to their research problems. However, they often lack knowledge of how to search for information effectively and lack the prerequisite skills for evaluation and interpretation of sources and data, which has a detrimental effect on the quality of their work (Eisenberg, 2008). Consequently, information literacy (IL) has become an integral part of engineering education curricula to teach students the skills required to find suitable high-quality information effectively.

Growing awareness of the need for environmental, economic and social sustainability and increasing digitalization of modern society have resulted in new challenges for IL education in higher education engineering institutions. For engineers, designing a specific product to fulfill a particular technological need is no longer wholly adequate. It is additionally necessary to consider the context in which the product is manufactured and used and to understand the effects the product has on the environment and society during its production and operational lifetime (i.e., product and process lifecycle management). Moreover, the use of modern computerized design tools and diverse and mostly electronic information resources requires considerable digital expertise and information literacy. For engineering students to gain the competencies necessary for the modern workplace requires interdisciplinary approaches from academic staff supported by library personnel assisting students with understanding and utilization of the increasingly complex information source environment.

In addition to taking sustainability aspects into account in engineering design, a sustainability mindset is becoming a necessary part of modern life. For example, driving an electric car requires a new way of thinking about travelling, as the distances between battery-charging stations do not equal the kilometers that can be driven without refueling a gasoline-powered car. Similar examples can be found in many other areas, for example, food choices, health, lifestyle, and construction. Mindset changes are, therefore, essential in many contexts, including information literacy.

During the past decades, the emphasis of IL education has been on searching skills. The basic learning outcomes have included knowledge of the content of appropriate databases and how to search relevant databases using suitable search terms (Dunn, 2002). However, from a macro perspective, when creating a search query, search sets are handled as mathematical sets in set theory. Therefore, the primary objective in teaching search techniques should be to promote students' logical thinking rather than giving instruction on search features of individual databases. Once the logical thinking and reasoning essential for effective information seeking has been learned, information about search tools can be found, for example, in help pages of the database.

In addition to knowing search techniques, the searcher should understand that he/she is engaged in a problem-solving activity and that retrieved documents should contain information that can be used to find a solution. Moreover, the searcher must be able to analyze the research problem in its context and understand which parts of solving the problem require information searching. The searches can then be performed in suitable databases by operating with search sets into which the information search questions have been divided. In this article, global understanding of the whole research problem and knowledge of what kind of information is needed to solve it together with knowing how to utilize available information sources and search techniques are referred to as the IL mindset.

In view of the extent of interdisciplinary sustainability related research and the abundance of available information, the challenges IL education encounters become obvious. In cases when IL teaching resources are limited, short integrated IL sessions form one of the few economically-feasible ways to arrange the IL education. Therefore, the main research question in this article is: How does integrated short-term information literacy education affect engineering students' understanding of interdisciplinary research problems and the way they formulate information search questions?

To clarify how IL education can enhance students' research, Ackoff's knowledge hierarchy (Ackoff, 1989) is used as a basis. According to Ackoff, data are symbols that represent the properties of objects and events. Information that consists of processed data provides answers to questions like who, what, when, where, and how many. Processed information is called knowledge, which answers how-to questions; for example, “how a process should be carried out to minimize energy consumption.” Understanding is the next notion in the structure. This concept answers why questions such as “why does a specific aspect of saving energy work as it does.” According to Ackoff, the pinnacle in the hierarchy is wisdom, which, unlike the lower level concepts, is dependent on the judgement of the actor. In IL education, students should be guided to focus on their research problem to find the right kind of source data and information and use it to create new knowledge and understanding. In some cases, students may develop new outcomes as a result of increased wisdom.

In the case of the interdisciplinary sustainability-related research considered in this work, the substance and IL education together help students to position their comprehension of the subject and their understanding of their information needs on tools such as the sustainability maturity curve of Fava (2015). They are thus able to see the problem in a wider perspective. Fava's curve follows the same basic principle as Ackoff's (1989) hierarchy but utilizes a slightly different categorization to produce a sustainability-related continuum.

Figure 1, which presents examples of information needs, illustrates how the sustainability maturity curve can be used to help the searcher position the depth of his comprehension of the subject and his information needs. In the figure, Fava's (2015) first step Complying deals with, for example, laws, regulations, and properties, and thus represents the Data and Information levels of Ackoff's (1989) hierarchy. Market driven is a step in which, for example, more information on environmental, economic, and social issues is needed to satisfy user needs. This step corresponds to Ackoff's Knowledge level, which also answers “how to” questions. Fava's Engaged and Shaping the future represent deeper maturity of sustainability, enabling the individual to pro-actively undertake actions that require Ackoff's Understanding and Wisdom.

Figure 1. Applied sustainability maturity curve. The curve illustrates how the depth of understanding of the subject and information needs can be positioned in the maturity continuum.

Complex interdisciplinary issues require understanding of how the issues involved relate to each other. Sustainability questions are always connected to a particular context. Therefore, a successful search for sustainability information in connection with a technical problem requires that the searcher has sufficient competency in both sustainability science and the technology concerned. For example, when mechanical engineering students need to focus on sustainability matters, they have to understand the context of their research, which can be visualized as the position on the maturity curve where their information needs and substance knowledge are located. The tasks of IL education in such a context are to help students establish their location on the curve, to provide students with the tools to find the information needed, and to guide them regarding the kind of information they can and should look for to solve their research problems.

This article discusses an example of short-term IL education integrated with a mechanical engineering seminar course in which students, using available literature, aim to find the most suitable construction materials for a given application while being cognizant of sustainability considerations. The course is taught at a Finnish university of technology and the language of instruction is English. The course curriculum includes lectures on applicable materials and information literacy, writing of a seminar paper, and production of a poster presentation. The substance teacher and the information specialist co-teach the students, who work in teams of four to five.

Literature review

In this article, new challenges to IL education caused by concerns associated with sustainability and increasing digitalization are discussed. The boundaries of the sustainability tolerance of our planet are discussed by Rockström (2015), who claims that an integrated social, environmental, and economic perspective is imperative to maintaining the planet's stability. Rockström argues further that humanity needs to understand its dominant role in the operation of the biosphere and how it shapes the Earth's systems. Sustainability research extends across numerous disciplines of the humanities, the social sciences, and the natural sciences, and, therefore, IL education, especially in higher education institutions, where students undertake research as a part of their studies, faces the challenge of widening students' views of their information search questions to contain sustainability aspects in addition to their own field of science.

The modern information environment has rapidly adopted new technologies and the volume of digital information is growing exponentially. For example, according to Gantz and Reinsel (2012), IDC, a company that tracks and analyzes the growth of digital data, estimates that from 2012 to 2020 the volume of digital information in the United States will double every two years. The growth in digital information and information and communication technologies (ICT) impacts people's life and work and thus sets new demands on the skills needed to manage the information environment. To teach students, who are used to searching mainly via nonscientific search engines, to filter relevant and useful sources of information from the abundance of available documents is a major challenge of IL education.

Effective ways of organizing IL education in higher education institutions to achieve optimal learning results have been widely discussed in the literature. The questions: what should be taught, by whom, when, and how much time should and can be allocated to IL education have all received attention. However, the question “does IL education support the development of students' research behavior” does not seem to be discussed.

Characterizations of information literacy have changed from underlining IL-related skills (ACRL, 2000; Gibson, 2008; Johnston & Webber, 2003; Maybee, Bruce, Lupton, & Rebmann, 2013; Rader, 2002) to a focus on creating new knowledge on the basis of retrieved information. This change in understanding of the concept can be seen in the new ACRL “Framework for Information Literacy for Higher Education,” which presents creation of new knowledge and understanding the value and use of information as fundamental to IL learning outcomes (ACRL, 2015).

IL education is almost always given by library personnel, and academic libraries often have designated staff, called branch, subject, or liaison librarians, to teach IL and serve the information needs of different research areas. Hardy and Corrall (2007) discuss the skills that such librarians feel that they need in their work. Most librarians taking part in their study listed professional, pedagogical, technical, and personal skills but none of the respondents mentioned “subject-related knowledge and understanding” as a key aspect of their competencies. Hall and Derek (2014) argue that “branch librarians” communicate closely with faculty and students, learn their customers' information needs and can use that knowledge to customize library services. In neither of the aforementioned studies, however, are librarians seen as co-teachers with faculty education.

A closer connection between faculty and librarians is presented by Wertz, Purzer, Fosmire, and Cardella (2013), who link IL to the engineering design process and thus see the need to integrate IL instruction with engineering curricula. They recommend collaboration between the engineering faculty and librarians and extend collaboration even to faculty from communication and English departments in order to integrate reinforcement of IL skills in existing engineering curricula. Linking IL and language studies to substance education is also presented by Talikka, Eskelinen, and Värri (2014) in a case-study related to materials science. Although these studies present collaboration practices, and the integrated IL and language education provides students with valuable skills in both cases, the wider aspect of connecting these skills to undertaking research is not considered.

One-shot IL sessions are inexpensive to arrange and are less labor-intensive. However, for example, Pausch and Popp (1997) consider them rather ineffective. They claim that participants attending such events are satisfied with the one-shot session only because they do not know enough to be dissatisfied. The authors do, however, note the need for further research to establish the validity of the assessment methods used to evaluate these brief IL sessions and other forms of IL education. Talikka, Värri, and Eskelinen (2015) found, however, that changes in students' information searching performance can be seen even after brief, 90-minute IL education sessions.

Thus far, research on the effects of IL education has mainly focused on changes in students' technical searching skills. Research results of both short one-shot sessions and extensive study modules are reported in the studies of Webber and Johnston (2000), Cisse (2016), and Juntunen, Lehto, Saarti, and Tevaniemi (2008). Departing from this focus, Artman, Erica Frisicaro-Pawlowski, and Monge (2010) connect IL education with scientific writing and, in a literature study, Fluk (2015) discusses the use of “research logs,” a kind of learning diary, as a way to determine the efficiency of IL education. However, discussions of IL education in connection with students' understanding of the nature of research and research practices are not found in literature.

The new and continuously expanding information environment, along with the need to take sustainability issues into account in engineering education, challenges IL education to adopt new actions to guide students in learning a new IL mindset. Thus far, research on IL education appears to have concentrated primarily on studying how IL education is organized, and what are the learning outcomes as far as information searching is concerned. However, although IL education is included in university curricula in order to provide students with knowledge that promotes their ability to do research, the effects of IL education on their research behavior does not seem to have been the subject of much study. This work addresses this gap in the literature by studying the effects of IL education on students' understanding of the interdisciplinary research process. The focus is on the effects of brief integrated IL sessions, which are a form of IL education found in both small and large universities.

Research framework

The effects of short-term integrated IL education on the IL mindset, research understanding and research outcomes of undergraduate (i.e., third to fourth year) students were investigated. The group studied comprised students attending a mechanical engineering seminar course. Their tasks were, via literary research, to find the best materials for a given solution with a special emphasis on sustainability, to report their results in a seminar paper, and to prepare a poster presentation in English.

The IL education was integrated into the subject course. It consisted of two 90-minute lectures given by an information specialist and a 20-minute feedback session in which the substance teacher and the information specialist together met the student teams to discuss their work from the IL viewpoint. There were 10 substance lectures in the course. The first IL lecture was after five substance lectures and the second one after the seventh substance lecture. The IL feedback session was right before the second IL lecture. Thus, the arisen information searching problems could be discussed during the lecture. The content of the IL education, which focused on the research process, consisted of the topics presented in Table 1.

Table 1. Content of the IL education.

	Content
1	Characteristics of different information sources. Information sources applicable to this task.
2	Selecting information sources for searching.
3	Selecting search words, alternative words, effects of truncating, using abbreviations, differences between British and American spelling.
4	Creating search queries by using logical (Boolean) and proximity operators, parentheses, and phrases.
5	Using the available information retrieval portal (Metalib).
6	Demonstration of searching in ProQuest, Science Direct, and Scopus. The advantages of using databases' own user interfaces.
7	Analyzing tool of Scopus, e.g., what the analysis of publication year, source, and subject area can show.
8	Impact factors in Journal Citation Reports (JCR) as a tool for finding subject-related high quality journals.

Applied methodology

The effects of the IL education sessions were observed from three perspectives: the IL expert, the subject teacher, and the students. Changes in students' information searching behavior and information use and development of the students' IL mindset were studied by the IL expert. Students' research behavior and knowledge of the researched topic were evaluated by the substance teacher. The third viewpoint was that of the students, whose responses were used to validate the teachers' impression of the extent to which learning outcomes had been achieved.

The study was carried out with a group (n = 35) of third and fourth year mechanical engineering students. The group consisted of both Finnish and foreign Master's level students who worked in teams. All teams had students of different nationalities. Therefore, the knowledge background and ability to understand multidisciplinary engineering problems of the teams were diverse, which means that their IL mindsets are not identical.

To carry out the research, four methodological steps were applied.

The first step aimed to analyze changes in students' ability to understand their research problem globally by asking them to describe their research question/s before and after IL education. Based on the sustainability maturity curve by Fava (2015), a four-part classification was generated to categorize the depth of students' research questions. The categories were:

I.	Questions concerning recognition of the limitations in the use of materials (may concern legislation, technical standards, etc.) • Complying with rules and regulations
II.	Questions concerning the sustainability of chosen materials • Satisfying sustainability requests
III.	Questions focusing on the sustainability of the applicable material options and their benefits • Taking sustainability issues proactively into account
IV.	Questions related to the most sustainable way to meet the needs set to the application • Developing solutions to meet future sustainability needs

In the second step, the development of students' information search questions was evaluated. The six-point classification approach by Vakkari (2000), which is based on the model of the stages of information searching presented by Kuhlthau (1993), was applied in the evaluation. In development of this classification, the six cognitive process categories of Bloom's taxonomy (Anderson & Krathwohl, 2001) were connected to Vakkari's definitions to link them to the profundity of students' understanding of the issues studied. The data were collected from surveys carried out at three stages of the IL education continuum: (a) before the IL education, (b) after the IL education, and (c) after the seminar paper and poster writing processes had been completed. The categories used in the classification were:

I.	Initiation (Bloom's cognitive dimension: remember) The searcher recognizes a gap in his knowledge and understanding.
II.	Selection (understand) The searcher identifies and selects a topic to be investigated, including comparison of different aspects.
III.	Exploration (apply) The searcher investigates information on the general topic or focuses on general viewpoints to extend his understanding.
IV.	Formulation (analyze) The searcher forms a focused perspective on the topic and a detailed analysis of the application being studied.
V.	Collection (evaluate) The searcher interacts with the applicable retrieved information.
VI.	Presentation (create) The searcher completes the information retrieval and uses the findings.

The third step of the methodology consisted of evaluation of the retrieved literature sources based on the students' own opinions. In a survey, students evaluated the success of their searches and named the criteria by which they had assessed the reliability and quality of their literature sources.

The fourth step considered the success of the students' research. The results of the categorization of the research problem descriptions, and information search questions were analyzed by cross-tabulation with the final grades given by the substance teacher. Additionally, the research problem descriptions and information search questions were studied by cross-tabulation to clarify possible interdependence.

Results of the four-step methodology

Changes in the formulation of the students' research questions were studied and the results are presented in Figure 2. Students' definitions of their research questions were categorized in four groups as described in the previous section.

Figure 2. Development in research question formulation. After IL education, students' research questions focus more on sustainability of the applicable material options and their benefits.

From the figure, it can be seen that before the IL education students' research questions dealt more with recognizing the limitations of the use of materials in the application (34%) and with the sustainability of the chosen materials for the particular environment (56%). After the IL education, their research questions still focused on sustainability issues of applicable materials in the given environment (48%) but the proportion of research questions concerning the sustainability of the applicable material options and the benefits of the materials had increased from 6% before IL education to 27% after it. However, according to this classification, which was based on the sustainability maturity curve, only one of the mechanical engineering students' research questions – both before and after IL education – could be classified as an examination of how to fulfill the presented need in the most sustainable way.

In the survey, students answered a question about whether their research problem had become clearer during the IL education process; 61% agreed that understanding of the research problem had improved. Students commented for example:

Yes it became [clearer] because we get more information on our subject.

Yes, before research problem it was too general. After this we came more close and deep to the topic.

Yes, at the start we consider to evaluate the whole assembly of tidal turbine but we saw that was too complicated so we narrowed down to only the blade assembly.

'During the process of development, it was possible to add more details and viewpoints to our research problems and it got more clear in the end.

Students who felt that their research problem was not clearer after the IL education (39%) commented, for example:

No, I don't think so. It was exactly what we wanted in the beginning of this seminar work.

For the most part it remained the same.

Well, our research problem was quite precise and focused from the beginning.

We had problems with our research problem.

The same student group was studied when evaluating students' understanding of the information search questions. The students replied to the question ‘What is/are your information search question/s?’ at two stages: before the IL education and after the IL education. After they had written their seminar paper and prepared their poster, they described the issues on which they based their final information searches. Their answers were evaluated using the previously presented categories. Analysis of students' answers is illustrated in Figure 3.

Figure 3. Development in information search questions. During IL education students' information search questions develop from seeking facts and comparison information to detailed, application based information. After the course, students express information needs for interacting with the information and creating new knowledge. During the course the emphasis of the information search questions moves from left to right, from basic to more mature questions.

Before the IL education, 64% of students presented information search questions that dealt with recognizing the gaps in their knowledge and selecting a topic to be investigated. After the IL education, 51% of the information search questions concerned forming a focused perspective of the topic or a detailed analysis of the application. At that stage, a total of 91% of the information search questions were classified to categories where questions concerned general or focused viewpoint of materials or interacting with the information. Thus, students' viewpoints in information searching had moved to extending their general and detailed understanding of the topic, and they had started to interact with the retrieved information. After the course, students' goals in information searching were to find literature for further discussion of the issue via the retrieved information and use of their findings to create new knowledge (77% of all answers).

The students evaluated the usability of their search results. In a survey answered by 27 out of the 35 students, 24 students reported that the results answered their needs, two graded the results as not so satisfactory, and one person thought that the references found did not meet his/her needs.

A total of 12 out of 27 students also provided information regarding how they judged the scientific quality or reliability of their information sources. Four quality criteria were found in the 12 answers. These criteria are presented in Table 2.

Table 2. Students' criteria of reliability or quality of documents.

Criteria of reliability or quality of information sources, # of students who mentioned the criteria
Cited by many authors	5
Peer reviewed journal	4
Verifying the information from different sources	2
Highly ranked journals	2

The interdependences between the final grade and understanding of the research problem or ability to present appropriate information search questions were analyzed using cross tabulation. The scale of passing grades on the course ranged from 1 (pass) to 5 (excellent). The grades awarded ranged from 2 to 5 with the grade average being 4.0.

The correlation between the final grade and understanding of the research problem appeared to be nonsignificant (p = 0.703). Equally, there seemed to be no significant correlation between the information search questions and the final grade. However, according to the results from the cross tabulation of classifications of the research problem description and information search questions, there was a positive correlation between these two factors (p = 0.037). The limit value of p was 0.05.

Discussion

As an example of IL education in connection with interdisciplinary research, a case of a mechanical engineering seminar course with a focus on sustainability is discussed. Based on the measured results and observations, the students' IL mindset change was evaluated from two viewpoints: change in the profundity of the research questions and change in the formulation of the information search questions. The developed research question classification was based on Ackoff's (1989) knowledge hierarchy combined with categories in the sustainability maturity curve by Fava (2015). The change in formulation of the information search questions was studied using a six-level classification that was generated for this research and which based on Kuhlthau's (1993) and Vakkari's (2000) model of six different stages in information searching and Bloom's cognitive process taxonomy.

Based on the presented evaluations, it was obvious that as a result of integrated short-term IL education a change in students' IL mindset could be verified. The profundity of the research questions improved during IL education, according to the applied analysis. The students showed both deepened understanding of the problem and a more distinct focus on sustainability. Moreover, the information search questions developed strongly from fact-dependent to focusing on creating new knowledge. According to students' own evaluation, their search results became more precise and useful. There was a significant correlation between understanding of the research problem and information searching. However, the final grades did not correlate with either activity.

Conclusion

This article discusses how integrated IL education affects students' understanding of research problems and formulation of information search questions in the context of an interdisciplinary mechanical engineering seminar course with a focus on sustainability issues. Sustainability threats facing modern society and the increasingly digital form of information influence IL education. Information must be searched in multiple sources and a vast number of sustainability and engineering design related factors must be taken into consideration in the research process. Search skills are essential, but for successful research, students need to learn a new IL mindset to master the entire research problem and understand what kind of information is needed to solve the problem.

Learning to do research, especially in an interdisciplinary context, can be facilitated by integrating IL in substance education. After attending IL education, students formulated their research problems in a more mature way and focused on interdisciplinary matters, in this case mechanical engineering and sustainability, instead of concentrating only on basic issues of their own discipline. Development in the information search questions also indicates a change in student IL mindset. Prior to IL education, students' information search questions concerned gaps in their knowledge and constructing a clearer picture of the topic. After attending IL education, students expressed the need to evaluate the retrieved information and use it to create a wider picture in connection with the previously acquired knowledge. After writing their seminar paper and making a poster, their information search questions dealt with using the new information in their research.

During IL education, students not only gain the ability to technically perform information retrieval processes but they also learn a way of thinking which is necessary for understanding research and information searching problems. The IL mindset change makes solving any interdisciplinary searching problem more efficient and finding reliable source material more feasible. In addition, integrating IL education teaches students how information retrieval and use are part of the research process.

Integrating short-term IL education in substance education appears to cause a change in student IL mindset. The new way of thinking can be seen in the presented analysis, which shows that after IL education, students have gained a more mature understanding of their research problems and information needs, which correlate significantly. The lack of correlation between either of the measured factors and the final course grade may be a result of the high grade average, which was 4 (on a scale of 0 = fail to 5 = excellent).

The novelty in this article is in seeing IL education as a means of developing student IL mindset. The key change concerns development in shaping information search questions and understanding the depth of the research question. In this case, students' worked on materials selection with a special focus on sustainability. After IL education, the formulation of their research questions showed development not only in their IL mindset but also in their sustainability mindset. Further research is needed to find those courses in engineering curricula where integrated IL education is most beneficial, which would enable creation of an IL education thread across the engineering education.

The results of this research on the effects of brief integrated IL education on formulation of the research problem and information search questions together with previous studies concerning the impact of IL education on searching skills can be used to create a model for integrating information literacy in engineering studies. The model could be used to address problems caused by limited teaching resources in university libraries. From the engineering education point of view, the model could encourage faculty to foster closer teaching collaboration with each other and library personnel.