Challenges of Effective Collaboration in a Virtual Learning Environment among Undergraduate Students

Web-based technology has influenced the way we perceive and use computer devices. One effect of the increasing variety of online web services has been a move away from “personal” computing toward networked or “social” computing. In this paper we discussed a research project which explored the potential in using a purpose-built social-networked environment to support learning in undergraduate education. In particular, we were interested in which tools would be adopted and the collaborative manner which would be used. A prototype web-based networked environment was built that allowed each student a personal space with their profile, access to resources and a group-project work space. The network incorporated a people-centric paradigm that integrated networking across a range of collaborative tools to support course activities. The system embraced an “open” philosophy where all students and staff were linked to all members and workgroup areas. Each work-group area consisted of a number of web-based tools that included a group wiki, group blog, file repository, weblink tool, discussion area, and calendar. The site allowed for both synchronized and asynchronized communication via messaging and chat facilities. The results showed that students did not work collaboratively and struggled to use the web-based tools to advance their learning. Further investigation revealed that students preferred individual offline approaches to the more demanding processes required in a collaborative online environment.


Introduction
The escalating nature of social networking technologies is a world-wide phenomenon as websites modelled on Facebook, YouTube, Blogger and Twitter continue to proliferate.It is assumed that the digital infrastructures aimed at networking information and communication (that these social technologies support) are being embraced by new generations keen to live a more "connected" lifestyle.In this study, we explored this assumption on two fronts.Firstly, we were interested in investigating the degree to which students would use a networked environment to support the course related activities.Secondly, we wished to gain some measure of whether connectedness/networking offered students an advantage within an academic environment.Given the literature on "digital natives" (Prensky, 2001) and the "Net generation" (Oblinger & Oblinger, 2005;Tapscott, 1998), we were confident that this "unique" generation of learners (born between 1980 and 1994) would embrace an environment that incorporated these two capabilities.

Rationale
At the time of this investigation our university used Blackboard as its learning management system.As an information-centric system, it offered little functionality for us to explore these two questions.Consequently, we opted to develop our own private people-centric web-based environment.Lecturers from two disciplines, Management and Physiology, at the University of Otago volunteered to collaborate in this endeavour.A virtual environment was created that offered a complex, but flexible, functionality that supported networking and the collaborative construction of information.The site was introduced to students as a support site that they could use if they wished.While we expected there would be some who would not engage in our project, we were confident many would see the potential of the site in advancing their studies.We were aware from ad hoc discussions with students on other projects that anything that made a course easier or had the potential to increase grade values would be seen as attractive.We also knew students enjoyed using new technologies.The two staff members, while voicing concern regarding their own computer competency, were eager to explore anything that allowed them improved contact with students.

Aim
Our aim was to capture emerging phenomena rather than starting with predetermined hypotheses.Our research framework was guided by the "social mediation participatory knowledge construction" described by Salomon and Perkins (1998).Within this approach learning is seen as a matter of participation in a social process of knowledge construction and knowledge, jointly constructed and distributed over the entire social system (rather than possessed by a single individual), and "intimately welded to the context and the activity in which and by means of which it is constructed" (Salomon & Perkins, 1998: p. 9).
In particular we were interested in what Lewin (1948) termed "the group dynamics".We wanted to identify the way groups, and the individuals within them, acted and reacted under changing circumstances.These groups were arbitrarily created by faculty with the sole purpose of collaboratively creating a short research paper and class presentation.We were aware of the work of Arrow et al. (2000) that recommends research of groups should be conducted at three levels: individual members, the group as a system, and various layers of embedding contexts.In line with this philosophy, we examined our data at both individual and group levels and considered the larger schema (i.e.landscape) in our interpretation of contextual influences.

Research Design
The project adopted principles of "emergent design" (Cavallo, 2000), an aspect of "design-based research" (Barab, Dodge, Jackson, & Tuzun, 2007;Bell, 2004;Cavallo, 2000;Sandoval & Bell, 2004).As shown in Figure 1, we developed a four-stage approach.The first stage involved working with the staff from the two papers to unpack their curricula in order to map the two courses into a digital environment.From here we created a basic skeleton site that offered the core functions and a list of the assigned "work spaces".Once students registered they then established their group areas and connected with their research colleagues within their allotted "work spaces".
Given our aim was to encourage staff and student participation in the design of the environment, part B was the point at which most of the development would be undertaken.It was this iterative phase where work spaces and tool sets were personalised to the needs of the members through observations of their use and feedback on

Creating the Web Environment
The site was designed to give the students considerable control over their work spaces and associated tool sets in order to allow them a high degree of autonomy.As a result, the site layout and tool sets changed over the course of the project.It was expected that the networking capabilities inherent in virtual sites would help minimize the apprehension and logistical difficulties often faced by students involved in courses that require group work.
Web Site Architecture: A social networking architecture was used in order to produce a people-centric rather than content-centric environment in order to stimulate valuable relationships among teachers and students.The site was a secure, private network that only the researchers, lecturers and students could access.Within the site, a sophisticated access-permissions system allowed differing levels of access to colleagues, work areas, folders and files within project areas.Most access controls were maintained by each individual member or leaders of the project groups.The software enabled the students to control the operation and customization of the environment.The site employed advanced Web 2.0 technologies which were designed to deliver simple and intuitive collaborative access to a range of productivity tools.The network was intended to become the virtual "glue" that bound together information, tasks, ideas and topics.The site was structured around a template schema that allowed customisation on the fly.This flexibility and adaptability allowed the site to be progressively tailored to the needs of the students.
A student's "home page" consisted of five main areas.Underneath the course banner was the site-wide menu which gave access to: settings, list of members, list of work groups, messages, and instant messenger.To the left a second menu gave students access to a variety of personal tools.As can be seen in Figure 2 below, these included: a blog, photo album, discussion area, message tool and a file repository.On the right-hand side was a list of colleagues and below this was a list of workgroups this member belonged to.
Web Site Structure: Although the Management and Physiology courses were from different disciplines, there was a high degree of similarity in their structures.Both courses involved weekly lectures, a weekly lab or tutorial, set study of specific themes or topics and a group project.Both group projects aimed to simulate future workplace environments and enable students to develop necessary professional skills.Initial consultations were held with teaching staff in order to develop a structure from their respective curricula.Figure 3 illustrates the seven main contexts that lecturers identified as necessary component areas within the site.
Within this series of spaces or contexts that were mapped out it become clear that the expectation on students' was predominately passive.The staff were somewhat surprised by this, they had thought their courses was interactive, requiring their students to be proactive and involved.It wasn't until they completed this mapping process that they realise this was not the case.In both cases it was only the "Group Project" that facilitated active participation by the students.Two sites were created for the two courses.The similarity of structure and elements meant that the variations between the two sites were minimal.The anticipated processes within each of the contexts depicted in Figure 3 were further delineated.The assigning of tool sets deemed necessary to these processes was then undertaken to create the following specific spaces or contexts within each of the sites (Table 1).

Participants
Two 300 level classes, one in Science (Physiology) and the other in Commerce (Human Resource Management) participated in this project.Each class had their own branded web site.Presentations were given to both groups outlining the project and potential benefits that students might find useful for supporting their studies, in particular their group project.The invitation made it clear that participation was voluntary and that the project was part of a research study1 .

Survey Data
Informal discussions with students revealed that most were regular and competent users of social networking software.However, it was decided to determine, through the use of a short online survey, a more objective indication of our cohort's ability with web technologies.The survey consisted of only two questions; one on World Wide Web proficiency and the other the degree of use (Table 2).The majority of respondents (89.0%) rated themselves as Expert or Proficient users of the internet, and 74.0% of students reported spending 10 hours or more per week online.
The primary focus of the study was on group projects.Students were organised into small groups comprising 4 -6 members.Each group was to work together on a particular topic over the semester and give a presentation to the class near the end of the course.Twenty-eight of these project-based groups were established across the two courses: Management (n = 11) and Physiology (n = 17).
The majority of the data were collected from the site logs and included both quantitative and qualitative sources (pseudonyms used).Final extraction of raw data was undertaken after members had finished using the site.Both datasets were combined for analysis and no distinction was made between the Management and Physiology groups.

Quantitative Data
System usage statistics were automatically collected by the hosting program/system software.Community membership and community activity statistics were extracted from these.The total numbers of group members, visits and posts, and total posts as a percentage of visits, were tabulated and descriptive statistics were generated.
As part of the analysis process, each group was classified according to its level of engagement within the online community (Figure 4).Table 1.Site contexts and purposes.

Context Purpose
Personal Space Each member's home base was their "personal space" where they could create, store and share content with other members of the site.

Information Area
An information area was created as a repository of all resources required for the paper.This included administrative material, laboratory notes, information pertaining to projects and assessment, and an assortment of web links and articles.
Q & A Area This area was set up as a question and answer area rather than a forum space.It was created for students to post questions directly to the lecturer.These questions could relate to content or course concerns.

Lecture Area
A blog tool was used to post weekly Power Point lectures.Files were first converted to flash and then embedded into the space.This allowed members to simply view the presentation without requiring a download.The blog format allowed the associated files, the original PowerPoint file and pdf file, to be attached alongside the flash presentations.

Module & Research Area
The term "Module" was used in the Management paper and "Research Area" was used in the Physiology paper."Research area" represented research material for Physiology, and 'Module' included topic material in Management.Despite name differences, these two areas were very similar.

Project Areas
Both Management and Physiology included a collaborative project as part of the paper.Each site had "project group" areas set up for teams of four to six people.Within these areas students could access a large range of tools including: group calendar, group file repository, blog, discussion, wiki, chat, weblink, and survey functions.These tools could be renamed and formatted to suit the needs of the group.For example, one group used the wiki tool as a brainstorming area, another group used it to post annotations from web research.

How to
A discussion tool was customised to present animated screen captures describing how to use the site.These were embedded within an area that required members to simply click "play to view".

Notices
This was a hidden area available to teaching staff only.This area was set up to broadcast messages across the site to all members.

Communication
Student and staff communications, that is the "student to student", "staff to student", "student to staff" and "staff to staff" interactions, were all facilitated via the social networking structure within the site.All members could post in various discussion areas depending on access permissions.Each member could post private messages to other members via an internal message system (similar to email).Finally, members could communicate in real-time via an instant message system.This system allowed two or more users to set up real-time chat areas.Table 3 gives a breakdown of the online activity of each group.Participation rates for physiology students (n = 50/51) were 98.3%; and participation rates were 72.3% for Commerce students (n = 60/83).These rates are based on actual registration, where the student registered to gain access to the site.A number of users registered but did not use the site (n = 8).Table 3 shows the activities of the project-based groups and is organised according to the level of engagement of each group as shown in the columns to the far right.
Level of Engagement: As can be seen from Table 3, for the purpose of analysis, groups were divided into two main categories-Passive and Active.Passive groups included Passive viewers (groups who made no postings) and Minimal users (groups who only had one discussion thread).Active groups were described as Low activity or Active producers according to the group's level of engagement and the interaction group members had within the site and the purpose and nature of their visits and posts.Passive Groups: A total of 19 groups (67.9%) met the broad criteria for Passive groups.Of these, 16 were considered to be "passive viewers" as they used the sites only to view content and, consequently, yielded no further data.Groups with members who used the sites primarily to view content were categorised as "minimal users" (n = 3)."Minimal user" groups' discussion threads included those where the site administrator or a group member initiated discussions without response, and threads which featured limited discussion between members.
Active Groups: Nine groups (32.1%) used the site tools to participate in processes related to the achievement of course projects and to arrange meetings and were thus classified as Active groups.Of these, "low activity" groups (n = 3) included those who used the site mainly to facilitate knowledge management processes, such as storing and sharing reference materials."Active producers" used the sites to manage workflow, manage knowledge, and to construct knowledge.
Processes: "Active" groups used digital tools to facilitate and undertake the processes of knowledge management, process management and constructing deliverables.The following sections present the findings in relation to these processes.Although there is some overlap and a degree of fluidity between processes, data has been considered in relation to one process for the purposes of elucidation.
Number of Members within each online group: Although all groups had five members, membership of the online groups was voluntary.The number of members who participated in the online groups ranged from two to five.
Number of Visits and Number of Posts to the sites made by group members: NB: These totals include visits and posts by non-student members of groups (site administrator, researcher and lecturers).Discussions generally constitute more than one post.These factors account for apparent discrepancies in numbers across the table.Dormant posts (i.e., posts made by the site administrator-document uploads or offers for help on using site facilities-which were not responded to by students) are omitted from this table.
The Percentage of (all) Visits in which posts were made: The number of times each group used the Digital Tools to make posts.Some available tools (for example, chat, IM) were not used by any groups and, consequently, these tools were not included in this table.

Qualitative Data
All postings that were made to the sites were systematically extracted.Online dialogue and conversational scripts (as manifest through discussion threads, wikis and blogs) were taken verbatim from the sites and categorised by group and use of digital tools.All data were then reorganized according to the following four Project-oriented processes, or themes, which were developed from observing the students' behaviours on the sites and from discussions with staff about their expectations of student behaviour: Identity Formation: On registration, students had the opportunity to establish a profile which included their name, contact details, an "about me" statement and a photo.Over 70% of students filled in all or many of the available profile fields.Over 80% created at least one "colleague" connection with their friends and group partners.The creation of a profile was not initially part of the study; however it became obvious that students were very comfortable with this aspect of the site.In fact many of the students changed aspects of their profile throughout the semester.
Knowledge Management: Seven active groups used the digital tools available on the sites to manage knowledge.The document storage tool (documents/files/resources) was the primary tool used for this purpose.This tool was used by all seven active groups to upload, store and share sourced information such as research articles and references.In addition to using the document storage tool to manage knowledge, two groups used the blog and discussion tools to post, store and share academic resources and links to online information.
Process Management: While teaching staff made suggestions regarding the potential of the site tools to facilitate the management of group processes over half of the groups remain passive viewers who did not use the site in ways suggested by lecturers.In addition, within those groups that did use the site for these purposes, tensions about site use arose between members who wished to work virtually from those who preferred to meet face-to-face.Tensions around using the site notwithstanding, seven groups used the website to manage work-flow processes such as arranging meetings, planning projects and allocating tasks.
Meeting arrangements.Although, as previously noted, the minimal user groups' use of the site tools was limited to arranging meetings with no clear purpose, three active groups used the site tools (blog and discussion) to arrange meetings with a purpose related to the achievement of their group projects and the discussion tool to facilitate project management processes (including project planning and the allocation of roles and tasks).
Only six groups leveraged the site (wikis, files, discussions) to post, discuss and amend their on-going work products.

Discussion
From a researcher perspective, the project offered a unique opportunity to observe how students managed their colleague interactions, and to quantify the degree to which these interactions/collaborations increased over the life span of the project.While there have been numerous studies examining the use of different techniques and tools within computer-mediated collaboration, these studies assume that participants know how to collaborate and that providing the right tools will enable effective computer-mediated collaboration.But as Richard Joiner, a prominent voice in this area states, "We need to do more than this because students do not know how to collaborate effectively... they need to develop these skills in order for them to use the tools productively" (Joiner, 2004).It was this inability to collaborate effectively that surprised us in this study.
The following diagram (Figure 5) outlines three distinct activity layers that emerged.All students established a profile, connected with their group members, visited other profiles and looked at the resources, lecture presentations and other material that was loaded onto the sites.We defined this level as "individual".While the student was online, they were simply "viewing" and not actually engaging with anyone.The second level, which we termed "collective", represented activities that were collaborative in nature.At this level we were looking for collaborative development of their project plan, arrangement of face-to-face meetings and general project structuring activities.It was assumed the students would have found both the individual and collective activities straightforward, given that they were familiar with these practices from their use of social networks.Level three or "production" represented unfamiliar practice.We were aware that the students would find these practices challenging.To collaborate at this level required the students to use a number of tools.
While most students were comfortable engaging in practices associated with level 1, very few moved to level 2 and even fewer to level 3 (see Table 3).While the tools were different across the levels, the degree of complexity was not.All tools were regarded as "easy" to use and support was offered to assist students keen to explore more productive approaches.What we did conclude was that students were comfortable operating at the individual but not at the collaborative level.As one student commented regarding collaboration "I really have to think about what I am going to say if I put it on the site... easier to just do the work on my own and put in with the others at the end".
It concerned us that students struggled to work collectively and to produce online.The core requirement of the group project was production: production of a presentation and production of a co-authored paper.Given the activity on the site (Figure 3) it appears students used it as a communication device.For example, a significant number of posts related to organising face-to-face meetings.It was not unusual to see groups taking days and even weeks to organise a face-to-face meeting.These posts were analogous to text messages, with very few posts exceeding 50 words.Most students were clearly unaware of the potential of the site to increase productivity and quality of work through access to higher levels of collaboration.
Three conclusions were drawn from their low activity: 1) They were intimidated working in an online environment.
2) They were unsure how to work in a collaborative manner.
3) They hadn't realised the degree to which the environment facilitated the production/construction of sustantial manuscripts.
Informal observations and discussions with students revealed that the common "group" method was to divide the presentation into sections, with each member focusing on their part.The parts were then pulled together for the final product.While a group project, this approach turns it into a series of individual projects.We did not find any group that was co-producing each topic collaboratively.The site is designed to support this latter approach.

Conclusion
From a design perspective the project offered a unique opportunity to explore how to create a virtual collaborative environment.While the building process started in a rather adhoc manner, a systematic approach to designing and developing virtual environments with input from the target community resulted (Figure 1).This approach allowed site development to occur "on the fly", which assisted us in addressing user concerns as they surfaced.Some of these requests were at the function or tool level, others at the level of structure and yet others concerned training and orientation.
Finally, an unexpected outcome was the manner in which staff structured their course online.As shown in Figure 3, the course structure was predominately the delivery of content and did not require any active input from the students.None of the teachers would have realised their course followed this structure if they hadn't been asked to unpack and map their course into virtual space.Both staff members felt the mapping process gave them a deeper understanding of course design.

Future Research
The findings of this study reveal that undergraduate students struggle with the practice of collaboration.Further research is needed to ascertain why students in their final year of a bachelor degree struggle with the fundamentals of collaborative project work: data management, project management, and co-authoring.It also reveals that we shouldn't take student proficiency in the recreational use of web technologies as an indicator, and they can apply these skills within an academic or professional context.This leads to the need for further research into the level of student literacy with computer technologies within academic contexts.

Figure 1 .
Figure 1.Design-based emergent approach used to develop the site.needs/experiences.

Figure 2 .
Figure 2. Screenshot showing the layout of a personal space.

Figure 3 .
Figure 3. Contexts identified by teaching staff as necessary components of the site.

Figure 4 .
Figure 4. Classification schema based on level of engagement.

Figure 5 .
Figure 5. Diagram showing the three distinct levels of action.

Table 2 .
Survey results measuring participants self-reported technology proficiency.

Table 3 .
Site usage data from system logs.