Design Archives < 4D Research Lab

Jitte Waagen, Hugo Huurdeman, Jill Hilditch

Introduction: 3D datasets and open science

In the field of material heritage we see an exponential increase in 3D datasets which provides the scientific community with interesting new possibilities. One of these possibilities is to share 3D models using online platforms, making use of so-called ‘3D-viewers’. Such platforms can really add value to 3D datasets, because they allow for presentation of scientific data in real-world dimensions, provide the possibility of annotating the models, and often feature tools to interact with the models. All these factors increase the impact of 3D datasets by making them insightful and creating a versatile medium for communicating in-depth knowledge on those datasets. Some useful platforms have already been developed specifically for archaeological collections, such as Dynamic Collections, a 3D web infrastructure created by DARK Lab, Lund University (https://models.darklab.lu.se/dynmcoll/Dynamic_Collections/) or PURE3D (https://pure3d.eu/), focusing on the publication and preservation of 3D scholarship.

However, the availability of online 3D datasets on such platforms also presents challenges: 3D datasets can be both complex to understand and interact with, and presentation tools often lack possibilities for bi-directional knowledge transfer, which can mean that the insights and narratives generated from the user-perspective are difficult to integrate and often ignored. From an Open Science perspective, it would be very interesting if platforms such as these would provide novel tools to create a rich multi-user workspace. These might include creating your own versions of 3D models and personal annotated collections, as well as multi-author 3D models or collections, and tools to enable discussions on those models and collections, and creating 3D-illustrated learning pathways.

The project that we introduce here, 3DWorkSpace, is an Open Science project funded by NWO (https://www.nwo.nl/projecten/203001026, see also the project announcement https://4dresearchlab.nl/3dworkspace-project-announcement/) and led by Jill Hilditch, Jitte Waagen, Tijm Lanjouw and Hugo Huurdeman. The goal of the project is to develop an online platform for interacting with 3D datasets and explore its potential to offer structured guidance, stimulate discussion and advance knowledge publication. This project is not so much aimed at creating yet another platform, but is intended as a pilot study towards the direct combination of realizing a platform and presenting case studies that will explore its potential, benefits and shortcomings. These case studies are focused on both deployment in the classroom as well as for peer-interaction in a research and professional context.

Voyager

The 3D viewer technology is something quite different than the eventual user-facing platform in which you’d like to integrate it. Depending on the case, building a viewer from scratch might not be a good idea - especially when many good examples already exist. Since our goal was to explore the potential for creating the platform and to evaluate that, we decided to work with an existing viewer. In our explorations, both within the 4D Research Lab as well as in the Tracing The Potter's Wheel project (https://tracingthewheel.eu/), we evaluated various 3D viewers and 3D web technologies, such as 3DHOP (https://3dhop.net/), Aton (https://osiris.itabc.cnr.it/aton/), and Potree (https://potree.github.io/). Each of these has its specific benefits and drawbacks in terms of features, usability and technology, but eventually we chose Smithsonian Voyager (https://smithsonian.github.io/dpo-voyager/), an open-source 3D toolset. We found especially attractive the focus of Voyager on providing both a web-based 3D model viewer (Voyager Explorer) and an extensive authoring tool (Voyager Story). This authoring tool allows a user without specific technical experience to enrich 3D models via a web browser. A user can add, for instance, annotations as well as articles and combine these into tours. These enriched 3D models can, requiring some technical expertise, be subsequently published by integrating Voyager Explorer into a website. Given this capacity, Voyager ticked quite some boxes on our wishlist. A final important benefit of Voyager is that behind its development are professionals working hard to bring their product to as many users as possible and increase flexibility. Direct communication with the 3D Program team of the Digitization Program Office of the Smithsonian has been of fundamental value to the 3DWorkSpace project.

Having decided to use Voyager as the 3D-viewing building block of our platform, we turned to designing a platform in which it could be integrated, allowing us to reach our goals related to the open science approach of multi-authoring, learning and discussing. The challenge was to not fall into the trap of ‘featureism’, i.e. thinking up as many cool features as possible to integrate into the single most fantastic tool. This approach could lead to potential issues, including usability problems and implementation difficulties. Instead, we opted for a theoretical and methodological discussion which led to a baseline set of features that would facilitate the type of use and case studies that we were working towards. Thus, in addition to the basic browsing and search functions of such a platform, users should be able to:

create and use their own 3DWorkSpace account (user authentication)
compare 3D models side-by-side using multiple viewer panels (comparing models)
annotate specific 3D models (annotating models)
create and save personal or public collections of 3D models (collection making)
add basic metadata to collections (describing collections)
add comments to collections and reply to comments (discussing collections)
create learning pathways for collections, incorporating textual content and hyperlinks to custom views of specific models (creating collection learning content)

Components of the 3DWorkSpace platform

The final 3DWorkSpace platform integrates the basic features we defined, such as collection making, annotation of 3D models and detailed discussions about collections. These features were implemented using three main elements: a storage server for 3D assets, the 3D viewer and authoring tools, and the 3DWorkSpace system itself.

The first crucial element of 3DWorkSpace platform entailed the storage and retrieval of the required 3D assets. These assets include 3D models, but also related annotations and additional metadata about the models. As we aimed for creating a bi-directional platform, these files had not just to be statically stored, but also dynamically editable. Voyager directly supports the WebDAV-protocol (https://en.wikipedia.org/wiki/WebDAV), which allows for editing files directly on a web server. Therefore, this WebDAV-server provided the foundation of the 3DWorkSpace platform.

Second, we integrated the Voyager toolset into the 3DWorkSpace platform. Specifically, we made use of two elements of the toolset: Voyager Explorer, the web-based viewer for 3D models, and Voyager Story, the authoring tool for creating the necessary files to display 3D models together with contextual information in Voyager Explorer (using Voyager’s structured SVX-format). Enrichments created using Voyager Story were automatically saved on the previously described storage server and could be visualized using the Explorer element.

Finally, the third crucial element was the 3DWorkSpace system itself, which seamlessly integrated the Voyager tools. The system made use of the Firebase app development platform (https://firebase.google.com/) for features such as user authentication and associated databases. The user interface (‘front-end’) was created via the React-framework (https://react.dev/), a framework to create interfaces using individual pieces (named 'components'). An advantage of React is that created components are highly adaptable, resilient and reusable, further contributing to the goals of the Open Science program 3DWorkSpace is part of.

3DWorkSpace overview

The three discussed components led to the platform illustrated in Figure 1, 2 and 3. Users can browse and search models, collections and learning pathways. A unique feature of 3DWorkSpace is that users can always directly interact with the 3D models; in search results list, collections as well as detail views (Figure 1). The addition of multiple models makes directly comparing features of models possible, which is potentially useful for education, research and professional purposes.

Within the detailed views of collections (Figure 2) users can view and interact with associated 3D models, for instance by rotating models or by toggling visible annotations. Logged-in users can also edit 3D models and metadata using Voyager Story. These edits are directly saved on the WebDAV-server providing storage, offering a seamless experience.

On the right-hand side of a collection, various tabs allow for inspecting and editing collection metadata, notes and comments, as well as learning pathways. These features allow for unique possibilities in terms of bi-directional knowledge transfer: for instance, discussions with peers or teachers. Furthermore, learning pathways (Figure 3) allow for directly linking learning content with specific model views, such as close-ups of forming traces on ceramics. In this case, the multi-model view also allows for direct comparisons. Learning pathways will be further discussed in Blogpost 3.

Challenges, solutions and future work

While the 3DWorkSpace platform prototype provides various novel features, a number of challenges arose during its design and implementation, including user roles, potential system requirements and the authoring of enriched 3D models.

User authentication is an important issue. In the prototype version of the 3DWorkSpace platform, users can register and log-in to access commenting and editing features. However, there is no differentiation between roles; any user can directly edit or even delete any model, collection or associated data. In a future version of the platform, different user roles should be distinguished, to include administrators (having full editing access), editors (upload and edit models or collections) or commenters (only being able to comment on collections). This is especially important for use of the platform within educational settings.

The unique feature of displaying multiple editable models on search result pages facilitates model comparisons, but also resulted in issues with regards to high memory usage; a potential issue for users with older or limited computers. We resolved this issue by including only six models on any given page (e.g., in a search result list). In future work, model display via Voyager can be further optimized, for instance by initially showing low polygon-versions of models, or by showing thumbnails of models which only load after clicking on them.

A final challenge was the inherent complexity of the authoring tool Voyager Story for enriching 3D models with metadata, annotation and tours. Voyager Story has many in-depth features which make it an incredibly useful tool. However, this leads to some difficulties for initial users of Voyager Story due to its complexity. It was not feasible to resolve this within the scope of 3DWorkSpace, but we were able to alleviate it by creating extensive screencasts explaining the authoring process.

Conclusion

We hope with this blogpost to have provided you with some insights into our ideas and how they steered the development of 3DWorkSpace. We will comment on the platform evaluation and practical case studies in the next few blogposts!

Tracing the Potters Wheel

Team

Project Lead

Jill Hilditch - ACASA
Jitte Waagen - ACASA / 4D Research Lab

Concept, development, evaluation

Hugo Huurdeman - Open Universiteit
Tijm Lanjouw - 4D Research Lab
Caroline Campolo-Jeffra - Immersive Heritage

Technical development

Markus Stoffer - 4D Research Lab
Ivan Kisjes - CREATE
Saan Rashid - CREATE

Funded by NWO Open Science Fund (203.001.026)

Screenshot of the Voyager app

Fig 1. 3DWorkSpace models page

Fig 2. 3DWorkSpace side-by-side comparison on a collection page

Fig 3. 3DWorkSpace - Learning Pathway

Jitte Waagen and Emma van de Goot

It has been 9 months since the official start of the Virtual Past Places project (VPP). The VPP project revolves around the development of Virtual Reality (VR) environments, tailor-made for selected courses in the Faculty of Humanities at the University of Amsterdam. As we are in full development, and many interesting steps have been taken, it is about time for an update!

Since the start of the project, we have been closely working together with lecturers on the implementation of Virtual Reality in the classroom. A total of eight courses from different fields were selected for the project and for each of these, we are creating VR environments optimized towards a specific learning objective. In this phase of the project, we focus on two specific challenges. The first is to further strengthen the embedding of the VR environments in the teaching modules, and the second is to develop a method to evaluate the effect of the VR activity on improving education (Waagen, 2021).

Embedding and evaluating

In order to properly design the VR environments and to develop an effective evaluation method, a close collaboration with the Teaching and Learning Center (TLC) at the Faculty of Humanities was established.

Together with the TLC, we organized a brainstorm session with all involved to plenary discuss the affordances of VR in the context of improving cognitive skills, as well as a workshop for course (re-)design. We will expand on both below. The end goals of these sessions have been to ensure all involved are aware of the possibilities of VR for their education and are able to select a proper place for it in their courses. Through this process of plenary discussion and development, we aim to bolster the alignment of their design and implementation to learning goals of the selected courses.

Evaluation of VR in higher education as reported in literature is limited, partly due to the practical complexities of doing VR with large numbers of students. An important step has been a literature study into the current state of the implementation of VR in higher education. A, perhaps surprising, conclusion is that there is relatively little information available on the effective use of VR in higher education in the Humanities. One of the reasons is that many of the applications of VR are geared towards the development of affective or practical skills. This means that with the implementation of VR in this project, which will be oriented towards cognitive skills, we are venturing into little researched territory. As for the evaluation, with the help of evaluation expert Nina van Stokkum from the TLC, we decided on a three-part evaluation approach, a combination of assessing EvaSys evaluations, executing pre- and post-tests and organizing in-depth interviews with students and lecturers. We are still working on finetuning the three methods of evaluation.

Brainstorm session

In November 2022, the first session took place; a brainstorm on the potential added value of VR in higher education. All involved lecturers were received by Jitte Waagen and Tijm Lanjouw from the 4DRL, and education innovation specialist Jolanda Broex of the TLC. The objective of the session was to inform the participants of the possibilities of the browser-based collaborative VR Mozilla Hubs. In addition, the involved lecturers were asked to share their views on their perception of the potential benefits of VR in education. This meeting provided the project with a first indication of where the implementation of VR in the different courses could be headed. In addition, it gave us the possibility to manage expectations where needed.

The main expectation of lecturers on the implementation of VR is that it could indeed potentially improve cognitive skills, spatial thinking, and increase motivation among students. Given the different courses, coming from Architectural History, Ancient Studies, Conservation and Restoration, and Archaeology, a nice range of very diverging VR environments and potential learning goals were discussed.

ABC learning design session

Jolanda Broex hosted the ABC learning design (Active, Blended, Connected) session with the participating lecturers to critically think about the design of the courses and the proper place of a VR teaching activity. The ABC learning design session, a method developed for UCL (Young & Perović, 2016), was selected for its hands-on approach that assists participants in reevaluating their course in a short time span.

The method starts with three actions, namely producing a tweet, drawing a learning activity graphic and evaluating the degree of blended learning in the course. This part is aimed at stimulating reflection on current course setup and materials. In a Tweet (max of 320 characters) participants described the main learning objective and unique qualities of the course. After this they were required to create a graph that reflected the balance of different learning activities: production, practice, investigation, discussion, collaboration, or reflection. Finally, they evaluated the level of blended learning they were practicing and indicated this on a scale bar.

In the following redesign phase, a storyboard with a timeline of the course was filled with cards representing different learning activity types. Then, on a second timeline, these cards can be reshuffled following insights into the desired sequence of learning activity types in a new course design. After this redesign process, the lecturer could turn these activity type cards and learn what kind of actual practical exercises can be implemented, i.e., at the back of the card ‘Investigation’ one can find exercises such as ‘literature study’, ‘lab observations’, ‘web search’, etc. During the sessions, lecturers would brainstorm with students about the course and related activities.

In this way, we hoped that by deconstructing and then reconstructing the course from its basic premises, a conscious design process was stimulated that ultimately benefits the project goals: namely, assessing how to embed the VR environments. At the end of the session all lecturers created an action plan, listing the necessary steps they would need to take before turning to the actual VR environment design. Experiences have been shared in this blogpost by Jolanda Broex.

VR environment design

Following the workshops, the VPP team engaged with individual lecturers in actual VR design sessions. In a series of meetings that we are currently undertaking, concrete learning scenarios are being developed and VR content envisioned, after which an intake will follow with the developers at the 4DRL. Currently five initial VR designs have been developed. An example of a VR design is that for the MA course Historical Archaeology: Theories and Concepts. The main idea here is to develop a basic reconstruction of a part of the 19^th century Amsterdam Neighborhood ‘De Jordaan’. Students will research archival material of the area and period and use the VR to contextualize historical photographs by locating the exact place where they have been taken. As such, the VR environment helps to improve understanding of the spatial context of historical information and provides a tool for data-integration, discussion, and reflection.

Conclusion

In this blog post, we hope to have taken you along on our journey of conceptualizing and designing Virtual Reality for higher education in the Humanities. The past months, we have been working on solutions to ensure a smooth VR design process, to create a seamless embedding of the VR in the course design and create solid methods to evaluate the VR elements. We are excited to see how the project develops and we will update you soon!

References

Young, C., & Perović, N. (2016). Rapid and Creative Course Design: As Easy as ABC? Procedia - Social and Behavioral Sciences, 228, 390–395. https://doi.org/10.1016/j.sbspro.2016.07.058

Waagen, J. (2021, 10 November). Virtual Past Places, a collaborative VR for storytelling and education. 4D Research Lab. https://4dresearchlab.nl/virtual-past-places-a-collaborative-vr-for-storytelling-and-education/

Experiencing the virtual reality environments available on virtualpastplaces.eu

Mind map of the brainstorm session on VR affordances and teaching potential in higher education

ABC course design session in actions (Prof. dr. Danielle Slootjes)

Preliminary VR design sheet for Historical Archaeology: Theories and Concepts

Category: Design

3DWorkSpace (b)log I – from idea to platform

Virtual Past Places progress report: VR embedding and evaluation