Category: Reconstruction

3D Restoration at the Bonnefanten Museum

Posted on by Alicia Walsh

The Bonnefanten Museum in Maastricht houses the largest ensemble of Hortisculptures by the Dutch artist Ferdi (1927-1969). These colourful and eclectic sculptures have unfortunately experienced damage over time and are undergoing restoration by independent conservators, both at the Bonnefanten and at a studio in Amsterdam.

Aims

One such sculpture, the Shigiory Torinata, is a tall flower made from silk and artificial fur. It has tentacle like petals, a long stem, and rests in a wicker basket with leaves protruding out of it. The stem shows the majority of the damage, with tears in the fabric and along the seamline, as well as discoloration due to sun damage. In order to conserve this piece, the tears have to be sown and the discolouration restored.

For these reasons, the Shigiory Torinata has been chosen to be restored with the help of 3D methods at the 4D Research Lab.  Due to the stem's altered material, the pattern needs to be first documented in 3D, then virtually reconstructed, and finally printed onto new fabric that can be sown back onto the sculpture - without removing or modifying the original fabric.

Documentation

In order to document the stem's intricate pattern, the Artec3D Leo was used to scan the geometry with the texture. This handheld, structured light scanner offers a fast and high-resolution result.

Fortunately, the colour patterns displayed on the stem coincide with the patterns on the petals, which have experienced less light damage. These preserved colours allow us to replicate the pattern from the stem. After 3D scanning, colour measurements were taken of these petals using the Nikon D5300 camera, ring flash AR400, and a double polarised lens.

Digital Restoration

The next step after documenting the stem is to UV unwrap the pattern on Blender. However, since the fabric on the sculpture has wrinkled and bunched up in areas over time, it does not virtually unwrap flat. This causes distortion in the pattern, which has to be taken into account while reconstructing it digitally.

A 3D cylinder was created and aligned with the original stem and seam lines were assigned to it so that when the texture is projected onto this reconstructed cylinder, it can be unwrapped along those lines. This results in a flattened rectangular projection of the fabric.

The projection was transferred into Inkscape, an open source graphics editor. There the pattern was traced using curved line tools. Images from the tentacles with preserved colours were transferred into this workspace, and using the eyedropper tool, the correct colour was identified from each pattern. Colour swatches were created and these served the basis for reconstructing the colours of the stem. Twelve colours were identified in total, which will be verified in accuracy by the conservators working with these Hortisculptures.

Results

Once the pattern has been reconstructed, we brought it back into Blender and reapplied to its cylindrical stem by baking the pattern onto the original fabric and applying a silk texture map. In this way, we can visualise how the stem will look with its vibrant colours restored once the fabric is printed.

 

Credits:

Bonnefanten Museum

  • Charlotte Franzen, Head of Collections
  • Paula van den Bosch, Senior curator contemporary art

4D Research Lab

  • Tijm Lanjouw, Senior 3D Modeller
  • Alicia Walsh, Junior 3D Modeller

Conservation and Restoration

  • Ellen Jansen, University of Amsterdam, Independent conservator
  • Kaltja van de Braak, Independent conservator

Shinkichi Tajiri Estate

Shigiory Torbinata, 1966, Photo by Peter Cox, credits: Bonnefanten Museum
Shigiory Torbinata, 1966, Photo by Peter Cox, credits: Bonnefanten Museum
Tracing the pattern on inkspace
Tracing the pattern on inkspace
Original (left) and restored (right) stem.
Original (left) and restored (right) stem.

Virtual Past Places, a collaborative VR for storytelling and education

Posted on by Jitte Waagen

With this blog post, we would like to introduce our online virtual reality platform Virtual Past Places. From early 2021 the 4DRL has started experimenting with the browser-based collaborative VR technology of Mozilla Hubs. This came as a natural follow-up from the experiments with various Blended Learning projects in ACASA and the current interest of the 4D Research Lab into extended reality applications for storytelling and educational purposes (see e.g. here and here). Initially, the 4DRL was contacted by Paul Melis and Caspar van Leeuwen at SURF (find them here) to provide a case study for a pilot with the XR ERA network, Centre for Innovation, Leiden University, which we picked up on. The experience generated a lot of enthusiasm and ideas to continue to explore the possibilities in the 4DRL field of expertise: material heritage studies in a broad sense. This is evidently very relevant in times of Covid that placed emphasis on the necessity of online and hybrid learning approaches. This blog post aims to provide a succinct overview of recent developments and plans for the immediate future, and stages the Virtual Past Places website.

Homepage of Virtual Past Places

Experiencing virtual Vlooienburg

For the initial experiment with the XR ERA network, we used (part of) the historical reconstructions of the Vlooienburg neighbourhood, which provided us with a first impression. In addition a second room, a virtual exhibition space, was created to introduce the research project and elucidate the reconstruction process while allowing participants getting acquainted with navigating the environment. The experiences have been written up here and here, where the proper credits for this collaboration can also be found. Some of the more interesting positive observations were on 1) the possibilities presented by the platform to create different VR settings, 2) a straightforward sense of being which was much more pronounced than with a regular slideshow, 3) the fact that the onboarding was not too hard, at least not for the tech-savvy group of participants of that meeting and 4) a general sense of enthusiasm about the potential to experience 3D content in a first-person life sized perspective.

After the initial experience with the XR ERA network group, the excursion was repeated in various settings, ranging from research meetings to social events, and even valorisation purposes, the contents of the tour expanding every time.

Screenshot of the Vlooienburg VR experiment with XR ERA and SURF

Expanding the range of VR places

Inspired by these possibilities, funds have been obtained to create other virtual experiences as well. Where the Vlooienburg reconstructions represent a lost neighbourhood and how that could have looked like, as a set of visualised historical hypotheses, we thought it would be nice to add other types of projects that can benefit from virtualization.

For example, for the iconic UNESCO world heritage site of Troy, Turkey, we used models derived through drone and terrestrial photogrammetry to create a digital twin of the archaeological remains. ACASA is currently executing the Archaeology of Archaeology project at this renowned site. Although Troy is physically accessible, it is at the same time a far-away site, where you cannot take students at any moment it is opportune for a teaching module. Furthermore, it is often flooded by tourists affecting the experience, and moreover fenced off so there are parts of the site actually inaccessible for all but the archaeologists themselves. In close collaboration with project director dr. Gert Jan van Wijngaarden, a virtual tour was created, focusing on the Archaeology of Archaeology project with a specific attention on the ‘silent’ unknown workmen that did the actual excavation in most periods. Virtual information panels were placed at the site that served as illustrative materials alongside the site and the excavation trenches themselves.

Another interesting option is to place a real excavation in VR space as well. Usually such archaeological material remains are only visible during the excavation, where in the rest of the year they lie under a protective cover. The VR could allow for presenting the excavated remains and discuss the complex interpretations of the archaeology present to students and colleagues. They can enter the actual trenches where they would physically not be allowed to do so. An environment such as this can serve interesting purposes in student preparations for coming fieldwork campaigns, as well as to teach them about basic archaeological excavation methods.

Grassroots innovation

All in all, these endeavors gave us a lot of hands-on problem solving experiences with the software, which has been very helpful in getting things to run smoothly for the various VR spaces. In addition, and most importantly, the ongoing experimentation and evaluation allows for a build-up of experience and ideas on how to use such a collaborative VR effectively for storytelling and teaching purposes. Currently, this has culminated in a UvA innovation (Grassroots) project, in which we will implement Mozilla Hubs experiences as class modules in close collaboration with Gert Jan van Wijngaarden and other teachers at ACASA.

Examples of these are courses such as Archaeology, Museums and the Public, where the experience of authenticity will be critically assessed for the virtual environments, or the Research lab: The palaces of Troy IV, where the Troy VR will provide students with a comprehensible and detailed visualisation of the study subject.

The subsidy allowed the installation of a Mozilla Hubs cloud for ACASA, that provides three key-advantages; 1) control over content, as we retain full ownership over the uploaded materials, which is an important aspect in the world of heritage and visual rights, 2) scalability, i.e. temporarily upscale server capacity to allow much more participants then the standard number, and 3) branding and creating a UvA landing page, so we have been able to develop an easily accessible visual portal into the virtual spaces, which became Virtual Past Places. Furthermore, the subsidy will provide for technical assistance necessary for the cloud implementation and application during classes.

Towards further integration

At the moment, opportunities for further research and implementation are being explored. Since the content and infrastructure are secured, we aim for a deepening the conceptual embedding of VR spaces in the teaching modules, and combine that with a thorough evaluation programme to gather empirical data on the advantages and disadvantages of this technology in an educational context.

In the meantime, we welcome you at the Virtual Past Places website, where all discussed VRs (and more) are accessible: https://www.virtualpastplaces.eu/

Screenshot of Satricum VR, excavation of a Roman villa

The challenge of digitally reconstructing colour and gloss: the UNESCO Pressroom case study

Posted on by Tijm Lanjouw

Project background

How can virtual visualisation support decision-making in the restoration of historical interiors? In 2018, conservator in training of historic interiors Santje Pander, won the '4D Research Lab' launch award for her project on the UNESCO Press Room, by the renown Dutch architect and furniture maker Gerrit Rietveld. The room was designed for the UNESCO headquarters in Paris in 1958, but had become redundant and old-fashioned by the 1980s, after which it was dismantled and shipped back to the Netherlands for safekeeping by the Cultural Heritage Agency of the Netherlands (RCE). In recent years, the room has been brought back into attention, and was revaluated, which led to ideas about its possible reconstruction (recently a space has been found for the interior by the RCE).

For her MA thesis, Santje studied the possibilities of reconstructing specifically the linoleum surfaces of the room, which were designed as a unique pattern of shapes and colour that covered both floor and furniture. She proposes various alternatives for the reconstruction of the floor. The main choice regards the reconstruction of the linoleum floor using linoleum from the current FORBO (the original manufacturer) collection, or using a newly produced reconstruction of the old linoleum. For the latter option, two alternatives were proposed: reconstruct the linoleum to match the aged and faded colours of the furniture, or reconstruct the linoleum 'as new', based on samples found in the FORBO archives. An important consideration is whether the reconstruction respects the original intensions of Rietveld, who designed the floor and furniture (and in fact the entire interior) as a unity. The concept of unity was especially important since the architecture of the room itself impeded a sense of unity due to its irregular shape, and awkward positioning of structural colums.

The UNESCO Pressroom, design and spatial divisions, render of the 3D model.

To access the full report of this study:

Lanjouw, Tijm (2021): 4DRL report series 2 - The UNESCO Pressroom by Gerrit Rietveld: digital reconstruction for a study of colour use. University of Amsterdam / Amsterdam University of Applied Sciences. Journal contribution.

The digital 3D reconstruction of room and furniture

Although Santje's main focus was on the elements covered with linoleum, it was clear from the start that in order to to gauge the effect of certain choices on the perception of the room, the entire space had to be digitally reconstructed. This included features such as walls covered in different vinyls, wooden painted cabinets of various types, mirrors, windows, furniture with vinyl upholstry, concrete architectural elements, and of course the TL-lighting. A unique object was the so-called 'world-map table', a table with a light box type tabletop, which featured a map of the world. Fortunately, the original design drawings were preserved, as well as many (but not all) of the original objects. During modelling, the designs were compared with the photographic evidence and the preserved pieces in the depot, which reveiled only small divergences between design and execution. Hence, certain details aside, the reconstruction of shape and dimensions is generally of a high degree of certainty. As an added benefit of the modelling process, we gained some insights regarding certain design decisions by Rietveld, which we discuss in more detail in the project report.

Work in progress. Integrating the original paper designs with the model.

Reconstructing colour and gloss

For the reconstruction of the colours, we used colour measurements that Santje performed on the original linoleum samples and cleaned surfaces of the original furniture. The colour measurements were originally done with a X-rite Minolta i7 spectrophotometer, but we noticed that these diverged from the colours as measured on photographed samples, even though the light conditions of the spectrophotometer were matched by the studiolights. So we used both, to see if there was a noticeable effect on the reconstruction.

In restoration science, much attention is paid to accurate recovery of material properties such as colour and gloss of a surface. Subtle differences may detract from the experience of the authenticity of an object. However, accurate digital reproduction of these properties is not an easy task. The scientific approach would be to objectively measure colour and gloss, and then to enter these values into the 3D modelling program. This is not as simple as it seems. Colour is nothing more than certain wavelengths of light being interpreted by our brain, which 'colour-codes' it for us on the fly. This helps us to distinguish different kinds of objects. Colour perception varies across our species, so it is is very hard to objectively define colour. Also, colour is dependent on light: the same object has a different colour or tint under different environmental lighting conditions. So when we 'measure' colour, we basically measure a surface under specific conditions. Usually, this is 'daylight', which is a soft whitish light that we arbitrarily define as 'neutral'. However, in 3D modelling programs you create another virtual environment with lamps with specific properties, which means that the surface with the measured colour value is lit again, but under different conditions (in the case of the Pressroom: TL-lighting), creating yet another colour. And it becomes even more complex, since we also have to deal with the fact that there exists no single system to store and represent colour ('colour spaces'), and the digital model we use on devices (RGB) is a strong simplification of our own perception. Long story short, to match the colour and appearance of an object in a 3D program with simulated lights is ultimately a subjective process of trial and error.

Gloss on the other hand is basically the result of the microscopic roughness or bumpiness of a surface. The rougher a surface is, the more light gets dispersed, the more matt a surface appears. The smoother it is, the more it reflects light back to the observer. The smoothest surfaces are mirrors. There are devices that measure gloss, which was used by Santje in her material study. However, the resulting values cannot be simply entered in the 3D program we used (Blender), since it uses an entirely different model for computing gloss. So our method was to closely observe the original linoleum samples and linoleum floors in the real world, and try to match this in the 3D modelling program.

Historical linoleum samples on top of a modern linoleum floor. Photo by Santje Pander.
The effect of using a different colour measurement method. Left: RGB measurement on photos. Right: photospectometric measurement.
Photo of the Pressroom by UNESCO/D. Berretty.

Rendering

We created multiple renders with different material settings from the same perspective in order to compare the effects on the perception of the room. On purpose we chose a viewpoint that matched one of the historical photographs, so it was possible to compare this directly to the digital reconstruction. As the 1958 colour photos have known issues regarding the representation of colour, the marked difference was an interesting result that calls for reflection on how accurate our reconstruction is and how faded colour photos can cause a wrong impression of the original room.

The perceptual difference between the room in which modern alternatives of the colours are applied and those in which original colours are applied is especially striking. The difference between the images which show variations of the original colours ('as new', and 'aged'), is less perceivable. Although the actual RGB values are notably different when viewed next to each other in isolation, if applied in the room itself, differences are only noted after very close examination. It may be that the multitude of visual stimuli in the entire picture make it very hard for our brains to perceive small differences.

Render of the Pressroom from the same perspective as the photo. Colours based on colour measurement on original linoleum samples.

Reliability

The question remains whether these results are reliable enough to be used in the restoration decision-making process. There are multiple factors of uncertainty, the method of digital colour and gloss reproduction being an important one. Another factor is that we do not exactly know the original light conditions inside the room. We know that TL-lamps were used, but not exactly their power and light temperature. Based on these uncertainties, it can be argued that it is questionable that we have accurately recreated the interior. The model should therefore be considered as such, a working hypothesis about the physical appearance of a lost space. But we must not forget that an authentic recreation has in this case never been the aim. Moreover, it is quite unlikely that modifying the uncertain variables within reasonable bounds would have changed the outcome of the study significantly. Nevertheless, to model colour and lighting more accurately based on real world measurements, the digital methods we use also must improve.

Render of the Pressroom using colours available in the current FORBO collection, with a modern, glossy coating.

A virtual visit

The project got a nice spinoff in the form of an online 3D tour through the room, made in collaboration with the RCE. For this application we expanded the model to complete the room, and it was integrated with stories about the room from a design perspective. Of course, for this application we can only show one of the versions that we recreated. As a side note in respect to the above, the modifications and conversions necessary to be able to render the model in the browser create again a slightly different version of the room. This underlines the importance for us, researchers in the humanities, to understand and be transparent about the technical procedures and cognitive processes that lead to the creation of such digital 3D representations.

 

 

Screen capture of the virtual tour

Click here for the virtual tour in the Pressroom