The 2nd Episode of the BBC Documentary Renaissance Revolution

Just share a brilliant documentary that I’ve just seen the third time. Talking about this painting for the whole episode, with the editing really full of motion to convey the mass jumps of weirdness as well as relations, and also some great modern background music instead of classical magnificent orchestral ones for the renaissance topic, which I feel express the idea that we understand this painting from our own age and thoughts, just as it was also modern or even advanced at its own time.

This is also where I got some of the knowledge/interpretations of this painting I used in my project, like the owls, the Tree-Man, the structure, and many other abstract ideas…. even before I thought about the topic of this final project.

https://dailymotion.com/video/x1ircew

The UI Sketches

Decided to stop the modelling to work on the UI system and programming part which will be the framework of the project.

I would describe the UI’s style as medieval,  Bosch-ish, and fit in the overall project style.

1. Medieval

Firstly as I have no experience with game UI design, I looked for how it should look like online. Most of them were too cartoon, while the flat ones seemed too technological or modern for the painting’s age.

Then I thought about some historic RTS games I played like Total War and Europa Universalis. The UI designs are much more delicate in these successful game. I also found a very nice design on Behance (from which the last pic came).

 

2. Bosch-ish

I found a fun app in Apple Store the other day called Bosch Painter, which offers many interesting element cutouts from Bosch’s paintings, as well as canvases for the users to compose their own Bosch styled images. It gave me great inspiration.

The idea in this app is that we can create Bosch-ish pictures using things he drew as they are just so recognisable; and this method is also used somewhere else, like the exceptional exhibition Hieronymus Bosch 500 took place in city of Den Bosch this spring.

What surprised me afterwards was that when I looked for some good elements to use, I found some of his paintings are “born with UI” – with some perfect decorative elements:

the exterior (Saints James the Greater and Bavo) of The Last Judgment

The Extraction of the Stone of Madness (Cutting the Stone)

The Seven Deadly Sins and the Four Last Things

What’s more, an unsolved problem that how I should include the exterior shutter of the painting the Garden of Earthly Delights from the very first when I was still planning to make all the 3 scenes this summer, seemed to have got a way.

 

3. The Overall Project Style

This is the hardest part of all. But first I should actually introduce what I’ll use my UI system for. As I have mentioned a little in my plans before, I’m going to create a collection system for all the messages the player will meet with during the exploration of the scene. So when the player has found one message, it will appear in the collection, while the uncovered ones will still remain a “???” status. This is the main game mechanics of this project. Also, I’d like to add the functions of viewing where the message was from in map (based on the original painting), marking the objects you like and reviewing the spinning models later, and taking photos from the perspective you like.

So the demo came out as below. I don’t have a model’s pic taking from a proper view so I’m using a previews work of mine to imitate at the moment. And I found a gothic font to use on that title scroll like what Bosch wrote on it originally (but in a more recognisable font). I didn’t even use any other resources; all of materials I used were from Bosch’s paintings.

This should be very Bosch-ish. However, I don’t find it fit in the game scene’s style at all. It’s too heavy and not at all flat, but the problem is if I want to use elements from the renaissance paintings, the style can’t be like today’s aesthetics.

Here I changed a bit of it and hopefully it could be better with the flat-shading models later. I quite like those ancient styled things I deleted from the last version though. And I still not really satisfied with this design… apparently the layout is quite unbalanced. But the important thing right now, is to go on with Unity to realise the graphics, which is going to be an extremely tricky thing! Bless me 😛

The Project 3D Night Cafe

The Night Cafe

Today I came across a project on Made With Unity about a 3D restoration of Van Gogh’s Night Cafe by Mac Cauley. It was just surprisingly great to see that really somebody is also doing the same kind of thing at the moment, and has quite already achieved the kind of sense that the 3D restoration of a painting should be like in my very initial imagination.

For The Night Cafe, I wanted the viewer to experience the feeling of stepping into this back alley cafe the way Van Gogh might have. I also wanted to express that feeling of wonder when you see his paintings, of being swept up by his exuberant colors where every brush stroke feels imbued with his lust for life as much as with his inner struggles.

I’m also so thankful that the artist talked detailedly about the methods in this project which is of course very useful and inspiring to me dying for the shading and texturing! Although it’s like hey it’s impossible that no one is thinking to make something as you do, I now feel there’s the motivation again and the visible aim I can head towards. Anyway, people are fond of Van Gogh who has been getting all kinds of creative tributes, but not the other genius of painting!

A Useful Website

http://www.esotericbosch.com/

The Esoteric Bosch
Commentaries on the esoteric meanings in the paintings of Hieronymus Bosch

Found an excellent resource of the explanations of the painting. Detailed, academic and quite comprehensive. In the slides the writer explains the painting in every small part and that is exactly what I was thinking to make interactions to my project: when you come to a place, see something and trigger the explanations or something.

I’m afraid I can’t quote this materials directly or maybe I should contact the writer for permission. I keep it now and will see how I could use it later when I start to do the interactions. I’ll definitely need something though and will keep looking for other materials as well.

Game Inspirations, Academic Explanations, and Design Concepts

These days I am reading the book Hieronymus Bosch Garden of Earthly Delights by Hans Belting, to necessarily understand the work better, and hopefully, to find answers to my own unsureness about my project.

The theme being “a digital garden of earthly delights”, it should be a must that there are elements from the original painting which are of great quantity and variety. However, as I am doing the modelling for the weeks, I now get really stuck in a confusion of what I am doing and what it is going to be. I thought it would be fabulous because the painting needs to be animated, explained and digitalised and it has tremendous space to be imagined and developed, while by time I found I am, and will be spending days and nights doing nothing but making models exactly as what has been set in graphics, even so slowly and cannot achieve the quality of the original. There will be really lack of the “design” and I cannot see where my value is other than a poor labourer.

I am now reconsidering what makes my project special. But the first thing I need to be sure about is that I am not making only a restoration, but thinking about what I can possibly add on and change, making it a project “based on” this triptych, considering the time I have. I need to make choices about the priority that what can remain in my project and where I am going – it seems that I have really been evasive about this.

 

---

 

Narratives & Interactions

From all the materials I have been looking at, there are actually few things scholars have reached consensus, even on the very basic things like whether it really conveys specific messages or it is just a personal artistic imagination; and there are only all kinds of interpretations, or people think it is overlooked in interpretations. In fact, even the title is not named by the painter himself: he really left no notion at all, calling for the viewers to undertake their own imagination.

And Belting explains the viewing as a consciously non-linear narrative as well:

We are given a view of them that was old-fashioned even then. It is a view that precludes a fixed viewing point which had become already the norm in painterly perspective. Instead, it creates a view from above, redolent of early illuminated manuscripts, that offers a survey of a broader narrative space. Indeed, as in a medieval simultaneous stage, spatial perception actually functions as a means of narrative rather than from a new spatio-temporal viewpoint… He registers the world as it is without subjecting what goes on in the world to any linear narrative.

This provides the opportunity for me to offer a free-exploration with an interactive “map”/ intelligent guide in this project. As my initial intention, it is not a storytelling or as many places of interests where you have to follow their audio guide, but you will be able to discover what appeals to you most. And at those certain points, the interactions are triggered.

So there are ways I am thinking or inspired for the interactions:

Firstly as a “treasure map”.

I saw the GIF of a game called Isoland, being a relation of a game scene from a certain perspective and a drawing of that. This reminded me of the drawings related to the Garden of Earthly Delights by Bosch himself. And this could be how the viewers look for the places according to what they have been given, which could also be a special impression deepening for those important points, or a kind of collection.

A Chinese indie game Isoland.

The Tree Man, Drawing, Bosch

And the online documents also show the possibility of doing this: the Wikimedia Commons page, and also the interactive documentary of this painting I mentioned in another post of related projects before.

The page on Wikimedia Commons. It has annotations and the interactive frames for details where important.

Secondly, as kind of digital annotations.

This HUD styled visual effect is getting popular but not only related to futurism. At first I was worried if this could be disruptive in style, but now that I decided to “design”, I would like to integrate all the possibilities to create my own things. Not necessary to be so much futurism though.

Works by N3 Design. I saved this kind of motion graphics long ago and once thought about making my final project in this field.

The interactive map in Titanic Museum.

 

---

 

Style and Aesthetics

I did not think much about how the environment would look like at first, just regarding it as a 3D version of the original painting, trying to have a quite good quality and not using the texture directly from the painting (like Bosch VR).

However, as I spent more time on modelling, I found it really painstaking and kill a lot of time to perfect my models. It would be impossible for me to make the realistic models as well as everything at the same time myself. Then I started to think about low-poly again: will I really lose details that I want to show if I use low-poly style?

In Game Design Studio course last semester I used low-poly and it turned out quite nice, but I did not make a lot of stuff there, just to simplify the environment and express the ideas in Calvino’s book Invisible Cities. Except that I did not have time to learn the software to create beautiful models, a main reason was that I thought it would destroy the imaginary space if I visualised that extremely abstract content in a very detailed and specific way.

When I was making models for the final project this time, I have been thinking that I am visualising something not abstract in its appearance, into a non-abstract look. I now realise this is probably why I am confused: if the triptych needs a serious 3D restoration, it would not be a task for me who is not a professional 3D modeller; if I am doing a design project, it cannot be that I just re-make it in Blender and Unity. And would it really damage something if the subdivision of a eccentric monster is 4 or 8 faces, or the shading is flat or smooth? What is the idea that Bosch wants to show, what the humankind is doing and the atmosphere or a realistic-looking world?

The answer is obvious if I do not stress the realistic restoration any more. But I may still draw from the academic analysis on this issue.

Belting describes this painting as “unfettered venture into the realms of the imagination“, and he also refers to the words of an American writer Peter S. Beagle, that it is “as a place filled with the intoxicating air of perfect liberty.“

Yes it actually can be seen as a compromise to the pressure, but also I do feel that it is a right choice since the work could seem better integrated if I am not able to make the realistic look (after all, the results of realistic style by some professional team of games in the market is not so satisfactory).

 

There is also the description “an erotic derangement that turns us all into voyeurs” by Beagle, which is the point I consider to define how the viewers/players exist in this project. So after everything is set at the position according to the painting, I will make a place at a higher position at the very front of the scenes for the player to start with, from where the view is just similar to what can be seen from the picture as possible; and then he or she can move down and into the scene as like.

 

So I would draw on some nice game aesthetics to illustrate the kind of style I am heading for.

This slideshow requires JavaScript.

1. Firewatch. I really love the layered colours and the lightings in this game. It is said that they wrote the scripts to make things in distance of different colour and lower poly to create this kind of look, because the fog function in game engines has only single colour and the high poly would look pixelated.

I also found a tutorial about making a 2D picture of Firewatch into 3D when I searched for images. The result is stunning and it might be useful for me later when I set my scenes. https://blog.sketchfab.com/art-spotlight-firewatch-fan-art/

 

2. The Witness. The world seems not very big when looked down but it is actually great space to explore. The non-specular shading, the decorative flora and the details are kind of what I am looking for. And there are also humans as sculptures – it could be a case how low-poly human will be like, and I think mine are going to move around though.

This slideshow requires JavaScript.

 

3. Ori and the Blind Forest. This one is not flat styled and it is also kind of magical. But I love the layering (not able to move in 3D though), and the style here has the possibility to import the introducing elements I mentioned above.

This slideshow requires JavaScript.

 

4. Inside. Almost exactly what I imagined for the hell panel!

This slideshow requires JavaScript.

 

5. HK Project. This is about a cat’s adventure in a cyberpunk styled space and is still in process. I love the way that the cat is going through the world as a unnoticeable and non-affecting spectator, which is quite similar to the role of the player in my project. I thought about making it third person controlled to make a broader, more fixed and higher point of view, but it would bring new problems of how the player should be like. It is still a unsettled problem.

 

Animal Modelling – a First Discussion, and the Horse

I started modelling animals after the human meshes done.

Obviously there are numerous species, real or unrecognisable, for me being the second most significant and noticeable part/category of the work. I think it is quite interesting that after some time of reading and observation I found generally what are the animals in the each panels have rules to follow. (Click to see original images)

It seems that, while there are only Adam, Eve and the God in the first panel, the exotic creatures are more eye-catching. But then you will find that most of them appear to be very weird, many with quite horrible eyes at the bottom – yes I do not think they look like good creatures, and there are those with multiple heads, creepy reptiles, a “dog” with only two limbs, birds with strange body, unicorns and a “giraffe” – I remember a statement saying that at that time giraffe had not been imported into Europe. That is to say most of the animals in the heaven panel are the mythological ones.

But they turn quite normal in the central panel. We can see most of the animals there are common ones. Mainly three kinds: birds, fish, and quadrupeds,  looks normal and real, and stay harmonious with the earthly delights. And then in the last panel, there are few animals – and even not like animal anymore but monsters with roles and characters, and seemingly they have the authority above human, or are as the way to punish human.

But anyway, for my own project, I decided to start from the central panel where the title comes from. Actually from the very beginning I was really not sure where I could finally reach and am especially worried after I started make practical things, and thus I have been kind of prepared that I would be only able to finish one or two panels if I want to do the aspects I talked about in my proposal, and then it should be the central one, with most people, looks quite ready for an animation and would be more fun to interact with, and the right panel, which could be a great material for fascinating lightings and atmosphere that I would like to explore. They seems much more narrative after all.

So for the central panel, as why I made human first, the horses around the central pond are of the greatest quantity and similarity so will be the most efficient way to start with. Not all under the riders are horses though; there are also goats, oxen, donkeys, camels, bears, and other strange creatures. But as quadrupeds it would be easy to edit the rig.

I would say that this area is undoubtedly the most lively part of the whole triptych: it is almost animated. A round shape in the centre of the central panel, with the biggest crowd being engaged in this single activity, you can imagine how dominant it would be if the painting is a animated picture.

For the horse modelling and the rigging I mainly followed tutorials below:

blender 2.5 horse modeling PART_1 – YouTube
Animate a Moose in Blender – Part 1 The Rig – YouTube

Having looked at every single animal and the rider’s pose, I found none of them are being in a high speed of run (humans are seated steadily, some lying prone and even birds are standing on them), although several do have their forelegs up high. So I chose a mild one among the searching results of horse running frames. And according to that tutorial, I finally chose the slowest way “walk” for my animation instead of the “trot” I clicked on.

Click the image to see the original page, an incredible tutorial with many illustrations.

 

The two basic sets of poses I have made: walking and prancing for the quadrupeds.

Some Related Projects

Here are some related projects to mine I have been looking at so far.

1. https://tuinderlusten-jheronimusbosch.ntr.nl/en#

This is an interactive documentary of the Garden of Earthly Delights based on web browser. It is a very delicate project with high-resolution picture, music and audio introduction within featured details. A brilliant way for those who want to know about this painting quite detailed, where the image is very clear and the experience is smooth. I will also use this as an important reference for the details and text introduction, and which are the significant parts to add interaction.

 

2. http://www.bdh.net/work/boschvr/

This is a VR app making the original painting in 3D animated scenes early this year. I was disappointed when I first saw the title and thought there had already been the project I was planning to do, but got quite released after I saw its trailer because it was not really the way I wanted to make it.

In this app the user rides on a flying fish to explore the environment. The fish follows a fixed routine. While the user is able to look around, it is not possible to go around or even stop moving as will. The primary is that although it is 3D and even also made with Unity, it seems that the team did not make new models but edit the original image in small pieces of figures, add joint so they can move (rotate limbs) with those axes. As a result, the animation is quite rigid and the whole project seems plain and not really 3D. It remains all of the original though; and I cannot keep everything as I am modelling by myself, would of course lose a lot of details because of the time and technique limit. And another thing is that I would also texture my project in my own way to create an integrate environment but not use the painting with the colour and material of the original brushes and cracks.

I have downloaded the app to try this most title-related project. But there was some problems with my iTunes payment so I did not make it to buy the locked scenes which is 2.99 pounds and are only able to see the free one which is the heaven panel, and it was quite simple. I saw a video about the their documentation of people’s trials on street, where there was a little segment of the hell panel which seemed quite nice though. I will definitely try it as soon as I solve the payment problem.

 

3. The 3D animated version of a ancient Chinese painting Riverside Scene at Qingming Festival (Qing Ming Shang He Tu)

I did not find an official website for this project but it has been quite known among Chinese people, and it is visually more like what I want to do to the Garden of Earthly Delights.

The painting is one of the most treasured paintings in Chinese history and is originally 528.7cm*25.2cm on silk scroll. The similarity of it and the one of Bosch is the great details of figures and the scene which can be too much for a common visitor to know, as well as that they are both painted in their own unreal style so there must be a new way of modelling and texturing to make them look fully integrated in a 3D environment.

Below are the electronic versions of the painting. Looking like a computer game, it has everything totally remade with the digital effects (look at the lighting and petals in the first image), and I found this is kind of what effect I would love to achieve for my final version (I am not confident that I can though, since these projects whether I like or not took teams of professional people and much more time, but I would give it a try).

A Digital Theme Garden of Earthly Delights, the Proposal

MSc Design and Digital Media Final Project, the University of Edinburgh.

 

Outline

The idea is about re-visualising the painting Garden of Earthly Delights by Hieronymus Bosch, in a way that can both restore the original painting in a 3D world, and have the thoughts/implications/background knowledge explorable and more understandable for modern people.

There are reasons why I choose the theme:

Firstly, I was really fascinated in the course Game Design Studio last semester when I was creating my own world with bizarre materials and textures, as well as trying to visualise those abstract thoughts through setting the space. Obviously I did not feel enough in that short period where I was not able to fix both details and narratives and still desire to make curiously beautiful things, and this painting Garden of Earthly Delights has just, for long, been the most surreal, mysterious and detailed world that I want to explore.

I have been to Prado Museum and saw the original work there. Too many people, too small painting and yet just unintelligible, especially for what I myself was wondering. Nowadays digital equipments like headphones (for audio guide), augment reality (like in Casa Batlló to restore the scene) and interactive screens (for example, in Aberdeen Maritime Museum I saw a simple game to help experience driving a submarine which obviously made by Unity3D, the nice interactive map in Titanic Museum in Belfast, and now the Palace Museum in Beijing has its digital gallery too) are used in museums to make deeper understanding with the exhibits. With the background of sociology, I have always been thinking about how digital tools can make things deep and multidimensionally understandable, rather than that people are now rushing everyday to produce fast-food topics, which are mean to be discarded tomorrow. And user experience and digital exhibition are indeed the fields that I would like to engage in the future. (But although the exhibit in Prado Museum is not special, I was amazed by the related souvenirs sold in the museum shop and also want to create some parallel surprises in my virtual theme park.)

Another challenge I am excited about in this project is the style. While paintings in 2D can be really free and abstract, normally 3D models are just rigid and too regular/realistic to be artistic, or as the paintings. I am very interested about how this 3D work could look delicate, and also finding a way to develop and animate it (doing the static painting does not need to consider what is the next frame and can just leave the imagination; I am still not sure if I am going to follow the original in every way or integrate my own preference).

I have done some research about the digital exploration about the painting. There was one VR project just made this year; I download the app and felt it quite rough and it had only one fixed and uncontrollable routine. But the 2D materials/documentaries are fairly ample which could be great resource for my later study.

 

Schedule

Research: documentaries, literature, related projects;

modelling & setting the scenes; texturing & detail polishing; animation; interaction; video trailer;

Writing along with the working process

Initially as below:

 

Resources

Computer & graphic tablet (of my own)

Blender, Unity3d, Photoshop

Documentaries (online), specifically:

An interactive documentary project Jheronimus Bosch, the Garden of Earthly Delights (https://tuinderlusten-jheronimusbosch.ntr.nl/en#)

A BBC documentary Renaissance Revolution (2010), the 2nd episode (http://www.bbc.co.uk/programmes/b00vmxzx)

An VR App Bosch VR (http://www.bdh.net/work/boschvr/)

and more.

Written researches (from the library or online).

 

Outcomes

The main outcome should be an interactive 3D project with a video trailer and also a written work. In the project I will mainly integrate these aspects specifically:

1. Scenes restored in 3D models of everything shown in the original painting Garden of Earthly Delights, with part of the work animated in a rather lively way, to let people look around as they like, give them the opportunity to know more about this masterpiece and discover the interesting details.

2. Some interactions to show what are behind the visible, e.g. the implications and the prototypes of certain characters and objects. I will do research about how museums and galleries are using digital tools to introduce their exhibits, some online projects and also the interfaces or information balloons of games and find out a non-abrupt but useful way to add interactive information to my own work.

3. The exploration about 3D materials and textures – to keep the painting style with the sense of brush texture and free colour flexibly when turning the 2D painting into 3D models, so that the 3D work will not look cheap.

I know this is going to be massive work. But this painting is the one I really love and curious about, and this project includes topics that I would love to discover more. So I want to contribute all my final project time to it seriously and see how far I can get, and hopefully find the proper answers to my questions above. (And actually it is also the 500th anniversary of Jheronimus’ death, and there are events and projects festively like https://www.bosch500.nl/. I hope I could get inspiration through this opportunity and pay my tribute as well.)