Rome Reborn: A Virtual Journey Through 3D Cities with Dr. Bernard Frischer

NOVEDGE: Tell us a little bit about yourself and what you do?

Bernard Frischer: My name is Bernard Frischer and I am a professor of Informatics at Indiana University, where in 2013 I started the first doctoral program in Virtual Heritage in the United States. Virtual Heritage is a new field that studies the ways in which 3D technologies can help scholars of cultural heritage in fields like Archaeology and Art History to better fulfill their mission of documenting, researching, and communicating cultural heritage sites and monuments.
I will retire from this position on December 31, 2023.  I am also the founder of Flyover Zone, Inc., a company I founded in 2022 and from which I have no plans to retire!

NOVEDGE: What is "Flyover Zone"? 

Bernard Frischer: *Flyover Zone is an Indiana EdTech / SaaS corporation with the mission of virtualizing and democratizing world heritage sites and monuments. Virtualization means 3D modeling of the current condition of the monument; it also means digital restoration of any damage that the monument may have suffered since it was first created. Democratization in our context means two things: we use the Internet to stream the sites and monuments to our users. This makes it possible for those who for whatever reason (time or economic constraints, disability, etc.) cannot travel to the monuments to at least see and experience their digital twin, and to do so at far less cost. Democratization also means that your virtual guide is one of the world's leading experts such as the director of an excavation or a research institute. This is not normally the kind of person with whom you will be able to take a tour in the real world. 

We fulfill our mission through our two services: our Virtual Museum has over 750 3D models of classical sculpture in European museums. The remit of the museum is, over time, to cover other periods and cultures. The collection is growing at the rate of about 100 models per year. 

Our second service is Yorescape. This is a word we coined based on "yore," as in "days of yore," and "-scape," as in landscape. This is our streaming platform for virtual tours exemplifying what I like to call Virtual Tourism (VT) 3.0. 

VT 1.0 is the kind of slide show on Zoom given by local guides with which we became familiar during the Covid-19 pandemic. So, VT 1.0 is not at all interactive from the user's point of view. VT 2.0 empowers the user to teleport from point of interest to point of interest and, when arrived, to listen to pre-recorded expert commentary explaining what is seen. So, VT 2.0 gives the user spatial interactivity: you can go where you want, when you want. In VT 3.0, we add an AI affordance making it possible for the user to pose any question at any time. So, VT 3.0 grants the user content interactivity: you can get just the piece of information you need to understand what you are seeing or what the expert has said in the recording you have heard. At the moment, we have implemented VT 2.0 and have a working prototype of VT 3.0.

NOVEDGE: How many "tours" are available on your website? 

Bernard Frischer: With our latest publications this month ("Egypt Reborn: Giza" and "Flight over Ancient Rome") we have reached 14 tours. Our fifteenth will be "Athens Reborn: Agora," which we will release by the end of 2023. We divide our tours into series called "Athens Reborn," "Egypt Reborn," "Mesoamerica Reborn," "Rome Reborn," etc.

NOVEDGE: How and when did you first start thinking about mapping ancient Rome in 3D?

Bernard Frischer: I vividly recall when the idea came to me. In September 1974, after earning my Ph.D. in Classical Studies at Heidelberg, I arrived as the Rome Prize Fellow in Classical Studies at the American Academy in Rome. The professor-in-charge, the distinguished American Roman archaeologist Frank Brown, took us to see the great physical model of ancient Rome created from the 1930s to the 1970s by archaeologist Italo Gismondi. Like most people who see this amazing model for the first time, I was blown away. I immediately thought: "wouldn't it be nice to find a technical solution to make it possible to scale up the model from 1:250 to 1:1 so that you could create the illusion of walking down the streets of ancient Rome?" I discussed this idea with some of the other people on the tour. One was Donald Appleyard, a UC Berkeley Professor of Urban Studies. Donald told me that he actually had already developed a solution that was being used to composite photographs and physical models for zoning hearings in San Francisco. He thought this might work for the model of Rome. Unfortunately, Donald returned to Berkeley and died before I was able to set to work with him on the idea. 

NOVEDGE: Tell us about the journey that took you from Rome Reborn 1.0 to  Rome Reborn 4.0.

Bernard Frischer: After Donald Appleyard's death, I never forgot the impact that the physical model had made on me. In the early 1980s, I read about the Aspen Movie Map of the MIT Media Lab. This used a videodisc with random access to create the illusion of driving through the city of Aspen, Colorado. I got a small grant to study the feasibility of doing something similar for Rome. Alan Kay, then a Research Fellow at Apple, was my partner. We discovered various problems that made that solution inadvisable. So, eventually, I teamed up with Gabriele Guidi, a brilliant Italian engineer at the Politecnico di Milano, and we made a laser scan of the Gismondi model. That was in 2004. Unfortunately, it turned out that when scaled up from 1:250 to 1:1, all the defects of the models of the various buildings became disturbingly evident. So, we had to throw away the scan data (well, not literally: I still have the files in case someone ever finds them useful!) and start over. Luckily, in the early 2000s I met ETH Zurich graduate student Pascal Mueller at a scientific conference and persuaded him to lend a hand with the software he was developing called the CityEngine. That worked beautifully for all the "filler" architecture of the city--the thousands of apartment buildings. For the other monuments, we used architectural modeling software such as AutoCAD and 3D Studio Max. I like to compare the process of going from Rome Reborn 1.0 to 2.0 and ultimately to 4.0 to the evolution from the Hubble to the James Webb space telescope. With every new version, we could see the same features of our database, but they appeared in more detail and with greater and greater photorealism. Given my age (74), but also given the incredible beauty of version 4.0 achieved by our 3D modeling team, I think that this version is likely to be the last. Except, of course, that we will have to continue tweaking the model as minor errors are discovered and new finds come to light. But I doubt there will ever be a Rome Reborn 5.0, at least in my lifetime.

NOVEDGE: Who else has been helping you along the way?

Bernard Frischer: This has not beena one person project. To the contrary, many, many people have generously helped during the past 27 years since we started in 1996. The second or third slide of every public lecture I give presents their names so that I can give them due recognition and thanks. The current list includes nearly 80 people. Needless to say, I am grateful to them all, and as you examine the list, you can gauge the universal appeal of the Eternal City by observing how it includes people from the USA Europe, the Middle East, and Asia.

NOVEDGE: What kind of technology tools are you employing in recreating these 3D reconstructions? 

Bernard Frischer: We divide the archaeological data into two categories. Class I elements of the model include those features whose name, design, phasing, function, and location are all known with great precision. They number about 250. Examples include the Colosseum and Pantheon. Class II elements are everything else known with less certainty. For Class II elements we use the CityEngine (now part of ArcGIS of ESRI). For Class I elements we tend to use Autodesk products such as AutoCAD, 3DS Max, and Maya. For complex organic forms, such as sculptures, we use Pixologic's ZBrush. To render our images and video we use the Chaos Cloud.

NOVEDGE: What is next for you and  "Flyover Zone"?

Bernard Frischer: Like the Virtual Museum, Yorescape has the mission of covering the four corners of the earth. So, we are developing new tours in places where we have never worked, such as Africa and Asia. We are also adding more in-depth coverage of the parts of the world where we already have tours. For example, next year should see the release of a tour of the Colosseum in Rome, adding more depth to the "Rome Reborn" series, and it will also see publication of at least one tour presenting a monument dating to the Middle Ages or Renaissance in Italy. 

More broadly, we would love to develop tours that are not the normal sort of thing you can visit in the real world. Here I am thinking of something like the Moon or Mars or--speaking of Mars--a SpaceX factory or a distant galaxy.  

Eventually, this could be made into a serious game to be called "Ancient Life" (on the model of "Second Life"), where each "generation" of player could try to rise up in social and economic status by learning a trade or profession, competing for productive or reproductive rights and status, and by bringing into the world a child whose education would determine their status in the next iteration of the game. In other words, you become your own child when the game is reset after a few months.
This way, the metaverse of "Ancient Life" could continue indefinitely, with players rising and falling in status with the ultimate goal being (in the case of Rome) to become a senator or even to succeed to the imperial throne.  All of this was sketched out in 1986 and published in 1988. It is just now barely feasible, though it would still take quite an investment to bring to pass (in case anyone with some deep pockets is reading this! :)...