[Interview] Professor Toshiaki Fujii of Nagoya University talks about the cutting edge of 3D imaging technology research and the innovativeness of the 3D display "Looking Glass"

"A revolution in the world of 3D imaging technologyLooking Glass" This innovative 3D display uses ray reproduction technology to create three-dimensional images that make objects appear to float in front of your eyes without the need for a VR or AR headset. Another major feature is that multiple people can experience 3D images at the same time.

One product that has attracted particular attention is the 65-inch 8K resolution "Looking Glass 65." Its overwhelmingly large screen and high definition greatly expand the possibilities of 3D images.

[Reference] Introduction to Looking Glass

In this interview, we will talk about how this cutting-edge technology is being used in research.Professor Toshiaki Fujii, Graduate School of Engineering, Nagoya UniversityWe spoke to Professor Fujii about the latest trends and future prospects for 3D imaging technology. We will introduce the new world that XNUMXD displays will open up, along with Professor Fujii's cutting-edge research.

Professor Fujii: The keyword in our laboratory is "3D," but it is a little different from the typical 3D. Usually, 3D refers to three dimensions (X, Y, Z), but what we are dealing with is 3D in the sense of "looking three-dimensional," such as in 3D movies, and we are conducting research in the field of "three-dimensional displays" and "three-dimensional images."

For example, a 3D movie creates a sense of depth by presenting different images to the right and left eyes. In other words, it is made up of two 2D images. However, if you consider an environment in which the viewer can move around freely, the 2D images need to be shot from 2D positions, which essentially makes it 2D information.

In our research, we are studying the entire process of this four-dimensional information: acquisition ⇒ processing ⇒ compression ⇒ transmission ⇒ decompression ⇒ display.

This envisions the realization of a system for sharing space in remote locations, for example. Specifically, it covers a wide range of areas, including the development of cameras that capture 4D data, data processing and compression, high-speed transmission technology, decompression, and research into displays that can display 4D data.

Professor Fujii: We use LookingGlass as the final display device in our research, and as a device for the final evaluation of each stage. It is very important in research to visually evaluate the results of each stage of research, such as the development of cameras, improvements to processing algorithms, and evaluation of compression technology. For this reason, it is important to use the largest and most beautiful image display device currently available in the world, and we use Looking Glass as one of them.

We are also using Looking Glass to research holography technology. Holography is said to be the ultimate 3D technology that uses the interference and diffraction of light, but at present, device development has not kept up and it can only be realized in small sizes. Therefore, by capturing images generated by holography with a camera and displaying them in a larger size with Looking Glass, we are able to experience what holography will look like in the future when holography technology has advanced.

We also use it to demonstrate our research to new students and visitors from outside the lab.

"Looking Glass 65" used in Professor Fujii's laboratory.

The 8K resolution, 65-inch (height: 843mm / width: 1462mm / diagonal: 1651mm) display is the largest in the Looking Glass series (as of November 2024).

Professor Fujii: I first learned about Looking Glass at a corporate exhibit at an academic conference in the U.S. The product that Looking Glass announced at the time did not have a large screen like today's, but I felt that it was a well-balanced 3D display that optically improved the poor image quality compared to conventional displays.

The unique feature of Looking Glass is that, although it is an old method, by placing an acrylic material in front of a lenticular lens, it is possible to optically display a beautiful image while emphasizing the three-dimensional effect. Another major feature is that it does not require special software, and you can view three-dimensional images by simply connecting it to a PC via USB, and it can be easily linked to general tools such as 3D-CAD and Unity. This has significantly lowered the barrier to creating and displaying 3D images, making them easy to use. In fact, I think that is its greatest achievement.

Currently, we have the largest Looking Glass model with an 8K panel, and although we use other output devices, we believe Looking Glass is currently the best product in terms of both size and resolution.

Another great thing about it is that it allows a large number of people to view 3D images at the same time, which I think makes it ideal for use in exhibitions and demonstrations.

Broadcasts up to 100 viewers for multiple groups of people: Multiple people can watch 3D content from different angles at the same time on the same screen.

Professor Fujii: Our research aims to develop new forms of communication and entertainment using 3D imaging technology. This will enable things like "realistic communication," a system that creates an environment where you can share images with remote locations and feel as if you are there, and "free viewpoint imaging," which allows you to enjoy sports with a sense of realism as if you were in the stadium.

In other words, we aim to create an environment in which the images displayed move when you move your head, just like in the ordinary spaces we inhabit.

To achieve this, we are constantly striving to find the best output device for each era. At the moment, we believe Looking Glass is the best in terms of size and resolution, but we plan to continue to actively incorporate various new display technologies in the future.

Professor Fujii: The first improvement to Looking Glass is the improvement in resolution. Even the current 8K panel is not enough to display 16D images. If a panel with a higher resolution, such as 32K or XNUMXK, could be realized, it would be possible to display more detailed XNUMXD images.

However, given that demand for displays with a resolution of 8K or higher is currently limited, we believe it is important to anticipate growing demand for 3D displays in the future.

Professor Fujii: The field of 3D imaging technology has experienced several booms and declines in the past, but is now attracting attention again with keywords such as the metaverse and XR (cross reality: VR, MR, AR, etc.).

The latest trend is the development of AI technology, especially generative AI, which is having a major impact. Traditionally, generating realistic 3D images required a huge amount of time and effort, but by using AI, it is becoming easier to generate high-quality 3D images from just a few photos.

Combined with improvements in hardware such as improved GPU performance, it is now possible to generate more realistic and complex 3D images. The combination of these technologies brings us closer to realizing more realistic virtual spaces and AR experiences that seamlessly blend with the real world.

I think Looking Glass will be positioned as one of the displays that can show and experience these virtual spaces.

Professor Fujii: 2D display technology like Looking Glass adds value as one of the human senses, so it can be used in almost all fields where XNUMXD displays are currently used. For example, on factory control panels, it is possible to reduce operational errors and enable intuitive operation by displaying the pressed button in XNUMXD. It is expected that technology that can display things that need to be displayed in XNUMXD will be used in all kinds of situations.

Professor Fujii: Not just in the field of 3D imaging technology, but in any cutting-edge research or technological development, broad knowledge and flexible thinking are important. Even something that at first glance seems to be an old technology or a mature field can suddenly attract attention due to the demands of the times or technological advances in other fields.

For example, neural networks, a core technology in the current AI boom, have actually been around for decades, but with improvements in computer performance and the increased availability of big data, they have suddenly begun to produce revolutionary results.

Therefore, I think it is important to have an interest in a wide range of fields and to continue to learn greedily. I would like students to always explore new possibilities while valuing basic knowledge and skills, rather than chasing only the latest, cutting-edge technology.

Toshiaki Fujii

He graduated from the Department of Electronic Engineering at the University of Tokyo in 1990. In 1995, he completed his doctoral studies in electronic engineering at the Graduate School of Engineering at the University of Tokyo. In the same year, he became an assistant professor at the Graduate School of Engineering at Nagoya University. From 2008 to 2010, he was an associate professor at the Graduate School of Science and Engineering at the Tokyo Institute of Technology, and in 2012, he became a professor at the Graduate School of Engineering at Nagoya University, where he remains to this day. He is currently the Cabinet Office's Senior Director of Science and Technology Policy, 2020-2021.
Fellow (concurrent).

He is engaged in international standardization activities for image processing, video coding, particularly 3D video processing and communication, 3D video systems, and video coding.

In 2016 and 2022, he has received the IEICE Paper Award, the 2017 Institute of Image Information and Television Engineers (ITE) Niwa Takayanagi Paper Award, the FIT2017 (16th Forum on Information Science and Technology) Funai Best Paper Award, the Image Recognition and Understanding Symposium (MIRU2021) Best Paper Award (MIRU Nagao Award), the 2022 IEICE Achievement Award, and other awards.

He has served as Executive Committee Chair of the 2016 2022D Imaging Conference, Chair of the Image Engineering Research Special Committee of the Institute of Electronics, Information and Communication Engineers, Editor-in-Chief of the IEICE Transactions on Image Coding and Visual Media Processing Special Issue, Chair of the Papers Committee of the Institute of Image Information and Television Engineers, Chair of the Steering Committee for the Image Coding Symposium and Visual Media Processing Symposium (PCSJ/IMPS) (29-), Expert of the SC29/MPEG/VIDEO Subcommittee of the Information Planning Research Committee of the Information Processing Society of Japan, Member of the SC1/WG2022 Subcommittee, Program Chair of FIT2024, Steering Committee Chair of FIT2023, President of the Information and Systems Society of the Institute of Electronics, Information and Communication Engineers in 2024 and Vice President of the Institute of Image Information and Television Engineers in XNUMX.

A desktop 3D display light reproduction type display series capable of displaying highly realistic 3D images. It is possible to check and display 3D without the need for a VR or AR headset, and it is also characterized by the fact that multiple people can check the same 3D at the same time.

Currently, 6-inch space photo frame Looking Glass Go, 16-inch 4K OLED display model Looking Glass 16″ Spatial, 32-inch 8K resolution Looking Glass 32″ Spatial, 65-inch 8K resolutionLooking Glass 65″There are four models in the lineup.

Key applications include visualizing complex 3D data, prototyping and design review, education and presentations, multi-person collaboration, entertainment and creative applications.

"Unipos" is an overseas product procurement and consultation service for research and development. It is a one-stop service that globally procures and provides the products you need, such as specialized software required for research and development that is difficult to obtain in Japan, and the latest hardware not yet released in Japan.