4K Fine Remaster Engine

Concentrating Comprehensive Imaging Technologies to Achieve True 4K Image Quality

4K Super Resolution Technologies in the 4K Fine Remaster Engine

The response from each company in the TV industry concerning 4K TV is presently a hot topic, but the full-scale 4K boom is just beginning. For the foreseeable future, viewers will mainly be watching video sources such as BD (Blu-ray Disc) movies with Full-HD image quality, DVD and internet videos because true 4K image sources are not fully available. When viewing these substitute video sources on a 4K screen, upconverting will be an essential part of the playback process. In other words, the 4K TV will increase the size and image quality of the video source being played. The process that will achieve this level of definition, called "super resolution," will have an important influence on the image quality of the 4K TV.
Here, we discuss this with the engineers who worked on the development of the 4K Fine Remaster Engine, the new technology in Panasonic's 4K TV.

lm_5_1_lc_1_130930
lm_5_1_lc_2_130925

Could you give us a little bit of the background of the 4K Fine Remaster Engine development?

Ishikawa: The first time we actually looked at 4K image content on a 4K compatible screen, we realized immediately how fantastic and beautiful it was. However, we also realized that, even after an ordinary person bought a 4K TV, he or she would be watching image sources that were not 4K quality for some time. To cope with this, it was necessary to, for example, upconvert the 2K image quality of a Blu-ray Disc movie to 4K quality inside the TV. So, in order to be able to recommend the remarkable beauty of the 4K TV to potential customers, it was necessary for us to complete the upconversion (super resolution) technology so we could provide it to the customers.

What exactly is Panasonic's 4K TV upconversion technology?

Sakaguchi: Presently, there are a variety of super resolution technologies, including example-based and multiple frame reference types. Our 4K TV uses the example-based type. In choosing it, we placed the priority on the fact that this technology is for family use at home. When watching 4K images at home, there will be a wide variety of sources. After considering the customers' usage, we decided to use the "example-based" type because it easily copes with a variety of sources, including Full-HD, SD images, online movies, and others. And for customers who already know how beautiful the Full-HD images are on a Blu-ray Disc, we want them to be able to enjoy even higher-quality images with their 4K TV. The type of super resolution technology that fits best, and most efficiently, with this line of thinking is the example-based type.

Can you tell us about some unique Panasonic innovations?

Sakaguchi: Well, we used the example-based method to achieve our main goal, which was upconverting 2K Blu-ray Disc images to 4K images. Putting it simply, we gathered a large amount of data samples in order to complement the parts of the original image that were incomplete, then we compiled them into a database, and at the same time, we enhanced the technology so it would efficiently and instantly retrieve the best possible samples from the database. The 4K Fine Remaster Engine does this all by itself. We also reworked the tuning process leading up to the finished image, over and over again, until we were satisfied with the end result.

Takita: The thing that determines whether the example-based method can achieve an excellent effect or not is the amount and quality of the database. The more data the database contains, the greater the super resolution effect.

But it must be pretty difficult to build an unlimited database into a TV.

Yamamoto: That's right. The fact that there's a limit to the price that a customer is going to accept when buying a TV is a critical point in developing a home-use TV. On the other hand, TVs are used all over the world. It's important to create a library (a database) that'll be capable of achieving the super resolution effect, without noticeable gaps, for the kinds of images that people will come across anywhere in the world. Naturally, we don't want to compromise on image quality. So we worked hard to keep that balance.

Takita: Initially, in the research stage, we were collecting tens of millions of images from the world's unlimited image sources and using them to compile a database. Deciding how to make that database practical was our starting point. Then, limiting it to a TV for home use, we started selecting data from the viewpoint of, "what kind of images are most customers going to watch?" This led us to create categories using statistical methods. By adding a number of filters, we were able to focus on what was really needed. This selection part of the process turned out to be very time-consuming, though. It was a very difficult job.

Sakaguchi: After that, in view of the visual features, we started emphasizing some of the factors that would help us to verify the imaging, such as the diagonal and curved lines that tend to stand out in an image. Once we got past that point, we got into the area where we were relying on our intuition as engineers. As a result of this narrowing-down process, we finally reached the level of 100,000 image patterns that we had set as our goal. Since we didn't want to compromise on image quality, though, we were still a little bit greedy, and ended up with 120,000 patterns in all.

What other kinds of difficulties did you run up against?

Ishikawa: Since we decided to take on the challenge of an entirely new field with 4K TV, everything we tackled was a first for us. Actually, our engine development also began before the 4K panel even existed. In the earliest stage, we were running verification tests on 1/4-size images with Full-HD quality on a 32-inch panel. They served as substitutes because they were exactly 1/4 the size of a 65-inch 4K TV. By the time the actual panels became available, the development was well under way. That's why we were so impressed by the incredible contrast and image definition. We didn't really have the time to sit back and enjoy those feelings, though. There were still some areas in our engine that required work in order to suit the panel's performance, so we threw ourselves into bridging those gaps.

What kinds of images did you use for the image quality verification while you were creating the 4K Fine Remaster Engine?

Yamamoto: We chose a number of evaluation images for checking the changes before and after adjustment, but we frequently watched particular scenes from major Hollywood movies on Blu-ray Discs to verify the super resolution effects. In order to make sure we were moving in the right direction in tuning the database images, we watched the same scene, as an evaluation tool, over and over again. We also compared the same scenes when the entire team ran checks together. There were certain scenes in some of these movies that had parts that were difficult to render beautifully, so we continued watching the same scenes again and again, right up to the final stages of development.

It must have been tough to watch the same scene so many times.

Ishikawa: Yes, it was. In order to understand the changes before and after adjustment, we had to keep watching the same scene until we literally lost count. Some people would be watching it without knowing anything about the other parts of the movie, and without ever hearing the soundtrack. At any rate, we didn't have the time to relax and enjoy the movie. The whole team had to stay focused on our goal of creating the ideal 4K image quality as we made adjustment after adjustment. As a result, though, we ended up with images that even the engineers were satisfied with, as evidenced by their sighs of approval. Thinking back on it now, those were really good moments. Oh, and I also got them to let me use some images of my favorite supercars for quality checking. (Laughs) As an engineer, my main goal is to achieve the best image quality for all the different kinds of scenes and sources, but I have to admit it was a lot of fun to be able to check the images using the supercars that I love so much. And the power, the dynamism, was so different than on an ordinary TV. I hope everyone who has that kind of love for images will try out our 4K TV!

What kind of images do you want the people who buy a Panasonic 4K TV to watch?

Sakaguchi: Personally, I like science-fiction movies that use a lot of computer graphics, so I'd love to have people watch Blu-ray Disc computer graphics where the definition has been raised to super resolution. That's one kind of image I hope people will check out.

Takita: I definitely want people to watch some of the animation that's popular now both in Japan and overseas.

Yamamoto: I want people to watch images of buildings and natural landscapes around the world, like the ones you often see in travelogues. I want them to see the outstanding definition and real-life effects that 4K gives you in both close-up and distant views.

Ishikawa: Naturally, our concept is to make every kind of image stunningly beautiful, but I think that people will be able to recognize the emotional impact of 4K best from the newest, most beautiful Blu-ray Disc images.

It's safe to say that years of product knowledge, technical expertise, and the passion of each of the members came together to create a 4K TV that Panasonic can be proud of.

Ishikawa: Yes, that's exactly right. We all devoted our utmost efforts to build a TV imaging engine called FRE (Fine Remaster Engine). Actually, though, this was accomplished by a lot more people than just our team. We relied on the considerable knowhow of the parts developers, the people who procured our components, and the members who set up our mass production lines. Each and every one of them was guided by the Panasonic spirit of innovative manufacturing, and they too devoted their efforts toward the goal. I think this new product was made possible by Panasonic's tremendous store of knowledge and wide-ranging basic research in a variety of fields, both for consumer and professional products.
I sincerely hope that every customer will be able to experience the wonder of 4K image quality in this new product. I think it has a level of emotional impact and stunning beauty that will please people of every generation.

lm_5_1_lc_20_130930
lc_23_1_130925

Yuichi Ishikawa
Leader

Principal Engineer
Entered the present TV Development Section of Panasonic in 1991.
Develops image processing algorithms and circuits for TVs.

lc_23_2_130925

Junya Yamamoto
Architectural Design

Engineer
Entered the present TV Development Section of Panasonic in 2008.
Develops image processing algorithms and circuits for TVs.

lc_23_3_130925

Kenji Takita
Basic Algorithm Development

Chief Engineer, Ph.D.
Entered the present Panasonic Head Office R&D Section of in 2004.
Transferred to the TV Development Section in 2012.
Develops image processing algorithms for AV devices.

lc_23_4_130925

Satoshi Sakaguchi
Database/Library Development


Chief Engineer
Entered the present Panasonic Head Office R&D Section of in 2002.
Transferred to the TV Development Section in 2012.
Develops image processing algorithms for AV devices.