We set our goals even higher to motivate all the engineers. That made it possible to create the kind of smooth motion that would amaze everyone
4K Intelligent Frame Creation greatly enhances 4K video performance
What makes Panasonic 4K TV different from other brands of 4K TV? To sum it up, it's the sheer beauty of its video images. TV video is expressed by continuously switching still images, so the more frames per second (fps), the smoother the motion. There are a variety of frame speeds in use today. For example, internet images are about 15 fps, Blu-ray Disc movies are 24 fps, and broadcast images are 30-60 fps.*
Panasonic 4K TV displays all of these at a smooth 120 fps.* When there are insufficient frames in the content, new ones are created by the TV. This is generally called "frame rate upconversion." Each company has its own special techniques and knowhow in this area, so differences in picture quality naturally appear.
Here we'd like to focus on "frame rate upconversion." In Panasonic 4K TV, it's called "4K Intelligent Frame Creation". We've asked the engineers who took part in this technical development. They will show us how unique Panasonic 4K frame rate upconversion is and describe the enthusiasm that led from development to sales.
* For NTSC.
Please give us some of the background of Panasonic's 4K TV development.
Mori: The strength and potential of the 4K images were clear, so we as Panasonic engineers started considering our approach to 4K at a fairly early stage. However, in terms of standards and technology, it does not catch up with the level that includes the ability to transmit 4K60p signals over a single HDMI cable. As a result, comparing the situation with Full-HD, in which we had achieved 1080/60p transmission over a single HDMI cable, the ease of use for customers would actually be taking a step backward. With this in mind, we absolutely had to realize the transmission of 4K60p signals over a single HDMI cable in order to offer customers a product that they would be able to reliably use far into the future. That was the minimal condition for 4K TV development.
Within 4K technology, please tell us the importance of the "moving image smoothness" that your team has worked for. And how enthusiastic were you all in tackling this project?
Nio: Well, as far as TV evolution goes, since we're looking toward an age of 4K definition and large-screen viewing, we all wanted to create smoother images. I often watch animation together with my child, and I find myself wishing that the large-screen images were less jerky. I started thinking that way when the mainstream TV size became 37 to 40 inches.
Mori: The maximum class for CRT models was 36 inches. Images that appeared fairly smooth back then started looking jerky when the screen size grew.
Nio: Today, people are accustomed to 40-inch large-screen TVs for daily use. And when you watch a large screen from closer up with 4K picture quality, the smooth flow of the motion is much more important. And especially for children, we want to provide smooth, easy-to-watch images that are gentle to their eyes. This is what made me want to work as hard as possible to achieve them.
Mori: We're engineers but at the same time we're also consumers, so we recognize the importance of aiming for smooth-flowing 4K motion. So we set our target high and started down the list of things we needed to accomplish.
What kinds of issues were there?
Morita: Our number one development goal was to reduce the halo effect (shimmering noise around moving objects) that degrades the picture quality. The reason for this problem is that a 4K screen has about twice the pixel density, both vertically and horizontally, compared to a 2K screen. In other words, it has about four times the definition of a 2K screen. On the other hand, the ordinary viewing distance is about the same as before — around 1.5 meters. As a result, image problems such as the halo effect occur as a side effect of the TV's frame creation function, and are extremely noticeable. While this kind of fault also happens on other TV types, it is even more noticeable in 4K.
Kiuchi: In 4K Frame Creation technology, the motion vectors of objects moving on the screen are detected and new frames are created. Since the original content only has 2K worth of data, even when the size of the detection target is the same, the pixels for the frames that are created are increased by four times. In order to just maintain the same performance as the 2K image, the created pixels and picture-quality processing data are all multiplied by four. Under these conditions, we had to thoroughly check past problem patterns in order to reduce the halo effect. Especially in scenes where an object "appears" or "disappears," it is difficult to detect the motion vectors, and this tends to make it easier for the halo effect to occur.
Not being able to detect motion vectors must make it difficult to create the frames for smooth-flowing motion.
Nio: That's right. Put simply, frame creation is a process where you compare two input image frames and detect the object's motion, then you use those motion vectors to create a new frame in between those two frames that will smoothly express the object's motion. For example, consider an image of a person walking in front of a wall. You can detect the motion vector by comparing the positions of the person displayed in the two frames. In other words, in this case you can predict how the person moved between the two frames. However, there's a particularly difficult point here. The point is, showing the area of a wall that is hidden by a person in either frame. The wall in front of the person is going to be gradually hidden by the person's shadow. On the other hand, the wall behind the person is going to gradually appear. When comparing the two frames for these parts, each one has only one part showing, so it's very difficult to detect the motion vector, and this is the kind of place where the halo effect is likely to occur. Even though a part of the wall area looks like it is being hidden by the person, the wall is stationary so it's easy for human beings to distinguish (meaning that it's easy for us to detect a motion vector of zero). Actually, it's difficult for a TV engine to do this. It just goes to show how incredible the human eye's capabilities truly are.
Our team made it possible to further analyze the detected motion vectors, and accurately find the places where images are hidden and appear. And this made it possible to resolve the halo effect.
Mori: The fact that we had been accumulating examples of problem scenes for several years really paid off this time. That's our heritage as a TV maker. We've been dedicated in TV development for years, and each year we've been working hard to improve our products in order to deliver the best possible picture quality to our customers. This has become one of our main strengths.
Kiuchi: During development, the simple fact that everyone had experienced 4K images motivated us so much. We've had the chance to watch "4K-equivalent" images, by using four 32-inch screens, professional monitors, or other configurations. However, when we looked at the images on a 4K panel that was actually produced as a prototype, we were all amazed at the excellent contrast and the level of definition. Since it was enough to surprise us engineers, who've been involved in producing TVs for years, we believed the customer would be even more excited. Our desire to surprise more and more viewers became even stronger.
Were there any particular points that you worked hard on for overall imaging?
Nio: In addition to smoothing out the jerky motion in animation, we were determined to show everyone smooth online movies. Other than animation, I also find myself watching a lot of promotion videos for popular TV entertainers. (Laugh) Since I'm able to watch them on a large screen with Panasonic VIERA TVs, I thought it must be necessary to make the motion even smoother. I'm sure you'll agree as soon as you watch a Panasonic 4K TV, the images are more beautiful, smoother, and more fun to watch than on a PC screen.
Kiuchi: Online movies are like a treasure chest of images. However, the picture quality of the videos that an ordinary user takes, are generally limited because of the equipment and media they used, and the frame rate is often not high enough. This isn't because the person shooting them intended it to be that way, it's simply due to equipment limitations. In order to express the full intent of the images that were taken, the TV can definitely help by making the images smoother and more beautiful. This was my thinking with regard to our 4K Frame Creation technology.
Morita: I was in charge of the operational timing and system management of the overall 4K Frame Creation, and I found that the final home stretch was the hardest. There were so many parameters in this technology, and we need to make sure it works in every single case. For example, this included the input image frames (frequency, Hz), the resolution, and the display settings (Off/Low/Medium/High processing strength). The number of parameters was so immense, it exceeded the number that could be handled by our spreadsheet software. We ran into development snags and had to triple the time we originally planned. Rather than having any regrets, though, we went to the lab every morning with the solid determination of creating a product that we would be proud of forever.
What other strengths did you achieve in addition to reducing the halo effect?
Mori: We handled the LSI design entirely in-house. This allowed us to utilize the knowhow that we've accumulated over the years, and made it easier to add new technology and mechanisms (architecture), do the tuning, and attain optimization. Being able to do all of this freely is one of our major strengths. At the same time, developing LSI in-house is very a tough task. Once you've produced an LSI, it can't be repaired. It would take too long and be hugely expensive to produce it again, so we were as careful as possible to produce it by making detailed future preparations.
As the leader, Mr. Mori, I imagine you were under a lot of pressure much of the time.
Mori: I sure was. (Laugh) After setting up our targets, we tried to stick closely to the schedule, but this was certainly not an easy project and some things took a lot of time. As a result, though, I feel that we were able to produce something that will satisfy people all over the world, and we all felt so proud that we've accomplished something.
In order to create a competitive and attractive product, it's essential that you set your goals higher. Then, once development starts, you need to aim for that target without compromising. In the picture quality development for a TV, if the image processing LSI team makes a compromise, there's no way to recover the product level regardless of how hard the downstream development teams work. In the end, the product would be mediocre, which is an insult to our customers. As a leader, there are times when I had to be pretty tough on the team members in order to maintain the performance and schedule that we had promised. But I think that's one of the most important parts of being a leader.
Of course, I'm not the only one with this 'no compromising' policy. It's shared by every member of the team. And it's what made it possible for us to create such an excellent product, and to achieve our new LSI. This policy was directly connected to our ability to complete this product.
Developing high-quality image processing algorithms and LSI circuits
Managing frame control for sync signal generation and motion vector detection/intermediate frame generation
Smoothing processes to improve motion image resolution and image content using minimal frames
Development of high-quality image processing algorithms, and image quality tuning
I hope everyone will try viewing your favorite movies on this 4K TV. For me, that means 'Erin Brockovich,' since I'm a big fan of Julia Roberts. (Laugh) I was lucky enough to have the person who was in charge of authoring this DVD (the version that came out at that time) describe the technical points of how they achieved the picture quality. That made the movie even more fun for me to watch.
I hope people will experience many of the high-definition, lifelike images in documentaries showing our world's nature and all of its plants and animals. A large number of high-quality 4K images are already available on the internet, and many major events, like the World Cup and the Olympic Games, will be coming to us in 4K broadcasts. The fun is just beginning.
I guess my favorites are Japanese animation and computer graphic dinosaur movies. With 4K, you can feel the atmosphere surrounding the original animated images, and see the realistic expression of dinosaurs, with their shiny, plate-like hide and all.
My hobbies are mountain climbing and scuba diving, so I especially like to watch beautiful landscapes in nature. With 4K picture quality on a large screen, travelogues with the world's landscapes are also great to watch. What could be better than watching extraordinary images with real-life quality right in your living room?