Presentation to IEC TC100 May 2010

Recent MPEG and JPEG activities

Prepared by Kate Grant for the AGS meeting

JPEG• Recent meeting:– 51st: March 15 - 19, 2010, at Franklin W. Olin College

of Engineering, Needham, Massachusetts, USA• Next meeting– 52nd: Brussels, Belgium July 12 - 16, 2010.

• Schedule:– 53rd 2010 Oct 11 - 15 Guangzhou, China (co-located

with WG11)– 54th 2011 Feb 21 - 25 Japan (city and dates to be

confirmed)– 55th 2011 Jul 11 - 15 Berlin, Germany

New groups – Innovations Group

• JPEG Innovations formed to increase awareness of JPEG family of standards, to identify, explore and investigate new image related standard topics, to engage with industry and academia to hold workshops and seminars on these areas and raise the momentum for starting new standardization work items.

• AHG meeting report WG1N5405

AIC• standardization of a new evaluation approach

and image compression system for any potential technologies can be identified which significantly would improve the current image coding standard

• AIC ad hoc group is pleased to announce the production of calls for technologies, use cases, requirement, evaluation procedures and test material for three targeted applications: medical imaging, camera sensors imaging and security applications

Progression at 51st meeting• updated call for Advanced Image Coding of Medical

Imaging in WG1N5386 (LS to DICOM, in WG1N5389)• updated call for Call for Medical image databases in

WG1N5404 (LS to DICOM, in WG1N5392)• Call for AIC of Camera Sensor Data in WG1N5387 (also

LS to ISO TC42, WG1N5390)• Call for AIC for security applications in WG1N5388 (LS

to ISO TC223 WG 5, in WG1N5391)• and invites National Bodies and delegates to provide

comments and feedback, and to distribute it through appropriate channels.

N5388 CfP security applications

• An area of special interest is the creation of new image coding scheme for security applications such as video-surveillance, forensics, biometrics, unattended cameras, …

Mandatory features of the candidate technologies are as follows:

• Support QCIF to Ultra HD resolution, including various aspect ratios.• Support of still and moving pictures.• Support for high sample precision integer data, sample resolutions of

8bpp to 16bpp per component.• Rate/distortion performance at least as good as MPEG-4 AVC/H.264 and

JPEG 2000 standards.• Progressive image mode only.• Support for gray-scale, color and multispectral sample data.• Embedding of lower quality, lower resolution images for rendering and

preview purposes. • Very low to medium complexity compared to existing standards and

potentially adapted to parallel architectures.• Support for compression factors up to visually lossless.• Region of interest or random access to the bitstream for partial decoding.

Additional features that are desired:• Error/loss protection and error/loss resilience features to ensure the

suitability for exchange over a variety of error prone channels as well as long-term storage.

• Support the reusability of intermediate coding results for external video analysis/processing.

• Ability to annotate the image data by suitable metadata• Support of computational complexity scalability at the encoder and the

decoder.• Support of lossless compression.• Support of stereoscopic and multi-view sources• Backwards compatibility to existing formats.• Support of spatial and quality scalability.• Handling of HDR sensors data.• Embedded security (watermarking, fingerprinting, steganography,

bitstream selective encryption, …) • Conditional access, data integrity and privacy protection.

N5387 CfP Camera sensor data

• An area of special interest is the creation of archival and interchange formats for unprocessed camera (“raw”) data, however interested parties should be aware that AIC is also interested in other coding and specifically evaluation technologies that are described in other calls.

• Proposed technologies will be evaluated under a uniformly-acceptable testing setup and considered for standardization if appropriate.

Most desirable features of the candidate technologies are as follows:• Ability to represent sensor data from a wide variety of existing and

expected solutions in the market (Bayer pattern and other types of sensors).

• Lossless compression of such data.• Constrained lossy compression, limiting the maximal tolerable

distortion defined by a well-accepted quality threshold of the demosaiced image.

• Embedding of lower quality, lower resolution images for rendering and preview purposes by a simplified demosaicing.

• Low complexity adapted to parallel architectures.• Ability to annotate the image data by suitable metadata including,

but not limited to color spaces, camera settings, optical and sensor characteristics, …

Additional features that could be considered:

• Backwards compatibility to existing formats and solutions.

• Spatial and quality scalability.• Relationship to, or inclusion of, demosaicing tools and

techniques.• Storage of HDR sensors data.• Conditional access, data integrity and rights protection.• Error/loss protection and error/loss resilience features

to ensure the suitability for exchange over a variety of error prone channels as well as long-term storage.

Ongoing work• amendment on guidelines for Digital Cinema enables

the determination of the data size of the compressed bit stream from the predefined bit rate and allows a wider range of frame rates.

• all four parts of the JPEG XR image coding system completed

• ISO/IEC 24800 JPSearch, Part 2, Registration, identification, and management of schema and ontology, and Part 4, File format for metadata embedded in image data at FDIS; Part 5, Data interchange format between image repositories at FCD

MPEG

• Recent meetings:– Kyoto, January 18-22 2010– Dresden, April 19-23 2010

• Next meeting:– Geneva, July 26-30, 2010

• Schedule:– China, October 11-15, 2010– Korea, January 21-28 2011

2 Joint collaboration teams with ITU

• N11112 Terms of Reference of the Joint Collaborative Team on Video Coding

• N11187 Terms of Reference of the Joint Collaborative Team on Advanced IPTV Terminal

HVC

• Final Call for Proposals (CfP) for HVC (N11113) published jointly by MPEG and VCEG as a “Joint Call for Proposals on Video Compression Technology” after Kyoto meeting

• Results of the CfP were analysed jointly by MPEG and VCEG at the April meeting in Dresden.

• The Vision, Applications and Requirements document was updated in N11096.

HVC – April timetable• 33 pre-registrations of proposals by January 15th cut off

date• April 12: Registration of documents describing the

proposals• April 13: Submission of documents describing the

proposals• April 15: Cross-checking of bitstreams and binary

decoders• April 16: Report of subjective test results available to

MPEG and VCEG• April 16-23: Evaluation of subjective test results

Five test classes of video sequence:

• A: 2560x1600 cropped from 4Kx2K, 2 sequences

• B: 1920x1080p 24/50-60 fps, 5 sequences• C: 832x480 WVGA, 4 sequences• D: 416x240 WQVGA, 4 sequences• E: 1280x720p 50-60 fps, 3 sequences

JCT-VC and HEVC• name "High Efficiency Video Coding (HEVC)" agreed for

the new joint standardization project• The JCT-VC chairmen propose to hold the 2nd JCT-VC

meeting during 21-28 July 2010 under ITU-T auspices in Geneva, CH.

• Further JCT-VC meetings are proposed to be held during– 7-15 October 2010 under WG11 auspices in Guangzhou, CN,– 21-28 January 2011 under WG 11 auspices in Korea.

11275Report of Subjective Test Results from the Call for Proposals on

High Efficiency Video Coding • Available publicly from 14.05.10• Proposals submitted for evaluation in response to the CfP on

High Efficiency Video Coding from 40 organisations: BBC, ETRI, Fraunhofer HHI, Fujitsu, Hisilicon, Hitachi, Huawei, Intel, JVC, LG Electronics, LM Ericsson, MediaTek, MERCE, MERL, Microsoft Research Asia, Mitsubishi Electric, MIT, NCTU, NEC, NHK, Nokia, NTT, NTT DOCOMO Inc, France Telecom, Panasonic, Qualcomm, Renesas, RIM Ltd, RWTH Aachen University, Samsung Electronics, Sejong University, SHARP, SK Telecom, Sony, Sungkyunkwan University, Tandberg Telecom, Technicolor, Texas Instruments, Toshiba and University of Science and Technology of China

N 11221 Possible future actions on standardization with Type 1 licensing• At the Kyoto meeting China stated a development team

had developed an open source video codec with performance close to MPEG-2 without adopting the patents in MPEG-2 or subsequent standards

• Some NBs not convinced there is a need for the activity, others want to explore the possibility

• IETF starting work on a royalty-free audio codec and W3C is planning to standardise audio and video tags for royalty fee free codecs.

• ISO/IEC cannot guarantee royalty-free or even RAND terms. But MPEG standards will become royalty-free eventually, e.g. a royalty-free MPEG-2 profile could be created after most of the essential patents expire in 2014.

Dresden Resolution 10.4.1• Given that there is a desire for using royalty free video coding

technologies for some applications such as video distribution over the Internet, MPEG wishes to enquire of National Bodies about their willingness to commit to active participation (as defined by Section 6.2.1.4 of the JTC1 directives) in developing a Type-1 video coding standard. MPEG would appreciate if NBs provide the names of individual organisations that will commit resources. MPEG will use the information gathered from the NB responses, particularly including the number of countries willing to actively participate, in order to decide at the Geneva meeting whether to request approval of a new Work Item Proposal. MPEG does not intend to reopen the issue, unless strong support of at least five national bodies is presented in the future

Advanced IPTV Terminal timeline• Publication of Call for Proposals: January 2010 • Publication of 2nd Call for Proposals: April 2010 • All parties that believe they have relevant

technologies for AIT are invited to submit these technologies for consideration by MPEG and Q.13/16

• Submission and study of responses: July 2010 • Development of the AIT standard: July 2010 - July

2011• Approval of the AIT standard (FDIS): July 2011

11336 2nd Joint Call for Proposals on Advanced IPTV Terminal

• The purpose of the AIT standard is to leverage on advanced technologies to bring into IPTV services exciting initiatives with new features such as open APIs and the possibility for third parties to provide applications to those APIs.

• Both MPEG and Question 13 of ITU-T SG16 (Q.13/16) feel that, by standardizing a flexible set of protocols and interfaces, it will enable “media service ecosystems” yielding new forms of advanced digital media services capable of innovating the IPTV landscape.

• Such a set of protocols and interfaces is the target of the Advanced IPTV Terminal standard that MPEG and ITU-T Question 13, “Multimedia Platforms and End-Systems for IPTV Services”, of Study Group16 (Q.13/16) plan to develop jointly for IPTV terminals more advanced (in terms of functionalities) than the current IPTV terminals standardized or being standardized by ITU-T.

• AIT will support the service providers’ drive to deploy innovative multimedia services by identifying a set of “Elementary Services” and defining the corresponding set of protocols and APIs to enable any user in an AIT value chain to access those services in an interoperable fashion.

• Note that an AIT value chain is a collection of Users, including Creators, End Users and Service Providers, that conforms to the AIT standard.

AIT Ecosystem: AIT standard-enabled digital media service eco-system underpinning and supporting the activities of content creators and consumers

Minimal Complexity Video Coding• Resolution 10.2.1

• The requirements group recommends that members consider document m17005 and bring technical contributions on “minimal complexity video coding” to the 93rd meeting.

• M17005 proposes the development of a lightweight compressor of progressively scanned video, with 7 identified characteristics such as – achieves the same or better compression as the

combination of interlace (2:1) and chroma subsampling– is simple enough to implement in or near a camera sensor,

or display – handles bit-depths greater than 8 bits per pixel – able to achieve a fixed compression ratio

MMT: MPEG Media Transport• MMT is intended to provide adaptive progressive

transport with cross layer optimization, suitable for both hybrid delivery and conversational services.

• There has been interest from 3GPP and OIPF. • MMT Workshop had been held during recent MPEG

meetings (London, Kyoto)• Not yet a single view on the requirements, and a target

architecture still needs to be agreed. • A CfP on MMT was to be published in April 2010, with

response due in July 2010 but with disruption to travel the call was split and only the first part CfP on HTTP-based streaming using MP4 and MPEG2/TS files was finalised and issued.

HTTP Streaming documents

• 11337 HTTP Streaming of MPEG Media Context and Objectives

• 11338 Call For Proposals on HTTP Streaming of MPEG Media

• 11339 Uses Cases for HTTP Streaming of MPEG Media

• 11340 Requirements on HTTP Streaming of MPEG Media

MMT documents

• 11341 Draft Requirements for MMT (Modern Media Transport)

• 11342 Draft Call for Proposals on MMT• 11343 Draft Modern Media Transport (MMT)

Context and Objectives• 11344 Draft use cases for MMT• Final versions of these documents should be

available after the Geneva MPEG meeting (July)