MERL Business Contributions 2006-2007
This section details the impact of MERL on MELCO's business in four areas: product features, system components, licensing, and standards contributions. In each of these areas, there is continuing revenue from MERL technology that had its initial impact in previous years. This section presents only those items whose first impact occurred in the 12 months covered by MERL's 2006-2007 annual report.
A dream of MERL is to create a new high volume product for MELCO. We have not yet achieved this, but we have contributed important new features to a number of products. For such features, we take the date on which the product with the feature was first produced for sale as the date of MERL's impact on MELCO.
A large part of MELCO's business is in the form of large custom systems for business or government. MERL has contributed components to a number of such systems. For these components, we take the date at which the first system using it was delivered to the customer as the date of impact.
A different way that MERL can impact MELCO is by making standard contributions. This may or may not lead to direct revenue via licensing. However, it allows MELCO to keep closely in touch with important standards and to shape these standards for maximum benefit to MELCO. For standard contributions, we take the date at which a contribution is included in a draft of the standard as the impact date.
A final way that MERL can impact MELCO is by licensing MERL IP to third parties and obtaining direct revenue as a result. For licensing, we take the date on which a license agreement is signed as the impact date.
The following subsections detail what MERL's impact on MELCO has been in the 12 months spanned by this report. In addition, they summarize how this impact was achieved. It is worthy of note that there are several distinct models of how impact can be achieved ranging from work specifically requested by MELCO to finding an application in MELCO for a technology developed independently by MERL. In addition, the typical time from the inception of a project at MERL until actual impact on MELCO is 3 years, with some projects taking twice as long.
ZigBee Stack for Layout-Free Light Control System
On June 22, 2006, The President of Mitsubishi Electric Corporation, announced its Layout-Free Lighting Control System on the market for sale starting in August 2006. The system is called MELSAVE NET.F and it consists of inverter lighting instruments with communication capability, area controller, communication units, lighting sensors and motion sensors, switches, and lighting set-up device. The system uses ZigBee technology to configure lighting devices freely without any limitation from the layout. The lighting devices can be turned on or off and adjusted one by one in a whole area. In addition, this system will save energy and control a whole system more effectively. For a building with 20,000 square
meters (200,000 square feet) the cost of the system starts at $300K. MERL developed the ZigBee technology and its stack, especially the network layer protocol (NWK), application sublayer (APS) and ZigBee device object (ZDO), which is the enabling technology for this world leading layout free lighting control system.
ZigBee has many other potential applications, such as home automation, building automation, automatic meter reading, equipment condition monitoring, and environment sensing and control. MERL is working with MELCO to apply ZigBee technology to several other business areas. Some new system products including ZigBee are expected to come in the near future.
Details: In October 2002, MELCO became one of the founding members of the ZigBee Alliance, which was established to develop the first standard for low power, low cost, short-range multi-hop wireless ad hoc wireless network for variety industry applications. Shortly after that, MERL was requested by MELCO to support their ZigBee activity and make technical contributions towards the world first ad hoc wireless network standard.
From December 2002 to December 2004, MERL focused on the ad hoc network layer protocol development and become one of pioneer members to make significant contributions to the standard. For example, due to its energy efficiency, MERL's channel quality indicator based routing protocol was included in the specification. Using this protocol, the routing path is selected not just based on number of hops or some other merits, but based on the channel quality so that the overall energy consumption of the network is minimized. In December 2004, version 1.0 the of ZigBee specification was published. Since then, MERL was involved in the development of the enhanced ZigBee specification version 2.0 to promote ZigBee applications in HVAC systems, industry plant monitoring, and other areas.
In 2004 MERL pioneered the development of a ZigBee stack in collaboration with Renesas. Within one year, MERL completed the development of the ZigBee network layer protocol software and delivered to MELCO and Renesas. In 2005, MERL continued the stack development and finished APS and ZDO as well as some basic application functions. By the end of 2005, MERL's ZigBee stack software successfully integrated with Renesas hardware platform, passed all necessary interoperability tests and got ZigBee v1.0 certified. Meantime, MERL and MELCO developed various ZigBee demo systems to demonstrate promising applications. All of this prepared necessary technology for introducing new products such as Layout-Free Lighting Control System (LF-LCS).
The most important features of ZigBee technology include self-organizing, self configuration network, built-in discovery of devices and services, self-healing, and reliable communication. After the LF-LCS network is installed, all those communication units (ZigBee nodes) will be automatically connected and form a wireless network. Although the communication range between any two nodes is short (10m-100m), with multi-hop routing, the network work can cover large area with large number of nodes. Therefore, in this LF-LCS, the setting node can communicate with many other nodes one by one without moving close to one particular communication unit. In this way, all the lighting devices in the network can be freely controlled and configured without any limitation from the physical allocation. Thanks to its self-healing capability, when one of the nodes is out of order, for example, it runs out of the battery, the rest of network will continue to operate normally. After it is recharged, it can re-join the network automatically. This is why ZigBee is a perfect solution for LF-LCS and many other applications. According to the press release, MELCO is planning to ship 15 LF-LCS systems in near future. Since 2006, MERL has been working with MELCO to develop ZigBee middleware and enhance the ZigBee stack. With the mass production of the ZigBee chip, the cost of the ZigBee will dramatically drop. Many more LF-LCS systems and other application systems are expected on the market in 2008 and beyond.
Sports Highlights Playback for D903i Cell Phone
In September 2006, MELCO began shipping a new cell phone for NTT-DoCoMo (model D903i). This cell phone has companion software that contains an interface concept provided by MERL. For sports programming in particular, MERL's "intelligent fast forwarding" makes it easy to scan through recorded content, skipping from one key play to the next.
MELCO has a long history of video related products from video transmission equipment to TV sets. One of the labs that became part of MERL was founded in 1993 by the Audio/Visual business unit and has been involved with video ever since. These researchers work both on projects specifically requested by MELCO and on more speculative work.
Details: In 1998-99, MERL worked on video indexing in the context of the MPEG-7 standard, culminating in the acceptance by MPEG-7 of a "Motion Activity Descriptor" that can be very efficiently computed directly from compressed video.
Building on this foundation, MERL did speculative work on automatic video summarization starting in 2000 and continuing through 2001 and into 2002. The general goal of video summarization is to locate a small subset of a video that can serve as a summary of the rest.
MERL's initial work in this area took the traditional approach of breaking video into segments and selecting "key frames" to represent each segment. However, over time, MERL's focus shifted to what is perhaps better described as intelligent fast forwarding. In particular, MERL created a prototype system featuring variable speed playback where "interesting" segments are played back at normal speed while other segments are played back faster than real time. In this system, "interesting" was defined primarily in terms of MERL's motion activity descriptor, with segments featuring highly varied motion being considered more interesting.
MERL demonstrated its summarization work to MELCO people on a number of occasions staring in mid-2001. In mid-2001, MERL's prototype was demonstrated to a group of people at MELCO who were designing DVD recorders and generated considerable interest from them.
When purchasing a DVD, people are accustomed to receiving indexing information such as a table of contents dividing the recorded material into "chapters". However, when a standard DVD recorder writes content, this is typically stored without any indexing information at all. MERL's work held the promise of (at least partially) filling this gap.
Starting in the last half of 2001, the DVD recorder group began to push the process forward to the look and feel of the prototype. In late 2002, they presented MERL's demo to the home entertainment products business group and started to get buy-in from them.
In parallel with the image-based work above, MERL began to experiment with using audio features for summarization. This was done using code developed separately by a MERL researcher as part of his work on an audio contribution to MPEG-7. This work got off to a slow start in 2001, but by 2002 had yielded tantalizing results. Driven by the needs of their specific application, the MERL researchers eventually re-designed the audio analysis algorithms. They went on to devise a classifier training technique that yielded the high classification accuracies that were required for the application. A key aspect of the algorithm that it enabled scalable summarization i.e. generation of summaries of any desired length.
Everything began to come together in 2003, with a strong push toward productization. While studying exactly how MERL's work could be included in a DVD recorder, it was discovered that the planned DVD recorders did not have enough processing power to support video analysis---not even highly efficient analysis based on motion vectors. Fortunately, it was discovered that satisfactory results could be obtained by using very efficient-to-compute audio features alone.
MERL's on-site collaboration with Japanese engineers revealed that the percentage of a characteristic mixture of the commentator's excited speech and cheering, in a sliding window centered at the current point in time, is a good indicator of the interest level of the program around the point. The summarization mechanism consists of playing back only the parts that exceed a certain specified interest level. The final algorithm therefore consists of audio classification carried out on the incoming audio stream in real-time followed by a percentage calculation that yields the "interest level." The graph of "interestingness" vs. time enables the user to modify the length of the summary by moving the aforementioned threshold up or down. The technique works well across a wide variety of sports content, as noted by several reviews in the Japanese press.
Close collaboration continued through 2004 and into 2005, leading to the joint creation of product-ready summarization code. This was included in the product released in late 2005.
The product has received critical acclaim in the Japanese press. The prestigious magazine HiVi rated it as the best buy in its category. It is the world's first mainstream PVR with such highlights playback capability.
In 2006, MELCO ported this functionality to the companion software of NTT DoCoMo's cell phone model 903i. The software extracts sports highlights from content recorded on the PC. The extracted highlights are in mp4 format and are transferred to the cell phone through a USB link. The cell phone is equipped to play the sports highlights with essentially the same interface as that of the DVD recorder.
In addition the new work on cell phones, MERL's contribution to MELCO's DVD recorder product continues. In September 2006, MELCO began shipping a new DVD recorder (model DVR-DV635), which includes MERL's "intelligent fast forwarding" implemented using purely the host embedded CPU unlike the previous model that employed an additional DSP. MERL succeeded by both adapting the core algorithm to reduce the computation and memory footprint, as well as by optimizing the software, thus achieving the functionality at no additional hardware cost.
In the future, we plan to extend our techniques to other genres of content to enable similar intelligent "trick play."
Contributions to the ZigBee Cluster Library
The ZigBee standard for short range communications includes a so-called "ZigBee Cluster Library" which defines a set of reusable device descriptions and attributes that can be used to develop advanced application profiles. MERL technology for thermostat and fan control was included in the Cluster Library (Document 075123r01ZB) at the end of 2006.
Since 2003, MERL has been involved in the creation of ZigBee Alliance Specifications. We initiated early attempts to draft Application Profiles in the domains of Industrial Plant Monitoring (IPM) and Heating, Ventilation, and Air Conditioning (HVAC), with MERL researchers acting as chairmen of these two groups. The IPM specification was balloted and approved by the ZigBee Alliance in 2005. However, subsequent releases of the underlying ZigBee protocol stack caused a fundamental change in the way that ZigBee handles specific devices. In particular, the approach was changed from profiles to a library of concepts that could be used by devices. Aspects of MERL's work (particularly on the HVAC profile) were subsequently combined into this library.
Face Recognition for Biometric Terminal
In January 2007, MELCO made the first installation of a new biometric access control system (ACS) for the Japanese domestic market, the "Integrated Face/Fingerprint Biometric Terminal OPG-FACE." The system consists of a client device (or devices) for user interaction--the "terminal"-- and a server that does the heavy computation and stores the various biometric data bases. The terminals are capable of capturing both face images and finger print images. They also contain a keypad for user entry of PIN codes if desired. When installed, a terminal is mounted next to a door that requires controlled access. To gain access, a person must be recognized based on one (or both) of the biometrics. The novelty of the system is the inclusion of MERL's face recognition to allow faster and more convenient user interaction with the ACS.
Details: MERL's work on face recognition started in 2002. Face recognition was a natural follow-on to the face detection work done by MERL staff Paul Viola and Mike Jones and now widely imitated throughout the world. Although the face detection work was started outside MERL, we went on to make an improved algorithm for detecting faces in images, and then to the problem of recognizing those faces. All this work is distinguished because of its speed and its light weight implementation. The detection speed derives from two things: first, faces are detected by sequentially testing and rejecting image patches. Most patches are rejected right away, so the analysis can go on to consider the next patch. Second, each test is extremely simple and fast to compute because it is based on simple sums and differences of rectangular pixel areas in the patch. Exactly which tests ("rectangle filters") produce good results is determined by a machine learning selection algorithm called Adaboost. The same rectangle filters and learning
engine were applied to the face recognition problem by pairing face images into "same person" pairs and "different person" pairs. The learning engine selected filters that best classified pairs as same or different. Of course, before faces could be compared for recognition, they had to be first detected, and then aligned so that features were in more-or-less constant positions. This alignment was also accomplished using similar detectors trained to locate eyes, nose, corners of the mouth, etc.
There are many applications of face recognition in MELCO businesses, so MERL was encouraged to benchmark and improve the performance of the method over several years. This effort became a close collaboration with MELCO domestic laboratories and ultimately resulted in error rate reductions by more than an order of magnitude. The process of continual improvement continues today as we strive to extend the range of successful face recognition to less controlled lighting and pose situations.
Human Tracking System Using RFID and Face Recognition
In the spring of 2007, Mitsubishi Electric began selling a human tracking system that uses both face recognition and RFID tags to keep track of the location of people in a secure facility. RFID is good for positive ID at specific checkpoints. Face recognition allows less formal verification between checkpoints that a person passing by is who the system expected and is indeed authorized to be in the location. MERL provided the face recognition algorithm used in the system.
Since 2005, various parts of Mitsubishi Electric have become increasingly interested in systems employing face recognition that build upon existing products and sales channels. The human tracking application grew out of existing business with some large, secure facilities, and it was natural to propose a system to raise the general security level. Development went on in 2005 and 2006, culminating in the product release in the spring of 2007. The face recognition algorithm is the same MERL algorithm that is used in the biometric terminal described above.
Object Tracking for Harbor Surveillance Systems
In February 2007, MELCO began installing the new harbor surveillance system that is built using high-end security DVR as a core (model TL-5000U). The integrated system contains other components such as PTZ cameras, information fusion and storage servers, video encoders, cable detectors, etc. to provide a complete solution for a wide spectrum of applications, ranging from high security areas to small businesses and residential houses. MERL's advanced detection ad tracking technologies make it possible to automatically find humans in variable frame rate video, track humans and vehicles, analyze their actions using motion histories, detect left-behind objects, and effectively control PTZ cameras.
MERL has been an award winning leader in computer vision area in the last decade and MELCO has a long history of video surveillance products from the world's first time-lapse video recorder to smart cameras. In 2000, MERL focused its related research on the understanding of people's and vehicles' movements in video with the goal of developing technology for surveillance and traffic products. This led to a wide range of technologies in the areas of motion detection, object tracking, and human detection.
Details: In 2001, MERL started automatic video object segmentation for consumer video to enable efficient encoding. This work naturally evolved into the background generation by motion detection for static camera setups in 2002, culminating in the development of a robust and accurate online learning method for static cameras that outperforms the conventional approaches proposed by MIT and other companies. MERL's initial work in this area has been later extended for larger surveillance systems that consist of multiple non-overlapping cameras in 2003.
Building on this foundation, MERL did speculative work on the low-frame-rate object tracking to achieve processing of multiple input videos on the same processor in 2004. The goal of object tracking is to extract discriminative information of moving objects that can empower automatic event detection.
Everything began to come together in 2005, with a strong push toward productization. By the diligent efforts of MELCO team, MERL's object detection and tracking solutions found their way into DVR, access control and human recognition system, wide areas security system, and intelligent camera terminal. In 2006, a parking lot monitoring system was another application that wanted to feature MERL's detection technology in its upcoming versions.
Driven by the needs of the harbor surveillance system and by constantly improving on the core technologies, MERL has won one of the best paper awards at CVPR out of 1300 papers for human detection on manifold technique in 2007.
Technologies developed for this system also benefit other projects. For instance, the human detection using the classification on Riemannian manifolds is now a core solution for pedestrian detection for on-vehicle cameras. The background generation based people counting algorithm is being used in Misubishi Electric's large visual system. The shadow removal and tracking is integrated in the DSSS system for identification of illegal passes at toll gates on highways. The object boundary and pose tracking methods are being revised for tracking of organs in medical data, for extraction of player statistics in sports videos for a potential collaborator in US, for controlling of the airborne camera in a next generation Heli-tele.2 system, and for correlating the visual information in MERL's own ambient intelligence system.
Our detection and tracking methods are shown to be superior to our competitors in terms of accuracy as they employ competent image descriptors and precise learning techniques. We are currently developing ultra-fast and economical implementations that make use of the of-the-shelf components.
Phase Unwrapping for Interferometric SAR
In early 2007, MELCO delivered to a customer Digital Elevation Maps (DEMs) produced using a new Ku-band airborne Interferometric Synthetic Aperture Radar (InSAR) system. A DEM is an elevation (height) map of terrain, providing a height for sampled locations on the ground (in our case on a 50 cm x 50 cm grid) and can be used for a variety of purposes. An InSAR system uses two radar receivers, one of which is also a transmitter, separated in height and traveling on a moving platform such as an airplane, to create DEMs of terrain from long range and in any weather conditions. The new system includes a crucial computational software component from MERL for 2-D phase unwrapping. The system and its resulting DEMs demonstrated less than 50 cm mean absolute height error, which is believed to be unprecedented in InSAR imaging and substantially due to MERL's phase unwrapping algorithm.
MERL plans to continue development of the software, primarily to improve the speed of the algorithm, which may become important to MELCO as more and larger datasets are processed in the near future.
Details: Two-dimensional phase unwrapping is the critical computational step in InSAR and has received a number of recent treatments. In 1998, Constantini published a method based on network programming using a Minimum Cost Flow (MCF) algorithm. In 2000, MELCO's previous phase unwrapping algorithm was developed with MIT and was based on a Weighted Least Squares (WLS) algorithm. In 2001, Frey, Koetter, and Petrovic published an algorithm for phase unwrapping using "loopy" Belief Propagation (BP). In 2002, Dias and Leitao published the Z-Pi-M algorithm.
In the same time period, MERL had been actively involved in research on Belief Propagation and in 2002-2003 undertook a basic research project to accelerate the very time-consuming operations of 2-D BP networks by using a Graphical Processing Unit (GPU). MERL chose the Frey phase unwrapping algorithm as a test case and demonstrated a 30x acceleration of the BP-based phase-unwrapping computation using a GPU. Also during the same time period, MERL was assisting MELCO on a separate project for satellite-based 3-D modeling, including the use of satellite-based SAR.
In 2004, inspired by both the MERL projects, MELCO asked MERL to pursue research on phase unwrapping itself.
By March 2005, MERL had implemented in-house software for the existing WLS-based and BP-based algorithms and was able independently to verify that the Frey BP-based algorithm was superior in many ways to the WLS-based algorithm. Although, despite better results in most cases, the BP algorithm had one important flaw in that it did not guarantee a residue-free solution. "Residues" in 2-D phase unwrapping are points of inconsistency that disallow the forming of a consistent phase solution throughout the pixel map. Removing all residues is one important requirement of a successful 2-D phase-unwrapping solution.
During this testing period, MERL undertook a research effort to understand the nature of existing algorithms and to design a better one. All existing algorithms had fundamental theoretical flaws. The MCF algorithm always produces a residue-free solution, but does not support a noise model. The WLS algorithm produces a residue-free solution and rejects noise statistically, but at the expense of large systematic local distortions caused the wide-scale smoothing inherent in the formulation. The Frey BP algorithm performs well locally, but does not guarantee a residue-free solution and does not use a noise model. The Z-Pi-M algorithm does not guarantee a residue-free solution.
By June 2005, MERL had developed a solution using a combination of BP and MCF. The BP part was based on a well-formulated factor graph that supports both a noise model and a terrain model. The MCF part guaranteed a residue-free solution at each step. The MERL BP-MCF algorithm was developed and tested over the Summer 2005 showing superior results to existing algorithms including a guaranteed residue-free solution, statistical noise and terrain models, and good performance in both low-noise and high-noise regimes.
The main drawback to the system was that the BP part of the implementation was large and slow. For a large realistic 100 MPixel dataset, the algorithm would require 400 GB memory and take 184 hours (8 days) to compute, which is probably impractical. This drawback led to the design of a more efficient Iterated Conditional Modes (ICM) approximation to the BP part of the algorithm and led to creation of the MERL ICM-MCF algorithm. It was believed that the results would be similar, but the same 100 MPixel dataset would require only 5 MB and 6.5 hours.
Between September 2005 and March 2006, a software engineering effort was made to convert the original research prototype into useable software that was efficient and could be maintained and tested more easily. This effort included converting certain parts of the software from floating-point to integer and replacing BP with ICM.
In April 2006, MERL internally released MERL ICM-MCF algorithm to be used for internal data processing. MERL received SAR data from a flight test made by MELCO earlier in the year, processed the results and delivered them to MELCO for analysis. The analysis found the results to be significantly superior to the existing phase unwrapping results. MERL made subsequent improvements in the accuracy of the algorithm and the performance of the software.
In March 2007, MELCO reported results of evaluating the latest software on new flight data collected on behalf of a customer. The results were compared against laser profiler data of the same area and were found to produce height errors under 50 cm, a very good result, believed by MELCO experts to be unprecedented and meeting the requirements of the customer.
In April 2007, researchers from MERL and MELCO collaborated on a report to be published in Japanese at IEICE and in English at the IGARSS 2007 conference in July in Barcelona.
MELCO anticipates new contracts in the near future which will require processing many more and larger datasets. To make this feasible, MERL intends to continue work in FY2007 primarily to make the software faster. Other planned improvements include refining the noise model of the algorithm and improving memory management and diagnostic reporting of the software.
Contributions to IEEE 802.15.4a UWB Personal Area Networks
In March 2007, the IEEE Standards Association officially approved the IEEE 802.15.4a standard, which provides an ultrawideband (UWB)-based physical layer for communications and ranging in Personal Area Networks (PANs). PANs have obtained great importance in recent years especially for sensor network applications and home automation; for example, the widely used ZigBee standard defines a good networking layer for such networks. The IEEE 802.15.4a standard is the first to provide explicit support for ranging, and also provides a modulation scheme that provides extremely energy-efficient communications.
MERL has been a key player in the development of the IEEE 802.15.4a standard, and has provided a number of important technologies. MERL technologies that were added during the past year are the "Start Frame Delimiter" (SFD) and a scaling factor for the range quality indication.
Details: The IEEE 802.15.4a standard defines an alternate physical layer for the IEEE 802.15.4 standard, also commonly known as low layers of ZigBee. While the original 15.4 standard used narrowband signaling, the 15.4a standard employs ultrawideband (UWB) signals, i.e., signals with a bandwidth of 500 MHz. This large bandwidth enables very precise ranging between devices; the range information between several devices can be converted into geolocation information. The use of UWB signaling also allows the design of low-complexity, energy-efficient communications approaches.
MERL has been a key player in the 802.15.4a standard ever since its inception in 2003. Members of MERL have played leading roles as subgroup chairmen and technical editors, over the years. Furthermore, a number of key technologies from MERL were included in the standard. After the official approval of the standard by the IEEE in March 2007, the first products from different manufacturers have been deployed.
In order to achieve precision ranging, the 15.4a standard defines a preamble that is sent at the beginning of every packet. This preamble, which also serves for acquisition of the packet, has variable length, and repeats a certain pre-determined sequence many times. Since the length of the sequence is not known a priori, and since a receiver does not know which repetition of the sequence it is observing after having acquired the signal, it is necessary to clearly demark the end of the preamble, and the beginning of the actual payload data of a communications packet. This demarcation is achieved by means of the "Start Frame Delimiter" (SFD). The SFD is a unique sequence of bits that allows easy detection. With a well-designed SFD, the receiver correlates output during preamble part of the received packet would not have side lobes as illustrated in the figure, hence a better SFD detection performance.
Since both coherent and noncoherent receivers are envisioned in IEEE 802.15.4a systems, the SFD has to be designed in such a way that good correlation properties are retained with both types of receivers. The IEEE 802.15.4a supports a mandatory short SFD (8 symbols) for the default mode (1Mbps) and an optional long SFD (64 symbols) for the nominal low data rate (106Kbps). Both short and long SFD sequences in the IEEE 802.15.4a standard are provided by MERL.
A further contribution to the standard was the design of the feedback for ranging reliability information. In order to provide precise geolocation, a node has to combine the ranging information from different neighbors. The best location estimate is achieved when the node knows the reliability (probable error) of each ranging estimate; this information is communicated in a quantized form in a feedback packet. But for different applications, different orders of magnitude are most probable for the error -- for example, for line-of-sight short-range communications, errors on the order of one millimeter are likely, while for long-range non-line-of-sight situations, one meter is a typical error. In order to accommodate those different ranges in a quantized feedback packet, we have introduced a scaling factor into the packet.
Both the SFD and the ranging scaling factor are now established part of the 802.15.4a standard. Research at MERL and MELCO is ongoing for exploiting the ranging capabilities of 15.4a in a variety of future MELCO products.
"Watch-List" DVR Software DX-PC55EXP
In the first quarter of 2007, MELCO released to customers of its Digital Video Recorders (e.g. the TL5000U) a new software suite, the DX-PC55EXP, that runs on a PC and allows automatic scanning of the stored video for people on a specified "watch-list."
Details: The DX-PC55EXP is the latest version of software that started with the PC55PRO. The goal of the PC55 series is
to allow more convenient access to video stored on the DVR. This first version was introduced in the February 2006 and included
video search based on face detection and on tracking--another MERL technology (see the MERL 2006 annual report for more information about the the PC55PRO.)
Detection and counting of faces can narrow down the possible video an operator might have to search through, but face recognition can be even more specific about exactly what should be retrieved: someone who "looks like this." In 2004 MERL made a system for video retrieval based on face recognition that we called "Face Browser" (see MERL 2004 annual report.) The idea of Face Browser was that, as faces were detected and recognized in real time, meta-data characterizing that person was recorded along with the video. At a later time, a user could select a person by clicking on a gallery of faces and then automatically jump to points in the video containing that person. Again our MELCO colleagues took over the development that resulted in the PC55EXP release this spring. Although Face Browser demonstrated the possibility, face recognition for identifying people in video had to substantially mature before it was ready to become a MELCO product. Recognition in free video is much more challenging than recognition for access control where a person is cooperating with the system in order to gain entry. In the case of identification in free video, lighting, pose, and expression all vary in complex ways. This complexity promises to keep MERL and MELCO researchers challenged in the near future and is the principle focus of our current face recognition work.
Network Key Management for ZigBee Chip
ZigBee is a wireless communications standard intended for home automation and sensor network applications. ZigBee builds upon the IEEE 802.15.4 standard which defines low-rate (<250kbps) physical and MAC (Medium Access Control) layers. This chip is based on a microprocessor running software written by MERL. MERL provided the network layer (NWK), application support sub-layer (APS), and ZigBee Device Object layer (ZDO). During 2006, MERL completed the ZigBee stack by implementing APL Security. Renesas first shipped a chip implementing the ZigBee communications protocol in 2005. APL security was incorporated into the chip in the first quarter of 2007. Renesas also sells a ZigBee Demo Kit (RZB-ZMD16C-ZDK) for both 900-MHz and 2.4GHz.
Besides completing the last piece of ZigBee stack with the implementation of APL Security, MERL has worked on and continues to work on ZigBee applications with projects like Image Over Zigbee, Voice Over ZigBee and ZigBee Middleware.
ZigBee work at MERL started in 2002 with participation in the ZigBee Alliance standard. MERL started implementing the NWK layer in 2004 and by 2005 had completed the basic stack implementation of NWK, APS and ZDO, which was delivered to Renesas and MELCO. In 2006 MERL completed the ZigBee stack by implementing APL Security. ZigBee APL Security builds upon the security of IEEE 802.15.4, providing an architecture for management of keys and implementation of security policies. The implementation of APL Security for the application layer includes methods for key establishment, key transport, frame protection, and device management.
Key Establishment: ZigBee uses 128-bit symmetric keys. A key can be associated with either the network (NWK and MAC layers) or with a link (application layer). Establishment of link keys is based on a master key which controls link key correspondence between two devices.
Key Transport: Key distribution is an important function in a secure network. A secure ZigBee network will designate one device, called the trust center, that all other devices trust for the distribution of security keys. APL Security includes services for transport of keys between devices and the trust center.
Frame Protection: The implementation of security for ZigBee includes protection of outgoing frames with encryption using the master, network and link keys which have been established and transported between devices. Frame protection is implemented at the APS layer and also includes decrypting of incoming frames.
Device Management: The APL security is responsible for maintaining the security policies of a device and in the network. This includes the addition and removal of devices as well as the propagation of changes in devices or the trust center itself.
The implementation of APL Security completes MERL development of the ZigBee stack producing a two-fold business impact. While Renesas sells MERL's ZigBee stack implementation with their MCU, MERL is also working to help MELCO business units apply ZigBee technology to their customer applications for a technically competitive advantage.
Spoken Interface Testing & Support for US & EU Car Navigation
In the second quarter of 2007, MELCO started shipping production ready voice enabled navigation head units. This is the first production product to come out of MERL's wide ranging support for the development of voice enabled automotive electronics for markets outside of Japan. MERL has provided testing and evaluation of speech recognition and synthesis engines, voice user interface translation and design, and customer support among other services.
The MERL speech team's primary focus has been in developing easier to use and safer voice interfaces for mobile applications. SpokenQuery (SQ) enhances the usability of commercial, automatic speech recognition (ASR) engines. Using processing of ASR output, SQ allows for natural, free-form spoken requests for information. Using SpokenQuery is like using Google to locate something you need, but you say the descriptive words instead of typing them.
Details: This project was started in response to a request in 2001 to support development of voice interfaces for automotive navigation systems outside of Japan. The MERL speech team's experience in developing products for these markets provided key expertise. The project started with the testing of commercial speech recognition and synthesis engines and giving voice user interface design advice.
In September 2004 MERL sent a team member to Europe for several months to help with product specification and support communication with the automobile manufacturer. MERL has provided ongoing advice on the voice interface specification in several languages.
