As embedded devices become faster, cheaper, and capable of integrating more peripheral functions, traditional product categories are merging to create unique, multifunctional, and convergent devices that address consumers’ desires for accessing information, education, and entertainment content. As a result, consumer electronics devices are moving into a whole new age of computing in terms of how digital content including data, voice, audio, and video can be transferred and shared between devices, and how this can be accomplished with unprecedented control, convenience, and security in mind.
The connected home, or digital home, is a new, evolving consumer market segment in the electronics space that is driving new technologies, standards, and innovative products aimed at delivering digital content over home networks. However, this emerging market opportunity poses various challenges for device manufacturers and software vendors alike. Common challenges seen in the deployment of digital home solutions include digital rights management, network bandwidth, content storage, interoperability, dynamically changing environments, and distributed control.
Embedded software technologies such as Universal Plug-and-Play (UPnP)1, Common Internet File System (CIFS)2, Network Attached Storage (NAS), and media server technologies have been developed to help solve the challenges device manufacturers and developers face when delivering integrated digital home solutions. These solutions are destined to become successful and pervasive in the next era of consumer applications.
Market revolution
The consumer electronics industry recognizes opportunities created by the computing and communications revolution and the mass adoption of digital content accessible by consumers around the home. A new market to store, render, and share digital content over home networking through connected devices and the PC is evolving, and is referred to as the digital home. Consumer electronic products manufacturers have recognized the importance of interoperability early on. As a result, numerous organizations have been formed to help create standards and guidelines for products so that they will operate as expected within the heterogeneous environment of the home. The leading organization driving much of the technology is the Digital Living Network Alliance (DLNA)3, 4, www.dlna.org. The main product of the work of the DLNA is a set of guidelines utilizing standards from many other organizations. These standards include UPnP, www.upnp.org, the World Wide Web Consortium (W3C), the Internet Engineering Task Force (IETF), and others.
Digital home primer
The digital home consists of three main logical device types: players, servers, and controllers. These types can be combined or integrated separately with other unique device capabilities into a nearly unlimited number of products for the digital home.
Digital Media Server (DMS): A DMS is a device that stores and records content and makes that content available to devices. A server can be a PC, a NAS disk, or a DVD or CD player. A DMS typically provides capabilities for browsing and searching content as well as maintaining content within a media library.
Digital Media Player (DMP): A DMP is a device such as a TV, stereo, MP3 player, or picture frame that can take a digital content stream and render it in some form for consumption. It is only concerned with setting up content streams, receiving content, and playing that content.
Digital Media Controller (DMC): A DMC is most logically similar to a remote control for a stereo or TV. The controller is able to locate content, select it for playback, direct it to a particular player and control the playback such as pausing, rewinding, and fast-forwarding the viewing.
By using these three devices in various combinations, valued end-user products can be created. For instance, a cable set-top box is essentially a player in that it can take digital content and render it to a display. By adding DMP capability into a set-top box, it can access content from a PC or even the Internet using UPnP technologies, and render that onto the display.
A DVR is typically a set-top box with the additional capability of recording and playing back content from an internal hard drive. By adding DMS capability to a DVR, recorded content would be available to other DMPs on the local network. The DVR could also be used to store digital content other than recorded broadcast content, such as music downloads and DVD content.
The digital home has also created a new type of product often referred to as a NAS media server, a device that reliably stores a massive amount of content that can be accessed either through DMPs or directly from PCs using traditional file sharing capabilities. So, in addition to storing digital media content, users can store their tax returns, e-mail, spreadsheets, and other documents.
In Figure 1, typical products for the digital home are displayed. A DMP similar to the D-Link DSM-520 Wireless HD Media Player is connected to a Sharp LC-32D4U HDTV using a standard HDMI connection. A Blue Peach Blue Stream BPMS-120 Media Server with 500 GB of digital content is located somewhere within the home. The link between the media server and player is a 54 Mbps wireless 802.11g connection. Using this simple configuration, a person sitting in front of the TV can view all pictures uploaded from their digital cameras, listen to any audio uploaded from a CD or downloaded from the Internet, and view any movie stored on the media server.
Figure 1
As more consumer electronic devices become available for the digital home, the variety and capabilities of these products will continue to increase and result in robust digital entertainment environments.
Digital home technology
A significant collection of technology is involved in implementing digital home products. As with any embedded product, there are technologies specific to the platform, but the fundamental technologies for digital home products involve networking, storage, and digitized encodings. Architecturally, these technologies interrelate as shown in Figure 2. A media server will integrate storage technologies; a player will integrate encoding technologies, while a controller will integrate user interface technologies. All three will integrate the UPnP and networking technologies. A digital home product will likely be designed to include one or more of the server, player, or controller personalities.
Figure 2
A significant advantage of this architecture is that nearly an unlimited number of products can be designed utilizing a combination of elemental digital home components along with custom value-added components. This allows significant reuse of digital home technology across digital home products.
The following highlights the architecture of the various digital home components as well as the technologies relevant to each of the components.
DMS technologies
In this architecture, a DMS stores and catalogs digital content, allows access to the catalog for searching and browsing, and serves up the content to DMPs on demand. A media server consists of a number of required and optional subcomponents that deliver the various functions of the server.
File system and storage driver
A media server’s most obvious requirement is to have access to some storage medium and to be able to read and write content to that storage medium. A media server has a number of choices for storage technology and file system. Logical choices for a drive subsystem include IDE, SCSI, or SATA. The tradeoffs are primarily among speed, cost, capacity, and mechanical issues.
Reliability of the storage subsystem is also a design concern. Data backup in the home is typically rare. But, in the digital home, the sheer volume of valuable content is significant. A loss of this content would be very disruptive and would invalidate the architecture in the consumer’s mind. To alleviate this concern, a number of RAID strategies can be deployed. The simplest and most common of these strategies is RAID 1, in which data is mirrored on a separate disk.
The choice of file system is also an issue. A number of file systems available offer differences in performance, reliability, and capabilities. Some of the common file systems are shown in Table 1.
File System
Max Volume
Max File
Journal
FAT32
2 TB
4 GB
N
NTFS
256 TB
16 TB
Y
EXT2
32 TB
16 GB
N
EXT3
32 TB
16 GB
Y
Table 1
FAT32 may be the simplest to implement, but has the lowest reliability and the lowest performance. NTFS is a licensed format of Microsoft. EXT2 and EXT3 may be the most logical choice due to high reliability and high performance. Other file systems are available as well and should be considered.
Content repository
A file system directory structure is a poor mechanism for cataloging file system content. A file system typically does not provide cross-references or meta data storage. Meta data can sometimes be stored within the content, but searching for content this way is cumbersome. A media server, therefore, will typically include a content repository that can be thought of as a database of meta data and cross references of fields that assist with content lookup. Though standards for the implementation of the content repository have not been established, there are standards for how the meta data may be stored in the repository, how content is searched, and how results are returned.
Meta data is accessed within the digital home architecture using a subset of an XML description called Digital Item Declaration Language (DIDL), which is defined in MPEG-215. The subset is referred to as DIDL-Lite. The repository need not store content in a DIDL form, but, if not, it will need to translate between the repository form and DIDL when interacting with other digital home devices.
Content Directory Service (CDS)
CDS is a UPnP service that exports a set of functions that provides access to the repository to other A/V devices on a network. The function of a CDS is to allow browsing and searching of content within the content repository. As a UPnP service, remote nodes interact with it using Simple Object Abstraction Protocol (SOAP)6 calls that allow resources to be added, deleted, and updated from the repository.
Connection Manager service (CM)
A CM is designed to set up and manage one or more content streams between the server and the renderer. How active the media server’s connection manager is and the role it takes in content playback depends on the push or pull model being used by the players and the method they use to retrieve the content.
First-generation media players using the pull method can access most of the content. In this method, content is referenced using URLs, and the player initiates URL access. When this is the case, the connection manager has minimal functions. In the push method, where media servers control the delivery or broadcast of the content, the connection manager service allows a mechanism to identify the various content streams.
A/V Transport Service (TS)
TS is used only when the media server supports pushing of the content out to players. This would be used in conjunction with the CM. Using the TS, a remote node or controller node can control the playback speed, position, quality, mode, and other characteristics. Most media servers today do not support the TS.
NAS server
A media server’s main role is to serve up content to players and other devices on a network, but before a server can serve up content, it must have access to the content. By including a NAS server in a media server, PCs, and other devices on the network have direct access to the file system in the server.
A NAS server can be based on the NFS protocol or CIFS. NFS is a UNIX-based networked file system while CIFS is the network file system used by Microsoft Windows.
NAS client
A NAS client allows the media server to access content directly from file systems located on other devices or PCs within the network. This provides additional capabilities to the server so that it can initiate the import of content automatically. This allows the server to become a media aggregator. An aggregator is a device that will browse available storage devices looking for media content. Upon finding content, a check is made to see if the content already resides in the server. If not, then a reference to the content is automatically added to the repository, and optionally, the content is copied to the server.
DMP technologies
A DMP or renderer receives content from a server and renders it on some output device. The simple model of a player pulls content from a server. That is, once it learns of the content that the user wishes to play, it goes out and accesses that content on the server and begins to read it, and continues to read it until the content is finished or the user has stopped playing. In the other model, a player is informed of some channel to connect to and simply plays whatever content is available on that channel. Depending on the model, the services required of a media player differ.
Rendering control service
Each DMP has a rendering control service or renderer, which can be thought of as the video and audio controls on a stereo or TV. Controls such as brightness, contrast, sharpness, color balance, temperature, volume, loudness, and others can be adjusted through a rendering service.
CM
A media player can have a CM service that performs the same functions as a CM for the media server. With the CM, a control point can query the player for its capabilities and whether it is currently connected to a server. For instance, some TVs have a Picture-In-Picture (PIP) capability. Each window can play a separate video stream. This capability can be mapped to the CM as two separate connections. The control point can determine which channels are available to render content and the capabilities of those channels, and determines what content to play on each of those channels.
Whether a renderer implements a connection manager depends on whether it is following the pull or push model. The normal case is a pull model where the renderer actively accesses content from the server. In this case, the CM is optional.
TS
A player can also have an optional TS. With an A/V TS, a renderer can allow a control point to control the playback of the content. A control point, therefore, could direct the renderer to pause, continue, or affect other controls on the playback stream.
Content decoders (codecs)
The representation of digital content in a stream or a file is pretty much arbitrary. The only way that a player can display content recorded by some other device is if there is an agreed-upon representation prior to the transfer of the content to the player. This representation is referred to as the contents codec. Through generations of digital media uses, there have been quite a few different codecs that have become fairly standard. Some codecs are implemented in hardware DSPs, while others are implemented in software. Some standard codecs are shown in Table 2.
Codec
Type
Supporter
Wmv
Video
Microsoft
Avi
Video
Real
Divx
Video
Divx
Quicktime
Video
Apple
Mpeg4
Video
MPEG
Mpeg2
Video
MPEG
Jpg
Photo
Joint Picture Group
Mp3
Audio
MPEG
Wma
Audio
Microsoft
Aac
Audio
Apple
Table 2
Graphics and multimedia drivers
In addition to being able to decode the digital multimedia content, the player needs to be capable of rendering this content through either an audio or video subsystem. Generally, this rendering is done with some assisting hardware, whether it’s a separate graphics card, an integrated DSP, or another approach. Some of these hardware solutions will combine the steps of digital decoding and content rendering so that software support for the codec may not be necessary.
Digital Rights Management (DRM)
Content providers have long been concerned with unauthorized copying of their works. One of the issues with digital content is that copying can be that much easier. In fact, every time the content is played, digital copies are essentially made while buffering the content for play. When content is transported over a network, between a server and player, or downloaded, the content is vulnerable to being copied. To protect this content, numerous schemes have been instituted. Some, like Microsoft’s DRM, are specific to Microsoft codecs (WMV) and protect the content from essentially the studio to the player, while other schemes protect the content only while being transported from one device to another.
For instance, using Microsoft’s DRM, the content is protected when published. It becomes the job of the player to obtain a license to play the content each time it wishes to play the content. An alternative rights management approach that is less tied to proprietary content formats is Digital Transport Content Protection7 over IP (DTCP-IP). In this scheme, the content is only protected through transport. Devices that support DTCP are licensed and will only allow legal copies to be made. In other words, content is not individually protected; rather, devices are authenticated for rights to handle DTCP content.
DMC
A DMC initiates browsing, searching, and playing of digital content. A controller is essentially the user interface to the digital home. By defining a controller device, the controller functionality can be distributed and replicated throughout the home. Media servers and media players can integrate controller functions into the products themselves, or they can rely on external devices to control playback and manage the repository.
Media browser
A media browser is the portion of a controller that interacts with the CDS within media servers. This component has no particular requirements on how it should be implemented other than it should provide a satisfying experience for interacting with the repository. It most likely is tightly coupled with the user interface component. There is one instance of a media browser per controller.
Renderer controller
A renderer controller is the portion of the controller that interacts with the renderer control service and the renderer A/V TS of the individual renderers. The renderer controller will be able to play, pause, and rewind the content through interacting with the transport service and will be able to control the volume, brightness, and so on through the renderer service.
UPnP
UPnP8 is the foundation architecture for the digital home. The digital home personalities (media server, media controller, media renderer) are all defined as a set of UPnP services. UPnP, in turn, is based on the Web Services architecture of the W3C.
UPnP defines two types of devices: an end point and a control point. While a media server and media player are both end points, a media controller is a control point. A physical device can implement both end points and control points. The behavior of an end point and a control point differ, and you will need to implement your UPnP services accordingly.
UPnP utilizes Hypertext Transport Protocol (HTTP). One way to understand this device difference is to relate the devices to Web servers and Web browsers. They both utilize HTTP, but implement it differently. Aside from exceptions to this rule, control points are essentially Web clients and end points are essentially Web servers. The technologies relevant to UPnP implementations include:
Simple Service Discovery Protocol (SSDP)9 is a protocol utilizing headers of HTTP packets. In UPnP, devices either advertise services or discover services. An end point (media server or renderer) will advertise services while a control point will discover services.
Generic Event Notification Architecture (GENA)10 is similar in architecture to SSDP that follows a subscription model. End points generate event notifications while control points subscribe to the events of an end point.
SOAP is an XML protocol that allows a flexible remote procedure call mechanism over HTTP. In a UPnP stack, it is useful to have some way to abstract SOAP’s service calls, responses, arguments, and return values in a generic way. In that way, an application can make these calls and receive the results in a very natural programming model.
HTTP: A UPnP stack needs to support HTTP as well as the extensions to HTTP for SOAP, GENA, and SSDP. A control point needs to issue HTTP GET packets in order to obtain UPnP device and service definitions. A control point also needs to issue HTTP POST packets to transfer SOAP requests.
Document Object Model (DOM) and streaming11 is a programming model used for XML document creation and parsing. For most UPnP interactions, representing the SOAP requests and responses as DOM objects is most natural.
Networking technologies
The connected home depends on the ubiquitous deployment of wireless and wired home networks. This infrastructure is predominately Ethernet- and IP standards-based. The digital home is designed to take advantage of this infrastructure.
In order to deploy a digital home application within a device, a standard TCP/IP stack can be used. This should include ARP, DHCP, IP, TCP, and UDP. Depending on the serviceability strategy, other protocols may be relevant, such as TFTP and Telnet. If media streaming is being implemented, then the RTP and RTSP protocols may be relevant as well.
There is one addition to standard IP protocols adopted by the DLNA called Auto-IP or ZeroConf. This is a cross between DHCP and ARP that allows a device to determine a dynamic IP address in the absence of a DHCP server. All digital home devices should support Auto-IP.
Clearly a network interface needs to be chosen and supported as well. Generally a 1080i HD video stream encoded as MPEG-27 or WMV requires under 10 Mbps. This implies that a 100 Mbps wired network or a 54 Mbps 802.11g wireless network will support multiple HD video streams.
Digital home’s longevity
The consumer electronics industry is leading the way in the development of new and innovative products that leverage the convergence of device technologies, enabling consumers to receive digital entertainment conveniently, comfortably, and securely throughout their homes. As a result, the digital home is not a concept anymore; instead, it is a reality and will continue to explode in the consumer market for years to come. Since the digital home consists of multiple technologies and standards, embedded software is playing a crucial role in the wide market penetration and adoption of the digital home.
C. C. Hung is the product marketing manager for the consumer electronics market in the embedded systems division of Mentor Graphics. In addition to C. C.’s more than 13 years of experience with embedded systems and software in consumer electronics, telecom, and datacom areas, he has served in a variety of senior positions in product management and applications engineering fields with Freescale, Inc., and holds numerous patents and awards. C. C. has an MS in Electrical and Computer Engineering from University of Texas at Austin, an MS in Mechanical Engineering from Penn State University, and a BS from the National Central University in Taiwan.
[Photo of Richard Schmitt]
Richard Schmitt is the CEO and CTO of Blue Peach, which he founded in 2002. He has been key in creating the architecture behind Blue Peach’s Connected Home product line. Prior to founding Blue Peach, Richard was CTO and cofounder of Viridien Technologies, a services firm for the telecommunications industry, and held various management and technology roles at Stratus computer and Digital Equipment Corporation. A published author and contributor to several standards initiatives, Richard graduated from The Johns Hopkins University with a BA degree.
For more information, contact C. C. or Richard. at: Mentor Graphics
739 N. University Blvd.
Mobile, AL 36608
Tel: 251-208-3400
Fax: 251-343-7074
E-mail: cc_hung@mentor.com
Website: www.mentor.com/embedded
Blue Peach
11 Main Street, Suite 16
Southborough, MA 01772
Tel: 877-258-73224 ext. 704
Fax: 508-486-9001
E-mail: rschmitt@bluepeach.com
Website: www.bluepeach.com
References
[1] Jeronimo, Michael, and Jack Weast. UPnP Design by Example: A Software Developer’s Guide to Universal Plug and Play. Intel Press, 2003. return
[2] Hertel, Christopher R. Implementing CIFS: The Common Internet File System. Prentice Hall, 2004.
return
[3] Home Networked Device Interoperability Guidelines, version 1.0, Digital Living Network Alliance, 2004. return
[4] Home Networked Device Interoperability Guidelines, version 1.0, Optional Media Format Guidelines, version 1.0, Digital Living Network Alliance, 2004. return
