As we continually capture snapshots of life in pixels, we are amassing an array of software tools and devices to view, edit, store, personalize, and share our digital photographs. A single, easy-to-use system that could reliably provide all of these functions would simplify and enhance the photo production process.

Using an FPGA-based System-On-a-Programmable-Chip (SOPC) solution, a student design engineering team from the Information Science Institute at Beijing Jiaotong University created a portable, multifunctional digital photo album. While traditional digital photo albums are primarily capable of storing and displaying images, this dynamic prototype also allows users to directly edit and personalize digital photos on-the-fly and transmit images to their home or office PCs via the Internet.

The underlying technology for this album consists of a highly integrated system that includes an embedded processor in an FPGA. The SOPC design integrates image processing capabilities, compression, decompression, transmission, and receiving. A top-down SOPC design flow (Figure 1) helped the team substantially shorten the design cycle.

 

 

Flexible system design

Using SOPC Builder, the design team integrated all of the application’s functional modules through an Avalon bus. The integration tool enabled the team to quickly build a stable system that combines software and hardware components. The development platform consisted of an Altera DE2 Development and Education multimedia board (Figure 2) with a Cyclone II FPGA and a Nios II embedded processor.

 

 

The team optimized the design to reduce power consumption while using fewer design resources. Designing with the Nios II embedded processor, the team enhanced performance using custom instructions and peripherals to accelerate system operation.

The design team also incorporated the ability to accommodate future enhancements and functional expansion. For example, with an integrated communication interface, the albums can send personalized digital images to mobile phones.

Powerful processing without a PC

The digital photo album features the following functions:

Digital photo storage

Taking advantage of the Secure Digital (SD) card interface on the development board, the team loaded the data, demand, and address line from the SD card to the Avalon bus, enabling the embedded processor to control the SD card data reading and writing.

Digital photo viewing and playback

Using the MicroC/FS embedded file system, the album can notify the embedded processor of available photo files in the attached storage media, so users can view their images whenever they want.

Special music effects

This capability uses compressed G.729 code streams as the music format and an embedded laboratory decoding algorithm to play selected tunes.

Photo format compression and decompression

The team integrated a JPEG decoder module into the embedded processor, conducted all processing with decoded RGB data, and compressed the photos into JPEG format for network transmission.

Digital photo network transmission

The album transfers the JPEG code streams received through the network to the FPGA for processing. Afterward, the album compresses the processed photo data into JPEG code streams through the FPGA and transmits these streams over the network, allowing users to share images. The design team created a circuit board with a network function that connects with the FPGA through general-purpose I/O pins (see Figure 3).

 

 

User interface

By equipping the development board with a PS/2 interface, the design team was able to load mouse and keyboard functionality onto the Avalon bus and modify the onboard circuit that allowed the embedded processor to respond to the PS/2 peripheral.

Digital photo and audio file management, photo editing and processing, and watermark embedding and extraction

The embedded platform provides the processing resources needed to support software that offers these capabilities.

System hardware design

The album application consists of these modules:

·    SD card file system

·    Keyboard input

·    Audio digital-to-analog converter

·    Network controller

·    LCD controller

·    VGA display

·    JPEG codec

·    Image processing

·    Audio decoding

For system compatibility purposes, the designers used an SD card as the primary storage media for photos and music. For faster read/write performance, the one-line SD card read setting was modified on the development platform to a four-line mode.

The keyboard and mouse modules utilize the same protocols. During each cycle, the data line transmits some data and the clock line transmits a pulse that is read. This enables the devices to transmit data to the host and vice versa.

Even though the prototype uses an LCD display, the design team also created a VGA interface with horizontal and vertical time sequences. SRAM memory serves as the image data memory, displaying images after decompression and processing.

System software design

The MicroC/OS-II embedded operating system runs on the embedded processor, providing the software platform for the digital photo album application. The operating system schedules functional tasks, serving as a stable platform for integrating multiple functions. The design team opted to use MicroC/OS-II for this application because of its efficiency, real-time performance, small size, and scalability.

A JPEG codec module converts the image format. For encoding, the system:

·    Reads a bitmap file and stores graph information in the data architecture

·    Transforms the color model

·    Divides data into 8x8 blocks

·    Performs Discrete Cosine Transform (DCT) inverse transform, inverse quantization, and decode to each 8x8 data block

·    Utilizes read information to generate all of the tables required by decode

·    Stores the data as JPEG files

For decoding, the system:

·    Reads the JPEG file and stores graph information in the data architecture

·    Utilizes the read information to generate all of the tables required by decode

·    Divides the data into 8x8 blocks

·    Performs DCT inverse transform, inverse quantization, and decode to each 8x8 data block

·    Transforms the color model

·    Stores the data as bitmap files

With these processes, the digital album complements the digital camera, allowing users to review and edit photos stored in the camera directly and in real time.

On-the-go editing and sharing

An SOPC-based approach to design provided the team with a great deal of flexibility and enabled system integration to shorten the design cycle. Using a multifunctional digital photo album, photographers have a convenient system for easily and quickly processing their images from virtually any location.

David Auyeung is a senior embedded product marketing engineer with Altera Corporation (San Jose, California), where he is responsible for products including the Nios II processors, embedded software tools, and the Nios II C-to-Hardware Acceleration Compiler. Before joining Altera in April 2006, David held engineering and marketing positions at Mentor Graphics and Cisco Systems. He has a BS in Computer Science and an MBA from San Jose State University.

Altera Corporation
408-544-7000
www.altera.com