The high-speed data recorder was designed to address the high-speed, high-channel-count data acquisition needs of the Defense Threat Reduction Agency (DTRA). In the interest of national security, the United States Department of Defense pooled its resources, expertise, and capabilities to create DTRA. The role of DTRA is to ensure that the United States remains capable of deterring, reducing, and countering the present and future threats from weapons of mass destruction. One of the many roles of DTRA is to evaluate the lethality of conventional, biological, chemical, and other advanced weapons against a broad spectrum of target types in real-world scenarios.
Implementation requirements
DTRA issued a request for proposal for developing a high-speed data recorder for recording data from explosive events. The requirements stated that the system be a Windows-based, rack-mountable system that could acquire simultaneously triggered and sampled 16-bit data at one million samples per channel, per second, from 64 channels for at least 30 minutes. All sampled channels were to be over-sampled at up to eight times with an input signal impedance of 10kOhm.
In addition, DTRA required that the system accept Global Positioning System (GPS) and/or IRIG B input signals so as to provide highly accurate data time stamping. It was also essential that the system be fully programmable to incorporate any future modifications that could accommodate additional data acquisition cards for expanded channel counts and/or software functionality expansion. With a variety of proposals submitted for review, DTRA chose B & B Technologies, Inc. (BBT) for having the most technically acceptable solution at the lowest cost.
The solution was to use National Instruments’ LabVIEW version 6.1 programming language to talk to and control a variety of hardware from a number of different manufacturers. BBT assembled a system consisting of two high-speed PCI analog data acquisition cards from Interactive Circuits and Systems, Ltd. (ICS) coupled together with a PCI-based, high-speed digital data management and recording device manufactured by Conduant Corporation, aptly named StreamStor. In addition to the aforementioned cards, a KSI Corporation PCI GPS card was implemented to accomplish the time stamping of the data.
All of the aforementioned hardware was enclosed in a product called the Tape Killer, TK200, also manufactured by Conduant. When the TK200 is used in conjunction with the StreamStor product, the result is a versatile digital alternative to using analog tape recorders. The TK200 consists of a standard high-performance PC motherboard with two removable drive banks each capable of housing eight IDE hard drives.
The problem
Before inception of this system, DTRA had been using high-speed analog tape to record test data, which was cumbersome to operate for storage, retrieval, and archiving, and which could not provide the same reliability, speed, or archiving ability as provided by today’s digital technology.
With a data acquisition rate of 1 MHz per channel on 64 16-bit channels for a minimum of 30 minutes, the biggest challenge was collecting and storing all of the data to disk. With a data throughput of 128 Mbytes/sec (64 channels*1,000,000 samples/sec*2 bytes/sample) from the data acquisition cards, it was clear that standard PC architecture would not be a feasible solution. In addition, the storage of data at that rate for a minimum of 30 minutes would require at least 230 Gbytes of data storage.
With a standard test day involving several tests, calibration routines, and other validation procedures, it was essential that all stored data be easily retrievable with respect to the test start time. For this reason, GPS time stamping of events was mandatory to provide an easy method of retrieving data from any point in the entire run of test data.
The solution
DTRA’s requirement of recording phase-synchronized, simultaneously triggered, high-frequency vibration data on 64 channels with GPS time stamping of events required that BBT seek out and implement the latest technology in data acquisition and storage devices.
Hardware
The time stamping of events was accomplished using the TPRO-PCI GPS card from KSI, which allowed a transistor-transistor logic (TTL) level input signal to go into its Time Tag Input function, thereby buffering a GPS time stamp for every TTL pulse received.
Providing an accurate GPS time stamp was essential for rapid retrieval of various events during the data acquisition and recording process as there were to be various stages of the test, such as a calibration routine or one of the explosive events. At the start of acquisition, a TTL pulse is issued to the GPS card to record the start time and then additional pulses are issued for each subsequent event in the recording process. Due to the extremely short duration of the events, the lengthy recording time, and the precision required, it was essential that the operator be able to pinpoint each event in the data stream.
On the acquisition side, the system required 64 16-bit channels that could be simultaneously triggered and sampled at rates of at least 1 MHz. BBT found that ICS manufactured an A/D converter card called the ICS-645, which provided 32 channels of simultaneously sampled 16-bit data driven by A/D converter clocks capable of running at speeds of up to 20 MHz. The ICS cards contained a local data port, the P4 port, which could pass timing and triggering signals to other ICS-645 cards over a 26-pin ribbon cable.
The P4 port allows the user to connect multiple ICS-645 cards together in a master, mid-slave, and end-slave configuration where the master board shares its timing and triggering signals with each of the other cards in the system. This configuration creates a seamless array of data acquisition channels all running off the same A/D converter clock and trigger signal, thereby meeting the requirement that all channels be sampled and triggered simultaneously.
Being able to acquire the data at such a high rate was the easiest hurdle to overcome, but getting the data off the boards and stored to disk was a much more challenging task. The ICS card and supporting drivers provided the user with two options for data retrieval. The first option available was to pull the data off the PCI bus, into system memory, and then to disk. However, with the quantity of data required, this was not possible even with the fastest of computers. The ICS-645 card also provided an additional 80-pin port called the Front Panel Data Port (FPDP), which provided the ability to transfer 32-bit (4-byte) words at rates of up to 40 MHz resulting in a maximum data throughput of 160 Mbytes/sec.
With each sample from each channel requiring 2 bytes of memory, the user can specify that the ICS boards pack the data where two samples from adjacent channels are sent in each 32-bit word transferred on the FPDP bus. When configuring the ICS-645 cards, the user can also specify the cards in a master, mid-slave, and end-slave configuration as with the P4 local port. This feature permits the master ICS-645 to control the transfer of the data from both of the data acquisition cards, thereby meeting the throughput requirement for getting the data off the card.
With the availability of the FPDP bus to transfer data at the required rate, the most important task was to find a means to read and store this data from the FPDP bus. This led BBT to Conduant’s StreamStor PCI-816XF digital recorder, which can read data off the PCI bus or directly from its FPDP data port at sustained rates of up to 200 Mbytes/sec. The StreamStor is a digital data recorder that records to disk any and all data it receives, in the order in which it receives it, to an array of up to 16 standard IDE disk drives.
The StreamStor recorder is located at the end of the 80-pin FPDP cable and with the ICS boards managing the data packaging, the recorder receives and stores this data to its array of disk drives. The StreamStor device, specifically designed for real-time data recording and playback, acts as an extremely efficient Hard Disk Drive (HDD) controller to stripe the data to as many drives as are hooked up to the card. In this application, 16 120-Gbyte IDE drives were connected to the data recorder card providing just under 2 Tbytes of storage or at the prescribed data throughput rate of 128 Mbytes/sec – just over four hours of consecutive storage. Of note is that the test duration can be extended to almost seven hours with the advent of 200-Gbyte disk drives.
With all core components selected, the Conduant TK200 chassis was chosen to house the hardware, disk drives, and other components. The TK200 consists of a standard Windows-based, high-performance PC complete with CD-RW and floppy drives, USB ports, six open PCI slots for data acquisition, data storage, GPS, or any other required devices. The TK200 also provides two bays to house all of the data storage drives in a convenient rack-mountable chassis. Each drive bank contains eight drives that can be easily removed from the system for data archiving and seamlessly reinserted into the system to access a previous run of data.
In addition to the aforementioned components, BBT assembled several BNC patch panels to provide easy connection and access of the input signals as well as an integrated LCD display, keyboard, and mouse to create the complete system as shown in Figure 1.
Software
With all hardware components selected, National Instruments’ LabVIEW software was chosen to communicate and control the entire test process. DTRA did not specify any significant requirements for the user interface or system functionality other than to provide the ability to specify several key test parameters and allow for the retrieval of data. It is significant to note that the capabilities and, subsequently, the complexity of this system were greatly reduced by assuming many test parameters as constants to meet the very specific test needs specified by DTRA.
Some of the more obvious parameters that BBT intentionally excluded from the LabVIEW software, and that the user was not given control over, include the number of channels to acquire, internal or external triggering, and scaling in addition to many others. In subsequent software written for similar systems, BBT has incorporated the ability to manage many of these features in a customized user interface. An example of added functionality is applications with user-specified scan rates approaching the ICS-645 maximum of 20 MHz and a reduced channel count while permitting hardware (external) or software (internal) triggering.
Upon launching the software, a great deal of card initialization and setup occurs that is transparent to the user, and then the software main panel appears (see Figure 2) providing the user with the means to set up and initiate a test.
From the main panel, the user is given the ability to initialize the KSI GPS card by launching a new screen that executes a procedure to ensure that the TTL input to the GPS card is functioning properly and that the queue of triggered events is empty. After initializing the GPS card, the user proceeds to the System Setup, which allows the user to specify a scan rate, over-sampling, or file path location for the event time stamps and several other key test parameters. Once performed, the software returns to the main panel for the user to then arm the system and then await an external trigger.
Once triggered, the system begins acquiring data at the prescribed rate, and the data is written to the StreamStor device. It is significant to note that even with data streaming to disk at such high rates over the FPDP bus, data can also be viewed in real time without interrupting the data stream to disk. This is done by setting up the system to stream the data over the FPDP bus to the StreamStor device while also setting the system to read data from the PCI bus. In essence, the ICS-645 can be set up to send the same data at the same moment to both the FPDP bus and the PCI bus.
One of the outstanding features of the TK200 system is that there is no burden placed on the PC even with the massive amount of data, 128 Mbytes/sec, being transferred to disk during a recording. Once the acquisition boards are triggered and data is flowing over the FPDP bus, no further operating system or software intervention is required to maintain the data flow until the user wishes to stop acquiring data. This permits the operating system to perform other tasks or for software to be written to analyze and display real-time data pulled off the PCI bus without interrupting the data flow to disk.
While data is streaming to disk, the user is given an indication on the main panel, in Mbytes or test time elapsed, how much data has been collected since the trigger was received. During the test, the user can issue further TTL pulses to the GPS card to mark important instances during the recording process. With the desired amount of data collected, the user can then stop the test from the main panel by pressing the Stop Acquisition button, which returns all cards to their default state and prepares them to run another test.
With the data collection completed, the user is able to retrieve data from the recorder and store it to the system disk(s). Upon pressing the Fetch Data button, the software launches the Fetch Utility shown in Figure 3, which gives the user the ability to retrieve any amount of data from any location in the record and store it to as many files on the system disk as desired. The data is fetched from the StreamStor device in its raw form and is simply written byte by byte to the specified file.
The Fetch Utility provided to DTRA provides no ability to extract data from individual channels, but BBT has recently written software in LabVIEW using the Application Programming Interface (API) tools provided by Conduant to extract data with greater ease. For example, knowing the event times, a user interface can be created to extract certain segments from specified channels, from a particular event, with only a few mouse clicks. By providing this functionality, the user can avoid having to calculate the specific byte (using the scan rate and elapsed test time) corresponding to the event in question.
After fetching the desired data to the system drive for post-processing, the user can then run a new test or view the retrieved data on a simple graph by pressing the View Data button from the main panel. Once pressed, the Data Viewer (Figure 4) is launched to allow the user to view the data from any of the retrieved data files. At the customer’s request, the Data Viewer has no data analysis tools but merely displays the data contained in the specified file to verify that it was accurately recorded.
As previously mentioned, the data is written in its raw form to disk and as a result, when reading data into any software package such as LabVIEW, the programmer must write an extensive algorithm to convert this raw data. Since the data is written as unsigned 32-bit words but contain two channels worth of signed data, the programmer must extract each channel and scale the data to the desired voltage or signal levels.
Successful implementation
After a great deal of testing and debugging, two identical high-speed data recorders were successfully delivered to DTRA. Each system is capable of scanning 64 16-bit channels at 1 MHz for more than four hours. With the implementation of these systems, DTRA has replaced its old analog tape systems and now has an easy-to-use, robust solution that allows them to confidently acquire and digitally store accurate, synchronized test data. Furthermore, DTRA is now able to maintain and archive digital test results that are immediately available at any time for further analysis or review.
. . . . .
Dean Cyphery has been a system integrator at B & B Technologies, Inc. for two years and has more than three years of experience doing LabVIEW programming. Dean has a BSME from the University of Waterloo in Ontario, Canada.
B & B Technologies, Inc. (BBT) uses LabVIEW whenever possible to design systems for a wide variety of industries and applications. LabVIEW’s creator and manufacturer, National Instruments, is the authority on the use of this language and has recognized B & B Technologies, Inc. as one of only 10 companies to be designated as “Select integrators” in the United States. These "Select Integrators" are recognized as the most experienced, reliable, and proficient LabVIEW programming firms in the world.
Although the system and all devices discussed in this article met the customer’s needs, the system has a great deal more potential. BBT can and has successfully implemented similar systems utilizing and expanding upon the immense processing and acquisition potential of this system. Additional LabVIEW software can be written to customize this system to nearly any application while providing more functionality in an easy-to-use software package. For any questions about this system or for any other system integration needs, contact B & B Technologies at:
B&B Technologies, Inc. (BBT)
6610 Gulton Court NE
Albuquerque, New Mexico 87109
Tel: 505-345-9499
Fax: 505-345-9699
Website: www.bbtechno.com
For further information, contact Dean at dcyphery@bbtechno.com.
