On-Detector Processing and Data Streaming

Abstract

Advancements in X-ray detector technology have enabled new scientific breakthroughs in many disciplines at synchrotron sources and free-electron lasers. Enhancements to detector systems that extend dynamic range through higher frame rate, increase pixel count, and increase overall frame rates also boost data rates. Today, X-ray applications utilize state-of-the-art mega-pixel detectors at 100s Hz to 10s of kHz frame rates, creating raw data rates that can exceed 100 Gbps. Simply writing raw data to local disks and reading it back from the disk for subsequent analysis is becoming less attractive and often impractical, given these large data rates. Modern architectures are beginning to tackle the data deluge at the source with on-detector signal processing, early lossless compression, and direct data streaming to downstream CPU or GPU clusters, where inline analysis can provide scientists with real-time experimental feedback.