Information Technology/Large-Scale Data Handling

Abstract

The experience of light source users has been transformed in recent years by large increases in flux and brightness, revolutionary new optics and detectors, and automation and advanced sample environments. Beamlines are producing data at rates and volumes that challenge the capabilities of even the most experienced user groups. Meanwhile, the community of synchrotron users continues to grow in size and diversity: researchers come from physics, material science, energy and battery research, geology, biology, chemistry, art history, and more. Almost every natural science domain is being advanced through the techniques employed at these facilities, but a significant fraction of these researchers are first-time or infrequent users of a particular beamline. The combination of an expanding base of new users and increased beamline capabilities is leading to an increase in the amount of “dark data” that is not analyzed fully (or, in some cases, at all).