Center Established to Tame Digital Data Deluge

Researchers at UNC have launched a center that will bring together scholars and technical experts to solve the problems of managing and sharing today’s deluge of digital data.

In addition to building shared collections, the new Data Intensive Cyber Environments Center supports software systems for data curation and data preservation so that today’s knowledge will be available for future generations.

The center, funded by external research grants, draws on data management technology whose generic nature makes possible an array of uses, from helping the National Archives preserve the nation’s digital information – including records that document our nation’s experience – to helping digital libraries cope with the size and complexity of digital knowledge.

Other uses include enabling the sharing of digital data by large-scale interdisciplinary scientific research collaborations across the nation and the globe.

UNC’s School of Information and Library Science, libraries, Information and Technology Services, the Renaissance Computing Institute and numerous research projects will use the center’s technologies to organize and curate collections, create digital repositories and federate data resources.

“As digital information expands at a staggering rate, there is a pressing need for this center,” said Tony Waldrop ’74, vice chancellor for research and economic development and chair of the center’s oversight board. “We are very fortunate that a world-renowned team of researchers with expertise in data management, curation and preservation will provide leadership in this field.”

The Data Intensive Cyber Environments, or DICE, group joined Carolina’s top-ranked School of Information and Library Science in fall 2008. The award-winning research group, formerly of the San Diego Supercomputer Center at the University of California, San Diego, has an international reputation in developing digital data technologies.

For more than 10 years, the DICE group’s data grid technologies have been used in research projects worldwide to manage large, distributed data collections and to support discovery, access, retrieval, replication, archiving and analysis tasks. The researchers most recently released iRODS, the open source Integrated Rule-Oriented Data System, which introduced user-settable rules that automate complex management policies, helping users handle today’s mushrooming collections of digital data.

iRODS harnesses the full power of cyber-infrastructure and virtual technologies to integrate functions that free digital data collections from the constraints of space – whether physical, administrative or disciplinary – and time through long-term preservation. The iRODS software assembles distributed data into shared collections. The rule-based architecture enables users to develop and enforce community policies, without changing core iRODS code, so that they can easily and flexibly manage data collections and verify policy enforcement to ensure appropriate access and authentic data.

“The indispensable role of digital data across society, and the increasing size and complexity of data collections, are reaching a critical point,” said Richard Marciano, founder and executive director of the center. “With the growing need for practical digital data technologies, the new DICE Center is already collaborating with many important projects across the UNC system as well as national and international partners, helping them harness their digital data collections and working with them to efficiently create, share and preserve new knowledge.”

Initially, the campuswide DICE Center includes three core research units that form the foundation for data-intensive and data-lifecycle activities. Marciano will direct the sustainable archives and libraries technologies unit, an interdisciplinary unit focused on developing information technology strategies and conducting research in the area of digital materials and records collection and preservation.