What's the future of data management? A needs assessment and blueprint help guide potential next steps.
What is a Data Management System or DMS? A DMS is a system of hardware and software that ingests data from air monitoring sensors and instruments, then storing, visualizing, and distributing that data.
As air quality monitoring is expanding, new instruments and sensors are emerging, and we are seeing an increase in the amount of public and private data. Many organizations are finding that it's both time and resource intensive to manage this deluge of air quality data. All these factors create staffing, funding, and sustainability challenges for organizations seeking to manage air quality data. There is also a significant amount of duplication - many organizations are spending resources to create their own data management system (DMS) from scratch, diverting resources from data analysis, visualization, and, most importantly, actions to improve air quality.
To address these challenges, Clean Air Fund worked with TD Enviro and partners at the University of Massachusetts Amherst and Valiant Solutions to investigate the current global DMS landscape, understand the need for and interest in DMS', and develop a blueprint for a potential path toward an air quality DMS. We share some of the major findings of this surveying and interviewing work below.
The assessment findings were important:
DMS’ are an urgent need in the air quality community - nearly 70% of those contacted said they needed a DMS now or very soon.
A basic DMS will meet most potential users’ needs - features like data ingest, data harmonization, quality control, visual display, and the ability to share data/visualizations. Customizations and advanced features were not perceived as necessary for an initial DMS.
Open-source and low-or no-cost options are preferred. Open-source systems cost less, and can promote community data sharing and collaboration.
A big challenge that air quality DMS users have is the lack of data harmonization. Too many data formats and parameter names, inconsistency in time formats, no universal methods to indicate data quality, and other issues came up in this project. A DMS that could harmonize data from a variety of sources would make data ingest, integration, and sharing easier, letting people spend time on other project outcomes instead of wrangling data
Capacity and training are an issue, with understaffed organizations with limited resources and significant turnover. A new DMS would require a lot of training and capacity building to make it easily accessible from day one of its release.
One of the most important findings was that a community of practice should be built before building a DMS to garner more input and buy-in. In addition, a community of practice will be needed to sustain an open source solution, build capacity and create training, and ensure that a DMS continues to flourish in the future.
A thriving community of practice will ultimately be needed to maintain and continue a DMS system and associated into the future. The data management world has many opportunities and challenges ahead! If you’re interested in learning more about DMS’ or have an air quality issue or topic that you need help exploring, get in contact.
