Dataset Value Taxonomy at AIES 2025: From Big Data to Valued Data

In October 2025, I presented our paper From Big Data to Valued Data: A Dataset Value Taxonomy for AI-Native Empirical Research at AAAI/ACM AIES 2025 in Madrid, Spain. This paper and project were led by my PhD student Scott Seidenberger. In the paper, we argue that as AI lowers the cost of downstream analysis, the central bottleneck in empirical research moves upstream to datasets.

That shift is where we start this paper. AI systems now automate more of the analytical pipeline, including coding, statistical workflows, visualization, and drafting. In empirical research, this means competitive advantage depends less on who can run analysis faster and more on who can build, steward, and govern high-value datasets with rigor.

Datasets as the New Frontier in Empirical Research

As AI makes downstream tasks like analysis, synthesis, and publication easier, we see the limiting factor in influential empirical research shifting to dataset creation and stewardship. The strongest contributions now come from high-value datasets that are temporally rich, broadly representative, and responsibly accessible. In that setting, the decisions we make about what to collect, how to curate it, and who can reuse it directly determine scientific influence, reproducibility, and long-term impact.

Example: Very recently, funding programs have begun placing stronger emphasis on dataset-centered empirical research. NSF’s CAMEL K-12 solicitation supports coordinated work to generate and share high-value, AI-ready datasets for K-12 mathematics learning. This direction is consistent with DVT’s perspective and reflects a wider shift in empirical research priorities.

Core Constructs of DVT

In this work, we present DVT as a discipline-agnostic framework for evaluating dataset value in empirical research through three orthogonal constructs.

• Temporal Investment captures the irreducible time and labor embodied in creating datasets, especially for longitudinal and event-driven empirical research.
• Scale and Breadth captures not only the record count of datasets, but also how well datasets represent units, modalities, and sampling frames in empirical research.
• Accessibility captures the legal, technical, financial, and logistical barriers that determine who can inspect, reproduce, and extend datasets in empirical research.

What DVT Adds Beyond Big Data Descriptions

Earlier Big Data taxonomies were useful for describing properties such as volume, velocity, and variety. In DVT, we address a different question in empirical research. We focus on evaluative value rather than descriptive scale. We treat temporal effort, representational breadth, and access barriers as measurable dimensions that can inform real decisions.

In this paper, we make four concrete contributions. We synthesize critical Big Data and data governance scholarship into a compact evaluative framework. We operationalize that framework with a transparent scoring protocol. We integrate an ethical lens that makes tradeoffs visible. We also show portability across empirical research domains, including astronomy, public health, climate science, finance, and digital humanities.

How Scoring and Calibration Work

We decompose each DVT construct into observable dimensions with ordinal class labels, then combine them through a calibrated scoring scheme. This structure supports cross-field comparison while still allowing each empirical research community to set thresholds that match its own evidence standards and governance norms.

In practical terms, DVT helps us distinguish between datasets that are large but shallow and datasets that are slower, harder, and more valuable to build. We believe that distinction is central for empirical research planning, review, and funding in an AI-native environment.

How Stakeholders Can Apply DVT

We designed DVT to support decision-making across empirical research roles, whether stakeholders apply it explicitly or implicitly.

• Empirical researchers can use it to prioritize dataset design and curation choices with stronger long-term epistemic value.
• Reviewers and evaluators can use it to compare dataset contributions with criteria that go beyond size.
• Funders and institutions can use it to justify investments in datasets that are more likely to produce durable empirical research impact.

Next Steps

We are extending this work by refining calibration strategies across domains and by building stronger empirical benchmarks for DVT-based evaluation. The broader goal is to make high-impact dataset work easier to recognize, justify, and sustain across AI-native empirical research programs.

Links

• Paper
• Slides