Overview
While benchmarking tabular machine learning has driven state-of-the-art results through dedicated architectures, real-world tables frequently contain string entries in addition to numerical values. Tabular learning with strings has remained understudied due to the absence of a dedicated benchmarking suite. To address this gap, researchers introduced STRABLE, a benchmarking corpus comprising 108 real-world tabular learning problems containing strings and numbers from diverse application domains.
Research Context
The presence of string entries alongside numerical data raises several foundational questions for tabular learning:
- Whether dedicated learners that model strings and numbers end-to-end are necessary.
- Whether encoding strings as numbers, such as through categorical encoding, provides sufficient performance.
- Whether the resulting processed tables behave like standard numerical tables and require the same learning algorithms.
Approach
To investigate these questions, the researchers carried out an empirical study evaluating 445 pipelines across STRABLE's 108 datasets. The evaluation covered two broad architectural paradigms:
- End-to-end architectures: Models that directly and jointly model string and numerical entries.
- Modular pipelines: Workflows where strings are initially transformed using an encoder, processed through an intermediate post-processing stage, and finally supplied to a tabular learner.
Findings
The evaluation revealed several patterns governing the interaction between string encodings, table types, and tabular learners:
- Categorical-dominant tables: Because most real-world tables are categorical-dominant, combining advanced tabular learners with simple string embeddings achieves strong predictive performance at low computational cost.
- Free-text-dominant tables: On tables dominated by free text, large language model (LLM) encoders become competitive.
- Post-processing sensitivity: The performance of LLM encoders appears sensitive to subsequent post-processing methods, with notable differences observed across distinct LLM families.
- Ranking reliability: STRABLE establishes generalizable pipeline rankings that closely approximate oracle rankings, indicating that the corpus provides a representative testbed for tabular learning with string inputs.
Why This Matters
Real-world datasets regularly combine numerical records with free text and categorical strings, yet machine learning research has lacked a solid benchmarking foundation to evaluate models under these conditions. STRABLE provides a dedicated resource to evaluate end-to-end systems against modular configurations across diverse tabular tasks.