Supported Data Formats & Schemas

Overview: Doushi is specialized for tabular datasets. You can upload CSV, Excel (.xlsx), and Apache Parquet files directly through the web UI, Python CLI, or REST API.

See Pricing for upload limits.

Supported File Formats

Format	Extensions	Recommended For
Comma-Separated Values	.csv, .tsv	General purpose, quick exports
Apache Parquet	.parquet	Large datasets, compressed columnar data
Microsoft Excel	.xlsx, .xls	Business spreadsheets, finance models
JSON Lines	.jsonl	Event streams, structured log exports

---

Supported Column Types

Doushi automatically inspects and infers data types for every column:

1. Numerical Features

• int64, float64, currencies, percentages, and scientific notation.
• Automatically scaled and normalized using robust standardizers resistant to outliers.

2. Categorical Features

• Low and high cardinality string columns (e.g. country, device_type, department).
• Automatically encoded using one-hot encoding (low cardinality) or target/frequency encoding (high cardinality).

3. Datetime Features

• ISO-8601 strings, UNIX epoch timestamps, or formatted dates (e.g. 2026-10-21).
• Doushi extracts temporal signals: day of week, hour of day, month, days elapsed, and is-weekend flags.

4. Text & Freeform Comments

• Short text columns (e.g., feedback notes, issue titles).
• Transformed using TF-IDF n-gram vectorization and sentiment polarity features.

5. Binary & Categorical Targets

• 0/1, true/false, "yes"/"no", or multi-class labels ("low", "medium", "high").

---

Columns Automatically Dropped

To protect against overfitting and data leakage, Doushi's schema analyzer automatically identifies and drops:

• Unique Row Identifiers: Columns where every row has a unique string or integer (e.g. user_id, uuid, row_number).
• Zero Variance Columns: Columns where all rows share the identical value.
• Pure URL Strings: Unparsed full web links (unless domain extraction is triggered).

---

Best Practices for Tabular Data Preparation

1. Include 1,000+ Rows for Reliable Generalization: While Doushi can fit datasets with as few as 100 rows, models trained on 1,000 to 100,000+ rows achieve superior out-of-sample accuracy.
2. Do Not Manually One-Hot Encode: Upload raw categorical strings (e.g. "Germany", "France", "Japan"). Doushi selects the best encoding strategy per model architecture during the tournament.
3. Keep Target Values Grounded: Ensure the target variable reflects actual outcomes rather than future information not available at inference time.