DataEngUtils for ML Engineers
ML Engineers live between data pipelines and model training: inspecting datasets, validating schemas and moving configuration between frameworks. You might need to check a Parquet training set before a run, confirm an Avro schema for a feature store, or convert a YAML config to JSON for a serving endpoint. Reaching for Python for each of those is friction, and web tools add privacy risk. DataEngUtils handles them in the browser on a Rust engine compiled to WebAssembly, so your training data and schemas never leave your machine.
Tools for ml engineers
- Parquet InspectorInspect Parquet metadata, row groups, column statistics and encodings without uploading the file to
- Parquet Data ProfilerProfile a Parquet file in your browser and get per-column statistics — null percentage, distinct cou
- Parquet QueryQuery a Parquet file in your browser — pick columns, filter rows and limit the result, then export t
- CSV → ParquetTurn CSV exports into columnar Parquet without shipping them to a random web converter. DataEngUtils
- Parquet → CSVConvert Parquet to CSV in your browser without uploading the file to a cloud converter. DataEngUtils
- JSON Schema ValidatorJSON Schema Validator is built for data engineers who need to validate and format pipeline configs,
- Avro Schema ValidatorAvro Schema Validator is built for data engineers who need to inspect, debug, and explain data artif
- Avro File InspectorAvro File Inspector is built for data engineers who need to inspect, debug, and explain data artifac
- YAML ↔ JSONYAML ↔ JSON is built for data engineers who need to convert between file formats and schema represen
- JSON → CREATE TABLEGenerate CREATE TABLE statements from JSON payloads without pasting sample records into a web form.
- UUID GeneratorUUID Generator is built for data engineers who need to generate identifiers, test rows, and mock pay
- Hash GeneratorHash Generator is built for data engineers who need to generate identifiers, test rows, and mock pay
Workflow: Inspect and profile a training dataset
Open a Parquet dataset to check its schema and per-column statistics before training, without writing a notebook cell.
1. Open the dataset
Drop the Parquet file into Parquet Inspector for schema, row groups and null counts.
2. Profile the features
Use Parquet Data Profiler to spot skewed distributions, missing values and outliers per column.
3. Slice for a quick look
Use Parquet Query to filter rows and export a CSV or Parquet sample.
Workflow: Validate pipeline schemas
Check Avro and JSON Schema definitions and inspect Avro records before they reach a feature store or model registry.
1. Validate a schema
Paste your Avro or JSON Schema into the matching validator for structural and type checks.
2. Inspect Avro records
Open an Avro file in Avro File Inspector to view the schema, metadata and a record preview.
3. Fix and re-validate
Edit inline until clean, then copy back to your project.
Workflow: Move configs and generate IDs
Convert YAML and JSON configs between experiment tracking and serving, and generate UUIDs or hashes for run tracking.
1. Convert a config
Use YAML ↔ JSON to move MLflow or Kubeflow configs into the format your serving API expects.
2. Generate run IDs
Use UUID Generator (v1, v4 or v7) for experiment and artifact identifiers.
3. Hash artifacts
Use Hash Generator for SHA-256 checksums to version datasets and check reproducibility.