End‑to‑end data solutions for code LLMs and AI agents

01 Non‑Public Code Repositories for LLM Pre‑Training

Production code from real companies — never indexed, never crawled.

Around 3,000 proprietary codebases that have never appeared in public training sets (GitHub, GitLab, HuggingFace). Production‑grade repositories from real companies — primarily sourced from a network of outsourcing agencies and startups whose products were discontinued or acquired.

Snapshot

Distribution: JS/TS 35%, PHP 30%, Obj‑C/Swift 12%, Java 8%, Python 4%, Other 11%.

Composition: 54% discontinued / 46% active or maintained. Full legal rights to license every repository.

02 SWE‑Bench‑Style Benchmarks

→ View sample (Hugging Face)

03 Post‑training data

• SFT dialogues — instruction‑tuning data authored by practicing engineers, not crowd workers.
• RLHF & DPO pairs — pairwise preferences, rejection sampling across custom rubrics.
• Prompt‑to‑Code datasets — generation‑focused training data.
• RAG evaluation — retrieval‑augmented generation evaluation datasets.
• Red teaming — adversarial multi‑step scenario datasets.
• Open‑source dataset expansion — popular open datasets extended with new sources, domains, and refined taxonomy.

04 Agent data

• Agent behavior trajectories — datasets of agent actions and reasoning for accuracy and performance improvements.
• Agent architecture analysis — evaluation datasets for architectures: search, integration, tools, and external systems.
• Dialog evaluation — datasets scoring dialog history across multiple criteria.
• Dialog safety — evaluation suites for politeness, honesty, and integrity.

05 Enterprise Data

Internal enterprise data sourced from real companies — active, acquired, or wound down — each with certified consent to license.

→ View metadata (Google Sheets)