How Lyrebird Generates Synthetic Data — Answering the Questions CTOs Ask

When a test environment needs realistic data, most teams pick one of two paths: copy production and mask it, or hand-write a few hundred rows of sample data. The first moves sensitive data into lower environments and creates fresh risk on every refresh. The second is safe, but it misses real-world distributions, relationships and edge cases — tests pass, and the bug shows up in production.

Lyrebird offers a third path: data that behaves like production, without copying production rows. This article explains how it works without getting lost in implementation detail, then answers the questions technical leaders ask during evaluation.

In Short: Connect Once, Extract the Recipe, Generate Offline

Lyrebird’s workflow has three steps.

1. Connect once and profile

Lyrebird connects to the source database read-only and learns its shape:

• Schema: tables, columns, types, keys, constraints, indexes.
• Relationships: which table points to which, how many orders a typical customer has, and why some customers have hundreds.
• Distributions: where amounts cluster, which years dates span, how often each status value appears.
• Cross-column rules: delivery dates follow order dates, two fields are always empty together, a code’s prefix derives from another column.
• Formats: the structure of phone numbers, document number patterns, text lengths.

All of this is written to a compact file we call a profile. A profile is not a copy of the data — it is a statistical recipe for it.

2. Disconnect

Once the profile exists, Lyrebird never needs the source database again. Connection credentials are held in memory only and never written to disk. Everything that follows runs from the profile file.

3. Generate as much as you want, wherever you want

Load the profile and choose a target size: a developer database at 1% of production, or a load-test dataset 100× larger. From the desktop app or a CI pipeline, fully offline, Lyrebird produces SQL files, native bulk-loader files, or a database loaded straight into Docker.

# Once, read-only
lyrebird profile --output app.lyre

# Check what the profile contains before sharing it
lyrebird audit-profile --input app.lyre

# Anytime, offline
lyrebird generate --profile app.lyre --scale 0.1 --out ./testdb --docker

Where Does the AI Come In?

One misconception comes up constantly, so let’s clear it first: Lyrebird does not ask a language model to write rows.

AI is used only during profiling, as a local model running on your own machine. Its job is to answer two questions about every column:

1. What does this column mean? An email, an IBAN, a city, a currency, an order status, free text?
2. Is it personal data?

That matters because column names lie. A field called ref can hold email addresses; a field called code can be a national ID. The model looks beyond the name — at the type, neighboring columns and the overall shape of the values. Values of columns judged personal never enter the profile; at generation time they are replaced with correctly formatted, fully synthetic values.

There is no model at generation time. Generation is a deterministic engine that executes the recipe in the profile. Two practical consequences:

• No hallucinations. The engine cannot “make up” a foreign key or invent a column that is not in the schema.
• Reproducible. The same profile, settings and seed produce exactly the same data every time.

Curious why the model does not run in the cloud? See Where Should the Model Sit?

The Questions CTOs Ask

“Does production data leave our network?”

No. Both profiling and generation run on your own machines, CI runners or private servers. No cloud service or external API is involved. License validation is offline too, so Lyrebird works in air-gapped environments.

“Could the profile file itself leak data?”

This is the right question, because the profile is a file that will travel inside your organization. We answer it in three layers:

• Default behavior: Values of columns flagged as personal never enter the profile. Non-personal, repeating vocabularies (order statuses, currencies, region lists) may be kept for realism.
• Audit: audit-profile lists what the profile keeps as written text and scans it for leftover personal data and credentials. You can make this part of a security review before a profile leaves your team.
• Strict mode: If your policy demands more, a profile can be extracted so that it keeps no text from any source row. Only the schema, numeric and temporal distributions, format patterns and public catalogs such as ISO countries and currencies remain. An independent check verifies this promise before the profile is written; a single violation and the file is not written.

Strict mode has a cost, and we state it plainly: realism drops somewhat in fields that depend on business vocabulary (custom category names, for example). Your organization’s risk appetite decides which trade-off is right.

Profiles are also encrypted with your organization’s key and open only on your organization’s activated computers.

“Is the data realistic enough to catch bugs?”

For tests to mean something, data must preserve two things: shape and relationships.

• Shape: Numeric and date distributions, frequent values, null rates, text lengths and formats are preserved. Phone numbers follow real numbering plans; ID and card numbers satisfy their validity rules — so your application’s validations do not reject synthetic data.
• Relationships: Skew in orders per customer, cross-column rules and table-to-table ratios are kept. Real-world extremes like “a few customers have thousands of orders” do not disappear — and that is exactly where performance problems tend to hide.

Lyrebird’s goal is not “sample data” — it is test-grade data: a database that passes your application’s validations, exercises your queries with production-like volumes and distributions, and makes your reports produce meaningful results. Generated records are not copies of source records; that is not a gap, it is the design. What you are testing is not individual customers but how your system behaves against the shape and relationships of that data.

Dataset size follows the job: scale the whole schema by a factor, set a total row target, or grow a single table on its own for a load test. Table-to-table ratios and relationship density are preserved. That way development, integration and regression tests, demo environments, load tests, and external teams with whom data can never be shared all draw from the same profile.

“How do we test a specific business scenario?”

Lyrebird data preserves statuses, date ordering and cross-table consistency, so most scenarios — “cancelled orders”, “overdue invoices” — already exist in the generated data, and your tests find them by querying.

For records a test must know exactly — verifying a computed total, a specific state transition, or an edge case that is very rare in the source — you provide the scenario as a SQL file. It runs after the Lyrebird output is loaded and adds the few records your tests expect on top of the realistic data:

-- scenarios/overdue-invoice.sql
INSERT INTO invoices (customer_id, status, due_date, total)
SELECT id, 'OVERDUE', CURRENT_DATE - 45, 1250.00
FROM customers
WHERE segment = 'CORPORATE'
LIMIT 1;

Letting the database assign IDs and attaching to existing records merges the scenario with Lyrebird’s data without collisions. Keep scenario files separate from your schema migrations; they should run only in test environments.

Lyrebird supplies volume and realism; the scenario file guarantees the records your tests assert on.

“Will we hit foreign-key errors on load?”

No. Lyrebird builds the schema’s dependency graph, generates parent tables first and binds every child row to a valid parent. Composite keys and constraints hold across the whole schema; no orphan rows are left behind. When loading into Docker, row counts are verified as well.

“Can we use it in our CI pipeline?”

Yes. The Enterprise CLI runs headless on Linux, macOS and Windows. Because the same profile and seed produce the same data, “why did yesterday’s passing test fail today?” is never a data change. Row counts and the seed can be set per scenario from the command line.

“Which databases are supported?”

PostgreSQL, MySQL, SQL Server and Oracle. You can also profile one engine and generate for another — for example, a PostgreSQL test database from the profile of an Oracle system. That is especially useful for teams testing engine migrations.

“What if the AI gets it wrong?”

It can; no classifier is perfect. That is why Lyrebird positions the model as an assistant, not the final authority. Users with profile-preparation rights can review and correct column decisions before publishing a profile, and the audit step supports that human review. Users with generation-only rights generate exclusively from the organization’s published profiles.

“What infrastructure does it need? Is a GPU required?”

No. Lyrebird runs on a developer laptop, a CI runner or a private server. It uses an NVIDIA (CUDA) or Apple Silicon (Metal) accelerator when available and falls back to CPU automatically. The model is loaded only during profiling and released when the job ends; generation needs no model at all.

“What happens when the schema changes?”

You refresh the profile. Profiling is a single command, and its output is a versionable file, so you can keep profiles in step with your schema migrations. Generating new environments never requires touching the live database — only refreshing the recipe when the schema evolves.

“What does this mean for GDPR / CCPA?”

Legal assessment belongs to your data protection officer, but the architecture gives clear answers that make that assessment easier: no production rows are copied into lower environments, no data leaves the network, and what a profile contains is auditable with a report to show for it. Taking test environments out of personal-data processing scope significantly reduces compliance load for most organizations.

“How do we evaluate before buying?”

The Community edition is free. You can profile your own database, inspect the profile and generate up to 5,000 rows per table from the desktop app — seeing realism and relational consistency on your own schema, on your own machine. Enterprise removes that limit and adds CLI automation.

Summary

Lyrebird rests on one idea: a test environment does not need production data — it needs production’s recipe.

• Connect to the source database once, read-only.
• A local model determines what each column means and whether it is personal; personal values never enter the profile.
• The profile is auditable and can be tightened to hold no row text at all.
• Generation is deterministic, offline and reproducible; relationships and distributions are preserved.

To try it on your own schema:

Explore Lyrebird · Request a demo · Blog

• Synthetic data
• Test data
• Privacy
• Security
• Lyrebird