Mock Magic vs asking an AI for test data

Typing “give me 20 test customers as JSON” into ChatGPT or Claude works, and that makes a chat assistant a real competitor to any data generator, Mock Magic included. The useful question is which jobs it does better, and which jobs it makes harder than they need to be.

What assistants do well

• Domain-specific content. Ask for 30 alchemy ingredients with rarity and a price in silver, a week of maintenance tickets for a wind turbine or chat messages between guild members, and you get plausible content that no general-purpose generator has in its word lists.
• Your schema, as written. Paste a C# class or a JSON schema and the answer follows its fields, including relations such as orders that point at existing customer IDs.
• Languages and cultures. Names, addresses and text in Japanese, Arabic or Polish, right-to-left text for layout tests, and number and date formats that match a country.
• Edge cases on request. An assistant can brainstorm what might break a field: very long names, diacritics, apostrophes, single-letter names, emoji.
• No setup. For a one-off mockup, a chat window is the fastest path.

Where it gets hard

• Reproducibility. The same prompt can return different rows, and the output changes when the model behind the chat is updated. A test that depends on generated rows needs them saved in the project, which turns the chat into a one-time fixture generator.
• Volume and cost. Every row is generated output, metered and limited in length per answer. A list of 5,000 records takes several requests, may stop early or skip rows, and costs the same again every time you change a field.
• Consistency across runs. Ask twice and the second batch may format dates, phone numbers or capitalization differently, or add a field you did not ask for.
• Checking the output. Values can look valid without being valid: card numbers that fail the Luhn check, dates such as 31 June, totals that do not add up, or duplicates in a column that should be unique. Someone has to check them.
• Real data in prompts. Pasting production records to get “more like these” sends them to an outside service. Describe the format instead.

What Mock Magic does differently

• Deterministic. Turn on Stable mode in the Generator, or call Mock.Seed(2026) in code, and the same calls return the same values on every run. Mock.UseDateReference pins the dates as well.
• Volume on demand. The Generator produces up to 1,000 values per click, and code generates as many as you need in the Editor, in Play Mode and in builds, without a network connection, an account or request limits.
• Valid where it matters. Card numbers pass the Luhn check, ISBNs have a correct check digit, IDs count up, and MockPerson builds the e-mail address and username from the person’s name.
• Shapes and time. 15 series patterns for charts, and live streams with Constant, Random, Burst or Drip rhythms, are data that a chat answer can describe but not run.
• The limits. Mock Magic’s vocabulary is fixed and general: 57 data types with names from many regions, US addresses and phone numbers, and no locale setting. It will not invent alchemy ingredients for you.

Best of both: let the assistant write the generator

Two patterns combine the strengths.

Ask for code, not rows. A prompt such as “write a MockOf builder for 200 orders with seed 7 and prices between 5 and 80” produces a few lines of C# that generate the rows on every run, at any count. Mock Magic ships six skill files in its AI/Skills folder that teach a coding assistant the API: setup and overview, values, typed objects, series, live streams and export. The coding assistant guide shows how to point an assistant at them and what to ask.

Ask for the vocabulary once. Let the assistant write the domain-specific word list, review it, commit it, and let Mock Magic draw from it with a seed:

using MockMagic;

public static class PotionShopData
{
    // Written once by an assistant, reviewed and committed
    static readonly string[] Potions =
    {
        "Draught of Embers", "Moonwell Tonic", "Bitter Rootbrew",
        "Elixir of Quiet Steps", "Frostleaf Philter", "Tincture of Second Wind"
    };

    public static string[] Rows(int count)
    {
        Mock.RegisterToken("potion", () => Mock.Pick(Potions));
        Mock.RegisterToken("silver", () => Mock.Integer(5, 120).ToString());

        Mock.Seed(7); // also seeds UnityEngine.Random
        string[] rows = Mock.Parse("{{potion}}: {{silver}} silver", count);
        Mock.ResetSeed();
        return rows;
    }
}

The word list costs one request. The 500 rows built from it cost nothing and repeat exactly. The templates and custom data types guide covers tokens and typed generators in depth.

The assistant stays useful afterwards, too: it can review a generated sample for anything implausible, or brainstorm the edge cases that deserve to become fixed test cases next to the generated ones.

Sources

• Mock Magic’s scripting API manual page for seeds, templates and MockPerson.
• The skill list in the AI/README.md file that ships with Mock Magic.

Which one to choose

Choose a chat assistant for a one-off sample, a domain that generic data types cannot describe, multilingual content, or a brainstorm of edge cases. Save whatever you keep.

Choose Mock Magic for anything that runs again, such as tests, demo scenes, screenshots and CI, and for large volumes, chart series and live streams.

Use both when the content is specific but the data has to repeat: the assistant writes the word lists or the generator code, and Mock Magic runs it on every build.