‘Hey Gemini, make me some data’

AI is more than just a buzzword, it’s everywhere and it's here to stay.  At Elixel we have been experimenting and learning, looking for new and interesting ways to enhance how we work and what we do.  There are a myriad of tools and it can be hard to work out where to start.  So when I asked myself, ‘how can AI help me?’ I wanted to start with something simple. How could I use the power of a tool like Gemini or ChatGPT to simplify a mundane task that would aid our development phase? That’s when I began working on AI data generation.

We live in a data economy, where generating and analysing data is the key to success. It plays a fundamental part in any business, identifying where things are going right and wrong, but only if it can be easily interpreted. From visualising the effectiveness of e-learning experiences, to measuring the impact of social prescriptions on mental health, we have designed and developed our fair share of dashboards over the years to help our clients gather data and understand their businesses. 

In most situations, you are working with pre-existing data and are mapping how you can bring that data together in a way that resonates with the user. But what if you are working on a greenfield project where the data doesn't yet exist?

‍

The power of AI for dummy data

This is where generative AI comes in.  AI has the power to combine natural language with the specifics of data structures and produce data at scale that has some resemblance to real world input.  This can then be imported as dummy data so graphs and statistics can be developed. Nothing beats real world data, but in the early stages of a system that data often doesn’t exist yet and you have to start somewhere. 

Imagine you’re building a dashboard to showcase how users feel and what activities they have done over 6 weeks (we were, see mysoc)…but you have no users. You need to generate data for at least 10 individuals covering 42 days worth of material, each with 5 different submissions including written journal entries that have sentiment.  If you do this by hand that’s 2100 records and over two days work (30 secs per record)!  However, switch this to an AI automation and those 3000 records can be generated and imported in a few hours allowing you to get on with the important stuff.

‍

The starter kit for generative AI data

Nothing comes from nothing, though, so when using AI to generate data there are still some initial resources that you need to get it right:

• A data structure - Before generating data you have to know exactly how you want the data to be formatted.  Providing a data outline to the AI gives it a fixed plan of how data should be returned.  I usually do this with a file upload that provides an example to learn from.
• Possible data responses - For data that is limited, such as a selection from a dropdown, it’s necessary to provide AI with all the variations so it knows what to choose from.  Providing specific datasets helps prevent the AI from generating its own values in order to fulfill your request.  Again I tend to upload files, usually in JSON format to define these.
• Context and a persona - For natural language results such as a free text journal entry, the AI can benefit from guidance on the kind of person that is meant to be responding and why.  By providing context the dummy data is likely to be more relevant and therefore more suitable to test with. 
• A thorough and effective prompt - AI prompt writing is the new skill to master.  Ensuring you provide enough detail and clarity to the AI for it to complete the task correctly is vital, especially if your data is complex.  For example, my prompt was just under 500 words in total, the same length as the first 5 paragraphs of this article.

‍

When genAI data goes wrong

There are still a few recurring gotchas though to be wary of.  Continuing to interrogate your results is important to ensure success, don’t ever expect it to be right first time.  Some things to keep an eye out for include:

• Too much specificity - it might sound mad, but I believe you can get too specific with AI. By actively trying to instruct the AI, small misinterpretations of your intent can lead to incorrect responses. For example ‘your job is to create dummy content’ can actively result in the output being less relevant and lower quality than what could be achieved. It is ‘dummy’ content, but AI doesn’t need to know that.
• Fixed data mutation - When working with data that should stay the same e.g. an ID, AI can occasionally mutate the data destroying potential connections you are trying to make.  We aren’t talking big changes here, but subtle stuff.  Examples I’ve seen include changing a single character in an ID from lowercase to uppercase, and translating the same category title in 2 different ways.
• Down the rabbit hole - It’s easy to keep poking and prodding AI in the hope that your output will get better and better, but sometimes this can be a vicious cycle.  It’s better to keep in mind that you are looking for something that aids progress, but not something that’s perfect.  If after a day you’re still fashioning your prompt, you’ve probably gone too far!

‍

With these elements in place it is possible to produce a fairly representative set of data that can be used in development.  Your data has variety, resembles human language (no lorem ipsum here), and exists at scale, allowing you to test for more edge cases, all in a much smaller amount of time.

Using AI for in-development data generation is only scratching the surface of its power, but it’s also a safe way to experiment. I’m not one for relying on AI too much (in fact this article is written by hand), but I know there are ways that AI technology can enhance what we do and speed up delivery…and data generation is one of those.  We’ll definitely be using it again in the future, and I hope this article might encourage you to experiment too. 

Do you have the data you need to grow?  Can you analyse it or is it all trapped in silos? If you need help accessing and visualising the data at the core of your business, get in touch to see how we can help.

‍