Starting a Project: Data Generator

Sometimes, there is a need to generate data to test your code. While "testing in production" is a sweet spot to be, there are times when you need to "force" (stub) data that reflects certain business phenomena in production. This does not mean "chaos engineering" but rather a finite set that can be ingested into a suite of integration tests to increase confidence in code change.

For example, creating a number of threads to generate load can be done trivially with Apache Bench (ab), I'm looking for a scenario builder with lots of traffic data. I'm referring to the "needle in the haystack" scenario builder.

I started with some ideas on how to create multiple "stubs" in a thread pool so that I can force concurrent payloads to hit a service to test. Using the Spring Framework and Spring Boot, this task became super simple. I ended up with the following:

and the following TaskExecutor bean:

Of course, this is only a start with the following potential test:

• "Transaction boundaries" defined in my code.
• Verify the "concurrency" setup.
• Create a set of scenarios (the needle in the haystack)

I'll update more on this journey as I add more to the data generator project I have been creating.