Starting a Project: Data Generator

Sometimes, there is a need to generate data to test your code. While "testing in production" is a sweet spot to be, there are times when you need to "force" (stub) data that reflects certain business phenomena in production. This does not mean "chaos engineering" but rather a finite set that can be ingested into a suite of integration tests to increase confidence in code change.

For example, creating a number of threads to generate load can be done trivially with Apache Bench (ab), I'm looking for a scenario builder with lots of traffic data. I'm referring to the "needle in the haystack" scenario builder.

I started with some ideas on how to create multiple "stubs" in a thread pool so that I can force concurrent payloads to hit a service to test. Using the Spring Framework and Spring Boot, this task became super simple. I ended up with the following:

	@Async("myThreadPoolTaskExecutor")
	public Object createPayload() {
	// return payload pojo
	}

view raw PayloadGenerator.java hosted with ❤ by GitHub

and the following TaskExecutor bean:

	@Bean(name = "myThreadPoolTaskExecutor")
	public TaskExecutor threadPoolTaskExecutor() {
	ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
	executor.setCorePoolSize(4);
	executor.setMaxPoolSize(4);
	executor.setThreadNamePrefix("executor");
	executor.initialize();
	return executor;
	}

view raw PayloadGeneratorConfiguration.java hosted with ❤ by GitHub

Of course, this is only a start with the following potential test:

• "Transaction boundaries" defined in my code.
• Verify the "concurrency" setup.
• Create a set of scenarios (the needle in the haystack)

I'll update more on this journey as I add more to the data generator project I have been creating.