How to request a CSV file as JSON with jMeter

I want to request and post more than 10,000 CSV data.

Right now, I'm making processing of large volumes of data, if not big data. Processing large data is limited in the number of times it can be tested per day, so

I want that environment even in my house, I wondered if I could do something with jMeter. just made it.

jMeter doesn't have the ability to make CSV a JSON request by default

JMeter is a multifunctional and high-performance request sending app, By default, there is no "function that converts CSV to JSON".

However, by running the Script when sending "** BeanPreProcesser **", There is a function that can be controlled in various ways.

Roughly do this. -Set the receiving variable in the HTTP request

-Use Bean PreProcessor to create a JSON string and assign it to the receiving variable

・ Send HTTP request with jMeter

Set JSON transmission

First, set the HTTP request transmission settings normally. Create an HTTP header manager, Set Content-Type to ** application / json **

image.png

Create a variable for assignment in the request body

First, normally, enter the target address in the target server or path.

Then put "** variable to store the result of the script **" in the request This time I named it ** $ {post_data} **

image.png

Create BeanPreProcessor

Then right-click to create BeanPreProcessor and create a JSON string.

BeanPreProcessor is written in Java, so if you are used to it, it's easy to use. It seems that JSON serialization and deserialization are also possible by loading the library in jMeter ヾ (.> ﹏ <.) ノ ✧ *.

image.png

The processing content is as you can see. Loop the CSV line by line

request.json


"csv list":[{
  "Hogehoge":"CSV 1st column"
  "somehow":"CSV 2nd column"
  "foo":"CSV 3rd column"
}
{
  "Hogehoge":"CSV 1st column"
  "somehow":"CSV 2nd column"
  "foo":"CSV 3rd column"
}]
...

Make a JSON like this!

BeanShellPreProcessor.java


//▼-----------BeanPreProcessor processing started-------------▼
log.info("BeanShell PreProcessor start");


StringBuilder requestBody = new StringBuilder();
//Make a JSON start
requestBody.append("{\r\n");
//Line breaks are also reflected properly ヾ(¡>﹏<¡)No ✧*。
//Start creating list object

requestBody.append("\"requestCSV\":[\r\n");

//Read CSV file
BufferedReader reader = new BufferedReader(new FileReader(new File("C:\\csvFileForBatche\\06YAMAGAcopied.CSV")));
String line;
//Loop line by line

boolean secondLoopFlg = false;
while ((line = reader.readLine()) != null) {
	//Divide the read CSV with commas
	String[] params = line.split(",");

		if(secondLoopFlg){
			requestBody.append(",\r\n");
		}
		requestBody.append("{\r\n");
		//Create CSV parameters for each item
    		requestBody.append("\"countryPublicOrgCd\":\"").append(params[0]).append("\",\r\n");
		requestBody.append("\"zipcodePare\":").append(params[1]).append(",\r\n");
		requestBody.append("\"zipcode\":").append(params[2]).append(",\r\n");
		requestBody.append("\"prefNameKana\":").append(params[3]).append(",\r\n");
		requestBody.append("\"cityNameKana\":").append(params[4]).append(",\r\n");
		requestBody.append("\"streetNameKana\":").append(params[5]).append(",\r\n");

		requestBody.append("\"prefNameKanji\":").append(params[6]).append(",\r\n");
		requestBody.append("\"cityNameKanji\":").append(params[7]).append(",\r\n");
		
		requestBody.append("\"streetNameKanji\":").append(params[8]).append(",\r\n");
		requestBody.append("\"var1\": \"").append(params[9]).append("\",\r\n");
		requestBody.append("\"var2\": \"").append(params[10]).append("\",\r\n");
		requestBody.append("\"var3\": \"").append(params[11]).append("\",\r\n");
		requestBody.append("\"var4\": \"").append(params[12]).append("\",\r\n");
		requestBody.append("\"var5\": \"").append(params[13]).append("\",\r\n");
		requestBody.append("\"var6\": \"").append(params[14]).append("\"\r\n");

		//Close object
		requestBody.append("}\r\n");

		secondLoopFlg = true;
}

//Close JSON object
requestBody.append("]\r\n");

requestBody.append("}\r\n");
//Close file
reader.close();

//Assign to a variable prepared by the request
vars.put("post_data", requestBody.toString());

// ▲-----------BeanPreProcessor processing started-------------▲
log.info("BeanShell PreProcessor start");

Let's run ヾ (.> ﹏ <.) ノ ✧ *.

I will actually send a request in this state image.png

We were able to send CSV to 30,000 requests b

reference Sending Complete JSON Data from the CSV file in one request in JMeter https://stackoverflow.com/questions/32628934/sending-complete-json-data-from-the-csv-file-in-one-request-in-jmeter

receiving json and deserializing as List of object at spring mvc controller https://stackoverflow.com/questions/23012841/receiving-json-and-deserializing-as-list-of-object-at-spring-mvc-controller

Recommended Posts

How to request a CSV file as JSON with jMeter
How to register as a customer with Square using Tomcat
How to make a jar file with no dependencies in Maven
How to read a file and treat it as standard input
How to open a script file from Ubuntu with VS code
[Xcode] How to add a README.md file
How to achieve file download with Feign
How to achieve file upload with Feign
How to start a Docker container with a volume mounted in a batch file
[Ruby] How to convert CSV file to Yaml (Yml)
To manually deploy Struts2 as a war file
How to store an object in PostgreSQL as JSON with MyBatis (Mapper XML)
How to ZIP a JAVA CSV file and manage it in a Byte array
How to create an Excel form using a template file with Spring MVC
How to test a class that handles application.properties with SpringBoot (request: pointed out)
How to make a factory with a model with polymorphic association
How to run the SpringBoot app as a service
How to jump from Eclipse Java to a SQL file
How to delete a new_record object built with Rails
How to deal with No template for interactive request
How to manually generate a JWT with Rails Knock
I want to monitor a specific file with WatchService
[Java] How to get a request by HTTP communication
[How to insert a video in haml with Rails]
How to download a file (Servlet, HTML, Apache, Tomcat)
How to convert a file to a byte array in Java
How to get started with creating a Rails app
[Java] How to start a new line with StringBuilder
How to save a file with the specified extension under the directory specified in Java to the list
csv file output with opencsv
[Ruby] How to split each GraphQL query into a file
How to decompile apk file to java source code with MAC
How to divide a two-dimensional array into four with ruby
How to use a foreign key with FactoryBot ~ Another solution
How to leave a comment
[Easy] How to automatically format Ruby erb file with vsCode
Learning Ruby with AtCoder 13 How to make a two-dimensional array
How to read a library from a JAR file with VS Code << How to not use Maven / Gradle >>
How to number (number) with html.erb
How to test a private method with RSpec for yourself
How to update with activerecord-import
How to move another class with a button action of another class.
Mapping to a class with a value object in How to MyBatis
How to install GNOME as a desktop environment on CentOS 7
How to insert a video
How to set up a proxy with authentication in Feign
How to create a method
How to change the file name with Xcode (Refactor Rename)
Convert Excel to Blob with java, save it, read it from DB and output it as a file!
How to create a jar file or war file using the jar command
How to cancel cell merging within a specified range with POI
How to realize huge file upload with TERASOLUNA 5.x (= Spring MVC)
Try to make a cross-platform application with JRuby (jar file generation)
Read the file under the classpath as a character string with spring
How to load a Spring upload file and view its contents
How to run a job with docker login in AWS batch
How to rename a model with foreign key constraints in Rails
How to output a list of strings in JSF as comma-separated strings
Things to forget when intercepting a request with Android's WebView # shouldInterceptRequest
[chown] How to change the owner of a file or directory
I made a tool to output the difference of CSV file