In the rush work, we released a web application that downloads data entered by multiple people in CSV format. I used the following OSS.
There was no particular problem on the day of the start of operation, and when I left the office and drank with my friends, my smartphone was noisy. .. .. When I answered the incoming call, I received a message saying "Download is not possible". When I checked the details, he said, "Until 3 hours ago, when I downloaded the file, the CSV file was output, but suddenly the CSV was displayed on the browser."
I made a mistake with a friend, stopped drinking, and started investigating the cause. For the time being, I knew that it would work if I fixed it like this, so I released it with service priority first. However, the principle remains unclear, so we conducted a continuous investigation.
Below is the simplified code for the download process that caused this problem and the modified code.
@PostMapping
fun export(response: HttpServletResponse) {
/**
*CSV response.Write to outputStream.
*/
response.contentType = "text/csv"
response.setHeader("Content-Disposition", "attachment; filename=export.csv")
response.outputStream.flush()
}
@PostMapping
fun export(response: HttpServletResponse) {
response.contentType = "text/csv"
response.setHeader("Content-Disposition", "attachment; filename=export.csv")
/**
*CSV response.Write to outputStream.
*/
response.outputStream.flush()
}
The point is whether Content-Type and Content-Disposition are after or before writing to the outputStream. Subsequent investigations have shown that if Content-Disposition is in front, the file will be downloaded.
The reason why I set Content-Disposition after writing to outputStream is that the article I referred to happened to be like this, and I think that setting the header and writing to the body (stream) are independent processes. That's why.
However, even if Content-Disposition is set after writing to outputStream, the file can be downloaded normally for a while, so it can be inferred that the size of the CSV output has an effect. So I entered the data size on the screen and created a reproduction program that outputs CSV of that size.
@PostMapping
fun export(@RequestParam size: Int, response: HttpServletResponse) {
(1..size).forEach {
response.outputStream.print("a")
}
response.contentType = "text/csv"
response.setHeader("Content-Disposition", "attachment; filename=export.csv")
response.outputStream.flush()
}
Then, the response header confirmed by Chrome's developer tools that the size is 8KByte
Connection: keep-alive
Content-Disposition: attachment; filename=export.csv
Content-Type: text/csv
Date: Sat, 08 Feb 2020 10:26:07 GMT
Keep-Alive: timeout=60
Transfer-Encoding: chunked
It was 9KByte
Connection: keep-alive
Date: Sat, 08 Feb 2020 10:27:48 GMT
Keep-Alive: timeout=60
Transfer-Encoding: chunked
Doesn't Content-Disposition disappear like this? In the end, 8191 bytes is the former, and 8192 bytes is the latter. I think it's because 8192, which is a power of 2, exceeds something set.
I think that the following is probably happening in Tomcat.
So, while quickly monitoring the packet with tcpdump, I tried step-by-step writing from 8190 bytes to 1 byte with the debugger, and it was confirmed that the HTTP header and body were sent to the browser when 8192 bytes were written. ..
By the way, when I stepped in when writing 8192 bytes, I found the following code.
java:org.apache.catalina.connector.OutputBuffer
public void append(byte src[], int off, int len) throws IOException {
if (bb.remaining() == 0) {
appendByteArray(src, off, len);
} else {
int n = transfer(src, off, len, bb);
len = len - n;
off = off + n;
if (isFull(bb)) {
flushByteBuffer();
appendByteArray(src, off, len);
}
}
}
bb is a member variable of ByteBuffer type, and the transmission data is stored in bb in the appendByteArray and transfer methods. In the isFull method, it becomes true when the size accumulated in bb reaches the allowable amount, and it is sent to the browser by the flushByteBuffer method.
If you follow the flushByteBuffer method further,
java:org.apache.coyote.http11.Http11OutputBuffer
@Override
public int doWrite(ByteBuffer chunk) throws IOException {
if (!response.isCommitted()) {
// Send the connector a request for commit. The connector should
// then validate the headers, send them (using sendHeaders) and
// set the filters accordingly.
response.action(ActionCode.COMMIT, null);
}
if (lastActiveFilter == -1) {
return outputStreamOutputBuffer.doWrite(chunk);
} else {
return activeFilters[lastActiveFilter].doWrite(chunk);
}
}
If it is not committed, it sends an HTTP header during the commit process to set the committed flag. After that, the HTTP body is written.
So, the code that I made easily was released without noticing it because it happened to behave as intended, but considering the difficulty on the side of Tomcat that processes various sizes, it is a natural implementation, there It was a story that I went all the way to get hooked.
(2020/2/8) Posted. (2020/2/9) Fixed an error in the size of the event switching before and after in the cause investigation. Added cause investigation 2 and consideration.
Recommended Posts