When I got a certain page with jsoup and got elements with selector, the result was less than the original number. Even if you look at the text () of the document, the acquisition elements are not enough ... (problem)
That was all.
Document doc = Jsoup
.connect("https://xxx.xxx.xxx/")
.userAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36")
.timeout(1000 * 60)
.maxBodySize(0)
.get();
·reference https://jsoup.org/apidocs/org/jsoup/Connection.html#maxBodySize-int- It said "The default maximum is 1MB". Is it surprisingly small? is not it.
・ The site that I referred to https://jmchung.github.io/blog/2013/10/25/how-to-solve-jsoup-does-not-get-complete-html-document/
Recommended Posts