[LINUX] Get html source when javascript is enabled using curl

There is a tool called domcurl. A very useful tool for web scraping. Recommended when the requirements cannot be met with phantomjs etc.

domcurl is a small NodeJS application that uses Puppeteer and can be installed by issuing the npm i domcurl command. Like the curl command, you can issue a simple domcurl [url] to get resources and run JS on your page.

https://paul.kinlan.me/domcurl/

$ npm i domcurl
$ ./node_modules/domcurl/index.js  --url https://example.com

Note: The title is not accurate, but it is written as curl for ease of searching.

Recommended Posts

Get html source when javascript is enabled using curl
Get soup on javascript enabled site
CPU usage is 100% when using pygame on Ubuntu 16.04