[PYTHON] Frequently used methods of Selenium and Beautiful Soup

I always get lost in coding Selenium and Beautiful Soup, so make a note for myself. I also post jQuery.

environment

• Python 3.7.4
• selenium 3.141.0
• beautifulsoup4 4.8.0
• requests 2.22.0

pip

pip install requests
pip install beautifulsoup4

Selenium (Python)

#Initialization
from selenium import webdriver
driver = webdriver.Chrome()
url = 'https://qiita.com/users'
driver.get(url)

#Get one element(css selector)
element1 = driver.find_element_by_css_selector('.UsersPage__header')
#Get text
print(element1.text)

#Get multiple elements(css selector)
elements = driver.find_elements_by_css_selector('.UsersPage__user')
for elem in elements:
    #Get attributes
    href = elem.find_element_by_tag_name('a').get_attribute('href')
    print('{}<{}>'.format(elem.text, href))
    
driver.quit()

Beautiful Soup (Python)

#Initialization
from bs4 import BeautifulSoup
import requests
url = 'https://qiita.com/users'
resp = requests.get(url, headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36'})
# resp.encoding = resp.apparent_encoding #Add if garbled characters
html = resp.text
soup = BeautifulSoup(html, 'html.parser')

#Get one element(css selector)
element1 = soup.select_one('.UsersPage__header')
#Get text
print(element1.get_text())

#Get multiple elements(css selector)
elements = soup.select('.UsersPage__user')
for elem in elements:
    #Get attributes
    href = elem.find('a').attrs['href']
    print('{}<{}>'.format(elem.get_text(), href))

jQuery (JavaScript)

//Initialization
location.href = "https://qiita.com/users";
var s=document.createElement("script");
s.setAttribute("src","https://code.jquery.com/jquery-2.2.4.min.js");
document.body.append(s);

//Get one element(css selector)
const $element1 = $(".UsersPage__header");
//Get text
console.log($element1.text());

//Get multiple elements(css selector)
const $elements = $(".UsersPage__user");
$elements.each(function(i,elem) {
  let $elem = $(elem);
  //Get attributes
  let href = $elem.find("a").attr("href");
  console.log(`${$elem.text()}<${href}>`);
});