I was a little addicted to getting the element whose attribute value is dynamic when scraping, so share it.
.html
<div class="hogehoge">
<input type="hidden" id="sample_1" value="fugafuga">
<input type="hidden" id="sample_2" value="fugafuga">
<input type="hidden" id="sample_3" value="fugafuga">
<input type="hidden" id="sample_4" value="fugafuga">
<input type="hidden" id="sample_5" value="fugafuga">
・
・
・
</div>
Suppose you want to get an element whose number of elements is indefinite and whose attribute value is dynamic in this way.
** ・ lambda function ver **
source = soup.find_all('input', id=lambda value: value and value.startswith('sample_'))
You can get the element by prefix search using the lambda function.
** ・ Regular expression ver **
source = soup.find_all('input', id=re.compile('^sample_'))
It can also be obtained with a regular expression.
** ・ CSS selector ver **
source = soup.select('input[id^=sample_]')
It can also be specified with the CSS selector.
How to find all divs who's class starts with a string in BeautifulSoup?
Recommended Posts