Log in to Yahoo Business with Selenium Python

Hundreds of Yahoo Business IDs and passwords. Is it possible to log in with this Excel list ID and password? Many things that the Excel list is not updated even if the password is changed. Therefore, use scraping to confirm the login.

I thought it would be easy, but I got stuck a little, so I'll make a note of it.

Conclusion

Do not log in directly from the login screen. Go to the login screen from another page in Yahoo and log in.

For example Without transitioning to the login screen directly https://business.yahoo.co.jp/ ↓ https://login.bizmanager.yahoo.co.jp/login Transition in the order of.

Looking at the source of the Yahoo! Business login screen, there are many hidden items. It seems that if this .crumb is empty, you cannot log in (nothing happens). So even if you enter the correct ID and password with the hidden .crub empty, nothing happens, and even if you enter the wrong ID and password, the "ID and password are incorrect" screen is not displayed. ..

Yahoo business login screen hidden.crumb has no value

<form method="post" action="/login.php" autocomplete="off" name="login_form" onsubmit="return checkMultipleSubmit();">
<input type="hidden" name="url" value="https://business.yahoo.co.jp/" data-rapid_p="1">
<input type="hidden" name="action" value="login" data-rapid_p="2">
<input type="hidden" name=".flow" value="" data-rapid_p="3">
<input type="hidden" name=".crumb" value="" data-rapid_p="4">

https://business.yahoo.co.jp/ ↓ https://login.bizmanager.yahoo.co.jp/login When transitioning in the order of A value has been entered in the hidden .crub.

Yahoo business hidden.After the session value is set in crumb

<form method="post" action="/login.php" autocomplete="off" name="login_form" onsubmit="return checkMultipleSubmit();">
<input type="hidden" name="url" value="https://business.yahoo.co.jp/" data-rapid_p="1">
<input type="hidden" name="action" value="login" data-rapid_p="2">
<input type="hidden" name=".flow" value="" data-rapid_p="3">
<input type="hidden" name=".crumb" value="dD14Q0R0ZkImc2s9eGlqMXh3dTRLcVJzM29KQ3pLMlIwaHNVcVZvLQ==" data-rapid_p="4">

If you log in after this state, you will be able to log in by scraping.

environment Windows10 Python3.8.3 Selenium Google chrome

-It is assumed that the chrome driver is set in the driver. ・ Time.sleep is included because it may be blocked if you proceed too fast.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

#Do not go directly to the login screen
driver.get("https://business.yahoo.co.jp/")
driver.get("https://login.bizmanager.yahoo.co.jp/login.php")    

#I created a function to make sure sendkey.
move_textbox_id(driver,"user_name","ID here")        
move_textbox_id(driver,"password","Password here")      
        
#I don't want to go too fast and be blocked, so go slowly.
time.sleep(1)     
    
element = driver.find_element_by_xpath("//*@id='bidlogin']/div/div/form/fieldset/div[3]/input")
element.click()

try:
　　#If you can find the link text for logout, you can log in.
   WebDriverWait(driver,8).until(EC.presence_of_element_located((By.LINK_TEXT, "Logout")))
except:
　　#What to do if you can't find the logout link for some reason

def move_textbox_id(driver, id, atai):
    start = time.time()
    while driver.find_element_by_id(id).get_attribute('value')!=atai:

        driver.find_element_by_id(id).clear()
        time.sleep(1)
        
        #Enter a value with the send key
        driver.find_element_by_id(id).send_keys(atai)
        time.sleep(1)

        if time.time()-start > 5 :
            return False   
    return True

It was hard to think that it was easy because I just logged in. Especially when operating in secret mode, if you log in directly from the login screen, you will never be able to log in. Certainly, the first login was rejected even when logging in manually instead of scraping. I wasn't worried because I thought I was messing around with copy and paste.