It is the 23rd day of Crawler / Web Scraping Advent Calendar 2015. It also serves as Adventer's Perfume Advent Calendar 2015 m (_ _) m
I would like to introduce one-liner web scraping using the text formatting tool "tse" by Python.
For more details, I think you can understand by looking at the slides presented by Mr. Ishimoto at this year's PyCon JP 2015. tse --Text Formatting Utility with Python
I wrote an introductory article about tse the other day, so please refer to that as well. tse --Introduction to Text Stream Editor in Python
Let's take a quick look at the latest news from Natalie as a way to organize our minds so that we can enjoy Perfume's performance in the Kouhaku Uta Gassen next week.
curl http://natalie.mu/music/news/list/artist_id/217 | tse -ms 'bs4' -b 'for dt in BeautifulSoup(sys.stdin.read(), "lxml").find_all("dt", class_="NA_title", limit=30):{{print(dt.string)}}'
output
V6 in red and white&Perfume co-stars with Mickey and Arashi tags with "Star Wars"
Arashi, AKB, Babymetal, X JAPAN and others "M Ste Super Live" song announcement
"Kohaku Uta Gassen" Participating Singer Song Announcement, Sachiko Kobayashi Shows "Senbonzakura"
CDTV New Year's special program aiko, Areki, KANA-BOON、T.M.R., Perfume, Gen Hoshino, etc.
Perfume lifts teaser ban on Anniversary Budokan live video collection
Screening of Perfume documentary, BiS Cano and others at music & subculture movie events
48 & 46, Stardust, Hello! Project mixed medley! Live-specialized "FNS Kayosai" 16th OA
Enon Kawatani's birthday is celebrated by gorgeous people, indigo la End's biggest one-man success in history
42 gorgeous groups such as Mr. Children, L'Arc, YUKI, Babymetal, Arashi, X JAPAN at the end of the year SP of "Music Station"
Perfume x Competitive Karuta, New Song Lifted in Trailer of Movie "Chihayafuru"
"Fake three daughters" have also come! Perfume, pornography, Ryunosuke Kamiki and other gorgeous co-stars in the 23rd year "AAA"
Original author & Suzu Hirose inspiring, Perfume's theme song for the movie "Chihayafuru"
There are tears, laughter, and impersonation! The 66th Red and White Contestant Conference of Delight
BUMP, Nogizaka, Gen Hoshino, Guess, μ in red and white's et al. 10 groups first appearance! X JAPAN is the first in 18 years
Perfume becomes the character of Mercedes-Benz, designed by Yoshiyuki Sadamoto
Perfume Budokan live is visualized, 10DAYS digest & sugoroku all performances in the first edition
Perfume, KANA for chat project "Konason Festival"-BOON, Theater Brook
29 groups including aiko, Ikimono, Kyary, Sekaowa, etc.
Arashi, Perfume, Sandaime J Soul Brothers, Momokuro, Gen Hoshino, etc. at "FNS Kayosai"
"MTV VMAJ" winners include Gen Hoshino, Sandaime JSB, Amuro and others
Next week at "Music Station", Ikimono, NMB, Sakurako Ohara, Kanjani, bknb, Perfume
"Perfume amazing comedian" again! Okite Porsche, Asagaya Shimai sister Watanabe on stage
Perfume, 5 albums from the Tokuma era are converted to analog
Shrimp in the next "Kai", TOKIO, Perfume, KANA-BOON, Good Morning America
Perfume, Minister of Internal Affairs and Communications Award/Winner of the ACC Grand Prix "I'm glad that the team is evaluated"
Perfume's Yahoo!Search Kisekae "STAR TRAIN" Ver.Appearance
Perfume, Ikimonogakari, and TOKIO will appear on the next "CDTV"
"Big fan for several years" Ken Shimura nominated Perfume as a conversation partner, on air on E-tele
"Three people together all the time" Perfume reveals future dreams at a film festival
Perfume, Paruru, ELT, Mamoru Miyano, GACKT and others appear on the red carpet at the Tokyo International Film Festival
"V6 & Perfume co-stars with Mickey and others in red and white, Arashi tags with" Star Wars ""
This news is terrible! Collaboration between Perfume and Disney! !! 2013/08/14 The excitement of NHK General TV "The magical melody that connects everyone" Disney & Ghibli's masterpiece "" is back! !! !! https://www.youtube.com/watch?v=qdPDeG_XxaU
tse -ms-import with <module name import * format> -b --Action to take immediately after startup {{}} --Specify the block to indent
Thanks to tse's -ms option, we were able to run BeautifulSoup in one liner!
So far, we have introduced the technique of one-liner web scraping using tse.
tse is very convenient because you can apply the description in Python that you are accustomed to to one liner! Please use it in scenes other than Perfume.
By the way, Crawler / Web Scraping Advent Calendar 2015 is just around the corner! Let's be careful not to be exposed as an Advent Calendar non-poster by @ jz5 on the last day.
Recommended Posts