python - Get xpath of a link in a webpage containing "sometext" -


i'm using scrapy (web crawling framework). there way can xpath of element (containing "sometext") in web page can extract elements similar xpaths? don't want xpaths hardcoded because crawling multiple websites.

i'm new scrapy , have been searching days , can't find :(

you have explicitly specify element want scrape either use xpath or regular expression or library beautifulsoup. 1 way of not explicitly specifying xpath traverse dom , extracting elements need. in case need kind of mechanism identifying elements want scrape. should write different spiders scraping different websites. scraping multiple website single spider make task harder , not practice either.

for deploying , running spiders can scrapyd


Comments

Popular posts from this blog

c++ - llvm function pass ReplaceInstWithInst malloc -

java.lang.NoClassDefFoundError When Creating New Android Project -

Decoding a Python 2 `tempfile` with python-future -