python - Half of json disappearing -
i trying scrape site in python prices , querying url api endpoint(??). when copy paste url parameters, complete json in browsers complete in chrome developers console. code looks this..
import requests import json api_woolies=('https://www.woolworths.com.au/apis/ui/search/products?ismultisearch=true&isspecial=false&pagenumber=1&pagesize=5&searchterm=chicken&sorttype=relevance') #the url woolies_data=(requests.get(api_woolies)) print woolies_data.text # woolies_data.text contains full json want woolies_data_=json.loads(woolies_data.text) # removes part want , leaves jsondata dont need print woolies_data_
the response content-type json. calling json on reponse object woolies_data.json()
, using json.loads(woolies_data.text)
or json.loads(woolies_data.content)
removes data @ front , starts corrections:none
can see in browser. tried using json.dumps
adds backslash , inverted comma @ start of key , values. researched error , came know happens if data jsonified. want relevant data convert python object can logical parsing calling products.name/price not able in present format of text since want bring query parameters client side. it's real dilemma can see data not useful. don't believe string manipulation suitable process. appreciate if me out.
assuming you're trying api response python dict, following need do:
import requests url = "https://www.woolworths.com.au/apis/ui/search/products?ismultisearch=true&isspecial=false&pagenumber=1&pagesize=5&searchterm=chicken&sorttype=relevance" r = requests.get(url) data = r.json()
the r.json()
function automatically decodes response's json content python dict datastructure, have assign variable, use regular dictionary operations access information you're looking for.
you being rather over-zealous in use of parentheses. example, in
woolies_data=(requests.get(api_woolies))
the parens surrounding requests.get(api_woolies)
not necessary. same true when assign api_wollies
variable. if accidentally add comma ,
inside parens, you'd create tuple, cause undesired behavior down line. finally, i'd suggest using spaces around assignments:
var = "value" # instead of var="value"
it improves readability, , recommended according python style guide (pep-8).
Comments
Post a Comment