python - Accessing JSON feeds -
i wanted access json feeds automatically.
suppose given this website, information in table obtained through link: http://www.cmegroup.com/cmews/mvc/productslate/v1/list/500/1?sortfield=oi&sortasc=false&venues=3&page=1&cleared=1&group=7&r=rsg3qvq3
(from xhr) there 3 other pages (shown above table), contain tables this. want access (suppose n pages) , (n tables, 1 each page). there automatic way so?
right now, have copied links each of json feeds , stored in them in txt file read file.
change fields 500/1
, page=1
in url. like:
page = 2 url = 'http:/ ... st/500/{0}?sortfield=oi ... page={0}&clear ...'.format(page)
to find out number of pages should scrap original page, example(not tested) using pyquery:
from pyquery import pyquery pq d = pq(html) selector = '#cmeproductslatepaginiationtop ul' n_pages = len(d(selector))-2
Comments
Post a Comment