rest - R, GET and GZ compression -


i building clients onto restful apis. links let me download attachments (files) server, , in best case these .txt. mention restful part since means have send headers , potentially body each post - standard r 'filename'=url logic won't work.

sometimes people bundle many txts zip. these awkward since don't know contain until download many of them.

for moment, unpackaging these, gzipping files (adds .gz extension) , re-uploading them. can indexed , downloaded.

i'm using hadley's cute httr package, can't see elegant way decompress gz files.

when using read.csv or similar files gz ending automatically decompressed (convenient!). what's equivalent when using httr or curl?

content(get("http://glimmer.rstudio.com/alexbbrown/gz/sample.txt.gz")) [1] 1f 8b 08 08 4e 9e 9b 51 00 03 73 ...

that looks nice, compressed byte stream correct header (1f 8b). need text contents, tried using memdecompress, says should this:

memdecompress(content(get("http://glimmer.rstudio.com/alexbbrown/gz/sample.txt.gz")),type="gzip") error in memdecompress(content(get("http://glimmer.rstudio.com/alexbbrown/gz/sample.txt.gz")),  :    internal error -3 in memdecompress(2)

what's proper solution here?

also, there way r pull index of remote .zip file without downloading of it?

you can add parser handle mime type. @ ?content , line you can add new parsers adding appropriately functions httr:::parser 

ls(httr:::parsers)  #[1] "application/json"                  "application/x-www-form-urlencoded" #"image/jpeg"                        #[4] "image/png"                         "text/html"                         #"text/plain"                        #[7] "text/xml"

we can add 1 handle gz content. dont have better answer @ point gave can incorporate function. 

assign("application/octet-stream", function(x, ...) {scan(gzcon(rawconnection(x)),"",,,"\n")},envir = httr:::parsers)  content(get("http://glimmer.rstudio.com/alexbbrown/gz/sample.txt.gz"), = "parsed")  read 1 item [1] "these not droids looking for" >

edit: hacked alternative:

assign("application/octet-stream", function(x, ...) {f <- tempfile(); writebin(x,f);untar(f);readlines(f, warn = false)},envir = httr:::parsers)  content(get("http://glimmer.rstudio.com/alexbbrown/gz/sample.txt.gz"), = "parsed") #[1] "these not droids looking for"

with regards listing files in archive maybe can adjust function somewhat. if try httr source files. have mime type "application/x-gzip"

assign("application/x-gzip", function(x, ...) {     f <- tempfile();      writebin(x,f);      if(!is.null(list(...)$list)){         if(list(...)$list){             return(untar(f, list = true))         }else{             untar(f, ...);             readlines(f)         }     }else{         untar(f, ...);         readlines(f)     } }, envir = httr:::parsers)  content(get("http://cran.r-project.org/src/contrib/httr_0.2.tar.gz"), = "parsed", list = true)  # > head(content(get("http://cran.r-project.org/src/contrib/httr_0.2.tar.gz"), = "parsed", list = true)) #[1] "httr/"                 "httr/md5"              "httr/tests/"           #[4] "httr/tests/test-all.r" "httr/readme.md"        "httr/r/"

Comments

Post a Comment

Popular posts from this blog

user interface - Python attempting to create a simple gui, getting "AttributeError: 'MainMenu' object has no attribute 'intro_screen'" -

complete newbie here python started learning language few days ago (with no programming experience beforehand). i'm bashing skull against desk here, trying create 1 menu button lead menu, supposed replace/hide/destroy previous menu (either works, long process can reversed). what i've come far: import wx class mainframe(wx.frame): def __init__(self): wx.frame.__init__(self, none) self.centre() self.main_menu = mainmenu(self) self.intro_screen = introscreen(self) self.intro_screen.hide() class mainmenu(wx.frame): def __init__(self, parent): wx.frame.__init__(self, parent=parent) self.main_menu = mainmenu panel = wx.panel(self) sizer = wx.boxsizer(wx.vertical) nextscreen = wx.button(panel, label='next screen', size=(150,30)) nextscreen.bind(wx.evt_button, self.nextscreen) sizer.add(nextscreen, 0, wx.center|wx.all, 5) self.show() self.c...
Read more

jquery - Common JavaScript snippet to share files on Google Drive, Dropbox, Box.net or SkyDrive -

this first assignment of file sharing. need share files our website above mentioned locations. currently, have been exploring each service’s api file sharing. question 1 : there javascript code snippet available (like addthis.com ) share files? question 2 : should use server side coding file sharing or javascript advisable? thanks viewers. found useful plug-in named "ink file picker". saves given http file resource directly users online storage engine. url: https://developers.inkfilepicker.com/docs/web/ live example: http://ctrlq.org/save/
Read more

Android Gson.fromJson error -

i've completed tutorial how connect json url , parse content using gson library. code works fine url specified in tutorial, tryed modify work this url . tryed display 1 field second ulr, base field, can't work. here code, please help: public class jsonparsingactivity extends activity { string url = "http://api.openweathermap.org/data/2.5/weather?q=london,uk"; @override public void oncreate(bundle savedinstancestate) { super.oncreate(savedinstancestate); setcontentview(r.layout.main); inputstream source = retrievestream(url); gson gson = new gson();// main gson class reader reader = new inputstreamreader(source); searchresponse response = gson.fromjson(reader, searchresponse.class); //gson.fromjson(string, class) toast.maketext(this, response.base, toast.length_short).show(); } private inputstream retrievestream(string url) { defaulthttpclient client = new defaulthttpclient(); httpget getrequest = new httpget(url);...
Read more