Can I load every item in a Google Cloud Storage bucket into a BigQuery Table without listing every filename? -


i write new log files google cloud storage bucket every 2-3 minutes data webserver (pipe-separated-values). have thousands of ~1mb files in single google cloud storage bucket, , want load files bigquery table.

the "bq load" command seems require individual files, , can't take entire bucket, or bucket prefix.

what's best way load thousands of files in gs bucket? have uri of every single file, opposed specifying bucket name or bucket , prefix bigquery?

you can use glob-style wildcards. e.g. gs://bucket/prefix*.txt.


Comments

Post a Comment

Popular posts from this blog

user interface - Python attempting to create a simple gui, getting "AttributeError: 'MainMenu' object has no attribute 'intro_screen'" -

complete newbie here python started learning language few days ago (with no programming experience beforehand). i'm bashing skull against desk here, trying create 1 menu button lead menu, supposed replace/hide/destroy previous menu (either works, long process can reversed). what i've come far: import wx class mainframe(wx.frame): def __init__(self): wx.frame.__init__(self, none) self.centre() self.main_menu = mainmenu(self) self.intro_screen = introscreen(self) self.intro_screen.hide() class mainmenu(wx.frame): def __init__(self, parent): wx.frame.__init__(self, parent=parent) self.main_menu = mainmenu panel = wx.panel(self) sizer = wx.boxsizer(wx.vertical) nextscreen = wx.button(panel, label='next screen', size=(150,30)) nextscreen.bind(wx.evt_button, self.nextscreen) sizer.add(nextscreen, 0, wx.center|wx.all, 5) self.show() self.c...
Read more

jquery - Common JavaScript snippet to share files on Google Drive, Dropbox, Box.net or SkyDrive -

this first assignment of file sharing. need share files our website above mentioned locations. currently, have been exploring each service’s api file sharing. question 1 : there javascript code snippet available (like addthis.com ) share files? question 2 : should use server side coding file sharing or javascript advisable? thanks viewers. found useful plug-in named "ink file picker". saves given http file resource directly users online storage engine. url: https://developers.inkfilepicker.com/docs/web/ live example: http://ctrlq.org/save/
Read more

Android Gson.fromJson error -

i've completed tutorial how connect json url , parse content using gson library. code works fine url specified in tutorial, tryed modify work this url . tryed display 1 field second ulr, base field, can't work. here code, please help: public class jsonparsingactivity extends activity { string url = "http://api.openweathermap.org/data/2.5/weather?q=london,uk"; @override public void oncreate(bundle savedinstancestate) { super.oncreate(savedinstancestate); setcontentview(r.layout.main); inputstream source = retrievestream(url); gson gson = new gson();// main gson class reader reader = new inputstreamreader(source); searchresponse response = gson.fromjson(reader, searchresponse.class); //gson.fromjson(string, class) toast.maketext(this, response.base, toast.length_short).show(); } private inputstream retrievestream(string url) { defaulthttpclient client = new defaulthttpclient(); httpget getrequest = new httpget(url);...
Read more