gate - How to randomly divide a huge corpus into 3? -


i have corpus(held in jserial datastore) of thousands of documents annotations. need divide 3 smaller ones, random picking. easiest way in gate?

a piece of running code or detailed guide welcomed!

i use groovy console (load "groovy" plugin, start console tools menu).

the following code assumes that

  • you have opened datastore in gate developer
  • you have loaded source corpus, , name "fullcorpus"
  • you have created 3 (or many need) other empty corpora and saved them (empty) same datastore. these receive partitions
  • you have no other corpora open in gate developer apart these four
  • you have no documents open

then can run following in groovy console:

def rnd = new random()  def fullcorpus = corpora.find { it.name == 'fullcorpus' } def parts = corpora.findall {it.name != 'fullcorpus' }  fullcorpus.each { doc ->   def targetcorpus = parts[rnd.nextint(parts.size())]   targetcorpus.add(doc)   targetcorpus.unloaddocument(doc) }  return null

the way works iterate on documents , pick corpus @ random each document added to. target sub-corpora should end (but not exactly) same size.

the script not save final sub-corpora, if messes can close them , re-open them (empty) original datastore, fix , re-run script. once you're happy final result, right click on each sub-corpus in turn in left hand tree , "save datastore" write disk.


Comments

Post a Comment

Popular posts from this blog

blackberry 10 - how to add multiple markers on the google map just by url? -

i have android application, want convert blackberry 10 , google map needed in app,but blackberry 10 not support googlemap , blackberry developer web site supply way replace it.the sample below: @override public void oncreate(bundle savedinstancestate) { super.oncreate(savedinstancestate); setcontentview(r.layout.main); webview webview = (webview) findviewbyid(r.id.mywebview); webview.getsettings().setjavascriptenabled(true); webview.loadurl("http://maps.google.com/?ll=36.97, -122&lci=bike&z=13&t=p"); } i cannot understand it,i not know how add multiple markers on googlemap via url? can me? this worked me map shown in webview.
Read more

php - guestbook returning database data to flash -

i've been struggling problem long , unable make work, hope can me it. first, know php files correct , mysql database works fine. in flash, able submit comment database comments not display. did many tests such making returned_database flash separately , worked fine - can see old comments. long combine submitting 1 , returning 1 together, flash doesn't show comments more. put lower version flash effects on frame thinking doesn't collaborate returning flash, submitting flash works fine. since pretty complex problem hard me copy actionscript code because on different frames in flash. if has ideas fixing maybe can send file you?
Read more

java - Using an Integer ArrayList in Android -

i needing creating arraylist , adding integers them. essentially, have card game pulls card , saves id (bcid). want add pulled id arraylist, can avoid pulling same card twice. have setup below, keep getting duplicates. know cards pulling correctly, displays fine - repeated cards, unfortunately. any can provide great! have done best include relevant parts. if need additional information, let me know. public static integer bcid1, bcid2, bcid3, bcid4, bcid5, bcdcid; public static list<integer> usedcards = new arraylist<integer>(); in example below, supposed detect duplicate initialize sequence draw different card. public static void setids() { try { bcid1 = integer.parseint(bc1); usedcards.add(bcid1); } catch(numberformatexception nfe) { } try { bcid2 = integer.parseint(bc2); if (usedcards.contains(bcid2)) { try { bluecard2(ctx); } ...
Read more