PHP, Splitting a Large File into parts incorporating a String search -
i have file on 400mb
it timetable database distributed in way.
in text file there string marks start of data record.
this string begins "bsn", , likewise there string marks end of data record starts "lt"
what i'm trying fathom how chop data file chunks, containing 1000 data records. when cycle complete, can import files sequentially.
the created files must numbered sequentially in new folder...
[edit] record set varies in length [/edit]
below sample of 1 of groups:
bsnc031551112111206240000001 << data record start >> bx emyem129000 losheffld 2235 2235 lidoresnj lispdn ltdrby 2326 23266 << data record end >> bsnc033501112111205130000001 << next record >> bx emyem118600
*the << >> tags added understanding, not exist in file.
i read in file using php fopen / fgets method here
something should work you
$fp = fopen($bigfile, "r"); $file_num = 1; $prefix = "file_"; $suffix = ".dat"; $buff = ""; $recno = 0; while ($rec = fgets($fp)){ if (substr($rec, 0,3) == 'bsn'){ $recno++; } if ($recno == 1000){ // reset record counter $recno = 1; // flush out file file_put_contents($prefix.$file_num.$suffix, $buff); // clear buffer $buff = ""; // increment file counter $file_num++; } // add buffer $buff.= $rec; } fclose($fp); // flush remainder if ($buff) file_put_contents($prefix.$file_num.$suffix, $buff);
Comments
Post a Comment