GridFS Chunk Size

I've been working on a project that involves storing large files in MongoDB's GridFS.  The files ranged in size from a few hundred megabytes up to several gigabytes.  These files are then streamed out of MongoDB by using the GridFS drivers file handle methods.  I began to wonder if the chunk size had any measurable effect on performance of either reading or writing.  Searching around I was not able to find any one who could provide anything beyond anecdotes about chunk size, so I decided to run some tests myself.

This is a very basic test only attempting to determine if chunk size has any measurable effect on reading and writing, and secondarily when reading as a file handle, does the amount read effect performance.  My test files were 50 2GB files generated from /dev/urandom.  I ensured that MongoDB had already mmap enough files to contain my entire test data files.  This would ensure I wouldn't be waiting on new file creation.  The basic procedure was to insert all of the filed in to gridfs, and then read them back out, measuring the time it took for each file.  Below are the scripts I used to generate the files, and to perform the read/write tests.

File Generator

 #!/usr/bin/php  
 <?php  
 $tmp_location = '/mnt/temp';  
 $tmp_filename = 'testfile';  
 $num_files = 50;  
 for ($i = 0; $i < $num_files; $i++) {  
  if (is_file("{$tmp_location}/{$tmp_filename}.{$i}")) {  
   continue;  
  }  
  exec("dd if=/dev/urandom of={$tmp_location}/{$tmp_filename}.{$i} bs=1M count=2000", $out, $ret);  
  var_dump($out);  
 }  
 ?>  

Write Test Script

 #!/usr/bin/php  
 <?php  
 $tmp_location = '/mnt/temp';  
 $tmp_filename = 'testfile';  
 $def_chunk = 262144;  
 $chunk_mul = 1;  
 $num_files = 50;  
 $c = new Mongo('mongodb://216.81.208.140');  
 $gfs = $c->selectDB('gridtest')->getGridFS();  
 $res = $gfs->drop();  
 echo sprintf("Write Test Chunk Size %u\n", $def_chunk * $chunk_mul);  
 for ($i = 0; $i < $num_files; $i++) {  
  $start = microtime(true);  
  $res = $gfs->storeFile("{$tmp_location}/{$tmp_filename}.{$i}",array('chunkSize' => $def_chunk * $chunk_mul));  
  $end = microtime(true);   
  echo sprintf("%f\n", $end - $start);  
 }  
 ?>  

Read Test Script

 #!/usr/bin/php  
 <?php  
 $tmp_location = '/mnt/temp';  
 $tmp_filename = 'testfile';  
 $def_chunk = 262144;  
 $chunk_mul = 1;  
 $read_size = 8192;  
 $num_files = 50;  
 $c = new Mongo('mongodb://216.81.208.140');  
 $gfs = $c->selectDB('gridtest')->getGridFS();  
 $null_fp = fopen('/dev/null', 'w');  
 echo sprintf("Read Test Chunk Size %u fread size %u\n", $def_chunk * $chunk_mul, $read_size);  
 for ($i = 0; $i < $num_files; $i++) {  
  $find = array('filename' => "{$tmp_location}/{$tmp_filename}.{$i}");  
  $res = $gfs->findOne($find);  
  $fstream = $res->getResource();  
  $length = $res->file['length'];  
  $start = microtime(true);  
  while(!feof($fstream)) {  
    fwrite($null_fp,fread($fstream,$read_size));  
  }  
  $end = microtime(true);  
  fclose($fstream);  
  echo sprintf("%f\n", $end - $start);  
 }  
 ?>  

Results

This is the average time it took to write all 50 files to GridFS.

This is the average time it took to read all 50 files back out.  For each chunk size I preformed 3 tests with different sizes passed to fread.

Conclusion

It seems that altering the chunk size for GridFS does have at least some effect for writing.  At sizes of 2 megs and greater there is a marked increase in write speed.  This may in due to how the various pieces of the disk subsystem, the ext4 journal, and journal in MongoDB are interacting.  Read speeds don't seem to be effected to a noticeable degree.  There's a slight drop at 4 & 8 meg, but except for the one odd skew, read differences are sub 1 second and don't indicate a general trend.  

Although this test didn't attempt to simulate multiple readers/writers to GridFS, and since files were written in or and then read out in order this is kind of a best case scenario for read ahead optimization, it seems that that chunk size makes no difference measurable difference when reading.