~mbuffer/
home of measuring buffer

Description

mbuffer is a tool for buffering data streams with a large set of unique features:
It is especially useful, if you are writing backups to fast tape drives or libraries. Those drives tend to stop and rewind if they have a buffer underrun. This so called tape screwing reduces the lifetime of the motors.
mbuffer can prevent buffer underruns, if used correctly and speed up the whole backup process. Please read the man page for details, how to use it.

Technology/Features

mbuffer is written in C and uses the POSIX API. Initially, I developed it under Linux to learn multi-threaded programming and speed up my own backup process, which used a DAT tape. Later it got enhanced features like network support. Today I develop mbuffer on Solaris and only use it from time to time to mirror some diretory tree from one machine to another.

Usage/Command line options

-b <num> : use <num> blocks for buffer (default: 11326)
-s <size> : use block of <size> bytes for buffer (default: 8192)
-m <size> : memory <size> of buffer in b,k,M,G,% (default: 2% = 88M)
-L : lock buffer in memory (unusable with file based buffers)
-d : use blocksize of device for output
-D <size> : assumed output device size (default: infinite/auto-detect)
-P <num> : start writing after buffer has been filled more than <num>%
-p <num> : start reading after buffer has been filled less than <num>%
-i <file> : use <file> for input
-o <file> : use <file> for output (this option can be passed MULTIPLE times)
-I <h>:<p> : use network port <port> as input, allow only host <h> to connect
-I <p> : use network port <port> as input
-O <h>:<p> : output data to host <h> and port <p> (MUTLIPLE outputs supported)
-n <num> : <num> volumes for input
-t : use memory mapped temporary file (for huge buffer)
-T <file> : as -t but uses <file> as buffer
-l <file> : use <file> for logging messages
-u <num> : pause <num> milliseconds after each write
-r <rate> : limit read rate to <rate> B/s, where <rate> can be given in b,k,M,G
-R <rate> : same as -r for writing; use eiter one, if your tape is too fast
-f : overwrite existing files
-a <time> : autoloader which needs <time> seconds to reload
-A <cmd> : issue command <cmd> to request new volume
-v <level> : set verbose level to <level> (valid values are 0..5)
-q : quiet - do not display the status on stderr
-c : write with synchronous data integrity support
-e : stop processing on any kind of error
-H
--md5 : generate md5 hash of transfered data
--hash <a> : use alogritm <a>, if <a> is 'list' possible algorithms are listed -4 : force use of IPv4
-6 : force use of IPv6
-0 : use IPv4 or IPv6
-V
--version : print version information

Download

Binaries

OpenCSW Package for Solaris

Bug Warning

Versions 20120505 to 20130209 have a bug that can lead to data corruption if you use multiple outputs or one output with hashing.

Sources

Version 20140310 (latest stable) md5=9937d7f666c19a2e6b2499b04dbecc44
Version 20140126 md5=5d6eb4da888fa205fd20ddb04dc0b5d9
Version 20130220 md5=69674ee0a06dbd8fb26c8d5b3e8c8a2d
Version 20130209 md5=5d987374ee8dfe4c152ce082ee07c648
Version 20121111 md5=c5ff2276797bffca68a7510fa20c9f41
Version 20120505 md5=f7fec4cb7525fd403a09a263b9c84934
Version 20110724
Version 20110317
Version 20110119 10 years anniversary release
Version 20101230
Version 20100526
Version 20100327
Version 20091227
Version 20091122
Version 20091110
Version 20091012
Version 20090628
Version 20090215
Version 20090113 (updated package with correct version tag on 2009-01-18)
Version 20090106
Version 20081207
Version 20081113
Version 20081023
Version 20081015
Version 20081014 (WITHDRAWN - DO NOT USE)
Version 20080910 (NEW FEATURE: multiple outputs)
Version 20080507 (previous stable)
Version 20080502
Version 20080329
Version 20080212
Version 20080122
Version 20080104
Version 20070911
Version 20070826
Version 20070518
Version 20070502
Version 20070401
Version 20070317
Version 20060728
Version 20060704
Version 20060626
Version 20060616
Version 20060421
Version 20060324
Version 20060217
Version 20060126
Version 20051006
Version 20051003
Version 20050730
Version 20050726
Version 20050510
Version 20050321
Version 20050125
Version 20011008
Version 20010807
This time no patch as it would be to big...
Version 20010716
Patch for Version 20010716
Version 20010704
Patch for Version 20010704
Version 20010703
Patch for Version 20010703
Version 20010702
Patch for Version 20010702
Version 20010628
Patch for Version 20010628
Version 20010626
Patch for Version 20010626
Version 20010624
Patch for Version 20010624
Version 20010622
Patch for Version 20010622
Version 20010621
Patch for Version 20010621
Version 20010619
Patch for Version 20010619
Version 20010619
Patch for Version 20010619
Version 20010528
Patch for Version 20010528
Version 20010518
Patch for Version 20010518
Version 20010515
Version 20010509
Version 20010507
Version 20010426
Version 20010208
Version 20010124
Version 20010119

LICENSE:

GNU GPL 3

DESCRIPTION:

mbuffer is a replacement for buffer with additional functionality:

TESTING:

to test the program I run an integrity check with gpg

ARCHITECTURES:

This software has been successfully tested under the following operatingsystems:
Thomas Maier-Komor