n!ckb > utilities
> cpiotool
cpiotool - wrapper for cpio
cpiotool -c <config file>
cpiotool --help
cpiotool -v
cpiotool [ --cpio <cpio binary> ]
[ --level <0-9> ] [ --dd <dd binary> ]
[ -h <remote host> ] [ -b <backup root dir> ]
([ -d <device> ] | [ -f <file> ])
[ --block-size <n> ] [ --reset-atime ]
[ --logdir <logdir>] [ --keep-logs <0-99999> ]
[ --rsh <rsh|ssh binary> ] [ --set <1-99999> ]
[ --maxsize <n> ] [ --ziplog [ <compression util> ] ]
[ --confdir <config dir> ]
[ --header <bin|odc|newc|crc|tar|ustar|ascii> ]
[ --notify <addr@domain> [ --notify <addr@domain> ] ... ]
[ --verbose ] [ --debug ]
cpiotool is a config-file-based wrapper for cpio(1L). the wrapper
combines several commonly-used cpio copy-out options with additional options
for supporting multilevel incremental backups, email notification using
mail(1), logging, and include/exclude lists. options specified on the
command-line override config-file parameters. include/exclude lists are only
available using the config file.
cpiotool displays usage and exits if no options are specified.
specifying a single option such as --verbose will start a level 0 backup
from the root directory with suitable defaults:
cpio -ov -H newc --block-size=128 -O /dev/rmt/0
- -c
-
absolute path of config file. see PARAMETERS.
- --help
-
print usage and exit.
- -v
-
print version and exit.
- --cpio
-
absolute path of cpio binary. if not specified, cpiotool will search
/usr/local/bin, /usr/bin, and /bin.
- --level
-
backup level 0-9, similar to dump(8). level 0 is a full backup. A level
number above 0 specifies an incremental backup of files modified more
recently than the highest-numbered previous level backup. defaults to 0.
- --dd
-
absolute path of
dd(1) binary. dd is used for remote writes if GNU cpio is
not detected.
- -h
-
remote host. name of remote host to write archive. the host must be
accessible via
rsh(1) or a drop-in replacement such as ssh(1). GNU cpio
only recognizes rsh, so the replacement should be called 'rsh' if GNU cpio
is used.
- -b
-
backup root directory. top-level directory where cpiotool recursively
searches for files to be archived. defaults to '/'.
- -d
-
device file to write archive. defaults to '/dev/rmt/0'. mutually exclusive
with -f.
- -f
-
absolute path of disk file to write archive. mutually exclusive with -d.
- --block-size
-
cpio option to set I/O block size to (n * 512) bytes. must be positive
integer. defaults to 128, i.e. 65536 bytes.
- --reset-atime
-
reset file access time after reading. default behavior updates atime
as file is read for backup.
- --logdir
-
absolute path of directory to write backup log. defaults to '/var/log/backup'.
- --keep-logs
-
number of days, 0-99999, to keep logs in this tapeset. logs in this
tapeset older than --keep-logs days are removed. setting this number to
0 disables logging. default is 365.
- --rsh
-
absolute path of alternate remote transport:
rsh(1) or ssh(1). alternate
remote transport is used if GNU cpio is not detected.
- --set
-
tapeset number. arbitrary integer, 1-99999, used to identify file groupings
that share the same incremental backup data and logfiles. tapeset number
determines which timestamp is used to set maximum file age for files in the
same backup group. log files are also named by tapeset number. defaults to 1.
- --maxsize
-
maximum archive size in MB, specified as integer or floating-point.
cpiotool keeps a running byte-count and uses this value to determine when
to request additional media. when using native cpio headers, cpiotool
subtracts a full block from the maxsize value for end-of-archive padding,
124 bytes for the final end-of-archive trailer, 110 bytes per file plus the
length of the filename for each header, and 3 bytes per file to allow for
each trailer. when using tar or ustar headers, cpiotool subtracts 512
bytes per file plus the length of the filename from the maxsize value to
allow for each header, and subtracts an additional number of bytes from
the maxsize value to allow for padding up to each block-boundry. the maxsize
value is only used when writing to a device. if set to 0 or unspecified,
the cpio binary will send media requests to the controlling terminal.
- --ziplog
-
if specified backup log will be compressed with compress(1). --ziplog
takes an optional argument of the absolute path to an alternate compression
utility, e.g. '--ziplog /usr/local/bin/bzip2'.
- --confdir
-
configuration directory. cpiotool writes incremental backup timestamps
in this directory. defaults to '/share/backup/conf'.
- --header
-
cpio option to set archive header type. must be one of bin, odc, newc,
crc, tar, ustar or ascii. see
cpio(1L) for details. defaults to
newc, or ascii if newc is unavailable.
- --notify
-
email address of recipient to send error report. cpiotool uses '/bin/mail'
to send the message, so the system must have a properly configured MTA for this
option to work. more than one address can be specified using multiple
--notify options.
- --verbose
-
display verbose output to STDERR.
- --debug
-
display very verbose output to STDERR.
the config file is divided in three sections. the [CONF] section lists
general configuration options. [DIRLIST] is a list of directories relative
to the backup root directory to be archived. [EXCLUDES] is a list of
directories relative to the backup root directory that should be excluded
from the archive. comments begin with '#'. config file parameters in the
[CONF] section can be overridden on the command-line. config file defaults
are the same as command-line defaults. see OPTIONS.
the [CONF] section accepts the following keys. values are specified after
whitespace.
- root-dir
-
top level directory where cpiotool recursively searches for files to be
archived.
- cpio
-
absolute path of cpio.
- level
-
backup level 0-9.
- dd
-
absolute path of dd binary. used if GNU cpio is not detected.
- remote-host
-
name of remote host to write archive.
- device
-
device file to write archive. mutually exclusive with file.
- file
-
disk file to write archive. mutually exclusive with device.
- block-size
-
positive integer that sets I/O block size to (n * 512) bytes.
- reset-atime
-
set to 1 to reset file access time after file is read by cpio.
- logdir
-
directory to write backup log.
- set
-
tapeset number, 1-99999.
- keep-logs
-
number of days to keep backup logs of specified tapeset, 0-99999. setting to
0 disables logging.
- rsh
-
absolute path of alternate transport. used if GNU cpio is not detected.
- maxsize
-
floating-point number that sets maximum archive size in MB. used only if
archive is a device. set to 0 to disable.
- conf-dir
-
directory where cpiotool writes timestamp files.
- header
-
sets cpio header type. must be one of bin, odc, newc, crc, tar,
ustar or ascii.
- notify
-
email
address(es) to send error report. requires '/bin/mail' and properly
configured MTA. multiple addresses can be separated by whitespace.
- ziplog
-
absolute path of compression utility used to compress backup log. if set to
1 cpiotool will use compress.
- verbose
-
set to 1 to display verbose output to STDERR.
- debug
-
set to 1 to display very verbose output to STDERR.
list of subdirectories relative to the root-dir to be backed up. if this
section is empty cpiotool will archive all files and directories under
the root-dir. note, directories must be specified relative to the
root-dir. using absolute paths will break the file-finding routine.
list of subdirectories relative to the root-dir to be excluded from the
archive. if this section is empty cpiotool will not exclude any files.
note, directories must be specified relative to the root-dir.
# cpiotool level 2 config
[CONF]
root-dir /home/nickb
cpio /usr/local/bin/cpio
level 2
#remote-host
device /dev/fd0
#file
block-size 20
logdir /var/log/backup
set 3
keep-logs 14
conf-dir /usr/local/backup/conf
header ustar
notify ops@domain.org backup@domain.org
maxsize 1.44
ziplog /usr/local/bin/bzip2
verbose 1
[DIRLIST]
# relative to root-dir
GNUstep/Library/AfterStep
site
[EXCLUDE]
GNUstep/Library/AfterStep/non-configurable
GNUstep/Library/AfterStep/start/Applications/Editors
trash
unused
# end config
the cpio binary supports archives that span multiple volumes. cpiotool
implements this in one of 3 ways, provided the archive is written to a
device:
-
if --maxsize is unspecified, the cpio binary will detect end of media
and send a request to the controlling terminal. this is easiest but requires
the backup to be run manually from a terminal. it is also time-consuming
and error-prone in the event a restore is needed, as each tape in the series
must be read in order to restore a file at the end of the archive. if
one of the tapes in the set is corrupt, it's possible that all data on
subsequent tapes in the archive will be inaccessible.
-
if --maxsize is specified but --notify is unspecified or a mailer
is not found, cpiotool will estimate the total size of all cpio headers
and keep a running total of the number of bytes written to the device.
when the number approaches --maxsize, IO to the cpio binary is closed
and a request for media is printed to standard out. sending a HUP signal
to cpiotool continues the backup. this option still requires the
backup to be run manually from a terminal, but has the advantage of writing
archives separately to individual tapes rather than writing one large archive
across multiple tapes. this decreases time required for restores and reduces
the potential for data loss.
-
if --maxsize and --notify are specified, and a mailer is found,
cpiotool writes archives in the same manner as 2 above, but sends
media requests to the email address specified in --notify. this
allows some degree of automation as no interaction with a terminal is
required.
not supported (yet). eventually cpiotool will store tape information in
a mysql database, but this is a project for the distant future.
the simplified concept of tapesets, along with the use of timestamp files,
determines which files to archive for incremental (level 1-9) backups.
if a configuration directory is specified cpiotool writes a timestamp file
called level[level]-s[set].timestamp. the last modification time of every
file in each directory in the [DIRLIST] is compared to the last
modification time of the timestamp file to determine if the file should be
archived. timestamp files are simply zero-length files with a naming
convention recognized by cpiotool. creating, removing, or changing the
modification times of timestamp files will affect the behavior of
cpiotool.
backup script that uses cpio
perl 5.6.0 or newer
cpiotool v0.65a, Copyright (C) 2001 Assentive Solutions
Nick Balthaser <nb9001@yahoo.com>
freebsd
solaris
-
if the remote transport (rsh or ssh) hangs up while using non-GNU cpio,
IO stops and cpiotool exits.
-
when running in a terminal in the background and writing to a remote host with
dd, sending SIGINT by pressing ^C can have unpredictable
results, even if cpiotool is running under a subshell or nohup.
-
error ``dd: unexpected short write, wrote 0 bytes, expected <n>'' while writing
to remote device. the remote dd detected end of media and quit prematurely.
this can mean the --maxsize value is greater than the
capacity of the media. now that
_padTrailer() is in place this
should happen far less often. adjusting --maxsize can
compensate for this in the interim.
-
no error checking on the --maxsize value. if an incorrect value is
specified, the cpio binary and/or dd may reach end-of-media before the
byte-counter. this will cause dd to hang up abruptly on remote backups
or conflicting media requests on local backups.
-
if a single file being backed up is larger than the media capacity,
the cpio binary could reach end-of-media before the byte-counter and
conflicting media requests would be sent. currently an exception is thrown
and the script exits if this happens. the only solution currently
is to disable --maxsize by setting it to 0 or leaving it unspecified.
this allows the cpio binary to create a single archive that spans multiple
tapes, which can be unreliable. see MULTIPLE-VOLUME ARCHIVES above.
-
the number of bytes subtracted from the --maxsize value to allow for
headers is an (improving) rough estimate and may be inaccurate. in the
meantime adjusting --maxsize can compensate.
-
the log-remover might not recognize logs with a compressed suffix (e.g.
.gz, .bz2).
-
for full functionality cpiotool relies on the presence of mail or mailx,
dd, and rsh or ssh. dd always exits on SIGINT. it's not always possible
to trap SIGINT.
-
no database interface, reliance on easily-manipulated timestamp files.
UNIX/System_administration
[top] [n!ckb]
[links]
[reading]
[utilities]
[resume]
[minutiae]
$Id: cpiotool.htm,v 1.10 2003/05/07 00:12:19 nickb Exp $