Simple and efficient reading/writing/modifying of complete files
use File::Slurp;
# read in a whole file into a scalar
my $text = read_file( 'filename' ) ;
# read in a whole file into an array of lines
my @lines = read_file( 'filename' ) ;
# write out a whole file from a scalar
write_file( 'filename', $text ) ;
# write out a whole file from an array of lines
write_file( 'filename', @lines ) ;
# Here is a simple and fast way to load and save a simple config file # made of key=value lines.
my %conf = read_file( $file_name ) =~ /^(\w+)=(.*)$/mg ;
write_file( $file_name, {atomic => 1}, map \*(L"$_=$conf{$_}\n\*(R", keys %conf ) ;
# insert text at the beginning of a file
prepend_file( 'filename', $text ) ;
# in-place edit to replace all 'foo' with 'bar' in file
edit_file { s/foo/bar/g } 'filename' ;
# in-place edit to delete all lines with 'foo' from file
edit_file_lines sub { $_ = '' if /foo/ }, 'filename' ;
# read in a whole directory of file names (skipping . and ..)
my @files = read_dir( '/path/to/dir' ) ;
This module provides subs that allow you to read or write entire files with one simple call. They are designed to be simple to use, have flexible ways to pass in or get the file contents and to be very efficient. There is also a sub to read in all the files in a directory other than \*(C`.\*(C' and \*(C`..\*(C'
These slurp/spew subs work for files, pipes and sockets, stdio, pseudo-files, and the \s-1DATA\s0 handle. Read more about why slurping files is a good thing in the file 'slurp_article.pod' in the extras/ directory.
If you are interested in how fast these calls work, check out the slurp_bench.pl program in the extras/ directory. It compares many different forms of slurping. You can select the I/O direction, context and file sizes. Use the --help option to see how to run it.
This sub reads in an entire file and returns its contents to the caller. In scalar context it returns the entire file as a single scalar. In list context it will return a list of lines (using the current value of $/ as the separator including support for paragraph mode when it is set to '').
my $text = read_file( 'filename' ) ; my $bin = read_file( 'filename' { binmode => ':raw' } ) ; my @lines = read_file( 'filename' ) ; my $lines = read_file( 'filename', array_ref => 1 ) ;
The first argument is the file to slurp in. If the next argument is a hash reference, then it is used as the options. Otherwise the rest of the argument list are is used as key/value options.
If the file argument is a handle (if it is a ref and is an \s-1IO\s0 or \s-1GLOB\s0 object), then that handle is slurped in. This mode is supported so you slurp handles such as \*(C`DATA\*(C' and \*(C`STDIN\*(C'. See the test handle.t for an example that does \*(C`open( '-|' )\*(C' and the child process spews data to the parant which slurps it in. All of the options that control how the data is returned to the caller still work in this case.
If the first argument is an overloaded object then its stringified value is used for the filename and that file is opened. This is a new feature in 9999.14. See the stringify.t test for an example.
By default \*(C`read_file\*(C' returns an undef in scalar contex or a single undef in list context if it encounters an error. Those are both impossible to get with a clean read_file call which means you can check the return value and always know if you had an error. You can change how errors are handled with the \*(C`err_mode\*(C' option.
Speed Note: If you call read_file and just get a scalar return value it is now optimized to handle shorter files. This is only used if no options are used, the file is shorter then 100k bytes, the filename is a plain scalar and a scalar file is returned. If you want the fastest slurping, use the \*(C`buf_ref\*(C' or \*(C`scalar_ref\*(C' options (see below)
\s-1NOTE:\s0 as of version 9999.06, read_file works correctly on the \*(C`DATA\*(C' handle. It used to need a sysseek workaround but that is now handled when needed by the module itself.
You can optionally request that \*(C`slurp()\*(C' is exported to your code. This is an alias for read_file and is meant to be forward compatible with Perl 6 (which will have slurp() built-in).
The options for \*(C`read_file\*(C' are:
binmode
If you set the binmode option, then its value is passed to a call to binmode on the opened handle. You can use this to set the file to be read in binary mode, utf8, etc. See perldoc -f binmode for more.
my $bin_data = read_file( $bin_file, binmode => ':raw' ) ; my $utf_text = read_file( $bin_file, binmode => ':utf8' ) ;
array_ref
If this boolean option is set, the return value (only in scalar context) will be an array reference which contains the lines of the slurped file. The following two calls are equivalent:
my $lines_ref = read_file( $bin_file, array_ref => 1 ) ; my $lines_ref = [ read_file( $bin_file ) ] ;
chomp
If this boolean option is set, the lines are chomped. This only happens if you are slurping in a list context or using the \*(C`array_ref\*(C' option.
scalar_ref
If this boolean option is set, the return value (only in scalar context) will be an scalar reference to a string which is the contents of the slurped file. This will usually be faster than returning the plain scalar. It will also save memory as it will not make a copy of the file to return. Run the extras/slurp_bench.pl script to see speed comparisons.
my $text_ref = read_file( $bin_file, scalar_ref => 1 ) ;
buf_ref
You can use this option to pass in a scalar reference and the slurped file contents will be stored in the scalar. This can be used in conjunction with any of the other options. This saves an extra copy of the slurped file and can lower ram usage vs returning the file. It is usually the fastest way to read a file into a scalar. Run the extras/slurp_bench.pl script to see speed comparisons.
read_file( $bin_file, buf_ref => \$buffer ) ;
blk_size
You can use this option to set the block size used when slurping from an already open handle (like \*STDIN). It defaults to 1MB.
my $text_ref = read_file( $bin_file, blk_size => 10_000_000, array_ref => 1 ) ;
err_mode
You can use this option to control how read_file behaves when an error occurs. This option defaults to 'croak'. You can set it to 'carp' or to 'quiet to have no special error handling. This code wants to carp and then read another file if it fails.
my $text_ref = read_file( $file, err_mode => 'carp' ) ; unless ( $text_ref ) {
# read a different file but croak if not found $text_ref = read_file( $another_file ) ; }
# process ${$text_ref}
This sub writes out an entire file in one call.
write_file( 'filename', @data ) ;
The first argument to \*(C`write_file\*(C' is the filename. The next argument is an optional hash reference and it contains key/values that can modify the behavior of \*(C`write_file\*(C'. The rest of the argument list is the data to be written to the file.
write_file( 'filename', {append => 1 }, @data ) ; write_file( 'filename', {binmode => ':raw'}, $buffer ) ;
As a shortcut if the first data argument is a scalar or array reference, it is used as the only data to be written to the file. Any following arguments in @_ are ignored. This is a faster way to pass in the output to be written to the file and is equivalent to the \*(C`buf_ref\*(C' option of \*(C`read_file\*(C'. These following pairs are equivalent but the pass by reference call will be faster in most cases (especially with larger files).
write_file( 'filename', \$buffer ) ; write_file( 'filename', $buffer ) ;
write_file( 'filename', \@lines ) ; write_file( 'filename', @lines ) ;
If the first argument is a handle (if it is a ref and is an \s-1IO\s0 or \s-1GLOB\s0 object), then that handle is written to. This mode is supported so you spew to handles such as \*STDOUT. See the test handle.t for an example that does \*(C`open( '-|' )\*(C' and child process spews data to the parent which slurps it in. All of the options that control how the data are passed into \*(C`write_file\*(C' still work in this case.
If the first argument is an overloaded object then its stringified value is used for the filename and that file is opened. This is new feature in 9999.14. See the stringify.t test for an example.
By default \*(C`write_file\*(C' returns 1 upon successfully writing the file or undef if it encountered an error. You can change how errors are handled with the \*(C`err_mode\*(C' option.
The options are:
binmode
If you set the binmode option, then its value is passed to a call to binmode on the opened handle. You can use this to set the file to be read in binary mode, utf8, etc. See perldoc -f binmode for more.
write_file( $bin_file, {binmode => ':raw'}, @data ) ; write_file( $bin_file, {binmode => ':utf8'}, $utf_text ) ;
perms
The perms option sets the permissions of newly-created files. This value is modified by your process's umask and defaults to 0666 (same as sysopen).
\s-1NOTE:\s0 this option is new as of File::Slurp version 9999.14;
buf_ref
You can use this option to pass in a scalar reference which has the data to be written. If this is set then any data arguments (including the scalar reference shortcut) in @_ will be ignored. These are equivalent:
write_file( $bin_file, { buf_ref => \$buffer } ) ; write_file( $bin_file, \$buffer ) ; write_file( $bin_file, $buffer ) ;
atomic
If you set this boolean option, the file will be written to in an atomic fashion. A temporary file name is created by appending the pid ($$) to the file name argument and that file is spewed to. After the file is closed it is renamed to the original file name (and rename is an atomic operation on most \s-1OS\s0's). If the program using this were to crash in the middle of this, then the file with the pid suffix could be left behind.
append
If you set this boolean option, the data will be written at the end of the current file. Internally this sets the sysopen mode flag O_APPEND.
write_file( $file, {append => 1}, @data ) ;
You can import append_file and it does the same thing.
no_clobber
If you set this boolean option, an existing file will not be overwritten.
write_file( $file, {no_clobber => 1}, @data ) ;
err_mode
You can use this option to control how \*(C`write_file\*(C' behaves when an error occurs. This option defaults to 'croak'. You can set it to 'carp' or to 'quiet' to have no error handling other than the return value. If the first call to \*(C`write_file\*(C' fails it will carp and then write to another file. If the second call to \*(C`write_file\*(C' fails, it will croak.
unless ( write_file( $file, { err_mode => 'carp', \$data ) ;
# write a different file but croak if not found write_file( $other_file, \$data ) ; }
This sub is just a typeglob alias to write_file since write_file always overwrites an existing file. This sub is supported for backwards compatibility with the original version of this module. See write_file for its \s-1API\s0 and behavior.
This sub will write its data to the end of the file. It is a wrapper around write_file and it has the same \s-1API\s0 so see that for the full documentation. These calls are equivalent:
append_file( $file, @data ) ; write_file( $file, {append => 1}, @data ) ;
This sub writes data to the beginning of a file. The previously existing data is written after that so the effect is prepending data in front of a file. It is a counterpart to the append_file sub in this module. It works by first using \*(C`read_file\*(C' to slurp in the file and then calling \*(C`write_file\*(C' with the new data and the existing file data.
The first argument to \*(C`prepend_file\*(C' is the filename. The next argument is an optional hash reference and it contains key/values that can modify the behavior of \*(C`prepend_file\*(C'. The rest of the argument list is the data to be written to the file and that is passed to \*(C`write_file\*(C' as is (see that for allowed data).
Only the \*(C`binmode\*(C' and \*(C`err_mode\*(C' options are supported. The \*(C`write_file\*(C' call has the \*(C`atomic\*(C' option set so you will always have a consistant file. See above for more about those options.
\*(C`prepend_file\*(C' is not exported by default, you need to import it explicitly.
use File::Slurp qw( prepend_file ) ; prepend_file( $file, $header ) ; prepend_file( $file, \@lines ) ; prepend_file( $file, { binmode => 'raw:'}, $bin_data ) ;
These subs read in a file into $_, execute a code block which should modify $_ and then write $_ back to the file. The difference between them is that \*(C`edit_file\*(C' reads the whole file into $_ and calls the code block one time. With \*(C`edit_file_lines\*(C' each line is read into $_ and the code is called for each line. In both cases the code should modify $_ if desired and it will be written back out. These subs are the equivalent of the -pi command line options of Perl but you can call them from inside your program and not fork out a process. They are in @EXPORT_OK so you need to request them to be imported on the use line or you can import both of them with:
use File::Slurp qw( :edit ) ;
The first argument to \*(C`edit_file\*(C' and \*(C`edit_file_lines\*(C' is a code block or a code reference. The code block is not followed by a comma (as with grep and map) but a code reference is followed by a comma. See the examples below for both styles. The next argument is the filename. The last argument is an optional hash reference and it contains key/values that can modify the behavior of \*(C`prepend_file\*(C'.
Only the \*(C`binmode\*(C' and \*(C`err_mode\*(C' options are supported. The \*(C`write_file\*(C' call has the \*(C`atomic\*(C' option set so you will always have a consistant file. See above for more about those options.
Each group of calls below show a Perl command line instance and the equivalent calls to \*(C`edit_file\*(C' and \*(C`edit_file_lines\*(C'.
perl -0777 -pi -e 's/foo/bar/g' filename use File::Slurp qw( edit_file ) ; edit_file { s/foo/bar/g } 'filename' ; edit_file sub { s/foo/bar/g }, 'filename' ; edit_file \&replace_foo, 'filename' ; sub replace_foo { s/foo/bar/g }
perl -pi -e '$_ = "" if /foo/' filename use File::Slurp qw( edit_file_lines ) ; use File::Slurp ; edit_file_lines { $_ = '' if /foo/ } 'filename' ; edit_file_lines sub { $_ = '' if /foo/ }, 'filename' ; edit_file \&delete_foo, 'filename' ; sub delete_foo { $_ = '' if /foo/ }
This sub reads all the file names from directory and returns them to the caller but \*(C`.\*(C' and \*(C`..\*(C' are removed by default.
my @files = read_dir( '/path/to/dir' ) ;
The first argument is the path to the directory to read. If the next argument is a hash reference, then it is used as the options. Otherwise the rest of the argument list are is used as key/value options.
In list context \*(C`read_dir\*(C' returns a list of the entries in the directory. In a scalar context it returns an array reference which has the entries.
err_mode
If the \*(C`err_mode\*(C' option is set, it selects how errors are handled (see \*(C`err_mode\*(C' in \*(C`read_file\*(C' or \*(C`write_file\*(C').
keep_dot_dot
If this boolean option is set, \*(C`.\*(C' and \*(C`..\*(C' are not removed from the list of files.
my @all_files = read_dir( '/path/to/dir', keep_dot_dot => 1 ) ;
prefix
If this boolean option is set, the string \*(L"$dir/\*(R" is prefixed to each dir entry. This means you can directly use the results to open files. A common newbie mistake is not putting the directory in front of entries when opening themn.
my @paths = read_dir( '/path/to/dir', prefix => 1 ) ;
These are exported by default or with use File::Slurp qw( :std ) ;
read_file write_file overwrite_file append_file read_dir
These are exported with use File::Slurp qw( :edit ) ;
edit_file edit_file_lines
You can get all subs in the module exported with use File::Slurp qw( :all ) ;
Same as Perl.
An article on file slurping in extras/slurp_article.pod. There is also a benchmarking script in extras/slurp_bench.pl.
If run under Perl 5.004, slurping from the \s-1DATA\s0 handle will fail as that requires B.pm which didn't get into core until 5.005.
Uri Guttman, <uri \s-1AT\s0 stemsystems \s-1DOT\s0 com>