22 Sept 2011
Net::LDAP::Security
Net::LDAP::Security
LDAP servers are vulnerable to LDAP query injection and DNS hijacking may lead to your query landing on the attacker's server. Attacker can sniff your username/password through your LDAP connection. Hacker may sniff your confidential information through LDAP connection.
So, How to prevent this? You can use Net::LDAPS and LDAP over TLS.
To know more about various security threats on LDAP and how to overcome those, read the document at Net::LDAP::Security
20 Sept 2011
Security Issues in Perl Scripts
Security Issues in Perl Scripts
system() and exec()
system() and exec() are potentially vulnerable to parameter tampering threat. Suppose if you have a CGI application which reads the the username parameter and displays the file he created. If you directly pass the parameter input to the system call it is a biggest security threat.
$user = param("user");
@out = system("cat /usr/upload/$user");
Imagine if some attacker is changing the param from "foe" to "test; cat /etc/passwd", your script will output the content of the user file along with the contents of /etc/passwd
As far as security is concerned, everything stated above with regard to the system() function applies to exec() too.
setuid scripts
Normally a Perl program runs with the privileges of the user who executed it. By making a script setuid, its effective user ID can be set to one that has access to resources to which the actual user does not (viz., to the owner ID of the file containing the program). The passwd program for example uses setuid to acquire writing permission to the system password file, thus allowing users to change their own passwords. Since programs that are executed via a CGI interface run with the privileges of the user who runs the web server (usually this is user 'nobody', who has very limited privileges), CGI programmers are often tempted to use the setuid technique to let their scripts perform tricks that they otherwise couldn't. This can be useful, but it can also be very dangerous. For one thing, if an attacker finds a way to exploit a weakness in the script, they won't only gain access to the system, but they will also have it with the privileges of the effective UID of that script (often the 'root' UID).
To avoid this, Perl programs should set the effective UID and GID to the real UID and GID of the process before any file manipulations:
$> = $< # set effective user ID to real UID.
$) = $( # set effective group ID to real GID.
and CGI scripts should always run with the lowest possible privilege.
Beware that just being careful in what you do inside your setuid script doesn't always solve the problem. Some operating systems have bugs in the kernel that make setuid scripts inherently insecure. For this, and other reasons, Perl automatically switches to a special security mode (taint mode) when it runs setuid or setgid scripts. We will discuss taint mode in our next article.
rand()
Generating random numbers on deterministic machines is a nontrivial problem. In security critical applications, random numbers are used intensely for many important tasks ranging from password generation to cryptography. For such purposes, it is vital that the generated numbers are as close to truly random as possible, making it difficult (but never impossible) for an attacker to predict future numbers generated by the algorithm. The Perl rand() function simply calls the corresponding rand(3) function from the standard C library. This routine is not very secure. The C rand() function generates a sequence of pseudorandom numbers based on some initial value called the seed. Given the same seed, two different instances of a program utilizing rand() will produce the same random values. In many implementations of C, and in all version of Perl before 5.004, if a seed is not explicitly specified, it is computed from the current value of the system timer, which is anything but random. Having some information about values produced by rand() at a given point and a sufficient amount of time, any self-respecting cracker can accurately predict the sequence of numbers that rand() will generate next, thus obtaining key knowledge necessary to compromise a system.
One (partial) solution to the rand() problem is to use one of the built-in random number generators on Linux systems -- /dev/random and /dev/urandom. Those are better sources of randomness then the standard library rand() function, but like anything else, they have their own imperfections. The difference between the two devices is that /dev/random stops supplying random numbers when its entropy pool runs out of randomness while /dev/urandom uses cryptography to generate new numbers when the entropy pool runs out. Another solution is to use a secure implementation of one of the more complicated cryptographic random number generators such as Yarrow.
system() and exec()
system() and exec() are potentially vulnerable to parameter tampering threat. Suppose if you have a CGI application which reads the the username parameter and displays the file he created. If you directly pass the parameter input to the system call it is a biggest security threat.
$user = param("user");
@out = system("cat /usr/upload/$user");
Imagine if some attacker is changing the param from "foe" to "test; cat /etc/passwd", your script will output the content of the user file along with the contents of /etc/passwd
As far as security is concerned, everything stated above with regard to the system() function applies to exec() too.
setuid scripts
Normally a Perl program runs with the privileges of the user who executed it. By making a script setuid, its effective user ID can be set to one that has access to resources to which the actual user does not (viz., to the owner ID of the file containing the program). The passwd program for example uses setuid to acquire writing permission to the system password file, thus allowing users to change their own passwords. Since programs that are executed via a CGI interface run with the privileges of the user who runs the web server (usually this is user 'nobody', who has very limited privileges), CGI programmers are often tempted to use the setuid technique to let their scripts perform tricks that they otherwise couldn't. This can be useful, but it can also be very dangerous. For one thing, if an attacker finds a way to exploit a weakness in the script, they won't only gain access to the system, but they will also have it with the privileges of the effective UID of that script (often the 'root' UID).
To avoid this, Perl programs should set the effective UID and GID to the real UID and GID of the process before any file manipulations:
$> = $< # set effective user ID to real UID.
$) = $( # set effective group ID to real GID.
and CGI scripts should always run with the lowest possible privilege.
Beware that just being careful in what you do inside your setuid script doesn't always solve the problem. Some operating systems have bugs in the kernel that make setuid scripts inherently insecure. For this, and other reasons, Perl automatically switches to a special security mode (taint mode) when it runs setuid or setgid scripts. We will discuss taint mode in our next article.
rand()
Generating random numbers on deterministic machines is a nontrivial problem. In security critical applications, random numbers are used intensely for many important tasks ranging from password generation to cryptography. For such purposes, it is vital that the generated numbers are as close to truly random as possible, making it difficult (but never impossible) for an attacker to predict future numbers generated by the algorithm. The Perl rand() function simply calls the corresponding rand(3) function from the standard C library. This routine is not very secure. The C rand() function generates a sequence of pseudorandom numbers based on some initial value called the seed. Given the same seed, two different instances of a program utilizing rand() will produce the same random values. In many implementations of C, and in all version of Perl before 5.004, if a seed is not explicitly specified, it is computed from the current value of the system timer, which is anything but random. Having some information about values produced by rand() at a given point and a sufficient amount of time, any self-respecting cracker can accurately predict the sequence of numbers that rand() will generate next, thus obtaining key knowledge necessary to compromise a system.
One (partial) solution to the rand() problem is to use one of the built-in random number generators on Linux systems -- /dev/random and /dev/urandom. Those are better sources of randomness then the standard library rand() function, but like anything else, they have their own imperfections. The difference between the two devices is that /dev/random stops supplying random numbers when its entropy pool runs out of randomness while /dev/urandom uses cryptography to generate new numbers when the entropy pool runs out. Another solution is to use a secure implementation of one of the more complicated cryptographic random number generators such as Yarrow.
13 Sept 2011
Perl 5.10 - Switch
Perl 5.10 - Switch
Perl 5.10 introduces a native switch statement into the language. Like other features in Perl 5.10, we can enable it with the
Finally, we can also set a default case, for when nothing has matched, although we have not shown it in our example.
It should be noted that
Perl 5.10 introduces a native switch statement into the language. Like other features in Perl 5.10, we can enable it with the
use feature
pragma.use feature qw(switch);It's also possible to enable all Perl 5.10 features with any of the following:
use feature qw(:5.10);
use 5.10.0; # Requires perl v5.10.0 or later.
use 5.010; # Same, with nicer errors on older versions.
Perl's new switch feature is best demonstrated with an example.
Let's take the classic guessing game, where the computer picks a number
and our user needs to try and guess it. use strict;
use warnings;
use feature qw(switch say);
# Pick our random number between 1 and 100
my $secret = int(rand 100)+1;
# An array of numbers guessed thus far
my @guessed;
say "Guess my number between 1-100";
# Get each guess from the user
while (my $guess = <STDIN>) {
chomp $guess;
# Check their guess using given/when
given($guess) {
when (/\D/) { say "Give me an integer"; }
when (@guessed) { say "You've tried that"; }
when ($secret) { say "Just right!"; last; }
when ($_ < $secret) { say "Too low"; continue; }
when ($_ > $secret) { say "Too high"; continue; }
# record the guess they've made
push(@guessed,$_);
}
}
The heart of our program is the given/when block, which is
enabled by the use of use feature qw(switch) at the top of our code.
Let's look at each part in detail:given($guess)- A
givenconstruct marks the start of our switch logic. It has the effect as assigning$_to$guessfor the duration of ourgivenblock. The changes to$_are not visible outside of thegivenblock (ie, they are lexical in scope). when (/\D/)whenprovided with a regular expression checks to see if$_matches that regular expression. In our case, if our input contains non-digit characters, then we have something that doesn't look like an integer, and execute the associated block.- After a successful
whenmatch Perl will automatically break out of thegivenblock unless told otherwise. In our case this repeats the loop asking the user for another guess. when (@guessed)- In this second test we're asking whether
$guessappears as a value in the@guessed. If it does we inform the user and go onto their next guess. when ($secret)- This is a direct comparison. Is
$guessthe same value as$secret?. If so the player has guessed correctly! Usinglastallows us to break out of ourwhileloop, ending the game. when ($_ < $secret)andwhen ($_ > $secret)- These final two tests are simple comparisons. Remember that
$_contains the item we weregiven.
We've usedcontinuefor these statements to say that Perl should not break out of ourgivenblock on a successful match. This means that if the user guesses too low or too high, we will eventually evaluate the line:
push(@guesses,$guess);
which remembers the guess as one we've already seen.
when with hashes (is $_ a key in the hash),
subroutines (does the subroutine return true with $_ as an argument)
and strings (tests for string equality). Furthermore, our
given expression need not be merely a number or a string, we can also
compare arrays, and hashes if we want to.Finally, we can also set a default case, for when nothing has matched, although we have not shown it in our example.
foreach / when
If you've enableduse feature qw(switch then you can also use
a foreach/when construct. Here's an example of tallying up
the number of times we see cool things in a list: use feature qw(switch);
foreach (@cool_things) {
when (/pirate/) { $pirate++ }
when (/ninja/) { $ninja++ }
when (/robot/) { $robot++ }
say "$_ doesn't look cool...";
}
If a when is activated, it automatically goes onto the next
iteration of the foreach loop. Just like given/while, we can
use the continue keyword to continue examining later options after
a successful match.It should be noted that
foreach/when only works when using Perl's
default $_ variable for iteration.Lexical $_
In Perl 5.10 we can write:my $_;This is allows us to do anything we like with
$_, but without the
possibility of changing $_ for code outside our current block. It's
strongly recommended for subroutines using $_ in Perl 5.10 to
avoid accidentally changing $_ for your caller.
16 Aug 2011
Compare two ldap branches
ldapcmp
ldapcmp is the tool which can be used to compare directory branches from two servers. This will compare all the entries in the branch including all the attributes in the branch
usage
ldapcmp -h hostA -h hostB -s sub -b "branch_path"
usage: ldapcmp -b basedn [options] [attributes...]
where:
basedn base dn for search
(if the environment variable LDAP_BASEDN is set,
then the -b flag is not required)
options:
-s scope one of base, one, or sub (default is sub)
-l timelim time limit (in seconds) for search (default is no limit)
-z sizelim size limit (in entries) for search (default is no limit)
-n show what would be done but don't actually do it
-v run in verbose mode (diagnostics to standard output)
-h host LDAP server1 name or IP address (default: localhost)
-p port LDAP server1 TCP port number (default: 389)
-h host LDAP server2 name or IP address (default: localhost)
-p port LDAP server2 TCP port number (default: 389)
-V n LDAP protocol version number (2 or 3; default: 3)
-ZZ start TLS request
-ZZZ enforce start TLS request (successful server response required)
-Z make an SSL-encrypted connection
-P pathname path to SSL certificate database (default: current directory)
-N name of certificate to use for SSL client authentication
-K pathname path to key database to use for SSL client authentication
(default: path to certificate database provided with -P option)
-m pathname path to security module database
-W SSL key password
-W - prompt for SSL key password
-I file SSL key password 'file' containing token:password pair/s
-Q [token][:certificate name] PKCS 11
-3 check hostnames in SSL certificates
-D binddn bind dn
-w passwd bind passwd (for simple authentication)
-w - prompt for bind passwd (for simple authentication)
-j file read bind passwd from 'file' (for simple authentication)
-E ask server to expose (report) bind identity
-g do not send a password policy request control
-R do not automatically follow referrals
-O limit maximum number of referral hops to traverse (default: 5)
-M manage references (treat them as regular entries)
-0 ignore LDAP library version mismatches
-i charset character set for command line input (default taken from locale)
use '-i 0' to override locale settings and bypass any conversions
-k do not convert password to utf8 (use default from locale)
-Y proxyid proxied authorization id,
e.g, dn:uid=bjensen,dc=example,dc=com
-H display usage information
-J controloid[:criticality[:value|::b64value|:
Example
ldapcmp -h exmaple.host.com -h example1.host.com -s sub -b "o=176,ou=grpStructure,o=com"
o=176,ou=grpStructure,c=com
different: type
1: SA 55
2: PA 55
ldapcmp is the tool which can be used to compare directory branches from two servers. This will compare all the entries in the branch including all the attributes in the branch
usage
ldapcmp -h hostA -h hostB -s sub -b "branch_path"
usage: ldapcmp -b basedn [options] [attributes...]
where:
basedn base dn for search
(if the environment variable LDAP_BASEDN is set,
then the -b flag is not required)
options:
-s scope one of base, one, or sub (default is sub)
-l timelim time limit (in seconds) for search (default is no limit)
-z sizelim size limit (in entries) for search (default is no limit)
-n show what would be done but don't actually do it
-v run in verbose mode (diagnostics to standard output)
-h host LDAP server1 name or IP address (default: localhost)
-p port LDAP server1 TCP port number (default: 389)
-h host LDAP server2 name or IP address (default: localhost)
-p port LDAP server2 TCP port number (default: 389)
-V n LDAP protocol version number (2 or 3; default: 3)
-ZZ start TLS request
-ZZZ enforce start TLS request (successful server response required)
-Z make an SSL-encrypted connection
-P pathname path to SSL certificate database (default: current directory)
-N name of certificate to use for SSL client authentication
-K pathname path to key database to use for SSL client authentication
(default: path to certificate database provided with -P option)
-m pathname path to security module database
-W SSL key password
-W - prompt for SSL key password
-I file SSL key password 'file' containing token:password pair/s
-Q [token][:certificate name] PKCS 11
-3 check hostnames in SSL certificates
-D binddn bind dn
-w passwd bind passwd (for simple authentication)
-w - prompt for bind passwd (for simple authentication)
-j file read bind passwd from 'file' (for simple authentication)
-E ask server to expose (report) bind identity
-g do not send a password policy request control
-R do not automatically follow referrals
-O limit maximum number of referral hops to traverse (default: 5)
-M manage references (treat them as regular entries)
-0 ignore LDAP library version mismatches
-i charset character set for command line input (default taken from locale)
use '-i 0' to override locale settings and bypass any conversions
-k do not convert password to utf8 (use default from locale)
-Y proxyid proxied authorization id,
e.g, dn:uid=bjensen,dc=example,dc=com
-H display usage information
-J controloid[:criticality[:value|::b64value|:
Example
ldapcmp -h exmaple.host.com -h example1.host.com -s sub -b "o=176,ou=grpStructure,o=com"
o=176,ou=grpStructure,c=com
different: type
1: SA 55
2: PA 55
27 Jun 2011
Comment your code with Smart::Comments
Normally we comment our code so that humans (including ourselves) can
understand it later on. However with ``Smart::Comments'' it becomes
possible to have comments which are not only useful in comprehending our
code, but also in *debugging* our code.
With ``Smart::Comments'' enabled, when a special comment sequence is
seen (``###'' by default) debugging output is produced:
use Smart::Comments;
### [] Running backups...
my $backup_dir = get_backup_host();
### Backing up files to: $backup_dir
foreach my $file (@manifest) { ### Copying files... % done
backup($file, $backup_dir);
}
### [] Backup complete...
With ``Smart::Comments'' enabled, output similar to the following would
be seen:
### [Fri Feb 25 12:14:34 2011] Running backups...
### Backing up files to: '/mnt/backup'
Copying files........ 64% done
### [Fri Feb 25 12:17:57 2011] Done...
One of great advantages of ``Smart::Comments'' is that the ``use
Smart::Comments'' line can simply be removed (or commented away) when
done, and all our smart comments are then simply ignored by Perl.
There's no requirement that ``Smart::Comments'' be installed on the
production systems, and no risk that that it may interfere with your
code.
You can read more about ``Smart::Comments'' on the
CPAN .
understand it later on. However with ``Smart::Comments'' it becomes
possible to have comments which are not only useful in comprehending our
code, but also in *debugging* our code.
With ``Smart::Comments'' enabled, when a special comment sequence is
seen (``###'' by default) debugging output is produced:
use Smart::Comments;
### [
my $backup_dir = get_backup_host();
### Backing up files to: $backup_dir
foreach my $file (@manifest) { ### Copying files... % done
backup($file, $backup_dir);
}
### [
With ``Smart::Comments'' enabled, output similar to the following would
be seen:
### [Fri Feb 25 12:14:34 2011] Running backups...
### Backing up files to: '/mnt/backup'
Copying files........ 64% done
### [Fri Feb 25 12:17:57 2011] Done...
One of great advantages of ``Smart::Comments'' is that the ``use
Smart::Comments'' line can simply be removed (or commented away) when
done, and all our smart comments are then simply ignored by Perl.
There's no requirement that ``Smart::Comments'' be installed on the
production systems, and no risk that that it may interfere with your
code.
You can read more about ``Smart::Comments'' on the
CPAN .
26 Jun 2011
local::lib
Work with local modules via local::lib
``local::lib'' helps solve the problem when a Perl package on your
machine isn't the version you need it to be. It allows you to create
your own directory of Perl packages into your own, or your team's
directory.
use local::lib;
# or
use local::lib '/path/to/team/directory';
``local::lib'' also sets up some useful shell commands which add our
preferred include directories to ``PERL5LIB'' in our environment. If our
code is not running under taint, we don't even need to add the ``use''
line to our code to use these newly installed modules.
``local::lib'' helps solve the problem when a Perl package on your
machine isn't the version you need it to be. It allows you to create
your own directory of Perl packages into your own, or your team's
directory.
use local::lib;
# or
use local::lib '/path/to/team/directory';
``local::lib'' also sets up some useful shell commands which add our
preferred include directories to ``PERL5LIB'' in our environment. If our
code is not running under taint, we don't even need to add the ``use''
line to our code to use these newly installed modules.
App::perlbrew
Experimenting with new versions of Perl and modules
Many developers have a chicken-and-egg problem when it comes to
upgrading their version of Perl. On one hand, the system version of Perl
cannot be upgraded because it might break existing scripts, on the the
other hand it has been difficult to install your own version of Perl to
see whether an upgrade is safe. The same happens with modules.
Pick your Perl version with App::perlbrew
Use ``App::perlbrew'' to install and switch between multiple versions of
Perl. ``perlbrew'' is easy to install, even if you don't have system
administration privileges, and very easy to use. Once you've installed
and initialised it, you can install whichever version of Perl you wish
to use.
perlbrew install perl-5.12.2
perlbrew install perl-5.10.1
``perlbrew'' will then fetch, compile and install that version of Perl
in your home directory (or other nominated location) and you then you
can just start using that version (``perlbrew'' will do all the path
magic required to make this your default version of Perl).
If you have more than one version of Perl installed by ``perlbrew'' you
can list the verions out:
perlbrew list
and switch between them:
perlbrew switch perl-5.12.2
You can also ask ``perlbrew'' to switch itself off so that you can go
back to using the system Perl.
perlbrew off
Many developers have a chicken-and-egg problem when it comes to
upgrading their version of Perl. On one hand, the system version of Perl
cannot be upgraded because it might break existing scripts, on the the
other hand it has been difficult to install your own version of Perl to
see whether an upgrade is safe. The same happens with modules.
Pick your Perl version with App::perlbrew
Use ``App::perlbrew'' to install and switch between multiple versions of
Perl. ``perlbrew'' is easy to install, even if you don't have system
administration privileges, and very easy to use. Once you've installed
and initialised it, you can install whichever version of Perl you wish
to use.
perlbrew install perl-5.12.2
perlbrew install perl-5.10.1
``perlbrew'' will then fetch, compile and install that version of Perl
in your home directory (or other nominated location) and you then you
can just start using that version (``perlbrew'' will do all the path
magic required to make this your default version of Perl).
If you have more than one version of Perl installed by ``perlbrew'' you
can list the verions out:
perlbrew list
and switch between them:
perlbrew switch perl-5.12.2
You can also ask ``perlbrew'' to switch itself off so that you can go
back to using the system Perl.
perlbrew off
Maintain your own CPAN with CPAN::Mini
If you wish to have a portable, off-line CPAN mirror, or wish to maintain the state of CPAN exactly as it was at a particular date, then ``CPAN::Mini'' is the way to go:
cpanmini -l ~/local-cpan \
-r http://search.cpan.org/CPAN
We can even use this with ``cpanm'':
cpanm --mirror ~/local-cpan --mirror-only Test::Most
``cpanmini'' only includes the most recent versions of the CPAN modules
(rather than their whole history) and as a consequence is only a few
Gigabytes big.
cpanmini -l ~/local-cpan \
-r http://search.cpan.org/CPAN
We can even use this with ``cpanm'':
cpanm --mirror ~/local-cpan --mirror-only Test::Most
``cpanmini'' only includes the most recent versions of the CPAN modules
(rather than their whole history) and as a consequence is only a few
Gigabytes big.
6 Feb 2011
Testing your perl code - Jacinta Richardson - Perltraining.com
==== Testing your code ====
Testing cannot prove the absence of bugs, although it can help identify
them for elimination. It's impossible to write enough tests to prove
that your program is flawless. However, a comprehensive test-plan with
an appropriate arsenal of tests can assist you in your goal of making
your program defect free.
== Test::Simple ==
Writing a test is just like writing regular Perl code, and checking that
what you got is what you expected. There are a huge number of modules in
Perl which are designed to make testing that much easier. These are all
built on ``Test::Harness'' underneath. ``Test::Simple'' comes standard
with Perl and provides a great starting point.
``Test::Simple'' provides one test routine: ``ok''. You pass it an
expression, and a description. If the expression, when evaluated, is
true, the test succeeds, otherwise it fails. For example:
use Test::Simple tests => 1;
ok( 1 + 1 == 2, 'simple arithmetic works');
If we save this into a file, and run that file, we get:
1..1
ok 1 - simple arithmetic works
We can add a failing test too:
use Test::Simple tests => 2;
ok( 1 + 1 == 2, 'simple arithmetic works');
ok( 1 + 3 == 2, 'of course, this should fail');
and then we get:
$ perl testscript.pl
1..2
ok 1 - simple arithmetic works
not ok 2 - of course, this should fail
# Failed test 'of course, this should fail'
# at test.pl line 4.
# Looks like you failed 1 test of 2.
So let's consider this code. First of all we must have a plan. This
tells ``Test::Simple'' how many tests we intend to run. If we get this
number wrong, our test suite will not pass; as ``Test::Simple'' cannot
tell whether we made a mistake in our plan, or if some tests didn't run
for some other reason.
Note that we didn't need to specify test numbers, just how many tests we
have. ``Test::Simple'' will keep track of the test numbers for us.
== Test::More ==
``Test::More'' provides a richer set of functions to make your tests
even easier to write. For example, ``is'', ``isnt'', ``like'' and
``unlike'':
use Test::More tests => 8;
# Load in my module, fail if it won't load.
use_ok 'Local::People';
# Test if simple addition is working.
is( (1 + 5), 6, "Simple addition behaves");
# Let's not compare apples to oranges.
isnt( "Malus domestica",
"Citrus sinensis",
"Inequal string comparison"
);
# Test that we can create a new person
my $self;
eval { $self = Local::People->new(cache_id => 1234) };
like( $@, undef, 'Correctly created $self' );
# Does my name match this regular expression?
like ( $self->name(), qr/Jacinta/, 'Name contains Jacinta' );
# We can also provide a regular expression in a string form:
like ( $self->hobby(), '/Perl/i', 'Name contains Perl' );
# Make sure my favourite foods don't contain broad beans.
unlike ( $self->favourite_foods(),
qr/broad beans/i,
"Don't like icky foods"
);
# Test that languages matches the expected list:
my @languages = qw/Perl C PHP C++ Bash/;
is_deeply( [sort $self->languages()],
[sort @languages],
'All the languages are there'
);
== Having a plan ==
With ``Test::More'', if you're just starting your test suite, or adding
an unknown number of extra tests you can tell ``Test::More'' that you
don't *know* how many tests there will be in total:
use Test::More qw(no_plan);
Of course, once your test suite has become more stable, you should
always provide a number of tests. This allows your testing modules to
tell if your tests have stopped prematurely (which usually indicates
something has gone wrong).
You can also delay defining your plan until later:
use Test::More;
# ...
plan( tests => 3 );
This is useful if you won't know how many tests you expect to run until
some condition is decided at run-time. For example, if you have some
tests which depend on an environment variable being set, or a module
being installed. It's a mistake to set a plan, and then change it later.
== ok() vs is() ==
If you make an error either in your module, or in your test script
``ok()'' will only tell you that your test failed. ``is()'' is a lot
friendlier and will tell you both what you got, and what you said you
were expecting. For example ``ok()'' might give us:
$ perl -I lib/ testprogram.pl
1..2
ok 1 - sum() works properly
not ok 2 - product() works properly
# Failed test 'product() works properly'
# at ok.t line 10.
# Looks like you failed 1 test of 2.
Which doesn't necessarily help us spot what we've done wrong. On the
other hand, if we change our tests to use ``is()'' we'll get:
$ perl -I lib/ testprogram.pl
1..2
ok 1 - sum() works properly
not ok 2 - product() works properly
# Failed test 'product() works properly'
# at ok.t line 10.
# got: '3628800'
# expected: '3628880'
# Looks like you failed 1 test of 2.
Here we can see that our error is in the number we said we're expecting
(we've mistyped it).
== Further resources ==
There's a whole lot more to testing, and ``Test::More'' does a lot more
than we've covered here. Many articles on testing in Perl and
specifically ``Test::More'' have been written, and we recommend these in
particular:
: * Building Testing Libraries
<http://www.perl.com/pub/a/2004/05/07/testing.html>
: * Introduction to Testing
<http://www.perl.com/pub/a/2001/12/04/testing.html?page=1>
: * Perl Testing: A Developer's Notebook; by Ian Langworth and
Chromatic; O'Reilly Media, 20005
Testing cannot prove the absence of bugs, although it can help identify
them for elimination. It's impossible to write enough tests to prove
that your program is flawless. However, a comprehensive test-plan with
an appropriate arsenal of tests can assist you in your goal of making
your program defect free.
== Test::Simple ==
Writing a test is just like writing regular Perl code, and checking that
what you got is what you expected. There are a huge number of modules in
Perl which are designed to make testing that much easier. These are all
built on ``Test::Harness'' underneath. ``Test::Simple'' comes standard
with Perl and provides a great starting point.
``Test::Simple'' provides one test routine: ``ok''. You pass it an
expression, and a description. If the expression, when evaluated, is
true, the test succeeds, otherwise it fails. For example:
use Test::Simple tests => 1;
ok( 1 + 1 == 2, 'simple arithmetic works');
If we save this into a file, and run that file, we get:
1..1
ok 1 - simple arithmetic works
We can add a failing test too:
use Test::Simple tests => 2;
ok( 1 + 1 == 2, 'simple arithmetic works');
ok( 1 + 3 == 2, 'of course, this should fail');
and then we get:
$ perl testscript.pl
1..2
ok 1 - simple arithmetic works
not ok 2 - of course, this should fail
# Failed test 'of course, this should fail'
# at test.pl line 4.
# Looks like you failed 1 test of 2.
So let's consider this code. First of all we must have a plan. This
tells ``Test::Simple'' how many tests we intend to run. If we get this
number wrong, our test suite will not pass; as ``Test::Simple'' cannot
tell whether we made a mistake in our plan, or if some tests didn't run
for some other reason.
Note that we didn't need to specify test numbers, just how many tests we
have. ``Test::Simple'' will keep track of the test numbers for us.
== Test::More ==
``Test::More'' provides a richer set of functions to make your tests
even easier to write. For example, ``is'', ``isnt'', ``like'' and
``unlike'':
use Test::More tests => 8;
# Load in my module, fail if it won't load.
use_ok 'Local::People';
# Test if simple addition is working.
is( (1 + 5), 6, "Simple addition behaves");
# Let's not compare apples to oranges.
isnt( "Malus domestica",
"Citrus sinensis",
"Inequal string comparison"
);
# Test that we can create a new person
my $self;
eval { $self = Local::People->new(cache_id => 1234) };
like( $@, undef, 'Correctly created $self' );
# Does my name match this regular expression?
like ( $self->name(), qr/Jacinta/, 'Name contains Jacinta' );
# We can also provide a regular expression in a string form:
like ( $self->hobby(), '/Perl/i', 'Name contains Perl' );
# Make sure my favourite foods don't contain broad beans.
unlike ( $self->favourite_foods(),
qr/broad beans/i,
"Don't like icky foods"
);
# Test that languages matches the expected list:
my @languages = qw/Perl C PHP C++ Bash/;
is_deeply( [sort $self->languages()],
[sort @languages],
'All the languages are there'
);
== Having a plan ==
With ``Test::More'', if you're just starting your test suite, or adding
an unknown number of extra tests you can tell ``Test::More'' that you
don't *know* how many tests there will be in total:
use Test::More qw(no_plan);
Of course, once your test suite has become more stable, you should
always provide a number of tests. This allows your testing modules to
tell if your tests have stopped prematurely (which usually indicates
something has gone wrong).
You can also delay defining your plan until later:
use Test::More;
# ...
plan( tests => 3 );
This is useful if you won't know how many tests you expect to run until
some condition is decided at run-time. For example, if you have some
tests which depend on an environment variable being set, or a module
being installed. It's a mistake to set a plan, and then change it later.
== ok() vs is() ==
If you make an error either in your module, or in your test script
``ok()'' will only tell you that your test failed. ``is()'' is a lot
friendlier and will tell you both what you got, and what you said you
were expecting. For example ``ok()'' might give us:
$ perl -I lib/ testprogram.pl
1..2
ok 1 - sum() works properly
not ok 2 - product() works properly
# Failed test 'product() works properly'
# at ok.t line 10.
# Looks like you failed 1 test of 2.
Which doesn't necessarily help us spot what we've done wrong. On the
other hand, if we change our tests to use ``is()'' we'll get:
$ perl -I lib/ testprogram.pl
1..2
ok 1 - sum() works properly
not ok 2 - product() works properly
# Failed test 'product() works properly'
# at ok.t line 10.
# got: '3628800'
# expected: '3628880'
# Looks like you failed 1 test of 2.
Here we can see that our error is in the number we said we're expecting
(we've mistyped it).
== Further resources ==
There's a whole lot more to testing, and ``Test::More'' does a lot more
than we've covered here. Many articles on testing in Perl and
specifically ``Test::More'' have been written, and we recommend these in
particular:
: * Building Testing Libraries
<http://www.perl.com/pub/a/2004/05/07/testing.html>
: * Introduction to Testing
<http://www.perl.com/pub/a/2001/12/04/testing.html?page=1>
: * Perl Testing: A Developer's Notebook; by Ian Langworth and
Chromatic; O'Reilly Media, 20005
Perl Programming Best Practices 2011
==== Perl Programming Best Practices 2011 by Jacinta Richardson (perltraining.com) ====
Over the next few tips, we'll be covering current best practices for
programming Perl.
== Introduction ==
In 2005, Dr Damian Conway published the very popular "Perl Best
Practices" book which details 256 "best practices" for Perl programming.
While some of these "best practices" have been further improved on in
the ensuing 5 years, many of them remain excellent practices for your
coding. If you have not yet read this book, we strongly recommend that
you do so! In this sequence of Perl tips we will cover a wide selection
of current best practices, taking into account the ideas and
modernisations that Perl has undergone..
== Some best practices are obvious ==
use strict;
use warnings;
If you're not already using ``strict'' in all of your scripts, and using
``warnings'' in almost all of your scripts, you have a lot to learn, and
should probably start with Conway's book.
= 10 year old features =
Perl has had scalar filehandles and three-argument open for more than 10
years now:
# Opening a file for reading
open(my $in_fh, "<", $input_file) or die "Failed to open: $!";
# Reading from that filehandle
while(<$in_fh>) {
# do something with $_
}
close $in_fh;
# Opening a file for writing
open(my $out_fh, ">", $output_file) or die "Failed to open: $!";
# Printing to that filehandle
print {$fh} "This text will go into my file.";
close $out_fh;
Three-argument open treats the filename literally, which protects you
from a number of potential security issues that are possible with
two-argument open. Scalar filehandles - because they are just scalars -
can be passed to subroutines, put in arrays, and used as values in
hashes. Additionally, when the scalar goes out of scope before you close
the file, Perl will automatically close the file for you. None of this
is easy with package filehandles.
= Coding standards =
Ideally every organisation you work for, or project you work on, has a
pre-established, formalised set of coding standards. While these may not
be as current as you'd like, consistency is a good thing. If you don't
have a coding standard, then now is a good time to develop one. Conway's
book is a great starting place.
== Upgrade your version ==
Perl 5.6.1 is almost 10 years old now (2001-Apr-08). Perl 5.8.2 is 7
years old (2003-Nov-05). Perl 5.10.0 is 3 years old (2007-Dec-18). Perl
5.8.9 is 2 years old (2008-Dec-14). Perl 5.10.1 is 1.5 years old
(2009-Aug-22) Perl 5.12.2 is 4 months old (2010-Sep-06).
Upgrade! At the very, very least, use one of the Perl 5.8.x versions,
but if you can, upgrade to either 5.10.1 or 5.12.2. Why? Because a whole
lot of wonderful, new, awesome and *modern* features have been
incorporated into Perl in the last few years.
To use the features of your modern version of Perl, just specify your
minimum version:
use v5.10.1; # get all the things from Perl 5.10.1
= Perl 5.12.x =
By specifying any of the Perl 5.12 versions, ``strict'' is automatically
turned on by default. Thus this:
use v5.12.2;
is the same as saying:
use v5.12.2;
use strict;
This is a great improvement.
== New features ==
With Perl 5.10.0 we received a host of new features. Some of these are
mentioned here:
= say =
``say'' gives us ``print'' with an attached newline. Instead of:
print "Hello World!\n";
print "$_\n" foreach @list;
we can write:
say "Hello World!";
say foreach @list;
it's shorter, neater, and gets rid of that ugly $_.
= // (defined-or) =
Consider this subroutine:
sub item_sale {
my ($product, $price) = @_;
$price ||= $product->cost;
...
}
This example is short, neat, and almost correct. But how do we signal
that this item is being provided for free without changing the $product
object? If we pass in zero, then that's a false value, so we're going to
get the full product cost.
We could rewrite this to be:
sub item_sale {
my ($product, $price) = @_;
unless( defined($price) ) {
$price = $product->cost;
}
...
}
Or, in Perl 5.10.0 and above we can use defined-or. Defined-or works
just like regular OR, except that it tests for whether the value is
defined, rather than true.
sub item_sale {
my ($product, $price) = @_;
$price //= $product->cost;
...
}
= state =
``state'' allows us to create variables which remember their state.
These are like ``static'' variables in C. Imagine the following code to
ensure that we only show 100 images:
my $images_allowed = 100;
sub view_image {
die "Too many images" unless $images_allowed-- > 0;
...
}
As it is, other subroutines here could be used to change
$images_allowed. To prevent that, we could wrap it into a closure:
{
my $images_allowed = 100;
sub view_image {
die "Too many images" unless $images_allowed-- > 0;
...
}
}
Or, we can use a state variable:
sub view_image {
state $images_allowed = 100;
die "Too many images" unless $images_allowed-- > 0;
...
}
= given-when =
There's now a native switch for Perl, except that it's very, very cool.
given ($has) {
when (/ninja/) { $ninjas++; }
when (/pirate/) { $pirates++; }
when (/robot/) { $robots++; }
when (@other) { $other_cool_things++; }
when (%interesting) { $interesting++; }
default { $boring++; }
}
This says smart match the value in $has. If it matches against any of
the first three regular expressions, increment that variable. Otherwise
if it is an element of @other increment ``$other_cool_things++'',
otherwise if it is a key in %interesting increment $interesting. If it
matches none of these comparisons, increment $boring.
The given-when can also be used with ``foreach'':
foreach (@input) {
when (/ninja/) { $ninjas++; }
when (/pirate/) { $pirates++; }
when (/robot/) { $robots++; }
when (@other) { $other_cool_things++; }
when (%interesting) { $interesting++; }
default { $boring++; }
}
= New features: smart matching =
given-when uses smart matching to work. Smart match allows us to compare
things to determine if they match, in the most useful way possible:
$foo ~~ undef; # is $foo undefined?
$foo ~~ @array; # is $foo in @array?
$foo ~~ %hash; # is $foo a key in %hash?
$foo ~~ $coderef; # does $coderef->($foo) return true?
$foo ~~ qr{$re}; # does $foo pattern match $re?
$foo ~~ $bar; # does $foo equal $bar?
== Next ==
There are many other best practices still to be covered. In our next tip
we'll discuss exception handling, and how you can test a wide variety of
Perl versions from your home directory.
Over the next few tips, we'll be covering current best practices for
programming Perl.
== Introduction ==
In 2005, Dr Damian Conway published the very popular "Perl Best
Practices" book which details 256 "best practices" for Perl programming.
While some of these "best practices" have been further improved on in
the ensuing 5 years, many of them remain excellent practices for your
coding. If you have not yet read this book, we strongly recommend that
you do so! In this sequence of Perl tips we will cover a wide selection
of current best practices, taking into account the ideas and
modernisations that Perl has undergone..
== Some best practices are obvious ==
use strict;
use warnings;
If you're not already using ``strict'' in all of your scripts, and using
``warnings'' in almost all of your scripts, you have a lot to learn, and
should probably start with Conway's book.
= 10 year old features =
Perl has had scalar filehandles and three-argument open for more than 10
years now:
# Opening a file for reading
open(my $in_fh, "<", $input_file) or die "Failed to open: $!";
# Reading from that filehandle
while(<$in_fh>) {
# do something with $_
}
close $in_fh;
# Opening a file for writing
open(my $out_fh, ">", $output_file) or die "Failed to open: $!";
# Printing to that filehandle
print {$fh} "This text will go into my file.";
close $out_fh;
Three-argument open treats the filename literally, which protects you
from a number of potential security issues that are possible with
two-argument open. Scalar filehandles - because they are just scalars -
can be passed to subroutines, put in arrays, and used as values in
hashes. Additionally, when the scalar goes out of scope before you close
the file, Perl will automatically close the file for you. None of this
is easy with package filehandles.
= Coding standards =
Ideally every organisation you work for, or project you work on, has a
pre-established, formalised set of coding standards. While these may not
be as current as you'd like, consistency is a good thing. If you don't
have a coding standard, then now is a good time to develop one. Conway's
book is a great starting place.
== Upgrade your version ==
Perl 5.6.1 is almost 10 years old now (2001-Apr-08). Perl 5.8.2 is 7
years old (2003-Nov-05). Perl 5.10.0 is 3 years old (2007-Dec-18). Perl
5.8.9 is 2 years old (2008-Dec-14). Perl 5.10.1 is 1.5 years old
(2009-Aug-22) Perl 5.12.2 is 4 months old (2010-Sep-06).
Upgrade! At the very, very least, use one of the Perl 5.8.x versions,
but if you can, upgrade to either 5.10.1 or 5.12.2. Why? Because a whole
lot of wonderful, new, awesome and *modern* features have been
incorporated into Perl in the last few years.
To use the features of your modern version of Perl, just specify your
minimum version:
use v5.10.1; # get all the things from Perl 5.10.1
= Perl 5.12.x =
By specifying any of the Perl 5.12 versions, ``strict'' is automatically
turned on by default. Thus this:
use v5.12.2;
is the same as saying:
use v5.12.2;
use strict;
This is a great improvement.
== New features ==
With Perl 5.10.0 we received a host of new features. Some of these are
mentioned here:
= say =
``say'' gives us ``print'' with an attached newline. Instead of:
print "Hello World!\n";
print "$_\n" foreach @list;
we can write:
say "Hello World!";
say foreach @list;
it's shorter, neater, and gets rid of that ugly $_.
= // (defined-or) =
Consider this subroutine:
sub item_sale {
my ($product, $price) = @_;
$price ||= $product->cost;
...
}
This example is short, neat, and almost correct. But how do we signal
that this item is being provided for free without changing the $product
object? If we pass in zero, then that's a false value, so we're going to
get the full product cost.
We could rewrite this to be:
sub item_sale {
my ($product, $price) = @_;
unless( defined($price) ) {
$price = $product->cost;
}
...
}
Or, in Perl 5.10.0 and above we can use defined-or. Defined-or works
just like regular OR, except that it tests for whether the value is
defined, rather than true.
sub item_sale {
my ($product, $price) = @_;
$price //= $product->cost;
...
}
= state =
``state'' allows us to create variables which remember their state.
These are like ``static'' variables in C. Imagine the following code to
ensure that we only show 100 images:
my $images_allowed = 100;
sub view_image {
die "Too many images" unless $images_allowed-- > 0;
...
}
As it is, other subroutines here could be used to change
$images_allowed. To prevent that, we could wrap it into a closure:
{
my $images_allowed = 100;
sub view_image {
die "Too many images" unless $images_allowed-- > 0;
...
}
}
Or, we can use a state variable:
sub view_image {
state $images_allowed = 100;
die "Too many images" unless $images_allowed-- > 0;
...
}
= given-when =
There's now a native switch for Perl, except that it's very, very cool.
given ($has) {
when (/ninja/) { $ninjas++; }
when (/pirate/) { $pirates++; }
when (/robot/) { $robots++; }
when (@other) { $other_cool_things++; }
when (%interesting) { $interesting++; }
default { $boring++; }
}
This says smart match the value in $has. If it matches against any of
the first three regular expressions, increment that variable. Otherwise
if it is an element of @other increment ``$other_cool_things++'',
otherwise if it is a key in %interesting increment $interesting. If it
matches none of these comparisons, increment $boring.
The given-when can also be used with ``foreach'':
foreach (@input) {
when (/ninja/) { $ninjas++; }
when (/pirate/) { $pirates++; }
when (/robot/) { $robots++; }
when (@other) { $other_cool_things++; }
when (%interesting) { $interesting++; }
default { $boring++; }
}
= New features: smart matching =
given-when uses smart matching to work. Smart match allows us to compare
things to determine if they match, in the most useful way possible:
$foo ~~ undef; # is $foo undefined?
$foo ~~ @array; # is $foo in @array?
$foo ~~ %hash; # is $foo a key in %hash?
$foo ~~ $coderef; # does $coderef->($foo) return true?
$foo ~~ qr{$re}; # does $foo pattern match $re?
$foo ~~ $bar; # does $foo equal $bar?
== Next ==
There are many other best practices still to be covered. In our next tip
we'll discuss exception handling, and how you can test a wide variety of
Perl versions from your home directory.
Subscribe to:
Posts (Atom)