README for Perl module String::Tokenizer This module provides a simple string tokenizer which takes a string and splits it on whitespace. It also optionally takes a string of characters to use as delimiters, and returns them with the token set as well. This allows for splitting the string in many different ways. You can read a nicely formatted version of the documentation for this module online: https://metacpan.org/pod/String::Tokenizer You should be able to install this using your usual method for installing modules from CPAN. If you don't have one, have a look at: http://www.cpan.org/modules/INSTALL.html This module was written by Stevan Little Copyright (c) 2004-2016 Infinity Interactive, Inc.