FileEvent Example 1 (Simplest use of FileEvent)

Introduction

FileEvent is one of those tools that you really need to see in action rather than just reading about it; the purpose is hard to explain, but easy to comprehend once presented with a few examples.

Firstly consider files with the following naming scheme:

file1_YYYYMMDD.txt

where:

• YYYY – 4 Digit year
• MM – 2 Digit month
• DD – 2 Digit day

The simplest way of getting FileEvent to notice such files is a configuration such as the following:

<?xml version="1.0" standalone="yes"?>
<FileEvent>
  <settings>
    <db>/tmp/testing.db</db>
  </settings>
  <event>
    <description>scp example1 transfer</description>
    <file_pattern>file1_%{4year}%{2month}%{2day}.txt</file_pattern>
    <directory>/home/venture/projects/SOURCE/fileevent/testing/example1</directory>
    <xfer_job_type>scp</xfer_job_type>
    <xfer_destination>test@lubuntu1:/tmp/%{f}</xfer_destination>
  </event>
</FileEvent>

The key points to notice here are:

• The “directory”is the directory where the files exist, and the “file_pattern” is the name of the files to look for.

• The “file_pattern” can make use of “%{value}” strings to represent patterns to search for – in this case parts of a date – such as “%{year}”.

Consider a directory with the following files in it:

-rw-rw-r-- 1 venture venture 27125 Aug 10 15:07 file1_20130819.txt
-rw-rw-r-- 1 venture venture 27125 Aug 10 15:07 file1_20130821.txt
-rw-rw-r-- 1 venture venture 27125 Aug 10 15:07 file1_20130822.txt

Running FileEvent is “list” mode will just list the files it will process for example:

$ fileevent.pl --config example1.xml --action=list 
Event #0 matched files:
  Directory: /home/venture/projects/SOURCE/fileevent/testing/example1
  Pattern  :  ^file1_%{4year}%{2month}%{2day}.txt$
Actual cut-down list to send now:
  file1_20130822.txt

By default it will only process a single file for the matching pattern, and also by default it will only match the “newest” file.

Notice that the newest file is not based on the modification time of the file, but from the date taken from the patterns of dates from the file – if such is available.

So to actually send the file:

$ fileevent.pl --config example1.xml –action=process

That will send the first file (though without verbosity not much will have been printed).

Notice that the file will still exist in the directory. Now run the same command again:

$ fileevent.pl --config example1.xml –action=process
2013/08/10 17:10:44.951 0000013-W File '/home/venture/projects/SOURCE/fileevent/testing/example1/file1_20130822.txt' already sent - removing from list to send or fetch.

This is one of features of FileEvent – it will (by default) not send the same file if it has already been sent successfully – so repeating the command has skipped the first file and sent the next one (based on file age again).

Run it again and it will skip the two files this time:

$ fileevent.pl --config example1.xml --action=process --verbose
2013/08/10 17:14:45.021 0000000-I Events to load from configuratione file: 1
2013/08/10 17:14:45.025 0000001-I Event #0 [scp example1 transfer] processing.
2013/08/10 17:14:45.026 0000002-W File '/home/venture/projects/SOURCE/fileevent/testing/example1/file1_20130822.txt' already sent - removing from list to send or fetch.
2013/08/10 17:14:45.026 0000003-W File '/home/venture/projects/SOURCE/fileevent/testing/example1/file1_20130821.txt' already sent - removing from list to send or fetch.

So all three files have been sent at this point. Running it again will send no files:

fileevent.pl --config example1.xml --action=process
2013/08/10 17:15:25.681 0000000-W File '/home/venture/projects/SOURCE/fileevent/testing/example1/file1_20130822.txt' already sent - removing from list to send or fetch.
2013/08/10 17:15:25.681 0000001-W File '/home/venture/projects/SOURCE/fileevent/testing/example1/file1_20130821.txt' already sent - removing from list to send or fetch.
2013/08/10 17:15:25.682 0000002-W File '/home/venture/projects/SOURCE/fileevent/testing/example1/file1_20130819.txt' already sent - removing from list to send or fetch.
2013/08/10 17:15:25.682 0000003-E Maximum wait time for event #0 passed.
2013/08/10 17:15:25.682 0000004-W ** No matching files found. **
2013/08/10 17:15:25.682 0000004-W  

Conclusion

This has been a very simple example of FileEvent showing just some of the basic features. It should be obvious now that the pattern matching is easy to use, the configuration files are easy to understand and the utility is stateful (in as much as it will not repeatedly send the same thing every time it is called).