Sunday, April 5, 2009

New Skulker Features under development

A couple of new features have been started for Skulker, both requirements for improving Skulker when dealing with enterprise environments.

Null Data Overwrite Support

More and more people are making use of virtualised storage - and particularly "thin provisioning". Whether this is provided at the file system level [think Solaris ZFS] or dedicated hardware storage arrays, one feature is becoming more common is the ability of the storage provider to reclaim storage that is no longer used. Several now support this implicitly if pages of storage are over-written with null data. Hence this null data support, initially for the delete action, will overwrite files will NULL's and then remove it. This will obviously incur a performance hit, so it is optional.

Parallel Processing Support
A couple of customers have rules which match a large number of files each time Skulker is run. Since these people have Skulker running on multi-cpu machines it makes sense to attempt to harness the additional processing power and perform several actions in parallel.

The mechanism now being developed will optionally allow allow functions that are applied to files to run across multiple "dispatchers". The key here will be flexibility to support 1-N dispatchers whilst not requiring much effort to make use of.