TP2 Security Improvements underway

I've recently started a series of changes to improve security and auditing facilities for TP2. First up are changes to implement secure repository management. Essentially the tp2 command set now provides tools to set and manage ACLs (Access Control Lists) on repositories. To support this the daemon process indirectly can be used to copy, remove, sign and un-sign packages in such repositories as well as provide repository index update services as before.

The range of services that the daemon provides will continue to grow as a lot more work on security and auditing is in the roadmap. Because of this the handling of requests by the daemon has also been re-written. In the previous versions it only handled requests in a single threaded manner. Now, however it supports an architecture for requests to be handled in the background - and all long-running requests now use this facility. The result is that the daemon can cope better with multiple requests issued at the same time.

TrueCL work

Now that the latest update for linuxha.net is out of the way I'm turning my attention to TrueCl once again.

TrueCl uses sqlite for two purposes; a status database and a configuration database. One of the most frustrating things is that depending on the platform, perl version and sqlite version I covered through up more subtle locking issues when attempting to read/write at approximately the same time from different programs.

Although still a work in progress I've now re-written the status database to make use of a running service - which seems to defeat the purpose of SQLite in some respects, but at last I have a working solution!

Next couple of days will involve flipping that functionality to the configuration database module too - and then that should be one of the few significant portability issues with the core code resolved.

Linuxha.net version 1.4.9 update soon

It has been more than a year since the release of 1.4.8 and so time has come around for a look over the code and minor tweaks and fixes to be performed. In the next couple of weeks version 1.4.9 will be released which will include the following changes, at the very least:

• Heartbeat Daemon statistics - it will be possible to report on the performance of the heartbeat process to ascertain whether any settings should be changed.
• Network and Heartbeat Daemon restart facility - a facility to restart these daemons completely rather than just reconfigure them will be available to improve online upgrading of software facilities.
• DRBD updates - The version of DRBD including will be bumped to 8.3.8.
• Daemon status reporting - The cluster status utility will include a facility to check the necessary daemons are all running.

Once this update is out I will be next turning to a new release of TrueCl the next generation clustering software which is working well at present.

Skulker v1.0.2 New Features

Over the weekend version 1.0.2 of Skulker will appear. This has a couple of interesting features:

• time_limit facility - it is possible to indicate the maximum amount of time a rule should run for before continuing with the next one. This is an additional "filter" over existing ones so you can indicate that, for example, up to 5000 files should be compressed, oldest version but only taking a maximum elapsed time of two hours.
• Random data scrubbing. The delete "file scrubbing" functionality has been extended to include a "random data" overwrite option (including multiple passes) if you might think it may be of use. A lot will depend on the OS and file system as to whether this does anything useful - absolutely not for copy-on-write file systems of course.

Code Manager 2 version 1.3.2 released.

The code is out there ... but not mentioned on freshmeat as yet since I've got to tidy up the user guide and push that out first. Documentation is never much fun, but most software needs at least a bit so I want to get that out too. Should happen in the next day or two.

1.3.2 has a few minor improvements but the main aim was to fix some problems. Nothing exciting or extras shiny to look at!

Code Manager 2 Work

Code Manager 2 is a strange beast really. Is it a cross between a single site version control system and a release management system. As with a lot of my work the key defining aim is to ensure the target audience finds the tool easy to use yet powerful enough to be genuinely useful.

Oodles of documentation on the Google code pages for it, but suffice to say it works with a single working directory structure shared by developers and a repository stored elsewhere. Various different repositories are supported and it works on UNIX and Windows.

All the standard features you might expect are there - working at the individual file level; but so are a lot more. It is possible to compare "projects" (a collection of directories making a single development environment), and even tag releases and automatically create packages from them for various package managers.

If you fancy a simple version control system, but powerful enough to work with a largish collection of sources and like the idea of easy package creation, take a look.

http://code.google.com/p/codemanager2/

It is currently at version 1.3.1 and a version 1.3.2 is probably going to happen in a few days time with numerous small fixes and tweaks.

Skulker v1 Up and Running

Wow - it has been some time since I last posted! The good news is that the software has continued to develop despite the lack of news here. Firstly Skulker...

Well 1.0.1 is actually out at present and the key features I really wanted have made it and are working well. These are:

• Multi-threading - the core code has been written to be thread friendly and the compress functionality supports threading across all recent (5.6 and above I reckon) versions of Perl. What this did show me is that lightweight threading in Perl is pretty inefficient but it seems to work OK for the compression module.
• Internal compression types - it does now support use of "internal" compression - that is rather than running external programs it can optionally (if configured and available) make use of Perl modules greatly improving performance - especially when handling a lot of files.
• Windows support - Working, including threading. Tested with Strawberry Perl and running in productions environments now just fine.
• Scrubbing - zero's out files before deleting them. Aim is to help with storage reclaims on sparse storage (certain high-end environments).