Upcoming TP2 release

One thing I never seem to do is concentrate on one project at a time! Hence I've picked up TP2 (my advanced packaging software suite) and looked at adding some features I was keen to make use of to aid package deployments in heterogeneous environments.

TP2 is cross-platform though mainly concerned with the main flavours of UNIX. Hence it is entirely possible to use it to deploy packages related to a particular OS and even a particular hardware architecture for an OS; or make them classed as completely generic and install anywhere.

More advanced sizable installs this generic approach typically needs to make use of the preinstall and postinstall scripts that TP2 packages may provide to customise the solution somewhat.

All the above works fine with the existing TP2 releases; however I wanted something more - the ability to dynamically alter the package content. Consider the following example:

The postinstall script could examine the existing architecture and generate some configuration files or additional scripts dynamically as part of the install process. Again this works - however any of these newly generated files are not considered part of the package and so verification of the package (via tp2 verify) would not cover them. 

This is now being addressed.

The upcoming release will allow the preinstall and postinstall to dynamically modified what files are considered part of the package - allowing you to trim or add the content to suit the environment.

The coding is virtually done; I'm really simply adding some more testing and will update the documentation and release it.

 

True Packager 2 Release 1.3.3 Features - Part 2

Passing Environment Variables

When you are running as the owner of a namespace when you attempt to install or remove a package the software will run the relevant programs natively; and hence have access to the current environment.

However if you are running as a user that is authorised (but does not own) the namespace in question, all activity is spawned via the daemon process. This has the following impact:

• The actual process doing all the work is initiated by the daemon; the process in the terminal is simply relaying the information it receives in this instance.
• Since the process installing the package is doing so via the daemon it is no access to the environment variables that the caller has available currently.

However from 1.3.3 onwards this problem can be overcome by making use of an additional optional section in the package configuration file, namely:

  <env>

    ENV_VAR1

    ENV_VAR2 

  </env>

The "env" element allows you to list a series of environment variables that the caller and the spawned install/remove session will pass between them. When such is part of the package those variables will be communicated from the calling shell and made available to the four following scripts:

preinstall

postinstall

preremove

postremove

Getting User Responses

This is still in the early stages of development but the idea is that the package should be able to configure a series of queries which the user can respond to and these responses are then made available to the two installation scripts; "preinstall" and "postinstall".

Again this is achieved via a new section in the package configuration file, namely the optional "response_info" element. This will have a separate "q" element for each query that the user is asked. Consider the following example:

  <response_info>

    <q text="Please enter your age" default="5" 

       type="integer" range="1 99" label="get_age"/>

    <q text="Please enter your email address" default="%USERNAME%@%HOSTNAME%" 

       type="text" label="get_email" pattern="^\S+@\S+" 

       hint="Please enter valid email address format"/>

  </response_info>

In this case two questions will be asked. The utility will attempt to perform validation and range checking for numbers, and for text queries it can optionally include a "pattern" attribute to ensure the response entered matches the pattern in question.

The "default" attribute is mandatory - this will allow installations to take default values if the user responses are not available. This is designed for headless installs; even though that is not yet available.

In such cases the preinstall and postinstall scripts have these values entered as part of the environment - these are in variables TP2_RESPONSE_label. Hence in the above example:

TP2_RESPONSE_get_age

TP2_RESPONSE_get_email

For more information see the available documentation on the google code downloads page: true packager 2 downloads.

True Packager 2 Release 1.3.3 Features - Part 1

Version 1.3.3 of True Packager 2 (or TP2 for short) has been released. This post includes a quick summary of the major differences that the software now supports.

Elevated Privileges
Early versions of TP2 made only two roles available; the namespace owner and "root". You had to be running a command as either of those for a package install to be successful.  Although this fulfilled the original design requirement of the packaging solution, recently it was requested that a namespace should allow certain other users to install packages; not simply the namespace owner.

The concept was simple; the namespace should be owned by a system/application account, but the installs should be done as normal admin accounts to provide better levels of security and auditing. This of course is often combined with only allowing signed packages to prevent just anything from being installed.

Thus as of 1.3.3 if a normal user attempts to install a package into a namespace they do not own they will get a rather cryptic message (needs to be improved!):

$ tp2 verify --namespace opt

Error: The ALLOW_VERIFIERS setting is missing for namespace 'opt'.

The owner of the namespace (or a user that has the capabilities to create or modify namespaces), for example as root:

# tp2 mod_ns opt ALLOW_VERIFERS root,test

Then examining the namespace configuration:

$ tp2 show_ns --namespace opt

Packages installed                       1

Compress Logs?                           Yes

Current log count                        1

File System Growth Unit (MB)             256

Log space usage (MB)                     0.01

Always repackage?                        Yes

Maximum space for logs (MB)              50

Log compression type                     gzip

Remove on reboot                         <None>

Users allowed to verify packages         root,test

Grow file systems automatically?         Yes

Maximum number of logs to keep           2000

Script to run after pkg install          /usr/bin/true

Namespace root directory                 /opt

Duration to keep preview logs (Days)     10

Perform auditing?                        Yes

Temporary files location                 /var/adm/tp2/ns/opt/spool

Script to run after pkg removal          /usr/bin/true

Namespace Owner                          root

Directory to extract packages in         /var/adm/tp2/ns/opt/spool

At this point the normal user "test" should have access to run the verification process on that namespace:

$ tp2 verify --namespace opt --verbose

Log  : Invocation of namespace verification via daemon starting.

Log  : Loading configuration of packages in "opt" namespace...

Log  : ...Done [1 packages]

Log  : Loading any immutable files details for "opt" namespace...

Log  : ...Done [0 files]

Log  : Checking package "tp2" [directories].

Log  : Checking package "tp2" [files].

Error: [tp2] Incorrect file checksum   : /opt/bin/tp2make_ns [is aaa9854607a9c98655cdebbc9d133c5d26daeac2, should be 6c1c1dbd3e2f37e6cbc5a8a97631262fa0c7d67f]

Log  : Total errors found: 1

At present there are four per-namespace privileges that can be configured:

ALLOW_VERIFIERS - allow the list of users configured to verify package configuration.

ALLOW_INSTALLERS - allow the users to install, upgrade, downgrade and clean packages.

ALLOW_REMOVERS - allow the users to remove package installations.

ALLOW_IMMUTABLES - allow the users to add/remove files to the namespace immutable files list.

What next?

Part two of this post will describe facilities to customise the package installation by allowing certain environment variables to be passed to pre/post install and remove scripts, and also to configure one or more queries that can be used to interact with the users prior to running the above scripts too.

TP2 Security Improvements underway

I've recently started a series of changes to improve security and auditing facilities for TP2. First up are changes to implement secure repository management. Essentially the tp2 command set now provides tools to set and manage ACLs (Access Control Lists) on repositories. To support this the daemon process indirectly can be used to copy, remove, sign and un-sign packages in such repositories as well as provide repository index update services as before.

The range of services that the daemon provides will continue to grow as a lot more work on security and auditing is in the roadmap. Because of this the handling of requests by the daemon has also been re-written. In the previous versions it only handled requests in a single threaded manner. Now, however it supports an architecture for requests to be handled in the background - and all long-running requests now use this facility. The result is that the daemon can cope better with multiple requests issued at the same time.

TP2 Uploaded

Tp2 has been uploaded to Google Code now too. This is a package management system with a unique of fairly unique features that will appeal to environments where package installations need to be carefully managed. The software is written in Perl and one of the strong points is the logging capability - each and every package installation and removal is logged. That facility is also tied to then ability to actual "preview" the operation.

Allowing a preview of a package installation is quite useful; it will only log files that are going to change as part of the installation. Hence if you were upgrading a package consisting of 1,200 files it will only show you those that will be changing - [being added, being removed, or being replaced], not those that are identical.

Many architectures do not allow binary files that are being executed to be removed ... TP2 works around this by renaming such files and deleted them at a later time - usually at the next reboot for example.

For any tool set to be useful I believe that documentation is a must; many people do not have the time or inclination to scan the source code or the Internet looking for nuggets of information. Hence TP2 comes with complete documentation covering installation, administration and generating packages.

TP2 has been used for quite some time and is pretty much complete; it does most things a package management suite should do and then some; for example the concept of "name spaces" for packages is quite unique [or at least certainly not common].