Upgrading to Drupal 7

I’ve recently upgraded a couple sites to Drupal 7. The first one (7.0 rc1) was a bit of a trial run to see how ‘ready’ 7 is. It was a simpler site, and in the process of an upgrade/retrofit already. It went pretty much according to plan with only a few slight hiccups, which are a subset of the items listed below. They certainly have made a concerted effort to do things the right way(kudos to the Drupal team and all contributors!), and to try to tie up the loose ends – and the effort shows. The overall impression one gets in the backend is of a tighter and more cohesive package. However – migrating my second site (using 7.0) over was a bit more challenging, as this site was quite a bit more complex.

The Process

Of course you need to follow the recommended paths on the www.drupal.org site – upgrade your D6 site to the latest point release, upgrade each module to the latest, look at all of your modules and determine which are now included in core, and which are ready for 7. You may need to dig into the issues or google it to see if people are successful with that module in 7 already. Do you really need that module? I’ve had to shift a few things around, and I’ve migrated from tinyMCE to the Wysiwyg module + CKeditor.

If you’re ready to do the upgrade, back everything up at least once, disable all your modules that aren’t in core or required, set your theme to garland, download the latest D7 release, and then follow the upgrade instructions. Hopefully all of your custom code, themes, modules and functions are under the sites dir.

So, many changes have been made to D7 – you’re going to find that you’ll probably have to tweak at least a few things to complete your upgrade.

Database

A number of the tables have been renamed, some fields have been refactored into different (new or renamed) tables, etc. If you don’t access the db directly, then this may not affect you(the actual D7 upgrade process does a pretty good job of converting all of your existing D6 db), but if you do…read on…

The table node_revisions is now node_revision and does not contain the fields teaser nor body – you’ll need to check out and join the new table field_revision_body (possibly on revision_id) and access fields body_summary and body_value. The naming conventions have migrated to a more human readable, sensible approach, and are an improvement upon the minimalist origins – but – it does lead to a bit of a ‘in-between’ feeling now – as it is a bit of both. This is by no means an exhaustive list, but the schema has also been modified concerning uploaded files as well as taxonomies, among other things.

db_query returns a result that is still a db object, but can be treated more like an array – so, to iterate through it,

you now (drupal 7):

foreach ($result as $object)

instead of (drupal 6):

while ($object = db_fetch_object($result))

and to get the number of items returned, call (drupal 7):

$result->rowCount()

instead of (drupal 6):

mysqli_num_rows($result)

Themes

If you’ve created your own theme, or tweaked an existing one, you’ll need to be aware of all the changes here – this isn’t a comprehensive list here, just a few of the things I ran into. Study the new Seven theme or D7′s Garland implementation. Dashes have been doubled in most of the filenames:

page-front.tpl.php becomes page--front.tpl.php

page-node-1.tpl.php becomes page--node--1.tpl.php

The outer shell of your html template – including DOCTYPE, html declaration, header and body, etc., is now kept in

html.tpl.php

(look for the default in modules/system/html.tpl.php)

For display of regions (drupal 6),
if($region_name) print $region_name;

becomes (drupal 7):
print render($page['region_name']);

If you want to suppress the printing of file uploads and taxonomy on your pages, override node.tpl.php (see default in modules/node/node.tpl.php) and explicitly hide those node fields:

hide($content['upload']);
hide($content['taxonomy_vocabulary_1']);

before the call:
print render($content);

To suppress the submitted by tag, I’m sure there’s a more elegant solution, but I just commented it out in node.tpl.php:

<?php //print $submitted; ?>

CCK

If you used CCK and some of its associated modules like Filefield to help create forms and or custom content types, you’ll need to be aware that the standard upgrade path will not migrate all of those CCK-related entities. Once your upgrade is complete, you’ll need to load the D7 CCK module from http://drupal.org/project/cck and enable it and its sub-entity ‘Content Migrate’. Then go to Structure >> Migrate Fields to actually migrate your custom fields and regain access to their content. This is evidently a work in progress – read more here: http://drupal.org/node/781088. The standard field types convert over correctly, but some of the less frequently used field types may present a problem. If you run into an error, try to determine what data was carried over correctly by examining the new field_data_field_* tables. Any fields that aren’t correctly populated can be rolled back, and then re-migrated.

I’ve found that the fields that you use to have in your custom content type are still there, but now the field contents will be saved into its own field-specific table. For instance, I have a custom content type named meeting and it was stored in the database in D6 with its own table: content_type_meeting. Any fields unique to meeting (for instance ‘Theme’ which would have been a field field_theme_value) would have been kept in that table provided that they weren’t also used by another custom content type, in which case they’d be kept in a separate, shared table. Now, in D7 – that field is always kept in its own tables: field_data_field_theme and field_revision_field_theme. So – now when you edit and save a node of type meeting, it will populate from(edit) and to(save) the new D7 field-specific tables, not the fields with the old content type table. This means that if you have custom code that picks up a list of meetings and formats it for display, you’ll need to modify your code to pick up meetings distributed in the database in the new format. A node_load() called on a meeting node will return the correct data in the affiliated fields, provided that the conversion has completed correctly.

Menus

All the menus came over fine as part of the upgrade, but custom, hierarchical menus situated in a block (that expanded and collapsed correctly in D6) would not display correctly out of the box in Drupal 7. I had to apply the following patch: http://drupal.org/node/942782#comment-3943328 (and running update.php after application) to get these menu items to correctly display – expanding and collapsing by clicking on the menu title.

When you edit a node now, have your Menu settings disappeared? Edit your content types, and down toward the bottom in Menu settings, enable the content type to be able to appear in the menus that you specify.

Imagecache

The image handling provided by ImageAPI and Imagecache is now contained within drupal core. See admin/config/media/image-styles for its admin in the backend.

D6 references to theme('imagecache', … need to be rewritten:

Drupal 6:

$imagecache_html = theme('imagecache',
    'meeting_screenshot',
    get_filename($screenshot),
    "screenshot of " . $image_title,
    $image_title);

Drupal 7:

$image_settings = array(
    'style_name' => 'meeting_screenshot',
    'path' => 'public://' . get_filename($screenshot),
    'alt' => 'screenshot of ' . $links->title,
    'title' => $links->title,
    'attributes' => array('class' => 'image'),
    'getsize' => FALSE,
    );
$imagecache_html = theme_image_style($image_settings);

My get_filename function just returns the filename – and the default public files area is set to ‘sites/default/files’ (configurable in the backend) – which is what ‘public://’ indicates to drupal. This is relative to the site root.

[note: if using 'getSize' => TRUE, need to apply this patch: http://drupal.org/node/1012416]

The new imagecache tmp files SHOULD be written out to (and referenced from):
sites/default/files/styles/{image_preset_name}/public
(check existence/permissions on this tree if the files aren’t showing up)

I found that not all of my imagecache presets from my D6 site got carried over (or correctly) when upgrading from 6 to 7.
I had to go in and delete and recreate some, and just create some in 7.

Custom Modules

I’ve created a number of custom modules on this site, and then I’ll utilize the output of that module in a given custom region, associating the two in the Blocks configuration screen. I found that those relationships survived the upgrade, but the relationships are not displayed correctly in the D7 Blocks configuration screen. Haven’t gotten to the bottom of that entirely yet…

The module_name.module file contains a series of required (and optional) functions -
hook_perm has become:

function hook_permission() {
    return array(
        'administer module_name module' => array(
        'title' => t('Administer module_name module'),
        'description' => t('Perform administration tasks for module_name module.'),
        ),);
}

the real action typically has happened in the function named hook_block – (where ‘hook’ gets replaced by your module_name) and this function has been split into a series of functions, including(with relatively common contents):

function hook_block_info() {
    $blocks['block_name'] = array(
        'info' => t('Human Readable Block Name'),
        'cache' => DRUPAL_NO_CACHE,
    );
    return $blocks;
}

function hook_block_configure($delta) {
    if ($delta == 'block_name') {
        $form['list_size'] = array(
        '#type' => 'textfield',
        '#title' => t('Number of module items to display in the list'),
        '#default_value' => variable_get('module_name_list_size', 3),
        '#size' => '3',
        '#maxlength' => '4',
        );
        return $form;
    }
}

function hook_block_save($delta, $edit = array()) {
    if ($delta == 'block_name') {
        variable_set('module_name_list_size', $edit['list_size']);
    }
}

and hook_block is now:

function hook_block_view($delta='')

and the code in your if view block would be kept in the new function – just delete section under if list

Entity module

I was getting errors on each node view:
Warning: array_flip(): Can only flip STRING and INTEGER values! in DrupalDefaultEntityController->load() (line 167 of includes/entity.inc).
Warning: array_flip(): Can only flip STRING and INTEGER values! in DrupalDefaultEntityController->cacheGet() (line 343 of includes/entity.inc).
Googling this problem did not help me find a definitive solution or root cause -it seems that the array values it is trying to flip are strings, so I think it may be an error in array_flip() – I’m not sure – but in order to get this working, I’ve hacked includes/entity.inc to convert the string vals to ints right before these array_flip() calls, and then everything appears to work…

$new_ids = array();
foreach($ids as $key => $value){
//    echo $value . " is of type " . gettype($value) . "<br>";

      $new_ids[$key] = intval($value);
}
$ids = $new_ids;

Taxonomy

I found that the taxonomies and node relationships survived the upgrade. However – there were a few changes in the db schema: term_node becomes taxonomy_index and term_data becomes taxonomy_term_data

where, in D6 I use to resort to this PHP snippet in ‘Page specific visibility settings’ to determine whether or not to display a block on a give node:

<?php
$content_area = 'some_content_area_term';

if (arg(0) == 'node' && is_numeric(arg(1))) {
  $nid = arg(1);
  $node = node_load(array('nid' => $nid));
  if(isset($node->taxonomy) && is_array($node->taxonomy)){
    foreach($node->taxonomy as $taxonomy){
      if($taxonomy->name == $content_area){
        return TRUE;
      }
    }
  }
}
return FALSE;
?>

Given the changes D7 brought, I had to craft the following function to determine if there existed a relationship between node and term id:
(reference http://drupal.org/node/733856)

function connection_exists($nid, $term){
    if (module_exists('taxonomy') && $nid > 0 && $term) {
        $node = node_load($nid);
        $vocabs_exist = TRUE;
        $vocab_index = 1;
        while($vocabs_exist){
            $field_name = 'taxonomy_vocabulary_' . $vocab_index;
            $term_index = 0;
            $terms_exist = TRUE;
            if(!empty($node->{$field_name})){
                while($terms_exist){
                    if(isset($node->{$field_name}['und'][$term_index]['taxonomy_term'])) {
                        $term_obj = $node->{$field_name}['und'][$term_index++]['taxonomy_term'];
                        if($term_obj){
                            if(strcasecmp($term_obj->name, $term) == 0){
                                return TRUE;
                            }
                        }
                    }
                    else{
                        $terms_exist = FALSE;
                    }
                }
            }
            else{
                $vocabs_exist = FALSE;
            }
            $vocab_index++;
        }
    }
    return FALSE;
}

I’m not sure if this is actually kosher Drupal – the suggested method of calling view_field_view on the node tag field_tags did not work for me – perhaps since I was using taxonomies that had just been upgraded from D6 and I wasn’t actually adding terms in 7?

Node Privacy Byrole

I was concerned about this module in D7, as I did not see much discussion or many issues(or recent updates for that matter) with it that were related to Drupal 7. It turns out to work fine. The db entries from the D6 site carried over fine, and I was able to verify that the users, roles and permission settings were pretty much intact. However – I did have to rebuild the content access permissions (admin/content/node-settings/rebuild) to get this to work correctly.

WYSIWYG

This module seems to work fine in D7. I’ve been impressed with the CKEditor, so I plug that in and it has been a good combination. Keep in mind – CKeditor installs out of the box with NO buttons displaying in toolbar. You must go in and configure presets to enable those menu items within the specific input formats that you want to expose.


user-profile.tpl.php

I needed to override this template, and in the past I was able to add regions by overriding template_preprocess_user_profile and adding a line like:
$variables['editable_node_list_region'] = theme('blocks', 'editable_node_list_region');

But this function has changed, and it’s not clear at all how to make this happen(If yo know, please tell me!). So for now I’m explicitly taking the output directly from the module that produces the html for this region, and displaying it from within the user-profile.tpl.php

$block_content =  editable_node_list_block_view();
        if(isset($block_content['content']) && $block_content['content'] != ''){
            echo "<br><h3>Editable Pages</h3>";
            print $block_content['content'];
        }

Undefined index: description

I also got these errors on each node edit:

Notice: Undefined index: description in field_multiple_value_form() (line 156 of /xxxx/modules/field/field.form.inc).
Notice: Undefined index: required in field_multiple_value_form() (line 178 of /xxxx/modules/field/field.form.inc).
Notice: Undefined index: required in field_multiple_value_form() (line 207 of /xxxx/modules/field/field.form.inc).

What this is not exactly clear(I’ve come across some involved descriptions), but if you go into each Content Type and click on Manage Fields and then Save, it will go away. This is a known bug and I expect it will
be fixed in the near term.

jQuery and collapsible fieldsets

I was using this in D6, but in D7 you must also make sure that your legend contains a span with class ‘fieldset-legend’, and that your content is contained in a div with class ‘fieldset-description’, and that you include ‘misc/form.js’ for your javascript. Then it’ll work.

drupal_add_js('misc/form.js');
drupal_add_js('misc/collapse.js');
<fieldset id='fieldset-id' class='collapsible'>
<legend><span class='fieldset-legend'>Fieldset title</span></legend>
<div class='fieldset-wrapper'>
<div class='fieldset-description'>Fieldset description</div>
Fieldset content
</div>
</fieldset>

Access for site editors

When assigning your editors permissions, make sure they have (Node) ‘Access the content overview page’, and if you want them to access ‘Publishing options’ (as well as ‘Authoring information’ and ‘Revision information’), give them (Node) ‘Administer content’ permission. Of course, give them edit/view/delete on any content types they own. Also give them (Overlay) ‘Access the administrative overlay’ and (System) ‘View the administration theme’ and (Toolbar) ‘Use the administration toolbar’

Posted in Drupal, PHP, Tech snippets | Tagged , | Leave a comment

Automating Object Ordering in Symfony

The next logical step to my last post Automating __toString in Symfony would be to automate the ordering of objects that are obvious – eg. – that have a name, title, (last_name, first_name), or precedence fields. By automatically adding a peer method that would order the objects, and would be referenced in the CRUD(both in selects in the edit forms, and in the lists) when needed, we provide yet another shortcut in our model preparation. I include context diffs below that accomplish this.

Index: plugins/sfPropelPlugin/data/generator/sfPropelAdmin/default/template/actions/actions.class.php
===================================================================
--- plugins/sfPropelPlugin/data/generator/sfPropelAdmin/default/template/actions/actions.class.php   (revision 3)
+++ plugins/sfPropelPlugin/data/generator/sfPropelAdmin/default/template/actions/actions.class.php   (working copy)
@@ -34,6 +34,8 @@
     $this->pager->setPage($this->getRequestParameter('page', $this->getUser()->getAttribute('page', 1, 'sf_admin/getSingularName() ?>')));
 getParameterValue('list.peer_method')): ?>
     $this->pager->setPeerMethod('getParameterValue('list.peer_method') ?>');
+getPeerClassName() . '::doSelectOrdered')): ?>
+    $this->pager->setPeerMethod('doSelectOrdered');
 
 getParameterValue('list.peer_count_method')): ?>
     $this->pager->setPeerCountMethod('getParameterValue('list.peer_count_method') ?>');

Index: helper/ObjectHelper.php
===================================================================
--- helper/ObjectHelper.php     (revision 3)
+++ helper/ObjectHelper.php     (working copy)
@@ -117,6 +117,16 @@

   $peer_method = _get_option($options, 'peer_method');

+  if(!$peer_method){
+      $ordered_select = 'doSelectOrdered';
+
+      $classPeer = constant($related_class.'::PEER');
+
+      if(is_callable(array($classPeer, $ordered_select))){
+          $peer_method = $ordered_select;
+      }
+  }
+
   $text_method = _get_option($options, 'text_method');

   $key_method = _get_option($options, 'key_method', 'getPrimaryKey');

Index: plugins/sfPropelPlugin/lib/vendor/propel-generator/classes/propel/engine/builder/om/PeerBuilder.php
===================================================================
--- plugins/sfPropelPlugin/lib/vendor/propel-generator/classes/propel/engine/builder/om/PeerBuilder.php (revision 3)
+++ plugins/sfPropelPlugin/lib/vendor/propel-generator/classes/propel/engine/builder/om/PeerBuilder.php (working copy)
@@ -63,6 +63,7 @@
                // consider refactoring the doSelect stuff
                // into a top-level method
                $this->addDoSelectOne($script);
+               $this->addDoSelectOrdered($script);
                $this->addDoSelect($script);
                $this->addDoSelectStmt($script);         // <-- there's PDO code in here

Index: plugins/sfPropelPlugin/lib/vendor/propel-generator/classes/propel/engine/builder/om/php5/PHP5ObjectBuilder.php
===================================================================
--- plugins/sfPropelPlugin/lib/vendor/propel-generator/classes/propel/engine/builder/om/php5/PHP5PeerBuilder.php        (revision 3)
+++ plugins/sfPropelPlugin/lib/vendor/propel-generator/classes/propel/engine/builder/om/php5/PHP5PeerBuilder.php        (working copy)
@@ -534,6 +534,54 @@
        }

        /**
+        * Adds the doSelectOrdered() method.
+        * @param      string &$script The script will be modified in this method.
+        */
+       protected function addDoSelectOrdered(&$script)
+       {
+           foreach ($this->getTable()->getColumns() as $col) {
+                $column_names[] = $col->getName();
+            }
+
+            if(in_array('name', $column_names)){
+                $orderByColumn = 'NAME';
+            }
+            elseif(in_array('last_name', $column_names)){
+                $orderByColumn = 'LAST_NAME';
+                if(in_array('first_name', $column_names)){
+                    $secondOrderByColumn = 'FIRST_NAME';
+                }
+            }
+            elseif(in_array('title', $column_names)){
+                $orderByColumn = 'TITLE';
+            }
+            elseif(in_array('precedence', $column_names)){
+                $orderByColumn = 'PRECEDENCE';
+            }
+            else{
+               return;
+            }
+
+           $script .= "
+       /**
+        * Method to select an ordered set of objects from the DB.
+        *
+        * @param      Criteria \$criteria object used to create the SELECT statement.
+        * @param      PropelPDO \$con
+        * @return     ".$this->getObjectClassname()."
+        * @throws     PropelException Any exceptions caught during processing will be
+        *               rethrown wrapped into a PropelException.
+        */
+       public static function doSelectOrdered(Criteria \$criteria, PropelPDO \$con = null)
+       {
+               \$critcopy = clone \$criteria;
+               \$critcopy->addAscendingOrderByColumn(".$this->getPeerClassname()."::$orderByColumn);".(isset($secondOrderByColumn) ? '
+                $critcopy->addAscendingOrderByColumn('.$this->getPeerClassname().'::'. $secondOrderByColumn.');' : '') ."
+               return ".$this->getPeerClassname()."::doSelect(\$critcopy, \$con);
+       }";
+       }
+
+       /**
         * Adds the doSelectOne() method.
         * @param      string &$script The script will be modified in this method.
         */
Posted in PHP, Symfony | Tagged , , | Leave a comment

Automating __toString in Symfony

Symfony is a powerful PHP based application framework, and has continued to improve through the hard work and support of its authors and contributors. However – there are a couple things that could be added that would greatly simplify the life of the user(me :) ). Once you carefully define your relational database schema, you run a series of commands to create your object model – a set of PHP based objects that map to your database tables; and also create your admin scaffolding – a set of CRUD pages that provide web access to maintain your database model. All of this happens within minutes. Pretty powerful.

Once you’ve gotten this far, though, all foreign key references in your data will show up in your CRUD pages with their given row id’s, not the name or title that you’ve given that object in its table. You can then create a __toString() method in that objects class file to return a human readable string for listings and edit pages. This is relatively straightforward, but if you have a large model, you may begin to find this somewhat tedious – as described in Michael Kimsal’s blog.

So – as many of the tables that I utilize in Symfony will typically have either a ‘name’ or ‘title’ field, I’d benefit from having the __toString() method automatically generated if my object has either the ‘name’ or ‘title’ field. Turns out this is not that difficult to accomplish. I’ve made the following modifications to a Symfony 1.2.4 install.

By adding an addToStringAccessor() method to /usr/share/pear/symfony/plugins/sfPropelPlugin/lib/vendor/propel-generator/classes/propel/engine/builder/om/php5/PHP5ObjectBuilder.php:

/**
 * Adds a __toString method in order to provide default human readable
 * representation of any table that has a field named 'name' or 'title'.
 * @param      string &$script The script will be modified in this method.
 * @param      Column $col The current column.
 */
protected function addToStringAccessor(&$script, $column_name)
{
        $script .= "
          public function __toString()
          {
              return \$this->$column_name;
          }
        ";

}



and then by modifying the addColumnAccessorMethods() method in /usr/share/pear/symfony/plugins/sfPropelPlugin/lib/vendor/propel-generator/classes/propel/engine/builder/om/ObjectBuilder.php from:

protected function addColumnAccessorMethods(&$script)
{
        $table = $this->getTable();

        foreach ($table->getColumns() as $col) {

                // if they're not using the DateTime class than we will generate "compatibility" accessor method
                if ($col->getType() === PropelTypes::DATE || $col->getType() === PropelTypes::TIME || $col->getType() === PropelTypes::TIMESTAMP) {
                        $this->addTemporalAccessor($script, $col);
                } else {
                        $this->addDefaultAccessor($script, $col);
                }

                if ($col->isLazyLoad()) {
                        $this->addLazyLoader($script, $col);
                }
        }
}



to:

protected function addColumnAccessorMethods(&$script)
{
        $table = $this->getTable();

        $to_string_column = '';

        foreach ($table->getColumns() as $col) {

                $column_name = strtolower($col->getName());
                if(($column_name == 'name' || $column_name == 'title') && $to_string_column != 'name'){
                    $to_string_column = $column_name;
                }

                // if they're not using the DateTime class than we will generate "compatibility" accessor method
                if ($col->getType() === PropelTypes::DATE || $col->getType() === PropelTypes::TIME || $col->getType() === PropelTypes::TIMESTAMP) {
                        $this->addTemporalAccessor($script, $col);
                } else {
                        $this->addDefaultAccessor($script, $col);
                }

                if ($col->isLazyLoad()) {
                        $this->addLazyLoader($script, $col);
                }
        }
        if($to_string_column != ''){
            $this->addToStringAccessor($script, $to_string_column);
        }
}



you’ll have your Symfony installation automatically generating __toString() methods in whichever objects have a name or title field. If you notice, I prefer the name field if there are both name and title fields in the table. Of course, you can change those keywords to whatever you use in your database schema. A useful modification would be to key off of an array that you’d defined in the settings.yml file, instead of having these column names hardcoded here.

This is part one of this solution. The second part involves automatically referencing these in your listing pages. Typically a listing, when automatically generated, will display the row id of any foreign key references in your table and corresponding PHP object.

If you utilize the powerful Admin Generator, you can quickly indicate which fields you want displayed(and how) in each edit or listing page. Utilizing this and a separate little PHP file called a partial, you can reference the object pointed to by your foreign key reference by calling its __toString() method. Each partial looks something like this:

$customer = CustomerPeer::retrieveByPk($customer_communication->getCustomerId());
echo $customer->__toString()

So – in an effort to not have to write a partial for each and every foreign key reference that you want to display in a listing, you can add the following clause to the getColumnListTag() method in /usr/share/pear/symfony/plugins/sfPropelPlugin/lib/generator/sfPropelCrudGenerator.class.php

    else if($column->isForeignKey())
    {
      $getter = 'get'.$this->getRelatedClassName($column);
      return sprintf('$%s->%s()', $this->getSingularName(), $getter);
    }

Once you have this in place, your listing will automatically now look like you intended:

But wait! There’s more. As an added bonus, or what’s called lagniappe where I grew up, you also get sortable columns in this modified approach. Typically, all of your columns that are resident in the given table/object will be sortable in the listing page. Once you introduce partials to display the human readable text instead of the row id for those foreign key references, you lose that ‘sortability’ on those ‘foreign’ columns. Employing this approach brings that sortability back, and you can now sort on any column in your listing.

Caveats: For any foreign key references whose tables do NOT have a name or title field, you will get an error when trying to display the listing, until you define a __toString() method.

These two changes are mutually exclusive, although they do work well together.

Posted in PHP, Symfony | Tagged , , | 1 Comment

Monitoring Windows with Nagios

Nagios is a great open source monitoring tool, and allows a wide range of options for monitoring servers and network devices of all varieties. SNMP is supported as are a custom set of plugins for the monitored clients. When it comes to Windows machines, there are also good options, but configuring it and getting it working is not a ‘one-click’ operation, thus the subject of this post.

Once you get this working, you’ll be able to view the Services page of your Nagios console, and view something like this:

Nagios Services GUI with monitored Windows machine

Nagios Services GUI with monitored Windows machine

This involves utilizing the NSCA facility in Nagios, which allows you to receive passive updates into your monitoring system via a proprietary protocol. This requires installing the monitoring software on the (client, monitored) Windows machine, configuring it to point at your Nagios server, then installing it as a Windows Service. Then you will modify your Nagios configuration and add this Windows machine, as well as the names of the services on the machine that you’ve chosen to monitor.

Setting up your client(monitored) machine

There are several options for the software that can be installed on Windows to enable this monitoring. Two of the better options are NC_Net and NSClient++ – I believe these are the ones that are being currently maintained. They both provide the ability to perform active checks (Nagios server contacts Windows machine to retrieve data), but NC_Net also provides the ability to perform passive checks(Windows machine sends data to Nagios server). The fewer open ports at the firewall, and fewer ways to get into each server is certainly preferable – not to mention lessening the load on your Nagios server. So – I prefer deploying passive checks for my Windows servers. NC_Net also appears to be more extensible, and offers more options. The only downside is that it requires dotNet – currently dotNet 2.0 – but that comes with the territory, so…

Be sure to download NC_Net from its Sourceforge project page I’m currently running NC_Net 4.4.0. Note: The original(?) version available from shatterit.com – which prominently claims to be the “Official Site” – hasn’t been updated for several years, and should really be taken down. Anyway – download the one from Sourceforge and install it on your Windows host. It will run on XP and WS 2003, I’m not sure about Vista or WS2008. You’ll need to modify several configuration files. At this point, it installs by default into C:\Program Files\Montitech\NC_Net

Within the config dir, there are two files that you need to modify – startup.cfg and passive.cfg. They are well documented internally, so you can read through them to fully understand all of the options. Or – if you just want it to work quickly, enable(uncomment) the following options:

startup.cfg:

active_check false
passive_check true
passive_alwayson true
embedded_send_nsca true
host_passive <windows_machine_nagios_host_id>
ip_passive <nagios_server_ip>

passive.cfg

C testrun false
2 cputotal -l 10,80,90,5,20,90
3 uptime
4 usedspace -l C -w 80 -c 90
5 servicestate -d showall -l NC_Net
7 Memory Use
8 Perf Counter -l "\Paging file(_total)\% usage","Paging File usage is %%.4f %" -w 50 -c 60
10 Instances -l System,Process,Memory,Processor
11 EventLog -l Application,any,10,1,NC_Net,-2,start,stop,0 -w 5 -c 20

I’ve enabled just a standard set of checks for illustrative purposes here. Read through the passive.cfg file to understand the different commands and their options.

If you do not already have the dotNet framework installed on your windows machine, d/l and install it. As of this writing, NC_Net requires dotNet 2.0 – that is what I’ve installed to get NC_Net 4.4.0 working.
It is currently available at dotNet 2.0

Then, from the command line within the NC_Net dir, enter:

Net Start NC_Net

This will start NC_Net as a service, which will attempt to contact your Nagios server at the default NSCA port(5667) once it has some data to report.

Setting up your Server

If you don’t have the NSCA addon installed in conjunction with Nagios, then download it from http://www.nagios.org/download/addons/, and install it. Note: I’m running Centos 5.2, and Nagios 3.0.3. One of the prereqs for NSCA is libmcrypt – if you’re missing that (locate libmcrypt.so), then you’ll need to d/l and install that prior to compiling NSCA.


mkdir /usr/local/src
cd /usr/local/src
tar xzf {your download dir}/nsca-2.7.2.tar.gz
cd nsca-2.7.2
sh ./configure
make all
cp src/*nsca /usr/local/nagios/bin/
chown nagios.nagios /usr/local/nagios/bin/*nsca
cp sample-config/nsca.cfg /usr/local/nagios/etc/

edit the nsca.cfg file and change the IP to your Nagios server’s IP (interface that you want NSCA listening on)
server_address=<nagios_server_IP_address>

You can then start the daemon:
/usr/local/nagios/bin/nsca –c /usr/local/nagios/etc/nsca.cfg
There are a couple ways to insure that the NSCA daemon starts automagically, and you can find a complete treatment of these here: http://nagios.sourceforge.net/download/contrib/documentation/misc/NSCA_Setup.pdf

Make sure the daemon is listening: netstat -an|grep 5667, and that you’ve configured firewall(s), as well as iptables and/or selinux on your Nagios server to be able to access your NSCA daemon at port 5667.

So – depending upon how the logging of your Nagios installation is configured, you should be getting some messages via syslog – typically by default in /var/log/messages, where, after at least five minutes, you should see some messages like the following:


Sep 6 18:43:13 nagiosadmin nagios: PASSIVE SERVICE CHECK: <windows_machine_nagios_host_id>;cputotal;0;OK - load average: 0%, 0%
Sep 6 18:43:13 nagiosadmin nagios: PASSIVE SERVICE CHECK: <windows_machine_nagios_host_id>;uptime;0;System Uptime - 23 day(s) 15 hour(s) 38 minute(s)
Sep 6 18:43:13 nagiosadmin nagios: PASSIVE SERVICE CHECK: <windows_machine_nagios_host_id>;usedspace;0;C: - total: 19.99 Gb - used: 7.23 Gb (36%) - free 12.75 Gb (64%)
Sep 6 18:43:13 nagiosadmin nagios: PASSIVE SERVICE CHECK: <windows_machine_nagios_host_id>;servicestate;0;NC_Net: Started
Sep 6 18:43:13 nagiosadmin nagios: PASSIVE SERVICE CHECK: <windows_machine_nagios_host_id>;Memory Use;0;Memory usage: total:2464.94 Mb - used: 146.97 Mb (6%) - free: 2317.97 Mb (94%)
Sep 6 18:43:13 nagiosadmin nagios: PASSIVE SERVICE CHECK: <windows_machine_nagios_host_id>;Perf Counter;0;"Paging File usage is %%.4f %" = 0.17 %
Sep 6 18:43:13 nagiosadmin nagios: PASSIVE SERVICE CHECK: <windows_machine_nagios_host_id>;Instances;0;System: - Process: alg,svchost#2,svchost#4,svchost#3,winlogon,svchost#1,svchost,_Total,wmiprvse,inetinfo,services,spoolsv,dllhost#1,smss,logon.scr,lsass,logonui,cygrunsrv,csrss,System,msdtc,dllhost,sshd,NC_Net,snmp,Idle - Memory: - Processor: _Total,0
Sep 6 18:43:13 nagiosadmin nagios: PASSIVE SERVICE CHECK: <windows_machine_nagios_host_id>;EventLog;0;OK: No entries in Application log recently.

Setting up your host in Nagios

You’ll need to configure Nagios to have entries mirroring the host and services that you’ve just set up in your NC_Net configuration. I’ve been using Nagios Administrator, which is a decent GUI interface into the Nagios command files. It is built upon Symfony, a powerful LAMP-based framework.

You should set up the following items in this order:

Add the Command

Add a ‘check_dummy’ command (if it doesn’t already exist)

Nagis Administrator check_dummy Command

Nagis Administrator check_dummy Command

Add the Services

Add a service to correspond with each passive check that you’ve defined in the NC_Net passive.cfg file. The Service’s Name field has to correspond with the name of the check specified as the second field of each enabled check in the passive.cfg. For instance, for the ‘cputotal’ passive check, define a service like:

Nagios Administrator cputotal service

Nagios Administrator cputotal service

Note that the ‘check_dummy’ command must be specified, and flag ‘use passive-service’ must be set in the ‘Special’ field.

Add the Host

Once you’ve added all the services that correspond to the passive checks you’ve enabled, add the host that corresponds to your Windows machine:

Nagios Administrator add Windows machine

Nagios Administrator add Windows machine

Note that – most importantly – the Name must match the name specified in the NC_Net startup.cfg – <windows_machine_nagios_host_id>, and that the flag ‘active_checks_enabled’ must be set to ’0′, and you must specify the correct IP address, host group, OS and contact group.

Nagios Administrator services for Windows machine

Nagios Administrator services for Windows machine

Scroll down and specify the correct services to match up to those in your NC_Net passive.cfg for that host.

Once you have these items all created successfully, go to the Generator screen and ‘Save’ your configuration. On your Nagios server, restart the Nagios server. service nagios restart

Now you should soon see the correct service entries on your Nagios GUI screen populated.

This is good treatment of monitoring Exchange Server with these tools.

Posted in Host Monitoring, Nagios | Leave a comment

iPhone and IMAP/SSL/self-signed cert

I’m the proud owner of a new iPhone 3G, but have had a hard time getting my SSL IMAP email working. A bit unusual, since I’ve been involved in supporting Internet email operations since the early 90′s. I run my own IMAP server (dovecot) on Linux, and utilize SSL encrypted IMAP on standard ports. I employ a self-signed cert as the price is right, and the folks that I provide email for all know who I am or who Blackfin Software is, so ‘verified identity’ is not an issue.

It turns out that you cannot accept and store a self-signed cert from within the iPhone mail application. So – what you need to do is to get it there from your syncing host. I’ve done this on a mac, so I’m not sure if it will work from a PC. You’ll need to go through the normal procedure of setting up your IMAP mail account utilizing Mac OS X Mail. The first time it contacts your IMAP server, it will complain that the certificate presented by your mail server is not trusted. Examine/display the cert, and then click and drag the actual cert to your desktop. Once it is there, double click on it. It should open it with the ‘Keychain Access’ app. Import the certificate into your (default) login keychain and once there, go into it and modify the trust settings such that it is ‘trusted always’. Exit Keychain access. Make sure that you can quit and restart Mail, and that it is able to both receive and send email with the Mac Mail app – without asking you if you want to trust the self signed cert.

Now you’ll need to sync your iPhone with your Mac. It should pick up these new email settings as well as the new trusted certificate from your keychain. If you have another account configured on your iPhone Mail, you may need to disable it. I had to actually reboot my iPhone (turn off, turn back on), to get these settings to work. This may or may not have had to do with the fact that I had been trying to configure the account directly on the iPhone. At this point it picked up my IMAP folders, etc. YMMV.

Hope this helps….

(iPhone 3G, Mac OS X 10.4.(?)7)

Posted in Tech snippets | 1 Comment

Saving those precious minds

In my capacity as a Dad, I find it a bit scary when my kids are online, and I don’t have any reliable monitoring software or – better yet – a good porn filter – in place to protect them from some of the more unseemly business out there. When we had our ISP – Gulf Coast Internet, based in the thriving metropolis of Pensacola, FL – back in the glory days of the net prior to the dot bomb, we provided a filtering service to some of our customers so that anyone using their dialup account would not be subject to the bad stuff. The provider of this filtering service – Nessus – did a remarkable job of maintaining a list of objectionable content. Subscribers to their service were pleased to have this net based filter available to them. It was 100%(well – extremely) reliable, in that it wasn’t software that Junior knew how to get around at night when you were in sleeping or out for dinner, etc. Also, and perhaps almost as importantly, this solution did not compel you to install yet another software package along with all the assorted and sundry bloatware that is already dragging your machine down to a crawl.

So, after we left the ISP behind, I was still able to use this filter for while, but when they finally reconfigured that box, or got rid of that service, we were left with the ‘look over their shoulder’ method of child-internet surf monitoring. Keeping the computer that the kids can use in a central part of the house, such as the kitchen, helps this process, but it is still a losing proposition.

So – what is available out there to help you? All kinds of software packages for the Windows environment that promise to prevent the kids from doing all of those things that they shouldn’t be doing online. Just install it after you’ve installed your antivirus, antiphishing, antipopup, antiadware, etc. – and watch that system slow down just a little bit more. I mean – how much time does the average parent spend trying to perform all of the administrative tasks that they need to do to try to plug all the holes left in MS software? I for one find it particularly frustrating, knowing that it does not have to be this way. Go buy a Mac already. Which is what we’ve done, a couple times now in the last three years. After OS X came out, and I became aware of how slick and polished the entire package is, we had to take the plunge. The family got a new iMac with the swivel screen and it sat in the kitchen on the bar, where the screen could be turned in for recipe perusal, and out for the kids homework and surfing. Apple is doing a good job locking down and maintaining security. It is really nice not having to worry about installing and maintaining all of the antivirus and anti-adware tools. The tools are all pretty good, and fairly intuitive – this greatly reduces the internal tech support load – which is not what computer people want to do after dinner. Ask me how I know.

Anyway, the point of this diatribe is that I needed a solution that would work for a small home network that consists of both PC’s and Mac’s – I use PC’s in my work as my customers do, so it’s not a matter of choice. So, it used to be that Belkin was providing very inexpensive routers that included the ability to use a third party filtering service – one that is now called Blue Coat – used to be Cyberian. What a great tool for families, I thought. After repeated attempts to get it working on my Belkin pre-N wireless router, I went through the normal tech support nightmare on steroids – particularly bad since Belkin is primarily a cable accessory company – only to find out that they stopped providing this service.

There are some more expensive routers that provide filtering, but after some peeking and poking, I found that some of the less expensive ZyXEL routers provide access to this third party filtering. So, I got a ZyXEL Zywall 2 plus, as that seemed to be the cheapest one that did provide this service. Still – at about $150, and $60 a year or so for the service, it is a bit more expensive than the $30 linksys or netgear router/wifi hubs that end up in most homes. But – peace of mind it does provide. I had some history with ZyXEL, apparently a Taiwanese company that appears to be much more engineering oriented as opposed to marketing driven. We had used their modems when we first started our ISP – they provided a technology that gave you a 16.8! kbps connection as opposed to the slower 14.4. Anyway – I’d read some good things about the ZyXEL modems and had some sucess with them, and we had a good experience with them at Gulf Coast. This router is a bit more complex than your consumer grade linksys, and there’s a little more configuration required, but it’s been working solidly for me now for several months, and I recommend it highly for all parents with more than one computer at home – irrespective of whether or not those computers are Macs or PCs. I found some of the best prices on the net at a small company called Nowthor – http://shopping.nowthor.com/zyxel-zywall-2-plus.html – I’ve ordered both the router and the subscription to the filtering service(what ZyXEL calls the iCard Silver) through them, and gotten good turn around time.

Posted in Kids, Tech snippets | Leave a comment