Updated documentation for Foundation 1.1

February 28th, 2006 by roger

The online documentation for Foundation 1.1 has been enhanced with topics like:

  • Foundation Architecture Overview
  • Dataflow
  • Data Integration approaches
  • Examples how to create feeders and adapters

You can get the latest document from the Foundation Home page on SourceForge or download the pdf directly.

GroundWork Foundation Milestone 2 Released

February 13th, 2006 by roger

GroundWork Foundation is an IT management data abstraction layer and development platform. Foundation 1.1 has been enhanced to integrate more diverse data sources into a single data store. This was achieved by introducing properties for describing the data points instead of using database fields for each measurement. The APIs and the existing feeders are backward compatible with previous versions.
The Foundation data model allows the integration of any state, event, and performance data, independent of the Monitoring Application that produces it. This offers the possibility to store data for additional systems, including open source and commercial monitoring systems, databases, and even hardware, such as detectors or sensors. It also allows the integration of Application Monitoring data known as MBeans.

GroundWork Foundation Milestone 2 (M2) Release

The current release includes the following enhancements and features:

  • Binary distribution for Java 1.4 and Java 1.5. No need to compile from source; just install configure and use it! For detailed instructions see the BINARY-INSTALL.txt included in the package.
  • Administration Adapter that allows configuration of the system at run time through data feeds
  • Event message consolidation. Similar messages are consolidated depending on criteria set by the user instead of creating a new entry for each event.
  • Upgrade to version 3.1 of Hibernate, an Object/relational persistence tool. The latest version improves the performance over previous releases, and is EJB 3.0 compliant
  • Better integration into the Spring framework. All adapters are now configurable through Spring assemblies.
  • Tutorial on how to create an adapter that plugs into the Foundation Framework. The tutorial shows how to create an adapter for SNMPTrap messages. The tutorial is available at: Tutorial

We plan to roll out the final release of Foundation 1.1 by mid March 2006. We are still looking for contributions from the Open Source community, so if you are a Foundation developer, please consider contributing your code for inclusion in the final release.
Below is a list of features under development (ToDos):

  • JMS Listener — Embedded topic server that would accept feeder messages from any JMS subscriber.
  • Adapters for any type of Monitoring data (JMX, log4j) or from other Open Source or commercial tools
  • Support for PostgreSQL, Derby, MS-SQL, Oracle, or other databases.

If you are interested in contributing to Foundation please contact Foundation development team

All Guava, All the Time

December 29th, 2005 by Administrator

Taylor has written up some excellent Guava docs. There’s one on installing Guava, another on the Guava framework and environment and yet another on building your first Guava app. If you want to see a Guava in action go to the demo page and log in as “guest” “guest”.

Event Broker Discussion: Part 2

December 28th, 2005 by taylor

This post took a bit longer than a week to write, as I’ve been heavily swamped by other projects. But enough about that - away we go.

We’re going to go over the basics of an event broker module. We’re not going to deal too much with the insides of Nagios and the functions that Nagios provides to make our life easier. We’ll go over those items in the future. What we’ll go over in this article, however, is building a basic event broker module skeleton which loads inside nagios. It’s not much. However, it does lay down the basic footprint needed for future discussion. Next week, we’ll deal with handling changes in status.

As stated in my previous article, an event broker is basically a shared library with at least two functions (entry-points) required. We’re going to start off with a skeleton of an event broker which is detailed below and go over what it contains:

#ifndef NSCORE
#define NSCORE
#endif
/* include the needed event broker header files */
#include "../include/nebmodules.h"
#include "../include/nebcallbacks.h"
#include "../include/nebstructs.h"
#include "../include/broker.h"
/* include some Nagios stuff as well */
#include "../include/config.h"
#include "../include/common.h"
#include "../include/nagios.h"
#include "../include/objects.h"
/* specify event broker API version (required) */
NEB_API_VERSION(CURRENT_NEB_API_VERSION);
// Our module handle
void *basic_module_handle=NULL;
// This function gets called when the module gets loaded by the event broker
int nebmodule_init(int flags, char *args, nebmodule *handle) {
	basic_module_handle = handle;
	write_to_logs_and_console("Loading Basic Event Broker Interface...",NSLOG_INFO_MESSAGE,TRUE);
	return 0;
}
// This is our unloading function which gets called by the neb
int nebmodule_deinit(int flags, int reason)
{
	write_to_logs_and_console("Unloading Basic Event Broker Interface...",NSLOG_INFO_MESSAGE,TRUE);
	return 0;
}

To compile properly, you’ll need access to the nagios header files. The easiest way to do this is to download the source of the Nagios version you are running and extract it. Then inside the nagios-2.xxx directory, create a new directory to store your event broker code. We’re going to store it in a directory called ‘basic’ for simplicity sake. Save it to basic.c, and compile it using gcc as so:

gcc -shared basic.c -o basic.o

This will compiled basic.c as a shared library, and save the compiled form as basic.o. Now you must decide where you want to keep your compiled modules. We’ll store our modules in /usr/local/libexec/nagios/modules. You can change this path based on your installation requirements. Next, you must modify your nagios.cfg to contain the proper event broker related options.

event_broker_options=-1
broker_module=/usr/local/libexec/nagios/modules/basic.o

The event_broker_options declaration dictates what events you want the event broker to manage. From current understanding, there is only 0, which means to monitor nothing, and -1 which means to monitor everything. The second directive broker_module, points to our broker module which we compiled and moved to a standard directory. The broker_module takes 1 or more parameters, seperated by spaces. The first is the path to the module, and the remaining parameters (none shown here), are space seperated parameters to pass to your broker module. Parameters will be covered in later articles.

Now, if you run Nagios by hand, using nagios <path to modified nagios.cfg>, you should see Nagios start up, along with the lines of:

Loading Basic Event Broker Interface...

If you look at the Nagios log file, you’ll see:

Loading Basic Event Broker Interface...
Event broker module '/usr/local/groundwork/modules/basic.o' initialized successfully.

This tells us our event broker module is configured properly in nagios.cfg, and that it is correctly loading.

In the future, changes to the basic.c file should be made, then the gcc command should be used to recompile. Then copy the newly created basic.o module to the module directory. Re-running nagios will start the changed event broker module.

Okay, we’re now going to go over our skeleton module.

#ifndef NSCORE
#define NSCORE
#endif

These lines will define the constant NSCORE, if not already done so. You must do this before including the following header files, as this will allow us to see additional structures. The name NSCORE probably comes from the combination of Net-Saint Core. As Net-Saint was the name of nagios before, and core is usually given to the files which make up the core of nagios, of which needs access to all data structures.

#include "../include/nebmodules.h"
#include "../include/nebcallbacks.h"
#include "../include/nebstructs.h"
#include "../include/broker.h"

These header files define the structures, constants, and functions which we will use as the basic level of being an event broker. We’ll go over the details of these header files and what they contain in later discussion.

/* include some Nagios stuff as well */
#include "../include/config.h"
#include "../include/common.h"
#include "../include/nagios.h"
#include "../include/objects.h"

These headers contain basic data structures and configuration parameters for the nagios system. config.h is created when you run the configure script inside the source directory. If gcc complains of a missing config.h, run configure (with the appropriate flags, especially -enable-event-broker), and try compiling again. We don’t use all the data structures and functions exposed by these header files in this module as it stands now; however, we will be using them in later discussion.

NEB_API_VERSION(CURRENT_NEB_API_VERSION)

This macro needs to be included for the purpose of making sure that when you load an event broker module with nagios, it will be loaded against the right version of the event broker api. If the event broker api differs in anyway, the CURRENT_NEB_API_VERSION will change, and Nagios will not load the module.

// Our module handle
void *basic_module_handle=NULL;

This global variable is a pointer to the memory address of our module. We’ll use this from time to time when registering callbacks. We don’t register callbacks currently, but we will require this later on.

// This function gets called when the module gets loaded by the event broker
int nebmodule_init(int flags, char *args, nebmodule *handle) {
	basic_module_handle = handle;
	write_to_logs_and_console("Loading Basic Event Broker Interface...",NSLOG_INFO_MESSAGE,TRUE);
	return 0;
}

Finally, some meat and potatoes. The nebmodule_init function gets called when the Event Broker attempts to load our module. There are three parameters which get passed to this function. flags is usually a constant which defines the context in which the modules are loading. In the beginning when nagios first loads all modules, this will be equal to the constant NEBMODULE_NORMAL_LOAD. The string args is the same space seperated parameter list that you provide in nagios.cfg (if any). The nebmodule pointer handle is the memory address of our module. The first line in this function assigns basic_module_handle the same address which handle points to. This will be used later when we register our first callbacks. The next line uses a function exposed by nagios. It’s one of the many logging functions that nagios uses and in turn we use. write_to_logs_and_console takes three parameters. The first is the text string of the message you want to write. The second is the type of message. This is going to be one of the many constants that define what type of message it is. But for now, use NSLOG_INFO_MESSAGE, which means this message is informational and doesn’t have any real severity. The last parameter is a boolean which states if we want the text written to the console (in this case, we do).

// This is our unloading function which gets called by the neb
int nebmodule_deinit(int flags, int reason)
{
	write_to_logs_and_console("Unloading Basic Event Broker Interface...",NSLOG_INFO_MESSAGE,TRUE);
	return 0;
}

This is the last required function needed in our module. nebmodule_deinit gets called whenever nagios needs to gracefully unload modules. If nagios is forced to shutdown; however, this function may not be called. So don’t count on it being called in all circumstances. (In fact, I really can’t find a scenario when this DOES get called; however, the sparse code that there is available in current nagios distributions does have this function, so we’ll keep it there for safety sake). In this, we use the same write_to_logs_and_console function to print a friendly message that we are shutting down.

That’s the basic structure of an event broker module. It’s very simple in nature; however, next time we’ll actually start registering callbacks so we can get told when certain events happen in the nagios process. Add some comments, let me know where you want this to go. Do you want to store status data in a database? In a directory? In some other fashion? Do you want to talk more about the logging functions, or the utility functions in nagios that we can use? Add comments, and we’ll discuss them in future posts.

Until next time (which will hopefully be a week), you should dig into the header files that were included in our sample module. This will give you a heads up during the next post when referencing constants and data structures.

If there’s anything you need to discuss which isn’t related to this article, feel free to contact me at [email protected]

Thanks for the read. Have a wonderful New Years.

ZDNet Blogs on GroundWork - Palamida Announcement

December 23rd, 2005 by Administrator

ZDNet blogger Dana Blankenhorn posted about the GroundWork - Palamida announcement regarding software transparency and learning the source of all your open source code. Also takes a look at Palamida’s new website, ipingredients.org. You can take a look at our announcement on December 7.

GroundWork Guava Update

December 22nd, 2005 by Administrator

GroundWork Guava is an Ajax’d PHP development framework that we’re really excited about. Guava will be the front-end basis of our next generation product coming out later next year. Taylor has built a small chat application with Guava to demonstrate its abilities. Log in to our Guava demo as “guest” and “guest”. You need to use recent versions of Firefox or IE.

In other news, Ajaxian has posted a a bit about Guava. Stop by our demo and chat a bit :@)

Anonymous Subversion Access to GroundWork Code

December 20th, 2005 by Administrator

Now you can get early access to some of GroundWork’s open source code repository via the Subversion source code management tool. Browse on over to our Ajax’d PHP framework, Guava and a chat applet based on Guava. These are bleeding-edge projects that will require you to roll up your sleeves, but they’re quite exciting. Look for a revamped and Guavafied Status Viewer and a new version of Foundation soon.

Jetspeed-2.0 Released

December 19th, 2005 by roger

The Apache Portals Jetspeed Team is pleased to announce the final release of the Jetspeed 2.0 Open Source Enterprise Portal. This final release is fully-compliant with the Portlet Specification 1.0 (JSR-168). Jetspeed-2 has passed the TCK (Test Compatibility Kit) suite and is fully CERTIFIED to the Java Portlet Standard.

The Jetspeed team will be presenting the new 2.0 release at ApacheCon US 2005
on December 10th in San Diego.

Highlight of main features:

Standardized:
* Fully compliant with Java Portlet API Standard 1.0 (JSR 168)
* Passed JSR-168 TCK Compatibility Test Suite
* J2EE Security based on JAAS Standard, JAAS DB Portal Security Policy
* LDAP Support for User Authentication

Foundation Component Architecture:
* Spring-based Components and Scalable Architecture
* Configurable Pipeline Request Processor
* Auto Deployment of Portlet Applications
* Jetspeed Component Java API
* Jetspeed AJAX XML API
* PSML: Extended Portlet Site Markup Language
- Database Persistent
- Content Management Facilities
- Security Constraints

Administrative Portlets:
* User, Role, Group, Password, and Profile Management
* JSR 168 Generic User Attributes Editor
- JSR 168 Preferences Editor
- Site Manager
- SSO Manager
- Portlet Application and Lifecycle Management
- Profiler Administration
- Statistics Reports

Application Servers Supported:
* Tomcat 5.0.x
* Tomcat 5.5.x
* Websphere 5.1, 6.0
* JBoss

The Nagios Event Broker Interface

December 6th, 2005 by taylor

Introduction to the Nagios 2.0 Event Broker Interface (NEB)
Of all the features to come out of the 2.0 branch of development for Nagios, the event broker interface is perhaps the most important and most powerful. The purpose of the Event Broker interface is to give 3rd party developers the capacity to develop callback routines which are performed whenever certain events occur during the Nagios monitoring process; however, there are additional possible benefits to this interface which can prove to be powerful but dangerous.

What is the Event Broker?
The Event Broker is an interface in Nagios which links shared code libraries into the Nagios core process at run-time. When an event occurs during the Nagios process, the Event Broker will be ordered to send notification of these events and their relevant data to registered callback procedures which are inside the shared libraries (known as NEB Modules).

What is a NEB Module?
A NEB (Nagios Event Broker) Module is a shared library (usually written in C, but support for C++ is now included). With a common entry-point procedure, it registers callback procedures with the Event Broker and asks to be notified whenever specific events occur. After the point of receiving the notification of an event occurring, the possibilities of what the NEB module can do at that point is endless. Common ideas for the purposes of NEB Modules are:

  • Store event data into a database (since db support is removed from Nagios)
  • Convert events into other formats (SNMP traps, syslog)

What kind of events can a NEB Module process?
When a Nagios Event Broker module initializes, and technically any other time the Nagios code pointer is within one of it’s procedures, it can register a callback to be processed whenever any of these events occur:

  • Related to Process Data (Nagios startup, shutdown, etc)
  • Timed Events (When they’re added, executed, or removed)
  • Logged Events (When written to log files, or logs rotated)
  • System Commands (plugins, notification commands)
  • Event Handlers (Host, Service and Global handlers)
  • Notifications (When they begin and end)
  • Service Checks (When initiated and processed)
  • Host Checks (When initiated and processed)
  • Comments (When added, deleted or loaded)
  • Flapping (When started or stopped for a host/service)
  • Downtime (When added or deleted, and started and stopped)
  • Status Updates (For the Nagios process, host or service)
  • “Adaptive Updates” (When configuration for the process/host/service changes during runtime)
  • External Commands (When they have started)
  • Aggregated Status Updates (When status dumps start and end)
  • Retention Data (When it has been loaded, and when it has been saved)

These events cover pretty much every aspect of the Nagios monitoring system. Whenever an event like the ones described above occurs, the Event Broker gets called into action. The event broker will look to see what callbacks have been registered for the event which has occurred. It will then call every callback in sequential order (based on what order they were added). When a callback has been entered, it is given pointers to the appropriate data structures which holds the information related to the event. For example, if the event type was that of a host status changing, the pointer would point to a nebstruct_host_status_struct, which contains various elements in regards to the status update. In that structure is yet another pointer to the main host structure. In this structure you can see every thing in regards to the host. The configuration, the status information and statistical information. Much more information than what you would find by reading the Nagios log files. This goes for every other event. You can deeply introspect the Nagios daemon.

Because you are linked into the Nagios daemon itself, you have the capacity of doing some major things to evaluate and possibly modify the Nagios monitoring system. Nagios has a habit of keeping all control variables in global scope. (This includes process configuration information, scheduling information, status information, etc). Nagios also has extension functions to add / remove scheduled events from the scheduling queue (more sophisticated event broker modules may ask to call a procedure every 5 seconds, for example). You also have access to all the various output procedures (to print things to the log messages, console, etc). Because there is no data abstraction in place, there is the possibility to change the configuration of Nagios during run-time. Possibilities are:

  • Reading configuration from different sources (databases, directories, etc)
  • Add/remove/edit host/service/hostgroup/contacts, etc during run-time (removing the need for restarting the daemon)
  • Saving information to different targets (databases, directories, flat files, traps, syslogs)

The event broker interface was designed with a specific purpose. To get event data and do with it what you want. However, because of the design, there is the chance of hacking the Nagios daemon to provide further functionality than designed. A lot of these ideas may be present in the 3.0 roadmap; however, with some tinkering, we can achieve these features now.

Join me each week as I post a new blog entry on extending the Nagios daemon with the Event Broker interface. The first blog entry next week will go into the development of a very small event broker to go over the required code to get you up and running.

Taylor Dondich
IT Groundwork Staff Developer

Roger Ruttimann Speaking at ApacheCON 2005

November 29th, 2005 by roger

The dominant topic for ApacheCON 2005 is application scalability and to no surprise security and dealing with spam. The first two days will be half day tutorials while the sessions run Monday-Wednesday.

As a member of the Jetspeed team, Roger Ruttimann from GroundWork will co-present the tutorial on building a Web Portal using Jetspeed.

The tutorial covers the basic architecture of Jetspeed, the Portlet specification (JSR-168), authentication, customization, Ajax pipeline, SSO, the application bridges (Struts/PHP/Perl) and how to build a customized Web Portal. The final release of Jetspeed-2 will be available for ApacheCON.