FIWARE Monitoring

Build Status Coverage Status

This is the code repository for FIWARE Monitoring, the reference implementation of the Monitoring GE.

This project is part of FIWARE. Check also the FIWARE Catalogue entry for Monitoring.

Any feedback on this documentation is highly welcome, including bugs, typos or things you think should be included but aren’t. You can use github issues to provide feedback.

For documentation previous to release 4.4.2 please check the manuals at FIWARE public wiki:

GEi overall description

FIWARE Monitoring is the key component to allow incorporating monitoring and metering mechanisms in order be able to constantly check the performance of the cloud infrastructure.

This involves gathering operational data in a running system, which usually requires collecting data from heterogeneous sources. Besides, the monitoring architecture should be easily extended to collect additional data for any other required needs.

FIWARE Monitoring is agnostic to the framework used to gather monitoring data. It just assumes there are several monitoring probes collecting information, which somehow must be forwarded to an adaptation layer, responsible for transforming data into a common representation (NGSI) and publishing through a Context Broker (see Orion).

Collected information can be used for several purposes:

  • Cloud users to track the performance of their own instances.
  • SLA management, in order to check adherence to agreement terms.
  • Optimization of virtual machines.

Components

Monitoring framework
It is up to the infrastructure owner which tool (like Nagios, Zabbix, openNMS, perfSONAR, etc.) is installed for this purpose.
Collector
Framework-specific component to forward monitoring data being gathered to the adaptation layer (i.e. NGSI Adapter). Monitoring GE provides a Nagios loadable module NGSI Event Broker as collector for such monitoring framework.
Adaptation layer
NGSI Adapter serves as generic adapter to transform monitoring data from probes to NGSI context attributes.

Build and Install

The recommended procedure is to install using RPM packages in CentOS 6.x, or DEB packages in Ubuntu 12.04/14.04 LTS. If you are interested in building from sources, check this document.

Requirements

  • System resources: see these recommendations.
  • Operating systems: CentOS (RedHat) and Ubuntu (Debian), being CentOS 6.3 the reference operating system.
  • RPM/DEB dependencies: some required packages may not be present in official repositories, or their versions are too old (for example, nodejs). In any case, checking for such dependencies and configuration of alternative sources is automatically managed by the package installation scripts when using the proper tool (yum in CentOS or apt-get/gdebi in Ubuntu).

Installation

Using the RPM/DEB files

Download the package(s) from the FIWARE Files area and use the proper tool to install it. Take into account that you may need to manually install dependencies, as some tools aren’t able to manage them when installing from file:

CentOS

$ sudo rpm -i fiware-monitoring-ngsi-adapter-X.Y.Z-1.noarch.rpm
$ sudo rpm -i fiware-monitoring-ngsi-event-broker-X.Y.Z-1.x86_64.rpm

Ubuntu

$ sudo dpkg -i fiware-monitoring-ngsi-adapter_X.Y.Z_all.deb
$ sudo dpkg -i fiware-monitoring-ngsi-event-broker_X.Y.Z_amd64.deb

Upgrading from a previous version

Unless explicitly stated, no migration steps are required to upgrade to a newer version of the Monitoring components:

  • When using the package repositories, just follow the same directions described in the Installation section (the install subcommand also performs upgrades).
  • When upgrading from downloaded package files, use rpm -U in CentOS, or use same dpkg -i command in Ubuntu.

Running

As explained in the overall description section, there are a variety of elements involved in the monitoring architecture, apart from those components provided by this Monitoring GE (at least, an instance of Context Broker is required and some underlying monitoring framework, such as Nagios). Please refer to their respective documentation for instructions to run them.

From the Monitoring GE components, only NGSI Adapter runs as standalone server. Once installed, there are two ways of running NGSI Adapter: manually from the command line or as a system service (the latter only available if installed as a package). It is not recommended to mix both ways (e.g. start it manually but use the service scripts to stop it). This section assumes you are using the system service (recommended): for the command line alternative, please refer to this document.

In order to start the adapter service, run:

$ sudo service ngsi_adapter start

Then, to stop the service, run:

$ sudo service ngsi_adapter stop

We can also force a service restart:

$ sudo service ngsi_adapter restart

Configuration file

The configuration used by the adapter service is optionally read from the file /etc/sysconfig/ngsi_adapter (in CentOS) or /etc/default/ngsi_adapter (in Ubuntu):

# ADAPTER_LOGFILE - Logging file
ADAPTER_LOGFILE=/var/log/ngsi_adapter/ngsi_adapter.log

# ADAPTER_LOGLEVEL - Logging level
ADAPTER_LOGLEVEL=INFO

# ADAPTER_LISTEN_HOST - The host where NGSI Adapter listens to requests
ADAPTER_LISTEN_HOST=0.0.0.0

# ADAPTER_LISTEN_PORT - The port where NGSI Adapter listens to requests
ADAPTER_LISTEN_PORT=1337

# ADAPTER_BROKER_URL - The endpoint where Context Broker is listening
ADAPTER_BROKER_URL=http://127.0.0.1:1026/

# ADAPTER_RETRIES - Maximum number of retries invoking Context Broker
ADAPTER_RETRIES=2

Most of these attributes map to options of the command line interface as follows:

  • ADAPTER_LOGLEVEL maps to -l or --logLevel option
  • ADAPTER_LISTEN_HOST maps to -H or --listenHost option
  • ADAPTER_LISTEN_PORT maps to -p or --listenPort option
  • ADAPTER_BROKER_URL maps to -b or --brokerUrl option
  • ADAPTER_RETRIES maps to -r or --retries option

Default values are found in /opt/fiware/ngsi_adapter/config/options.js.

Checking status

In order to check the status of the adapter service, use the following command (no special privileges required):

$ service ngsi_adapter status

API Overview

To transform monitoring data into NGSI attributes, probe raw data should be sent as body of a POST request to the adapter, identifying the source entity being monitored in the query fields.

For example, if using the check_load Nagios probe to measure CPU load, then the request would look like:

curl "{adapter_endpoint}/check_load?id={myhostname}&type=host" -s -S --header 'Content-Type: text/plain' -X POST -d @- <<-EOF
OK - load average: 5.00, 7.01, 7.05|load1=5.000;10.000;10.000;0; load5=7.010;15.000;15.000;0; load15=7.050;30.000;30.000;0;
EOF

This would result in an invocation to Context Broker updating the context of an entity of type host identified by myhostname with a new attribute cpuLoadPct with value 5.00.

Please have a look at the API Reference Documentation section bellow and at the programmer guide.

API Reference Documentation

Testing

End-to-end tests

Please refer to the Installation and administration guide for details.

Unit tests

The test target is used for running the unit tests in both components of Monitoring GE:

$ cd ngsi_adapter
$ grunt test

$ cd ngsi_event_broker
$ make test  # synonym of standard 'check' target

Please have a look at the section building from source code in order to get more information about how to prepare the environment to run the unit tests.

Acceptance tests

In the following documents you will find a business readable description of the features provided by the components of the Monitoring GE, as well as automated tests for them:

License

(c) 2013-2015 Telefónica I+D, Apache License 2.0