To visualize the schemas you can copy paste them to http://chris.photobooks.com/json/
collectd json schema
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Collectd Mapping",
"description": "A mapping from collectd meters and events to *other* meters and events",
"type": "object",
"properties": {
"value list": {
"description": "Statistics in collectd consist of a value list",
"type": "object",
"properties": {
"values" : {
"type": "array",
"anyOf" : [
{ "type" : "object", "properties" : { "absolute" : { "type" : "number" } } },
{ "type" : "object", "properties" : { "counter" : { "type" : "number" } } },
{ "type" : "object", "properties" : { "derive" : { "type" : "number" } } },
{ "type" : "object", "properties" : { "gauge" : { "type" : "number" } } }
]
},
"value length" : {
"description": "The number of values in the data set",
"type": "number"
},
"time" : {
"description": "Time stamp at which the value was collected",
"type": "number"
},
"Interval" : {
"description": "interval at which to expect a new value",
"type": "number"
},
"host" : {
"description": "used to identify the host",
"type": "string"
},
"plugin" : {
"description": "used to identify the plugin",
"type": "string"
},
"plugin instance" : {
"description": "used to group a set of values together",
"type": "string"
},
"type" : {
"description": "unit used to measure a value",
"type": "string"
},
"type instance " : {
"description": "used to distinguish between values that have an identical type",
"type": "string"
},
"metadata" : {
"description": "an opaque data structure that enables the passing of additional information about a value list",
"type": "string"
}
}
},
"notifications": {
"description": "Notifications in collectd are generic messages",
"type": "object",
"properties": {
"severity" : {
"description": "can be one of OKAY, WARNING, and FAILURE",
"type": "string"
},
"time" : {
"description": "Time stamp at which the event was collected",
"type": "number"
},
"message" : {
"description": "The notification message",
"type": "string"
},
"host" : {
"description": "used to identify the host",
"type": "string"
},
"plugin" : {
"description": "used to identify the plugin",
"type": "string"
},
"plugin instance" : {
"description": "used to group a set of values together",
"type": "string"
},
"type" : {
"description": "unit used to measure a value",
"type": "string"
},
"type instance " : {
"description": "used to distinguish between values that have an identical type",
"type": "string"
},
"metadata" : {
"description": "an opaque data structure that enables the passing of additional information about a value list",
"type": "string"
}
}
}
}
}
collectd simple mapping schema:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Collectd Mapping",
"description": "A mapping from collectd meters and events to *other* meters and events",
"type": "object",
"properties": {
"resource_id": {
"description": "Mappings from collectd resource_id to other framework resource_id",
"type": "object",
"items": {
"type": "array",
"items": {
"collectd_resource_id": "string",
"new_resource_id": "string"
}
}
},
"meters": {
"description": "Mappings from collectd meters to other framework meters",
"type": "object",
"items": {
"type": "array",
"items": {
"collectd_meter": "string",
"new_meter": "string",
"new_meter_unit": "string"
}
}
},
"events": {
"description": "Mappings from collectd events to other framework events",
"type": "object",
"items": {
"type": "array",
"items": {
"collectd_event": "string",
"new_event": "string",
"collectd_severity": "string",
"new_severity": "string"
}
}
}
}
}
collectd to ceilometer mapping example
I think 2 things are required. The message format schema for the end point and the mapping schema.
- Unknown User (maryamtahhan) will examine 1 combined schema for next week
Ceilometer old samples schema
{
"counter_name": "instance",
"counter_type": "gauge",
"counter_unit": "instance",
"counter_volume": 1.0,
"message_id": "5460acce-4fd6-480d-ab18-9735ec7b1996",
"project_id": "35b17138-b364-4e6a-a131-8f3099c5be68",
"recorded_at": "2015-01-01T12:00:00",
"resource_id": "bd9431c1-8d69-4ad3-803a-8d4a6b89fd36",
"resource_metadata": {
"name1": "value1",
"name2": "value2"
},
"source": "openstack",
"timestamp": "2015-01-01T12:00:00",
"user_id": "efd87807-12d2-4b38-9c70-5f5c2ac427ff"
}
Currently data normalization or mappings are managed manually in the ceilometer write plugin:
A snippet from https://github.com/openstack/collectd-ceilometer-plugin/blob/master/collectd_ceilometer/units.py:
# Unit mappings in alphabetical order
UNITS = {
'apache.apache_idle_workers': 'Workers',
'apache.apache_bytes': 'B/s',
'apache.apache_requests': 'Req/s',
'apache.apache_scoreboard': 'Slots',
'apache.apache_connections': 'Connections',
'apcups.timeleft': 's',
'apcups.temperature': '°C',
'apcups.percent': 'Load',
'apcups.charge': 'Ah',
'apcups.frequency': 'Hz',
'apcups.voltage': 'V',
...
and the meter code itself can be found here: https://github.com/openstack/collectd-ceilometer-plugin/blob/master/collectd_ceilometer/meters/base.py fields are mapped as follows:
| ceilometer meter field | collectd field |
|---|---|
counter_name | plugin, type |
counter_type | "gauge"-> "gauge", "derive"-> "delta "absolute"-> "cumulative" "counter"-> "cumulative" |
counter_unit | unit |
counter_volume | value |
message_id | - |
project_id | - |
recorded_at | - |
resource_id | hostname, plugin_instance, type_instance |
resource_metadata | meta |
source | collectd |
timestamp | time |
user_id | - |
using the new schema