Barometer OS Summit
Feedback was good
Questions
Starting to use collectd, Barometer
What different plugins available? Turns out most are already in Barometer
Prometheus was mentioned -- Mostly about host to get data into Prometheus from collectd. Not necessarily replacing collectd.
Virt Plugin
3 missing variables.
Added to JIRA
Ovs-stats
Extended DPDK stats
JC to check
PMU Plugin Wiki updates
Al Morton
List of VES metrics meeting this week
Going to agree with Al that IPMI are important. What will we do with those on ETSI side?
Give feedback to Al on VES.
NFVI Monitoring White Paper
Create google doc
Start outline
Should cover why to monitor specific metrics / conditions, not just these are the things that are available to monitor.
Sunku --
Had a chat Sando - Paul
Call at end of week
Self healing Sig
List of projects (OPNFV, etc…, )
Sunku to e-mail...
(MS3) Installer integration with OpenStack
3.0 Integration / nosdn deploy / run HC 6/15
3.1 deploy SDN / pass HC 6/29
PMU Plugin
Evaluating options
JIRA item to track virt plugin metric additions (gaps with ceilometer) -- https://jira.opnfv.org/browse/BAROMETER-98
JIRA item to track virt plugin VM name enhancements -- https://jira.opnfv.org/browse/BAROMETER-99
Ceilometer / Collectd parity
Can Ceilometer be replaced with collectd?
NFVI (OpenStack Services) monitoring
NFVI Monitoring Whitepaper -- https://jira.opnfv.org/browse/BAROMETER-97
How to collaborate? Google docs?
Gambia release planning
Gambia release will concentrate on maintenance and support of plugins delivered in Fraser release.
Need to check on status of ovs-dpdk-stats, ovs-dpdk-events plugins
One or two features might be added -- TBD
Need code review of container build changes! Gerrit # -- 53805
Standards
VES
Prometheus
Gambia Release -- traditional release
Need scenarios and deliverables defined by May 11
Gambia Additions
More information added to virt plugin?
Cpu pinning
NUMA node characteristics
What to do about (mostly) static information?
Additional plugins?
May need modification to RAS
MCElog might be incomplete
Looking at out of band platform hardware
Additional containers?
Ceilometer replacement?
How to get metadata?
Need code review of container build changes! Gerrit # -- 53805
Standards
VES
Prometheus
Interest in using Prometheus as polling engine / TSDB
Prometheus_collectd_exporter vs. Prometheus_node_exporter
Gambia Release
Traditional, XCI, both?
Not clear on XCI requirements.
When do the tests need to be in place for XCI?
What are tentative development goals for the Gambia release?
RPMs / Container build
New pull request - https://gerrit.opnfv.org/gerrit/#/c/53805/9
Standards
All the info we’re collecting will end up in analytic engines
Analytics processing tasks need info all the way down to the agent layer
Need to make sure there’s a measurement timestamp to provide context
Does test008 spec have the flexibility to talk about measurement of network interface using a particular VLAN tag?
Yes, scope parameter makes that possible
Scope must be conveyed with the measurements
VES
6.0 specs on the way (next week)
Multi-VM reqs
Compass support -- not for Gambia or Fraser?
Barometer will be supported in Gambia
HA scenario issues with build
A problem with OOO?
Fails every third build or so.
Will be looked at soon (next week)
Edge Cloud
Requirements
Categories
Use cases
Latency
Use cases where latency is critical
Example feedback loop?
What does lightweight mean in the context of edge telemetry?
Spoke with DMA project team. They are ok with working in a separate container. They will swing back in a few weeks with a demo.
Standards with Al Morton
No updates
VES with Bryan Sullivan
Most work done by Goku
Bryan will update VES for latest release to make sure it works with everything
Plan is place to move items forward in Barometer
RPMs with Leif
Making progress
Func tests
Will not support Compass this cycle
Collectd pull requests
Used to be tracked on wiki
We need to track open pull requests and put them on Barometer development update page: https://wiki.opnfv.org/display/fastpath/Barometer+Development+Updates
Release
Gambia path or something else?
Which track do we follow?
Continuous delivery
Flexibility with milestones, since you establish them yourself?
Bryan doesn’t think we should do traditional path -- not enough resources
FCI could be a good thing if it marries VES and Barometer
Edge Cloud
What are the requirements?
Need categories
Always about latency for faults and metrics
Framework and edge computing use cases would be good
Is VES a good fit?
Good for cloud-native
What is a cloud? What value does opensource ecosystem get from:
Common telemetry date model
Consistent set of mechanisms by which data delivered to collector
How lightweight?
Less than 1% CPU, and 5% memory
Two VES encodings: C and Java
Goku: tokenized VES data structure
Overhead of data and complexity of stack
Go from binary XML to tokenized JSON
Current implementation is not lightweight but is “easy”
Use gzip, as it is natively available to HTTP stack (works out of box)
DMA Project discussion
Positive feedback on DMA proposal. Want to move forward.
Work should be done in a separate container.
Standards
Al has been reviewing Alok’s VES proposals. Found some VMWare specific items. Vendor-specific provisions are discouraged. Al to talk with Alok ;).
VES is starting to incorporate some TOSCA information.
Collectd
Emma suggested that all collectd PRs be from the same fork, Barometer, in order to make collectd aware of who the reviews are for.
Also, should have more Red Hat people reviewing collectd. Need to talk with local Red Hatters.
VES
Bryan returned. Wants to keep the VES agent development moving forward. Did not know about DDS messaging stuff in ONAP. However, see’s that Kafka is not set in stone. What to do at the edge.
General
Next week I will be out. Andrew Bays will run meeting.
DMA Project
Presentation given by Tomofumi Hayashi. Slides available on the wiki.
Major takeaways…
Run time reconfiguration of collectd. There were some questions as to whether this should be a feature of something like an Ansible playbook.
Topology annotation. Questions about security and what information would be included. It was stated that only infrastructure information would be annotated.
Data reduction and analysis at the edge.
DMA project should be an add-on. Might need to be able to “unwind” project in the future.
OPNFV Bottlenecks project using Barometer container
Development
Thanks to Emma for updating the docs and helping track down Funtest issues
Still seeing an error with Functest… seems to be related to Functest, not Barometer. Emma tracked down the internal Barometer issues!
Barometer container -> multistage build from RPMs progressing slowly..
ONS recap
Made MS6...
Next upcoming milestone, MS7, April 6th -- last day to request a stable branch. Can request one anytime until then.
After than MS8, April 16, completion of testing
Current Fraser status
test_add_remove_volume
test_attach_volume_nowait
test_detach_volume_nowait
….
Perhaps it was a system / connectivity failure?
Emma good catch on Hugepages failing fix
Gnocchi failures
Results not appearing quickly enough due to Gnocchi archive policy
Change default policy to High
AODH failures
Need to modify the tests to perform specific failure action
REDFISH Plugin
Purpose -- node or rack?
Design review
Standards Update
VES Updates
Progress on ETSI?
VES Mobile edge platforms?
Is there a cadvisor plugin for collectd?
C, Java VES libraries
Microservice heatbeat
ONAP flow
Vendor VNF
Closed loop control
Services goes into VNF
Any missing heatheats
At the FM layer
Functest debug and then power loss ;(
Milestone 5 complete
Scenario deployment passed
Functest failing
MS6 03/16/18
Test case implementation completed.
Preliminary documentation completed.
Complete documentation compliance table for MS6.
First day that projects may request creation of the stable branch for their project.
APEX
Functests failing
Virtual deploy -- RDT, Hugepages… Should we skip during virtual deploy?
APEX deployment help -- Aaron has a test setup running and can answer questions.
Collectd
PRs
pmu plugin / pmu core -- Reviewer comments -- 03/06/2018
ovs link status -- Is this still relevant?
AMQP1 -- Needs review
Connectivity Event -- Reviews in progress
Process Event -- Reviews in progress
System Event -- Reviews in progress
DMA project would like to have another meeting (possibly at ONS)
Did you get your feedback to Alok? I know I didn’t yet :(.
Want to get comments by the 15th
Produce a spec update by end of March
Have a weekly update
VES Update
VES Project into ETSI
Parts of the project, metric / event definitions, transport
Pair things down to what’s important
10’s of collectors at present
VES mobile edge platforms
CAdvisor plugin for collectd?
Maybe try in the Barometer container?
Milestone 5 approaching
Thanks for the help on Apex and Barometer
Tracked down issue with deploy
Standards Update
VES Update
Barometer containers to be part of RDO release
Team to work on building Kolla containers
AMQP1.0 container being built
Collectd on Cumulus
Open Compute / Networking
Standards Updates?
VES Updates?
Development
Apex scenarios are currently failing
Ansible script reviews in progress
MS5 approaching
Barometer next?
Should Barometer Graduate? (https://www.opnfv.org/software/technical-project-governance/project-lifecycle)
Are we a part of enough scenarios (another installer)?
Would be a good recognition of the work done by Barometer
Have had good recognition at shows (OpenStack Summit, OPNFV, etc)
Bryan would be supportive of applying for graduation
No definitive answer, but positive reception
Standards Update (Al)
Meeting with IFA working group completed work on all measurement content. Al provided an edit / rewrite of the document and resubmitted.
Discussion of GAPs in Baromter
VES Update
Bryan working on Barometer Kubernetes scenario. Should be ready soon. Barometer as DaemonSet
Conversion of VES to gRPC in the testing phase.
Development Update
Ansible scripts submitted to Gerrit
Clarification that plugins not in collectd yet should still make it into Barometer / Fraser
(MS5) Scenario integration and feature freeze -- 03/02/18
Good discussion on “What’s next for Barometer”
Scope expansion -- Should Barometer move beyond metrics / events to policy
Scope continuation into container space --
OPNFV Barometer people going to ONS in March
Aaron
Al
Bryan
??