Meeting info can be found in the calendar. Caution, the time in calendar is given in UTC.
Meeting is held on zoom: https://zoom.us/j/5014627785 webex and IRC #opnfv-meeting.
June 09th, 2020, 4pm UTC
- Collectd bi-weekly meeting this week
- Topics for discussion: collectd-python repo, and questions regarding downstreaming of those plugins
- Updates from Closed Loop automation group
- highlighted discussions on Zabbix and prom for closed loop, with focus on observability as opposed to automated remediation
- Feedback welcome on direction, e.g. focus on analytics, etc
- POD13 assignments
- Node 3&4 for a demo
- Node 5 will be Jenkins slave
- CI updates
June 02nd, 2020, 4pm UTC
- Welcome
- We have Vani and Vishal joining the LFN mentorship program for 3 months, they are going to work on the CI
- Collectd bi-weekly meeting update
- DPDK v18.11 patch is up for review: https://gerrit.opnfv.org/gerrit/c/barometer/+/69491
- Closed Loop automation working group has their bi-weekly meeting this week
May 26th, 2020, 4pm UTC
- Agenda
- Container update
- SNMP agent config
- Q: Why are the configs different for ansible and docker sample configs?
- TODO(Emma): Resolve differences and determine functional differences;
- Intern update
- Starting Monday which is a public holiday in Europe
- We'll put together an onboarding doc for Monday, and sync on Tuesday morning.
- Patch for DPDK version update
- https://gerrit.opnfv.org/gerrit/c/barometer/+/69491
- Some file path changes moving from 16.11 to 18.11, do we need to support both versions?
- If the changes are trivial, both versions can be supported in the build
- If the updates are non-trivial, then instructions can be included for v16.11 in the docs, and only support v18.11 in the automation.
May 19th, 2020, 4pm UTC
- Agenda
- Intern update
- Collectd bi-weekly meeting updates
- AoB
May 12th, 2020, 4pm UTC
- Agenda
- Intern update
- Collectd updates
- Container updates
- Release plan updates
- Minutes
- Intern update
- Interviews in progress, thanks to KK and Jabir for helping with application reviews and interviews
- Collectd updates
- Container updates
- Release plan updates
- Discussions ongoing; release date is not finalised yet;
- Partly dependant on CNTT requirements
- Expect announcements once it's finalised
- Alignment with functest
May 05th, 2020, 4pm UTC
Apr 28th, 2020, 4pm UTC
Meeting today is on webex: https://meetingsemea13.webex.com/meetingsemea13/j.php?MTID=m06e819e9beb3cec98713b8425a5a3e32
Apr 21st, 2020, 4pm UTC
Meeting today is on webex: https://meetingsemea13.webex.com/meetingsemea13/j.php?MTID=m06e819e9beb3cec98713b8425a5a3e32
- Agenda
- Intern update
- Collectd update
- Minutes
Apr 14th, 2020, 5pm UTC
Meeting today is on Jitsi: https://meet.jit.si/barometer-weekly-meeting
- Agenda
- Proposal for new meeting time (4pm UTC or 5pm UTC): https://doodle.com/poll/i6pg4hhphgg5qsi3
- Interns
- Volunteers for interviews
- Additional intern available - do we have work for two interns?
- AMQP patch
- Virtual plugfest
- Minutes
- Actions
- Emma to update project proposal
- Emma to e-mail Jim re: work for a second intern
- Emma to update the meeting time on OPNFV meetings page
Apr 07th, 2020, 5pm UTC
- Agenda
- amqp patch
- collectd CI status
Mar 31st, 2020, 5pm UTC
Mar 24th, 2020, 5pm UTC
Mar 17th, 2020, 5pm UTC
Mar 10th, 2020, 5pm UTC
- Agenda
- Minutes:
- Actions
- ACTION: Emma to follow up with CNTT folks and send meeting invite
- ACTION: Move HLDs/test plans to collectd wiki
Mar 03rd, 2020, 5pm UTC
- Agenda
- Minutes
- Actions
- Look out for the code freeze e-mail and do your usual testing
Feb 25th, 2020, 5pm UTC
- Agenda
- Collectd meet-up summary
- Project graduation
- Minutes:
Feb 18th, 2020, 5pm UTC
- Cancelled due to collectd meetup
Feb 11th, 2020, 5pm UTC
- Agenda
- Action items
- sunku ranganath to check how vm stall metrics can be gathered; – can update on March 3rd
- Emma Foley send out info on committer nominations
- Emma Foley Prepare proposal for project graduation
- Minutes
Feb 3rd, 2020, 5pm UTC
- Agenda
- FOSDEM
- Collectd CI
- Review failing patch
- minutes
- <collabot`> Minutes: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2020/opnfv-meeting.2020-02-04-17.04.html
<collabot`> Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2020/opnfv-meeting.2020-02-04-17.04.txt
<collabot`> Log: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2020/opnfv-meeting.2020-02-04-17.04.log.html Meeting summary
- attendees (efoley, 17:05:49)
- efoley (efoley, 17:05:59)
- mrunge (mrunge, 17:06:06)
- Al Mortion (acmacm, 17:06:35)
- Pierre Lynch (acmacm, 17:06:55)
- Jabir (efoley, 17:07:18)
- FOSDEM (efoley, 17:08:08)
- efoley and kkepka attended fosdem and presented in the SDN devroom (efoley, 17:09:17)
- https://fosdem.org/2020/schedule/event/do_you_really_see_whats_happening_on_your_nfv_infra/ (efoley, 17:09:39)
- there were some questions about cloudifying collectd (efoley, 17:10:17)
- question: were there any volunteers to work on collectd? (efoley, 17:11:33)
- hopefully after the meet-up, when there is a roadmap (efoley, 17:12:23)
- Collectd CI (efoley, 17:12:40)
- kkepka shared a proposal for the CI (efoley, 17:13:29)
- this is a WIP and will be a working document to gather information and collaborate (efoley, 17:13:59)
- CI will should build collectd artifacts, open questions remain about where we will host the built packages (efoley, 17:15:40)
- statics analysis jobs are planned too (efoley, 17:15:56)
- This could include performance testing in the future (efoley, 17:16:25)
- Chat where you can join the discussion: https://docs.google.com/presentation/d/1eJT1iIviUxekuFyQMifsM0nqRxJzfN51H3agKCXSUHM (acmacm, 17:17:29)
- the presentation included an overview of current testing infra (efoley, 17:18:19)
- some other communities already run shared functional test in third party CIs, without the need for public infrastructure -- multiple CIs can run tests on private hardware and report to collectd github (efoley, 17:19:51)
- travis could be used for building packages and posting them to github, and to trigger nightly jobs (efoley, 17:21:25)
- some SUSE infra could be used too. (efoley, 17:21:47)
- Centralising the set of tests, and running a "standard" may require some trade-offs vs private CIs that are currently running tests (efoley, 17:24:07)
- there was a suggestion that we provide a place to host the logs from third party CIs; this would require some funding or dinated infra (efoley, 17:25:32)
- Discussion of LF CI/CD Evolution (In Prague) https://wiki.lfnetworking.org/pages/viewpage.action?spaceKey=~mbeierl&title=OPNFV+Prague+Highlight+Reel#OPNFVPragueHighlightReel-CI/CDEvolution(TrevorBramwell) (acmacm, 17:28:06)
- ACTION: sunkuranganath to look at criteria for joining LF (efoley, 17:39:34)
- ACTION: mrunge to look at criteria for joining apache foundation (efoley, 17:39:49)
- code review (efoley, 17:40:31)
- https://gerrit.opnfv.org/gerrit/#/c/barometer/+/69491/ (efoley, 17:40:36)
- patch looks okay, and the issue might be due to random CI failures (efoley, 17:43:14)
- patch is being rechecked (efoley, 17:43:28)
- ACTION: efoley to follow-up if it fails again (efoley, 17:43:38)
- AoB (efoley, 17:43:56)
- acm and sunkuranganath discuss how to collect vm stall metrics. (efoley, 17:46:41)
- ACTION: sunkuranganath to check how this can be gathered; possibly available from virt or pmu (efoley, 17:47:35)
- acm invites everyone to review the vsperf requested metrics list (efoley, 17:48:30)
- CFPs (efoley, 17:50:51)
- CFP is closed for ONES NA (efoley, 17:51:16)
- CFP is open for open source summit NA (efoley, 17:51:40)
- https://events.linuxfoundation.org/open-source-summit-north-america/ (mrunge, 17:52:11)
- plynch submitted to ONES on MANO testing work (efoley, 17:52:22)
- https://events.linuxfoundation.org/open-source-summit-north-america/ (efoley, 17:52:47)
- https://events.linuxfoundation.org/open-source-summit-europe/ (mrunge, 17:53:17)
- https://events.linuxfoundation.org/open-source-summit-europe/program/cfp/ (mrunge, 17:53:32)
- open source summit is in Dublin this year (efoley, 17:53:46)
- https://events.linuxfoundation.org/open-source-summit-europe/ (efoley, 17:53:59)
- https://events.linuxfoundation.org/open-source-summit-europe/program/cfp/ (efoley, 17:54:08)
- sunkuranganath had a CFP for ONES which can be resubmitted for OSS NA (efoley, 17:55:55)
- iruya release (efoley, 17:58:25)
- ACTION: efoley to check on release status (efoley, 17:58:34)
- ACTION: efoley check on plans for point releases (efoley, 17:58:50)
Meeting ended at 18:02:03 UTC (full logs).
Action items
sunku ranganath to look at criteria for joining LFUnknown User (mrunge) to look at criteria for joining apache foundationEmma Foley to follow-up if it fails again- sunku ranganath to check how this can be gathered; possibly available from virt or pmu
Emma Foley to check on release statusEmma Foley check on plans for point releases
Jan 28th, 2020, 5pm UTC
- Agenda
- Jerma planning
- Collectd meet-up
- Mostly ready to go
- Agenda needs to be finalised and prioritised
- Highest priority is updating the bus factor
- Review of action items
- IRC meeting
- Attendees
- Minutes
- Jerma release planning
- Collectd meet-up
- Review of action items
- IRC meeting
- Starting next week, we can use IRC to record meeting minutes
- TODO(emma): send reminder
- Iruya release status
- TST001 metrics
- ONES summit deadline is upcoming
- Suggest a CFP similar to the FOSDEM submission, and collaborating with CNTT folks
- TODO: sunku to send CFP to CNTT folks
- CI (comments from other discussions)
- We should get the one click install running post-build to verify the images are built correctly
- Actions
- Emma: Create patch to disable SNMP
- Jabir: Verify that baro-134 exists on other systems
- Emma: send a reminder about IRC
- Sunku: Send CFP for ONES
Jan 21st, 2020, 5pm UTC
Agenda
Attendees: elfoley, acm, kkepka, Mark Shostak, mrunge, Pierre Lynch, Sunku, PiotrD, Jabir, bgrabarc
Minutes
Jerma release plan is still a WIP, please add your requests to the Jerma planning page.
Collectd meet-up (Unknown User (mrunge) )
- Plugfest updates
- Discusson on OPNFV direction (Al Morton )
- ETSI-NFV MANO API testing (Pierre Lynch )
- We have a few tools for that, which have been used in previous demos. (sunku ranganath )
- It should be possible to generate generic fake events, but this would be used for testing only, not generally distributed.
- Pierre is requesting help for the ETSI-NFV testing of MANO APIs
- Can barometer be used simulate events?
- The request is to define an API, so that the development can be done on the ETSI-side
- There is some interest in collaborating from the CLA-WG
- Pierre's presentation is attached, it contains two examples of TCs that could benefit from these simulated events (part of TST-010)
- FOSDEM pres
- vsperf team update Al Morton
- TST-001 has a few missing metrics.
- Thanks to Sridhar Rao for highlighting these, and Al Morton for sharing
- Some of the missing metrics exist in collectd under a different name, others can be derived from existing metrics.
- TODO(sunku ranganath ) draw up a list comparing the TST001 metrics to collectd metrics
Jerma release plan is still a WIP, please add your requests to the Jerma planning page.
- Actions
- TODO(Emma Foley ): Follow up with CNTT folks on the Jerma action items
- TODO(sunku ranganath ) draw up a list comparing the TST001 metrics to collectd metrics
- (Emma) Investigate Barometer-134 (Jan 14th)
- (Emma) Look at healthcheck for collectd containers (Jan 14th)
Jan 14th, 2020, 5pm UTC
- Agenda
- Attendees:
- Minutes
- Iruya 9.0 release is January 31st
- There's a bug with the one click install with SNMP (
)- no resolution yet
- TODO(Emma) investigate
- Collectd meetup
- most likely Feb 17-18
- unknown if rooms are available
- Release planning
- vsperf team had some requests for missing metrics
- QPI, virt cpup steal, IRQ latency per CPU, context switches, DPDK per queue statistics, SRIOV metrics (kk has PoC)
- Collectd CI
- we need to get a list of tests/create a test suite people can run
- docs updates will be needed for new plugins
- Build acceptance tests for Barometer - make sure the containers are running as some builds failed recently and got published
- TODO(emma): Healthcheck for collectd.
- Actions
- (Emma) Investigate Barometer-134
- (Emma) Look at healthcheck for collectd containers
Jan 7th, 2020, 5pm UTC
Dec 17 2019, 5pm UTC
Dec 10 2019, 5pm UTC
- Stable branch: https://gerrit.opnfv.org/gerrit/c/releng/+/69322
- Backport needed for additional changes
- collectd meetup
- no feedback from Florian
- plan for a 1 day meetup without Florian
- Actions
- Do barometer reviews and backports - Emma
- Get access for new contributer - Emma
Dec 03 2019, 5 pm UTC
Nov 26 2019, 5 pm UTC
Nov 19 2019, 5 pm UTC
Oct 15 2019, 4pm UTC
Oct 01 2019, 4 pm UTC
- collectd 5.9.2 released today
- still relying on python 2?
- need to look at functest as well
Sep 17 2019, 4 pm UTC
- names for J and K release
- MS1 is coming up on Sept 30. Please submit your release plan here.
- collectd 5.9.2 release due asap, please test collect-5.9 branch.
- mrunge: I will be out Sep 24 and on Oct 08.
Sep 10 2019, 4pm UTC
- Participate in Iruya, traditional release
- integrated into any installer
- use stable releases
- Collectd user community: End User working group
- No meeting on Sep 24th, unless we have someone to host the meeting.
Sep 03 2019, 4pm UTC
- Participate in Iruya release?
- traditional release track
- continous delivery track
- both?
- action: pull more info from David re: continuous delivery track and requirements for us.
- Iruya to be released in Dec.
Aug 27, 2019, 4pm UTC
- question was brought up on PMU and RAS included in Barometer and how to measure QPI.
- RAS is named mcelog plugin in collectd.
Aug 20, 2019, 4pm UTC
Aug 06, 2019, 4pm UTC
- opens?
- under discussion: delay Iruya release to 2020?
July 30, 2019, 4pm UTC
- any opens?
- collectd-5.9.1 was released July 24th
- collectd-5.9.2 soon to bugs (related to ssnprintf-fix)
- rrdtool broken
- iptables broken
- release tarball contains dir named collectd-collectd-5.9.1
- send out a mail to collectd mailing list regarding collectd relases 5.9.0 and 5.9.1, recommend to use 5.9.2, and that it is due soon.
July 23, 2019, 4pm UTC
July 09, 2019, 4pm UTC
July 02, 2019, 4 pm UTC
June 18, 2019, 4 pm UTC
- reviews needed
- narrow scope for Barometer
- add analysis for metrics
- discovery of platform metrics
- CLA updates?
- weekly meeting: Thursday 7 AM Pacific
- telemetry tagging
- collectd release proposal
June 11, 2019, 4 pm UTC
- reviews needed
- narrow the scope for Barometer, leave space e.g for CLA
- CLA updates?
June 04, 2019, 4 pm UTC
- reviews needed
- narrow the scope for Barometer, leave space e.g for CLA
- CLA updates?
May 21, 2019, 4 pm UTC
- report from CLA WG
- remove outdated containers
- write up a vision document until next meeting
- add a note about supportability etc. in docs.
- feature planning for next release
May 14, 2019, 9 am UTC
- fix for anteater https://gerrit.opnfv.org/gerrit/#/c/67799/ still unreviewed
- Hunter stable branched tonight, Hunter was released last Friday
- this meeting slot not much used, attendance later (in the US slot) much higher.
- agreement: move the meeting back to 4 pm UTC, have async meetings for APAC folks.
- VESPA, small bridge between prometheus and VES
May 07, 2019, 4 pm UTC
- anteater issue, eg, with https://gerrit.opnfv.org/gerrit/#/c/67787/
- Hunter release, changes due soon
- missed creating Hunter stable branch, discussed pros and cons of merging patches and then create a branch.
- CLA WG update
- request for demo
- request for elasticsearch use case desription
- will move keep the meeting in this (late) slot, APAC meetings were rarely attended by people from that region.
- Idea is to have an additional meeting for APAC every 4(?) weeks to accommodate.
Apr 30, 2019, 9 AM UTC
Apr 09, 2019, will be in the late slot.
Apr 02, 2019
- Open Networking Summit Apr 3 - 5
- collectd release due, send a follow-up on to the mailing list, changelog is complete.
- VES-update still in the queue
Mar 26, 2019
- This kind of meeting notes is hard to follow. I/we should improve
- Sunku shared the closed loop platform architecture, we talked about the system architecture, and since this touches lots of other OPNFV projects, also how to integrate with these projects.
Mar 19, 2019
Mar 12, 2019
- ONS
- collectd wiki
- collectd release,
- barometer integration, the remaining installer seems to be xci
Mar 05, 2019
- dma update (policy draft for policy-agent)
- how about to filtering vm elements (if tenant is 'do-not-care', notificatoin will not be sent)
- discussion: collectd config change on-the-fly API (to interworking dma local-agent)
- collectd upcoming feature freeze (March 1st)
Feb 26, 2019
Feb 12, 2019
- https://wiki.opnfv.org/display/SWREL/Hunter+Scenario+Status scenarios: do we want them?
- apex integration "planned", but not listed right now.
- Follow-up with Akhila on that topic
- docs: what is up to date, and what is not?
- jenkins
- evolving scope of barometer?
- into analytics
- concerned about missing innovation
- evolution required
January, 29 2019
- Caution: next meeting will be at 10 am UTC (alternating with late meeting)
January 22, 2019
- demonstrate SA in a box using a set of metrics/events internally to AT&T to cut down the costs
- Take it through ECOMP control loop -> then define set of metrics for automation
- Alok will share findings as they progress
- working with Intel team to go through setup details
- its in line with how TST008 spec written. The spec provides what additional info to be included for a set of technology/feature to go with the plugin
January 15, 2019
- barometer-kolla integration
- ves updates still pending
- alerting via prometheus/alertmanager
January 08, 2019
- minimal service assurance demo working out-of-the box
- collectd container on dockerhub is 5.8.0
- bar-collectd vs. barometer-collectd (duplication, description needed)
- benchmarking of collectd?
December 18, 2018
- barometer-7.1.0 released
- reminder: stable branches and master branch
- upcoming conferences
- no meeting on Dec 25th, January 1st
December 11, 2018
- conferences?
- demo?
- release this week
- hack fest week of June 10th
Lightning Talk Notes
- What is Barometer?
- A project for building a scalable metrics platform for NFV environments and enables the monitoring of infrastructure that hosts our virtual network functions (VNFs).
- We can monitor platform components like CPU, memory, load, cache, thermals, fan speeds, voltages and machine check exceptions
- We’re including both the tools such as collectd and the installation utilities along with the definition of metrics and events that are critical for NFV infrastructure monitoring, because the tools without definitions is not a solution.
- Focus is on collection of metrics, not necessarily the storage or analysis of data. We leave that up to higher level orchestration systems to leverage the data for business logic and decision making. We can leverage things like machine learning and big data analytics to provide proactive changes rather than reactive (traditional monitoring), but our involvement in that is at the collection of metrics and events.
- Standards involvement -- TST_008 / IFA_027
- Why use barometer?
- Leverages collectd for metrics and events gathering. It’s robust with a strong upstream community and has been around for several years, meaning that it’s a stable application to build from with a lot of functionality.
- Extensible through the plugin framework, and we’ve been adding functionality over the last several months to solve for use cases that weren’t previously solved for.
- Can be used wherever Linux is available, including whitebox switches and routers.
- What are we solving for?
- Providing an implementation of collectd in cloud platform environments that are implementing for NFV so that we can find deficiencies and further enhance collectd through the plugin system
- Who is using barometer?
- Red Hat is leveraging the work done in the barometer project within the service assurance framework in order to provide the various metrics and events required by telco operators, edge providers, cloud providers, and other highly available environments.
- Used by both OPNFV Yardstick for test analysis and OPNFV Bottlenecks
- How can I use barometer?
- Barometer is being packaged as a Docker container, so it’s easy to install into your environment without a lot of risk.
- Also have influxdb container image to show an implementation with data storage
December 04, 2018
November 27, 2018
November 20, 2018
- There's going to be a Plugfest in early January. Unclear if anyone from Barometer is going to attend
- Release plan discussion
- Barometer Release Plan
- Improvements in documentation
- Add support for alternative TSDB like prometheus
- apex integration?
- install mcelog manually/from ansible; Ubuntu 18.04 apparently dropped the daemon from default install
- change ansible playbooks to allow configuration of "container type" to be installed: master (from collectd master), experimental (collectd master + selected patches), stable (latest collectd release)
- Barometer cloud native support: gRPC plugin of collectd to integrate with projects like clover
November 13, 2018
October 23, 2018
October 16, 2018
Meeting Minutes:
- Another week for finishing documentation of docker container install
- Verifying the manual steps now.
- Target end of week, beginning of next week.
- Agenda:
October 09, 2018
Meeting Minutes:
- collectd documentation
- DPDK
- Is it considered open source?
- Is there anything specific to DPDK or are they just mirroring standard parameters?
- Should add DPDK specific metrics!
- DPDK telemetry
- DPDKSTATs replacement
- Maybe have a small presentation at a later date for input.
- REDFISH?
- MCELOG
- RAS – x86 specific
- RDT – x86 specific
Agenda:
- Release status
- Tour of Prometheus
October 02, 2018
Meeting minutes:
- Need a JIRA item for Ansible scripts
- Remaining documentation for Ansible scripts
- 1,2 issues – scaling, etc...
- One command install
- Moved to Centos 5.7 / Ubuntu 18.04
- Show internal – One click install
- Demo
- Video – 5 minutes to monitoring your NFVi
- Prometheus waiting on Ansible
- Participation in https://openmetrics.io/
Notes from last meeting:
Demo from Barometer:
- Possibly create a demo and record it?
- Demo around containers/VM with metrics
- Float around few ideas, get comments over next calls
- Find a new way to use Barometer, would be good
- Demo ideas:
- Barometer for system tests/debugging.
- Look at network interfaces, OVS stats, etc., to track whats happening in an openstack deployment
- Need to know if there are any openstack deployments already available in opnfv pods
- Have a real time demo to show graphana dashboards in opnfv pods
Agenda:
- Status of Release
- What do we want in the release?
| virt plugin enhancements | Virt plugin should be enhanced to provide the following additional details: | Release G | X | X |
|  BAROMETER-99 - Virt plugin enhancement -- VM names OPEN
|
| ovs-dpdk-stats | The ovs-dpdk-stats plugin should include extended dpdk stats per interface | Release G | X | X |
| BAROMETER-96 - Enable OVS-DPDK PMD Statistics OPEN |
ovs-dpdk-events link status | The ovs-dpdk-events plugin should indicate link status change events | Release G | X | X |
|  BAROMETER-109 - ovs-events link status OPEN
|
| DMA Functionality | The DMA Local Agent will provide the following features: - Local processing of metrics – Allows metrics to be sampled at a higher rate than storage rate for detection of short-lived conditions.
- Runtime configuration of collectd – Collectd configuration can be managed remotely with the Local Agent providing the writing of collectd configuration and collectd restart.
- Simple OpenStack metadata discovery – Provide runtime information related to the mapping between virtual and physical elements. The metadata can be attached to metric sent by collectd.
| Release G | X | X |
| BAROMETER-95 - The DMA project will provide additional functional for the Barometer project OPEN |
| DMA Container | Add DMA container build | Release G | X | X |
| BAROMETER-94 - The DMA project will exist as a container separate from the collectd container OPEN |
| Collectd ovs_stats plugin does not collect dpdk interface statistics if under bond interface | The ovs stats plugin collects the right set of data in case the dpdk interfaces are mapped directly under the ovs bridge. But for the use case where a bonding interface is needed underneath the ovs bridge, ovs stats cannot pull the dpdk interface statistics.
| Release G | X | X | | BAROMETER-92 - Collectd ovs_stats plugin does not collect dpdk interface statistics if under bond interface OPEN |
- Original plan stalled by no release from Collectd.
- virt plugin enhancements – Complete waiting on release
- ovs-dpdk-stats –
- DMA Functionality - In progress
- DMA Container - In progress
- Collectd ovs_stats plugin does not collect dpdk interface statistics if under bond interface – completed waiting on release
- Participation in https://openmetrics.io/
Updates from last week:
- Changed the Kolla container to include almost all Collectd plugins, taken from latest stable branch
- Plugins are still taken from RPM build, relying on release version of Collectd
- Not included plugins would gaming plugins, etc.
- This is a great progress
- No new updates about community release of Collectd
- Intel team will prioritize backporting once Ansible script work is done
- Collectd code reviews:
- Lot of PRs, will take some time
- The DPDK patch got merged, need to backport to 5.8
- Sunku will ask the patch author to backport
- Ansible scripts work:
- Testing/validation on Ubuntu 16.04, 18.04 & Centos 7.5
- Centos 7.3 had docker issues, related to docker missing packages
- Latest docker Images is based on 7.5.3
- Centos 7.5 images (qcow2) have different naming scheme, very similar to Ubuntu 18.04
- There shouldn’t be any regressions, upgrading from 7.3 to 7.5.
- We can look in to if we need to on 7.3
- Require your help to review as patches are upstreamed
- Milestone 8 – Oct 30th
- Its not being pushed back
- Review help:
- Have 4 to 5 reviews pending in Barometer gerrit
- Require reviews
- Prometheus and Barometer
- Should Barometer participate in the standardization effort under way in Prometheus. (Yes!)
- Driven by Prometheus for data format
- Mentioned at Fosdem & Prometheus conference
- Standard is named as open metrics, submitted to IEEE
- Gone quite far, need 3 implementations of metrics format for this standard
- Google is doing one implementation, 2nd one by Prometheus community
- Trying to standardize data exchange format
- While Prometheus only supports metrics, standards also supports logs.
- Value of all this info is the correlation part
- Matthias would send more info.
- Need to make sure it wont impact our implementations of metrics in hardware/infrastructure level
- The community is not generally attuned to lower layers.
- Bring Barometer’s experience in monitoring the infrastructure.
- End result could be too PaaSy ;)
09/25/2018
Meeting minutes:
Demo from Barometer:
- Possibly create a demo and record it?
- Demo around containers/VM with metrics
- Float around few ideas, get comments over next calls
- Find a new way to use Barometer, would be good
- Demo ideas:
- Barometer for system tests/debugging.
- Look at network interfaces, OVS stats, etc., to track whats happening in an openstack deployment
- Need to know if there are any openstack deployments already available in opnfv pods
- Have a real time demo to show graphana dashboards in opnfv pods
Updates:
- Changed the Kolla container to include almost all Collectd plugins, taken from latest stable branch
- Plugins are still taken from RPM build, relying on release version of Collectd
- Not included plugins would gaming plugins, etc.
- This is a great progress
- No new updates about community release of Collectd
- Intel team will prioritize backporting once Ansible script work is done
- Collectd code reviews:
- Lot of PRs, will take some time
- The DPDK patch got merged, need to backport to 5.8
- Sunku will ask the patch author to backport
- Ansible scripts work:
- Testing/validation on Ubuntu 16.04, 18.04 & Centos 7.5
- Centos 7.3 had docker issues, related to docker missing packages
- Latest docker Images is based on 7.5.3
- Centos 7.5 images (qcow2) have different naming scheme, very similar to Ubuntu 18.04
- There shouldn’t be any regressions, upgrading from 7.3 to 7.5.
- We can look in to if we need to on 7.3
- Require your help to review as patches are upstreamed
- Milestone 8 – Oct 30th
- Its not being pushed back
- Review help:
- Have 4 to 5 reviews pending in Barometer gerrit
- Require reviews
- Gambia release behind
- Please have check-ins in good shape before submission
- Milestones
Release Plan Items - https://jira.opnfv.org/browse/BAROMETER-99
- https://jira.opnfv.org/browse/BAROMETER-96
- Ansible scripts update
- https://gerrit.opnfv.org/gerrit/#/c/61575/5
- Need to have the experimental Barometer container
- Need a place to send interested parties so they can try out newest features
- Early feedback is key
- Showing the host naming in virt plugin in experimental container would be good.
- If we have hardware, we can have virtualized RDO installed quickly in barometer opnfv pod
- A simple 1 controller/1 compute node running barometer container each, also on the pod host itself would be good to show metrics collection across all layers.
- Listing the steps in barometer wiki would be helpful
- Possibly have a packstack installation on a single machine.
- Prometheus and Barometer
- Should Barometer participate in the standardization effort under way in Prometheus. (Yes!)
- Driven by Prometheus for data format
- Mentioned at Fosdem & Prometheus conference
- Standard is named as open metrics, submitted to IEEE
- Gone quite far, need 3 implementations of metrics format for this standard
- Google is doing one implementation, 2nd one by Prometheus community
- Trying to standardize data exchange format
- While Prometheus only supports metrics, standards also supports logs.
- Value of all this info is the correlation part
- Matthias would send more info.
- Need to make sure it wont impact our implementations of metrics in hardware/infrastructure level
- The community is not generally attuned to lower layers.
- Bring Barometer’s experience in monitoring the infrastructure.
- End result could be too PaaSy ;)
09/18/2018
- Filed exception for MS5
- Sept 18th meeting for approval
- Gambia release behind
- Milestones
- MS6 9/07/18 (PAST)
- Test case implementation completed.
- Preliminary documentation completed.
- Complete documentation compliance table for MS6.
- First day that projects may request creation of the stable branch for their project.
- Exception request is approved
- MS7 9/21/18
- Last day that projects may request creation of the stable branch for their project.
- Any project not yet branched will be branched by LF.
- Exception request is approved
- We could cherrypick items and create a stable branch.
- Agree on "definition of done" →
- Have the Collectd PRs tracked through Barometer showcased thru experimental containers
- Experimental container are WIP
- Need to follow this for individual items in Jira too, update DoD value in every Jira item
- https://jira.opnfv.org/browse/BAROMETER-106
- https://jira.opnfv.org/browse/BAROMETER-92
- https://gerrit.opnfv.org/gerrit/#/c/61591/1
- Add support for 'collectd-master' container
- Need reviewers
- The CI integration part is WIP, Radek/Emma are following up
- Review: Ansible scripts
- Release Plan Items
- Introductions: Vinay Kumar
- Joined with RedHat, Metrics & Monitoring.
- Looking in to Gnocchi, AODH, Ceilometer, etc. Interested in integrating Collectd with OpenStack.
- Necessity of gRPC interface for collectd.
- Clover project and cloud native WG have proposal to integrate most of OPNFV projects with gRPC interface.
- GRPC is hot, good thing is binary format is very compact, while http is very verbose using up a lot of bandwidth
- ONAP talks a lot on using GRPC instead of kafka
- No downside on having GRPC
- Good to float the idea within Collectd community
- Florian & others added in write_prometheus, to enable cloud native
- It creates a scrape target
- Superior from Collectd exporter from Prometheus itself
- Real key is to provide correlation between app metrics and infrastructure metrics. Have them be in common data definitions & timestamps.
- Collectd timestamps are when metrics are generated
- Prometheus timestamp is when it gets scraped
- There is a disconnect
- We need to keep reminding CN folks about correlation on time/locality, between virtual/physical
- Need to check for direct integration between Jaeger and Collectd
- A list of metrics/capabilities through node_exporter in Prometheus might be helpful.
- Strengths, missing metrics?
- Get stuff in to node_exporter?
- CN folks might know as they are from different community than OpenStack, as there are different tools, etc.
- Aaron to help us out here to get started!
- Al to contribute here.
- Importance of having “hostname” as uniform reporting instead of IPv4 address:
- Memcached plugin is special -> provide both hostname & IPv4 address. Opened a PR in PuppetCollectd to fix proper reporting.
- It would be good to have additional text along with hostname, like rack name, tag datacenters
- Need to work with plugin authors
- Require following PR Reviews contributed by Intel:
September 11, 2018
Good discussion last week around what service providers really want to use related to telemetry. Do they need to install additional agents for monitoring?
Not everything is nor will be cloud native -- Providing native poll interfaces may not be possible
Node Exporter is the Prometheus / cloud native agent for collecting data from the hardware.
Collectd is the historical metrics / events collection agent.
Need a solution across all environments
Need a solution for baremetal, switches, SDN, etc...
Filed exception for MS5
Gambia release behind
Milestones
Test case implementation completed.
Preliminary documentation completed.
Complete documentation compliance table for MS6.
First day that projects may request creation of the stable branch for their project.
MS6 9/07/18 (PAST)
MS7 9/21/18
https://jira.opnfv.org/browse/BAROMETER-106
https://jira.opnfv.org/browse/BAROMETER-92
https://gerrit.opnfv.org/gerrit/#/c/61591/1
Release Plan Items
September 04, 2018
08/28/2018
Filed exception for MS5
Milestones
Scenario integration completed
All scenarios have deploy jobs in OPNFV CI
Scenario status page is locked. No new scenarios may be added. However, scenario owners may request the removal of a scenario.
Test case implementation completed.
Preliminary documentation completed.
Complete documentation compliance table for MS6.
First day that projects may request creation of the stable branch for their project.
MS5 8/24/18
MS6 9/07/18
MS7 9/21/18
Dev Updates
Fix functest
Waiting internal review
08/21/2018
Dev Updates
Waiting internal review
Week 33?
Self-healing Sig Update
08/14/2018
08/07/2018
Dev Updates
MS6 -- August 24
Test cases implemented, preliminary documentation completed, and stable branch open
Major feature should at least be proposed and partly reviewed at this time.
Functest support should be mostly done (tests upstreamed for collectd-plugins), bug fixes are okay after this date but no new tests.
MS7 -- September 14
Contact maintainers and ask “what can I do to help?”
Also, join #collectd on Freenode if possible
Won’t make Gambia, given that we would need to provide config, docs, validation in Barometer once it is closed; However, we should continue to help.
07/30/2018
· Dev Updates
[ELF] Patches created; waiting internal approval to upstream
§ Write redis plugin enable
§ Need +1
o Upcoming milestones
§ MS6 -- August 24
· Test cases implemented, preliminary documentation completed, and stable branch open
· Need to have development “mostly” complete by then
[ELF] Major feature should at least be proposed and partly reviewed at this time.
Functest support should be mostly done (tests upstreamed for collectd-plugins), bug fixes are okay after this date but no new tests.
§ MS7 -- September 14
· Stable branch window close
· Need feedback on the following list of collect plugins
o Feedback will help prioritize work
· Need to offer help to collectd to get the next release out (5.9)
o Contact maintainers and ask “what can I do to help?”
o Also, join #collectd on Freenode if possible
[ELF] Not likely to be available for Gambia, given that we would need to provide config, docs, validation in Barometer once it is closed; However, we should continue to help.
· Openstack voting has begun
[ELF] There was a very short voting period this time around, and it’s already closed
AOB:
- Collectd container split: Need to check whether the separate container build jobs (master and stable) are configured correctly; so we can start adding config and build support for newer features (Emma/Kamil)
07/30/2018
Dev Updates
MS6 -- August 24
Test cases implemented, preliminary documentation completed, and stable branch open
Need to have development “mostly” complete by then
MS7 -- September 14
Need feedback on the following list of collect plugins
Need to offer help to collectd to get the next release out (5.9)
Openstack voting has begun
07/24/2018
Dev Update
Late comment
Moving forward?
MS6 -- August 24
Test cases implemented, preliminary documentation completed, and stable branch open
Need to have development “mostly” complete by then
MS7 -- September 14
Need feedback on the following list of collect plugins
Need to offer help to collectd to get the next release out (5.9)
Openstack voting has begun
07/03/2018
06/12/2018
Development
ETSI NFVI Testing group might be interested in using Barometer
Might use Barometer to collect metrics during API compliance testing
Would also like to use fault triggering code from one of the Barometer demos
06/05/2018
DMA team (Tomo) to present a demo of Local Agent collecting OpenStack metadata.
One of the features of the DMA project is to collect virtual infrastructure information from OpenStack and make it available to collectd so that metadata can be added to metrics / events.
Tomo gave a demo of Local Agent using OpenStack API to gather metadata information for use by Collectd.
Development
Nomination of new contributors
Self Healing Sig
Use case templates about DMA project and local healing
What about 5 9’s
White paper on Barometer’s role in self-healing / availability
Originally discussed in Sydney
What should be included and what people care about
NEC, Verizon, etc…
Focus is HA.
Go over monitoring paper
New dedicated resource for collectd.
05/29/2018
05/22/2018
05/08/2018 --
05/01/2018 --
04/24/2018 --
04/17/2018 --
04/10/18 --
DMA Project discussion
Standards
Al has been reviewing Alok’s VES proposals. Found some VMWare specific items. Vendor-specific provisions are discouraged. Al to talk with Alok ;).
VES is starting to incorporate some TOSCA information.
Collectd
Emma suggested that all collectd PRs be from the same fork, Barometer, in order to make collectd aware of who the reviews are for.
Also, should have more Red Hat people reviewing collectd. Need to talk with local Red Hatters.
VES
General
04/03/18 --
DMA Project
Run time reconfiguration of collectd. There were some questions as to whether this should be a feature of something like an Ansible playbook.
Topology annotation. Questions about security and what information would be included. It was stated that only infrastructure information would be annotated.
Data reduction and analysis at the edge.
DMA project should be an add-on. Might need to be able to “unwind” project in the future.
Development
Thanks to Emma for updating the docs and helping track down Funtest issues
Still seeing an error with Functest… seems to be related to Functest, not Barometer. Emma tracked down the internal Barometer issues!
Barometer container -> multistage build from RPMs progressing slowly..
ONS recap
03/20/18 --
Made MS6...
Next upcoming milestone, MS7, April 6th -- last day to request a stable branch. Can request one anytime until then.
After than MS8, April 16, completion of testing
Current Fraser status
test_add_remove_volume
test_attach_volume_nowait
test_detach_volume_nowait
….
Perhaps it was a system / connectivity failure?
REDFISH Plugin
Purpose -- node or rack?
Design review
Standards Update
VES Updates
ONAP flow
Vendor VNF
Closed loop control
Services goes into VNF
Any missing heatheats
At the FM layer
03/13/18 --
03/06/18 --
Milestone 5 complete
MS6 03/16/18
Test case implementation completed.
Preliminary documentation completed.
First day that projects may request creation of the stable branch for their project.
APEX
Functests failing
Virtual deploy -- RDT, Hugepages… Should we skip during virtual deploy?
APEX deployment help -- Aaron has a test setup running and can answer questions.
Collectd
DMA project would like to have another meeting (possibly at ONS)
Did you get your feedback to Alok? I know I didn’t yet :(.
VES Update
VES Project into ETSI
Parts of the project, metric / event definitions, transport
Pair things down to what’s important
10’s of collectors at present
VES mobile edge platforms
02/27/18 --
02/20/18 --
Collectd on Cumulus
Standards Updates?
VES Updates?
Development
Barometer next?
02/13/18 --
Should Barometer Graduate? (https://www.opnfv.org/software/technical-project-governance/project-lifecycle)
Are we a part of enough scenarios (another installer)?
Would be a good recognition of the work done by Barometer
Have had good recognition at shows (OpenStack Summit, OPNFV, etc)
Bryan would be supportive of applying for graduation
No definitive answer, but positive reception
Standards Update (Al)
VES Update
Development Update
(MS5) Scenario integration and feature freeze -- 03/02/18
Good discussion on “What’s next for Barometer”
OPNFV Barometer people going to ONS in March
· Filed exception for MS5
o Sept 18th meeting for approval