netdata v1.20.0 Release Notes

Release Date: 2020-02-21 // about 4 years ago
  • Netdata v1.20.0

    📚 Release v1.20.0 contains 3 new collectors, 54 bug fixes, 89 improvements, and 38 documentation updates.

    At a glance

    Our first major release of 2020 comes with an alpha version of our new eBPF collector. eBPF (extended Berkeley Packet Filter) is a virtual bytecode machine, built directly into the Linux kernel, that you can use for advanced monitoring and tracing.

    🚀 With this release, the eBPF collector monitors system calls inside your kernel to help you understand and visualize the behavior of your file descriptors, virtual file system (VFS) actions, and process/thread interactions. You can already use it for debugging applications and better understanding how the Linux kernel handles I/O and process management.

    The eBPF collector is in a technical preview, and doesn't come enabled out of the box. If you'd like to learn more about_why_ eBPF metrics are such an important addition to Netdata, see our blog post: Linux eBPF monitoring with Netdata. When you're ready to get started, enable the
    📚 eBPF collector by following the steps in our documentation.

    This release also introduces host labels , a powerful new way of organizing your Netdata-monitored systems. Netdata automatically creates a handful of labels for essential information, but you can supplement the defaults by segmenting your systems based on their location, purpose, operating system, or even when they went live.

    🚑 You can use host labels to create alarms that apply only to systems with specific labels, or apply labels to metrics you archive to other databases with our exporting engine. Because labels are streamed from slave to master systems, you can now find critical information about your entire infrastructure directly from the master system.

    📄 Our host labels tutorial will walk you through creating your first host labels and putting them to use in Netdata's other features.

    Finally, we introduced a new CockroachDB collector. Because we use CockroachDB internally, we wanted a better way of keeping tabs on the health and performance of our databases. Given how popular CockroachDB is right now, we know we're not alone, and are excited to share this collector with our community. See our tutorial on monitoring CockroachDB metrics for set-up details.

    🌲 We also added a new squid access log collector that parses and visualizes requests, bandwidth, responses, and much more. Our apps.plugin collector has new and improved way of processing groups together, and our cgroups collector is better at LXC (Linux
    container) monitoring.

    📚 Speaking of collectors, we revamped our collectors documentation to simplify how users learn about metrics collection. You can now view a collectors quickstart to learn the process of enabling collectors and monitoring more applications and services with Netdata, and see everything Netdata collects in our supported collectors list.

    Acknowledgements

    🚀 We're extremely grateful to the following contributors for their help since our last major release in November 2019. Whether it's their first or fiftieth contribution, insights from our users not only help make Netdata better, but also remind us why we're so lucky to be part of a vibrant open-source community.

    • 🔌 k0ste and DefauIt for improving the application groups of the apps plugin.
    • ⚡️ gmeszaros for a fix to the broken updater.
    • blaines for an elastisearch collector fix.
    • 📚 stevenh for adding freeipmi support to our Docker image and lassebm for related fixes and documentation.
    • yasharne for helping us improve the httpcheck collector.
    • candrews for the introduction of -fno-common in CFLAGS.
    • Jiab77 for fixing a typo in the installer options.
    • amishmm for improvements to the systemd service files.
    • tnyeanderson for continuing to improve his multi-host sample dashboard.
    • 📄 yasharne and especially schneiderl for corrections to the docs.
    • lucasRolff for improvements to the litespeed collector.
    • Ehekatl for the improvements to the Prometheus remote write API and the fix to thesoftnet alarm.
    • 📄 wonsangki for translating several docs into Korean.
    • 🔧 candrews for fixing the option to disable the Prometheus remote API from configure.
    • kkoomen for improvements to the Apache proxy guide.
    • 🔌 vzDevelopment for assistance with the unicode support in the python.d plugin.
    • 🔌 hexchain for the addition of pressure stall information to the proc plugin.
    • 📚 nabijaczleweli and rex4539 for documentation fixes.

    💥 Breaking Changes

    • Removed deprecated bash collectors apache, cpu_apps, cpufreq, exim, hddtemp, load_average, mem_apps, mysql, nginx, phpfpm, postfix, squid, tomcat #7962 (ilyam8). If you were still using one of these collectors with custom configurations, you can find the new collector that replaces it in the supported collectors list.
    • 👀 Modified the Netdata updater to prevent unecessary updates right after installation and to avoid updates via local tarballs #7939 (prologic). These changes introduced a critical bug to the updater, which was fixed via #8057 #8076 (prologic) and #8028 (gmeszaros). See issue 8056 if your Netdata is stuck on v1.19.0-432.

    👌 Improvements

    Host Labels

    🆕 New Collectors

    Collector improvements

    • 🔌 apps.plugin
    • varnish: Added SMF metrics (cache on disk) #7926 (ilyam8)
    • 🛠 phpfpm: Fixed per process chart titles and readme #7876 (ilyam8)
    • python.d: Formatted the code in all modules #7832 (ilyam8)
    • 👍 node.d/snmp: - Added snmpv3 support #7802 (ilyam8) - Formatted the code in snmp.node.js #7816 (ilyam8)
    • cgroups: Improved LXC monitoring by filtering out irrelevant LXC cgroups #7760 (vlvkobal)
    • 👍 litespeed: Added support for different .rtreport format #7705 (lucasRolff)
    • 🐳 freeipmi: Added support to the docker image #7081 (stevenh)
    • 🔌 proc.plugin: Added pressure stall information #7209 #7547 (hexchain)
    • sensors: Improved collection logic #7447 (ilyam8)
    • proc: Started monitoring network interface speed, duplex, operstate #7395 (stelfrag)
    • 🛠 smartd_log: Fixed the setting in the reallocated sectors count, by setting ATTR5 chart algorithm to absolute #7384 (ilyam8)
    • nvidia-smi: Allow executing nvidia-smi in normal instead of loop mode #7372 (ilyam8)
    • 🚤 wmi: collect logon metrics, collect logical_disk disk latency metrics
    • 🚚 weblog: handle MKCOL, PROPFIND, MOVE, SEARCH http request methods
    • scaleio: storage pools and sdcs metrics. (#294)

    Exporting Engine

    • Implemented the main flow for the Exporting Engine #7149 (vlvkobal)

    Streaming

    📦 Installation/Packages

    • 🛠 Fixed missing directory when creating the symbolic link during eBPF installation and remove future options. #8133 (prologic)
    • 🛠 Fixed NetData installer on *BSD systems after libmosquitto and eBPF functionality was enabled. #8121 (prologic)
    • 🛠 Fixed issues with the RPM nightly builds resulting from the bundled libmosquitto functionality that was recently merged. #8109 (Ferroin)
    • Corrected the invocations of mktemp so that they produce temporary directories in $TEMPDIR instead of the current directory, in a way that is compatible with busybox. #8066 (Ferroin)
    • 👌 Improved CI/CD workflow to install required packages and build the agent across all the OS/Distro(s) we support #7969 #7949 (prologic)
    • ⚡️ Updated the installer to download go.d.plugin, only if we have a new version #7946 (ilyam8)
    • Assorted cleanup items in the RPM spec file. #7927 (Ferroin)
    • ➕ Added a new, simpler, Alpine based Dockerfile for quick dev and testing #7914 (prologic)
    • ➕ Added minor fixes and improvements to the installer/updater shell scripts. #7847 (prologic)
    • ➕ Added ReviewDog CI checks
    • Stopped removing netdata groups/users during uninstall (Debian postrm) #7817 (prologic)
    • Started using the system service manager to shut down Netdata. #7814 (Ferroin)
    • 👌 Improved the systemd service files, by removing unecessary ExecStartPre lines and moving global options to netdata.conf #7790 (amishmm)
    • ✂ Removed unnessecary echo calls from the updater. #7783 (Ferroin)
    • 🛠 Fixed warnings in the Debian package build process and enabled the builds to work with older versions of dpkg-buildpackage by modifying the formatting of the trailer line in the Debian changelog template. #7763 (Ferroin)
    • 🏗 Cleaned up static build process, by using /bin/sh and removing use of sudo #7725 (prologic)
    • ➕ Added auto-updates to kickstart-static64 installations. #7704 (Ferroin)
    • ➕ Added static build support for Prometheus remote write #7691 (Ehekatl)
    • 📦 Moved the script for installing required packages into the main repo. #7563 (Ferroin)
    • ⚡️ Updated the distribution support matrix. #7636 (Ferroin)
    • ➕ Added Ubuntu 19.10 to packaging and lifecycle checks. #7629 (Ferroin
    • ✂ Removed EOL distros from CI jobs. #7628 (Ferroin)
    • 🍎 Made the netdata installer more flexible, to accommodate install with ssl on MacOS #6922 (paulkatsoulakis)
    • 👌 Improved shutdown of the Netdata agent on update and uninstall. #7595 (Ferroin)
    • ➕ Added Fedora 31 CI integrations. #7524 (Ferroin)
    • ✂ Removed CentOS 6 package building and lifecycle tests #7425 (knatsakis), #7430 (ncmans)
    • ✂ Removed -f option from groupdel in uninstaller. #7507 (Ferroin)
    • 🏗 Injected archived backports repository on Debian Jessie for CI package builds. #7495 (Ferroin)
    • 🚀 Set the default release channel to stable #7399 (ncmans)
    • ✂ Removed EOL'd Ubuntu Trusty (14.04) from build #7481 (ncmans)
    • Corrected installer instructions during a non-privileged install #7393 (julidegulen)

    📚 Documentation

    Privacy

    • Added support for opting out of telemetry via the DO_NOT_TRACK environment variable #7846 #7929 (prologic)
    • 🛠 Fixed typo in the installer options to disable telemetry #7843 (Jiab77)
    • 👌 Improved documentation of opting out of anonymous statistics #7597 (joelhans)
    • ➕ Added anon tracking notice for installers #7437 (ncmans)

    Other

    🐛 Bug fixes

    • 🛠 Fixed problems reported by Coverity for eBPF collector plugin. #8135 (thiagoftsm)
    • 🛠 Fixed invalid literal for float\(\): NN.NNt error in the elasticsearch python plugin, by adding terabyte unit parsing. #8013 (blaines)
    • 🛠 Fixed timeout failing in docker containers which broke some python.d collectors #8002 (ilyam8)
    • 🛠 Fixed python collectors to work on synology6 #7980 (ilyam8)
    • 🛠 Fixed problem with the httpcheck python collector not being able to check URLs with the POST method, by adding body to the URLService #7956 (ilyam8). Also record the new options in httpcheck.conf #7952 (yasharne)
    • 🛠 Fixed netdata-updater.sh appearing to fail #7955 (ilyam8)
    • 🛠 Fixed error/warnings found by shellcheck for the netdata-updater.sh #7938 (prologic)
    • 🛠 Fixed editing configuration via edit-config, when NetData is installed to a symlinked /opt #7933 (prologic)
    • 🛠 Fixed installation failures due to .keep files #7829 (prologic)
    • 🛠 Fixed installation on FreeBSD systems with non GNU sed #7796 (prologic)
    • 🛠 Fixed Source0 URL in RPM spec #7794 (prologic)
    • 🛠 Fixed text if current version is >= latest version and already installed #8078 (prologic)
    • 🛠 Fixed CentOS 7 RPM build failures. #7993 (Ferroin)
    • 🛠 Fixed wrong messages during the build process #7989 (Ferroin)
    • 🛠 Fixed the unit tests for the exporting engine #7784 (vlvkobal)
    • 🛠 Fixed a Coverity issue with an unchecked return value #7780 (vlvkobal)
    • 🛠 Fixed port in use after uninstall issue, by resolving a libuv IPC pipe cleanup problem #7778 (mfundul)
    • 🛠 Fixed dbengine repeated global flushing errors and collectors being blocked, by dropping dirty dbengine pages if the disk cannot keep up #7777 (mfundul)
    • 🛠 Fixed issue with alarm notifications occasionally ignoring the configured severity filter when the ROLE was set to root. #7769 (thiagoftsm)
    • 🛠 Fixed Netlink Connection Tracker charts in the nfacct plugin #7727 (vlvkobal)
    • 🛠 Fixed support for read-only /lib on SystemD systems like CoreOS in static build installation #7726 (prologic)
    • 🛠 Fixed invalid shell installer error and netdata not starting from its installed location. #7698 (Ferroin)
    • 🛠 Fixed metric values sent via remote write to Prometheus backends, when using average/sum #7694 (Ehekatl)
    • 🛠 Fixed unclosed brackets in softnet alarm #7693 (Ehekatl)
    • 🛠 Fixed SEGFAULT when localhost initialization failed #7663 (underhood)
    • 🛠 Fixed the handling of permissions in the installer script and the RPM spec file so that theya re consistent with each other and with a clean install done with make install. #7632 (Ferroin)
    • ⬇️ Reduced the number of broken pipe error log entries, after a SIGKILL #7588 (thiagoftsm)
    • 🛠 Fixed a syntax error in the packaging functions. #7686 (Ferroin)
    • 🛠 Fixed Coverity errors by restoring support for protobuf 3.0 #7683 (vlvkobal)
    • 🛠 Fixed inability to disable Prometheus remote API #7674 (candrews)
    • 🛠 Fixed SEGFAULT from the cpuidle plugin #7664 (Saruspete)
    • 🛠 Fixed samba collector not working, due to inability to run sudo #7655 (ilyam8)
    • 🛠 Fixed invalid css/js resource errors when URL for slave node has no final / on streaming master #7643 (underhood)
    • 🛠 Fixed keys_redis chart in the redis collector, by populating keys at runtime #7639 (ilyam8)
    • 🛠 Fixed UrlService bytes decoding and logger unicode encoding in the python.d plugin #7601 #7614 (ilyam8), #7376 (vzDevelopment)
    • 🛠 Fixed a warning in the prometheus remote write backend #7609 (vlvkobal)
    • 🛠 Fixed not detecting more than one adapter in the hpssa collector #7580 (gnoddep)- Fixed race condition in dbengine #7565 (thiagoftsm)
    • 🛠 Fixed race condition with the dbenging page cache descriptors #7478 (mfundul)
    • 🛠 Fixed dbengine dirty page flushing warning #7469 (mfundul)
    • 🛠 Fixed missing parenthesis on alarm softnet.conf #7476 (Steve8291)
    • 🛠 Fixed race condition in the dbengine #7533 (mfundul)
    • 🛠 Fixed "Master thread EXPORTING takes too long to exit. Giving up" error, by cleaning up the main exporting engine thread on exit #7558 (vlvkobal)
    • 🛠 Fixed rabbitmq error "update() unhandled exception: invalid literal for int() with base 10" #7464 (ilyam8)
    • 🛠 Fixed some LGTM alerts #7441 (jacekkolasa)
    • 🛠 Fixed valgrind errors #7532 (mfundul)
    • 🛠 Fixed monit collector LGTM warnings (#7387 (ilyam8)
    • 🛠 Fixed the following go.d.plugin collector issues: - mysql: panic in Cleanup (#326) - unbound: gather metrics via unix socket (#319) - logstash: pipelines chart (#317) - unbound: configuration file parsing. Support include mechanism. (#298) - logstash: pipelines metrics parsing (#293) - phpfpm: processes metrics parsing (#297)