« Changelog History


Pravega v0.8.0-rc1 Release Notes

Release Date: 2020-09-18 // over 3 years ago

  • 🚀 This release contains exciting new features and significant enhancements over the previous major release line (0.7.x). This page provides an overview of the major ones.

    Key-Value Tables

    In 0.8, we have introduced an entirely new primitive that enables efficient insertions, updates and lookups of arbitrary key-value pairs. Using the same underlying infrastructure as Streams, Key-Value Tables provide the same features one would expect of a typical key-value store, plus some more.

    Here are a few highlights:

    • Distributed Key-Value Store is built on top of append-only Pravega Segments.
    • Keys are sharded uniformly across underlying Segments or can be grouped by means of Key Families.
    • ⚡️ Conditional or unconditional updates and removals.
    • ⚡️ Single or multiple entry batch (atomic) updates or removals.
    • Single or multiple entry retrieval.
    • Listing of keys/entries in bitwise lexicographical order (by key).
    • Java Map wrapper provided for easy integration with existing applications.

    🚀 Key-Value Tables are built on top of Table Segments - the same underlying primitive that Pravega has used to store its own Stream, Segment and Transaction metadata since release 0.5. The Pravega Controller and Segment Store rely on the consistency, durability and atomicity guarantees of Table Segments for all of their metadata needs - the same guarantees are also extended to Key-Value Tables as well.

    For more details, please refer to PDP-39 - Key Value Tables.

    As of 0.8, Key-Value Tables are released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

    Schema Registry

    With Pravega 0.8, we are also releasing Pravega Schema Registry 0.1. The Schema Registry service is designed to store and manage schemas for unstructured data stored in Pravega streams.

    Here are a few highlights:

    • The service provides a RESTful interface to store and manage schemas under schema groups.
    • 👉 Users can choose from different compatibility policies for safely evolving their schema.
    • 👍 The service has built-in support for popular serialization formats in Avro, Profobuf and JSON schemas. However, users can store schemas of other formats too.
    • 🚀 In v0.1 we have compatibility policies enabled only for Avro. Subsequent releases will support compatibility checks for Protobuf and JSON.
    • The service stores and manages encoding information in form of codec information. Codecs could correspond to different compression or encryption used while encoding the serialized data at rest. The service generates unique identifiers for schemas and codec information pairs that users may use to tag their data.

    For more details, visit the Schema Registry repository.

    As of 0.1, Schema Registry is released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

    🐎 Performance Improvements and Optimizations

    🚀 Pravega 0.8 contains significant improvements over the previous release. These are the main improvements included in 0.8.

    • ⚡️ Issue #4774: (Client) Saves significant compute cycles for a reader by minimizing and optimizing costly calls for managing its internal state.
    • Issue #4653: (Segment Store) Control the number of container starts/recoveries performed in parallel.
    • 🚚 Issue #4294: Remove Netty from the client.

    🔧 Configuration Names

    🔧 Pravega 0.8 changes the names of almost all configuration items, based on a new uniform configuration naming scheme.

    👀 For instance the property controller.minRebalanceIntervalSeconds is replaced by controller.rebalance.interval.seconds.min and pravegaservice.storageImplementation is replaced by pravegaservice.storage.impl.name. See this Wiki page for new names of old properties.

    🚀 Old properties are deprecated but will continue to work for now. So, upon upgrade from previous version your Pravega deployment will continue to work. You can use a combination of new and old properties to Configure Pravega components, although it is highly discouraged. If you specify a property using both new and old names, the old property will take precedence.

    ⬆️ Fresh installations must use new properties (even though old properties will work too for now). For upgrade from previous versions, we highly encourage migrating to new properties at the earliest opportunity.

    👀 See issue #4712 for more details.

    ⚡️ Other New Features and Updates

    • Issue #4893: (Controller, Metrics) Adds metrics for created and deleted scopes.
    • Issue #4910: (Segment Store, Metrics) Adds two new metrics to report the size distribution of appends and reads.
    • ✅ Issue #4680: (Client) Add a new truncation api to ByteStreamWriter.
    • Issue #4791: (Client, Controller) Add a new controller API to fetch segments for a given epoch
    • Issue #4845: (Client, Usability) StreamInfo obtained via io.pravega.client.admin.impl.StreamManager indicates if the Stream is sealed.
    • ⬆️ Issue #4846: (Library upgrades) Upgrade Netty to 4.1.50Final.
    • ⬆️ Issue #4883: (Library upgrades) Upgrade gRPC to 1.24.2.
    • ⬆️ Issue #4531: (Library upgrades) Upgrade Bookeeper client to 4.9.2.

    🐛 Bug Fixes

    • Issue #4631: (Controller, Metrics) Report failed transaction commit metric only in case of non-retryable failures.
    • ✅ Issue #4847: (Client) Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

    🚀 For a comprehensive listing of issues resolved in this release, refer to the attached commits.txt file.

    ⚡️ Ecosystem Updates/Links

    Connectors

    - Pravega Boomi Connector

    Kubernetes Operators