All Versions
17
Latest Version
Avg Release Cycle
29 days
Latest Release
1562 days ago

Changelog History
Page 1

  • v0.8.1-rc0 Changes

    December 08, 2020

    🚀 This release contains several bug fixes and performance improvements over 0.8.0:

    Segment Store

    • Issue #5294: Bookkeeper Client may cache old DNS entries for too long
    • 🛠 Issue #5269: Fixed a deadlock in StreamSegmentReadIndex

    Controller

    • Issue #5341: Fix thread leak in controller

    Client

    • 🚚 Issue #5137: Remove dependency on netty-transport-native-epoll

    General

    • 🚀 Issue #4775: Add snapshot stage for all release branches

    🚀 To see all issues resolved in this release, refer to the attached commits.txt file.

  • v0.8

    August 19, 2020

  • v0.8.0-rc2 Changes

    September 25, 2020

    🚀 This release contains exciting new features and significant enhancements over the previous major release line (0.7.x). This page provides an overview of the major ones.

    Key-Value Tables

    In 0.8, we have introduced an entirely new primitive that enables efficient insertions, updates and lookups of arbitrary key-value pairs. Using the same underlying infrastructure as Streams, Key-Value Tables provide the same features one would expect of a typical key-value store, plus some more.

    Here are a few highlights:

    • Distributed Key-Value Store is built on top of append-only Pravega Segments.
    • Keys are sharded uniformly across underlying Segments or can be grouped by means of Key Families.
    • ⚡️ Conditional or unconditional updates and removals.
    • ⚡️ Single or multiple entry batch (atomic) updates or removals.
    • Single or multiple entry retrieval.
    • Listing of keys/entries in bitwise lexicographical order (by key).
    • Java Map wrapper provided for easy integration with existing applications.

    🚀 Key-Value Tables are built on top of Table Segments - the same underlying primitive that Pravega has used to store its own Stream, Segment and Transaction metadata since release 0.5. The Pravega Controller and Segment Store rely on the consistency, durability and atomicity guarantees of Table Segments for all of their metadata needs - the same guarantees are also extended to Key-Value Tables as well.

    For more details, please refer to PDP-39 - Key Value Tables.

    As of 0.8, Key-Value Tables are released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

    Schema Registry

    With Pravega 0.8, we are also releasing Pravega Schema Registry 0.1. The Schema Registry service is designed to store and manage schemas for unstructured data stored in Pravega streams.

    Here are a few highlights:

    • The service provides a RESTful interface to store and manage schemas under schema groups.
    • 👉 Users can choose from different compatibility policies for safely evolving their schema.
    • 👍 The service has built-in support for popular serialization formats in Avro, Profobuf and JSON schemas. However, users can store schemas of other formats too.
    • 🚀 In v0.1 we have compatibility policies enabled only for Avro. Subsequent releases will support compatibility checks for Protobuf and JSON.
    • The service stores and manages encoding information in form of codec information. Codecs could correspond to different compression or encryption used while encoding the serialized data at rest. The service generates unique identifiers for schemas and codec information pairs that users may use to tag their data.

    For more details, visit the Schema Registry repository.

    As of 0.1, Schema Registry is released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

    🐎 Performance Improvements and Optimizations

    🚀 Pravega 0.8 contains significant improvements over the previous release. These are the main improvements included in 0.8.

    • ⚡️ Issue #4774: (Client) Saves significant compute cycles for a reader by minimizing and optimizing costly calls for managing its internal state.
    • Issue #4653: (Segment Store) Control the number of container starts/recoveries performed in parallel.
    • 🚚 Issue #4294: Remove Netty from the client.

    🔧 Configuration Names

    🔧 Pravega 0.8 changes the names of almost all configuration items, based on a new uniform configuration naming scheme.

    👀 For instance the property controller.minRebalanceIntervalSeconds is replaced by controller.rebalance.interval.seconds.min and pravegaservice.storageImplementation is replaced by pravegaservice.storage.impl.name. See this Wiki page for new names of old properties.

    🚀 Old properties are deprecated but will continue to work for now. So, upon upgrade from previous version your Pravega deployment will continue to work. You can use a combination of new and old properties to Configure Pravega components, although it is highly discouraged. If you specify a property using both new and old names, the old property will take precedence.

    ⬆️ Fresh installations must use new properties (even though old properties will work too for now). For upgrade from previous versions, we highly encourage migrating to new properties at the earliest opportunity.

    👀 See issue #4712 for more details.

    ⚡️ Other New Features and Updates

    • Issue #4893: (Controller, Metrics) Adds metrics for created and deleted scopes.
    • Issue #4910: (Segment Store, Metrics) Adds two new metrics to report the size distribution of appends and reads.
    • ✅ Issue #4680: (Client) Add a new truncation api to ByteStreamWriter.
    • Issue #4791: (Client, Controller) Add a new controller API to fetch segments for a given epoch
    • Issue #4845: (Client, Usability) StreamInfo obtained via io.pravega.client.admin.impl.StreamManager indicates if the Stream is sealed.
    • ⬆️ Issue #4846: (Library upgrades) Upgrade Netty to 4.1.50Final.
    • ⬆️ Issue #4883: (Library upgrades) Upgrade gRPC to 1.24.2.
    • ⬆️ Issue #4531: (Library upgrades) Upgrade Bookeeper client to 4.9.2.

    🐛 Bug Fixes

    • Issue #4631: (Controller, Metrics) Report failed transaction commit metric only in case of non-retryable failures.
    • ✅ Issue #4847: (Client) Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

    🚀 For a comprehensive listing of issues resolved in this release, refer to the attached commits.txt file.

    ⚡️ Ecosystem Updates/Links

    Connectors

    - Pravega Boomi Connector

    Kubernetes Operators

  • v0.8.0-rc1 Changes

    September 18, 2020

    🚀 This release contains exciting new features and significant enhancements over the previous major release line (0.7.x). This page provides an overview of the major ones.

    Key-Value Tables

    In 0.8, we have introduced an entirely new primitive that enables efficient insertions, updates and lookups of arbitrary key-value pairs. Using the same underlying infrastructure as Streams, Key-Value Tables provide the same features one would expect of a typical key-value store, plus some more.

    Here are a few highlights:

    • Distributed Key-Value Store is built on top of append-only Pravega Segments.
    • Keys are sharded uniformly across underlying Segments or can be grouped by means of Key Families.
    • ⚡️ Conditional or unconditional updates and removals.
    • ⚡️ Single or multiple entry batch (atomic) updates or removals.
    • Single or multiple entry retrieval.
    • Listing of keys/entries in bitwise lexicographical order (by key).
    • Java Map wrapper provided for easy integration with existing applications.

    🚀 Key-Value Tables are built on top of Table Segments - the same underlying primitive that Pravega has used to store its own Stream, Segment and Transaction metadata since release 0.5. The Pravega Controller and Segment Store rely on the consistency, durability and atomicity guarantees of Table Segments for all of their metadata needs - the same guarantees are also extended to Key-Value Tables as well.

    For more details, please refer to PDP-39 - Key Value Tables.

    As of 0.8, Key-Value Tables are released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

    Schema Registry

    With Pravega 0.8, we are also releasing Pravega Schema Registry 0.1. The Schema Registry service is designed to store and manage schemas for unstructured data stored in Pravega streams.

    Here are a few highlights:

    • The service provides a RESTful interface to store and manage schemas under schema groups.
    • 👉 Users can choose from different compatibility policies for safely evolving their schema.
    • 👍 The service has built-in support for popular serialization formats in Avro, Profobuf and JSON schemas. However, users can store schemas of other formats too.
    • 🚀 In v0.1 we have compatibility policies enabled only for Avro. Subsequent releases will support compatibility checks for Protobuf and JSON.
    • The service stores and manages encoding information in form of codec information. Codecs could correspond to different compression or encryption used while encoding the serialized data at rest. The service generates unique identifiers for schemas and codec information pairs that users may use to tag their data.

    For more details, visit the Schema Registry repository.

    As of 0.1, Schema Registry is released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

    🐎 Performance Improvements and Optimizations

    🚀 Pravega 0.8 contains significant improvements over the previous release. These are the main improvements included in 0.8.

    • ⚡️ Issue #4774: (Client) Saves significant compute cycles for a reader by minimizing and optimizing costly calls for managing its internal state.
    • Issue #4653: (Segment Store) Control the number of container starts/recoveries performed in parallel.
    • 🚚 Issue #4294: Remove Netty from the client.

    🔧 Configuration Names

    🔧 Pravega 0.8 changes the names of almost all configuration items, based on a new uniform configuration naming scheme.

    👀 For instance the property controller.minRebalanceIntervalSeconds is replaced by controller.rebalance.interval.seconds.min and pravegaservice.storageImplementation is replaced by pravegaservice.storage.impl.name. See this Wiki page for new names of old properties.

    🚀 Old properties are deprecated but will continue to work for now. So, upon upgrade from previous version your Pravega deployment will continue to work. You can use a combination of new and old properties to Configure Pravega components, although it is highly discouraged. If you specify a property using both new and old names, the old property will take precedence.

    ⬆️ Fresh installations must use new properties (even though old properties will work too for now). For upgrade from previous versions, we highly encourage migrating to new properties at the earliest opportunity.

    👀 See issue #4712 for more details.

    ⚡️ Other New Features and Updates

    • Issue #4893: (Controller, Metrics) Adds metrics for created and deleted scopes.
    • Issue #4910: (Segment Store, Metrics) Adds two new metrics to report the size distribution of appends and reads.
    • ✅ Issue #4680: (Client) Add a new truncation api to ByteStreamWriter.
    • Issue #4791: (Client, Controller) Add a new controller API to fetch segments for a given epoch
    • Issue #4845: (Client, Usability) StreamInfo obtained via io.pravega.client.admin.impl.StreamManager indicates if the Stream is sealed.
    • ⬆️ Issue #4846: (Library upgrades) Upgrade Netty to 4.1.50Final.
    • ⬆️ Issue #4883: (Library upgrades) Upgrade gRPC to 1.24.2.
    • ⬆️ Issue #4531: (Library upgrades) Upgrade Bookeeper client to 4.9.2.

    🐛 Bug Fixes

    • Issue #4631: (Controller, Metrics) Report failed transaction commit metric only in case of non-retryable failures.
    • ✅ Issue #4847: (Client) Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

    🚀 For a comprehensive listing of issues resolved in this release, refer to the attached commits.txt file.

    ⚡️ Ecosystem Updates/Links

    Connectors

    - Pravega Boomi Connector

    Kubernetes Operators

  • v0.8.0-rc0 Changes

    September 11, 2020

    🚀 This release contains exciting new features and significant enhancements over the previous major release line (0.7.x). This page provides an overview of the major ones.

    Key-Value Tables

    In 0.8, we have introduced an entirely new primitive that enables efficient insertions, updates and lookups of arbitrary key-value pairs. Using the same underlying infrastructure as Streams, Key-Value Tables provide the same features one would expect of a typical key-value store, plus some more.

    Here are a few highlights:

    • Distributed Key-Value Store is built on top of append-only Pravega Segments.
    • Keys are sharded uniformly across underlying Segments or can be grouped by means of Key Families.
    • ⚡️ Conditional or unconditional updates and removals.
    • ⚡️ Single or multiple entry batch (atomic) updates or removals.
    • Single or multiple entry retrieval.
    • Listing of keys/entries in bitwise lexicographical order (by key).
    • Java Map wrapper provided for easy integration with existing applications.

    🚀 Key-Value Tables are built on top of Table Segments - the same underlying primitive that Pravega has used to store its own Stream, Segment and Transaction metadata since release 0.5. The Pravega Controller and Segment Store rely on the consistency, durability and atomicity guarantees of Table Segments for all of their metadata needs - the same guarantees are also extended to Key-Value Tables as well.

    For more details, please refer to PDP-39 - Key Value Tables.

    As of 0.8, Key-Value Tables are released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and expect to graduate them out of Beta in a future release.

    Schema Registry

    With Pravega 0.8, we are also releasing Pravega Schema Registry 0.1. The Schema Registry service is designed to store and manage schemas for unstructured data stored in Pravega streams.

    Here are a few highlights:

    • The service provides a RESTful interface to store and manage schemas under schema groups.
    • 👉 Users can choose from different compatibility policies for safely evolving their schema.
    • 👍 The service has built-in support for popular serialization formats in Avro, Profobuf and JSON schemas. However, users can store schemas of other formats too.
    • 🚀 In v0.1 we have compatibility policies enabled only for Avro. Subsequent releases will support compatibility checks for Protobuf and JSON.
    • The service stores and manages encoding information in form of codec information. Codecs could correspond to different compression or encryption used while encoding the serialized data at rest. The service generates unique identifiers for schemas and codec information pairs that users may use to tag their data.

    For more details, visit the Schema Registry repository.

    🐎 Performance Improvements and Optimizations

    🚀 Pravega 0.8 contains significant improvements over the previous release. These are the main improvements included in 0.8.

    • ⚡️ Issue #4774: (Client) Saves significant compute cycles for a reader by minimizing and optimizing costly calls for managing its internal state.
    • Issue #4653: (Segment Store) Control the number of container starts/recoveries performed in parallel.
    • 🚚 Issue #4294: Remove Netty from the client.

    🔧 Configuration Names

    👀 Pravega 0.8 changes the names of almost all configuration items, based on a new uniform configuration naming scheme. For instance the property controller.minRebalanceIntervalSeconds is replaced by controller.rebalance.interval.seconds.min and pravegaservice.storageImplementation is replaced by pravegaservice.storage.impl.name. See this Wiki page for new names of old properties.

    🚀 Old properties are deprecated but will continue to work for now. So, upon upgrade from previous version your Pravega deployment will continue to work. You can use a combination of new and old properties to Configure Pravega components, although it is highly discouraged. If you specify a property using both new and old names, the old property will take precedence.

    ⬆️ Fresh installations must use new properties (even though old properties will work too for now). For upgrade from previous versions, we highly encourage migrating to new properties at the earliest opportunity.

    👀 See issue #4712 for more details.

    ⚡️ Other New Features and Updates

    • Issue #4893: (Controller, Metrics) Adds metrics for created and deleted scopes.
    • Issue #4910: (Segment Store, Metrics) Adds two new metrics to report the size distribution of appends and reads.
    • ✅ Issue #4680: (Client) Add a new truncation api to ByteStreamWriter.
    • Issue #4791: (Client, Controller) Add a new controller API to fetch segments for a given epoch
    • Issue #4845: (Client, Usability) StreamInfo obtained via io.pravega.client.admin.impl.StreamManager indicates if the Stream is sealed.
    • ⬆️ Issue #4846: (Library upgrades) Upgrade Netty to 4.1.50Final.
    • ⬆️ Issue #4883: (Library upgrades) Upgrade gRPC to 1.24.2.
    • ⬆️ Issue #4531: (Library upgrades) Upgrade Bookeeper client to 4.9.2.

    🐛 Bug Fixes

    • Issue #4631: (Controller, Metrics) Report failed transaction commit metric only in case of non-retryable failures.
    • ✅ Issue #4847: (Client) Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

    🚀 For a comprehensive listing of issues resolved in this release, refer to the attached commits.txt file.

    ⚡️ Ecosystem Updates/Links

    Connectors

    - Pravega Boomi Connector

    Kubernetes Operators

  • v0.7.3-rc0 Changes

    December 07, 2020

    🚀 This release contains several bug fix and performance improvements over 0.7.2:

    Segment Store

    • Issue #5294: Bookkeeper Client may cache old DNS entries for too long.
    • 🛠 Issue #5269: (SegmentStore) Fixed a deadlock in StreamSegmentReadIndex.
    • Issue #5119: (SegmentStore) Copy-on-Read for Table Segment Compaction
    • Issue #5155: (SegmentStore) Enabling copy-on-read for all Segment Reads
    • 🛠 Issue #5154: (SegmentStore) Fixed a stats concurrency bug in the Read Index (#5170)
    • 🛠 Issue #5207: (SegmentStore) Read Index Bug Fixes

    Controller

    • 🚚 Issue #4757: Remove stream reference from watermark bucket upon stream deletion.
    • Issue #5341: Fix thread leak in controller.
    • ⏱ Issue #4915: Add timeout to all calls to segment store from controller.

    🚀 commits.txt

  • v0.7.2 Changes

    September 18, 2020

    🚀 This release contains several bug fix and performance improvements over 0.7.1:

    Segment Store

    • Issue #5102: TableSegment Rollover Size not properly set.
    • Issue #5039: RollingStorage can trim Header Chunk on FileSystemStorage.
    • Issue #3637: Enable compaction on all Table Segments.
    • 🚀 Issue #4784: Release throttle in the event an unhandled exception comes from netty

    Client

    • Issue #5062: Handle SegmentSealedException during Append Setup.
    • ✅ Issue #4817: Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

    Controller

    • Issue #4476: Truncate internal streams to reclaim storage space.
    • 🔊 Issue #4985: Additional Logs for tracking controller periodic retention.
    • Issue #4944: Change granularity of Time based Retention Config in REST API.
    • 👻 Issue #4920: From noteTime api, throw exception rather than set the response code.
    • Issue #4913: Filter partially created streams from list of streams.
    • Issue #4815: Ensure TxnFailedException is thrown only if Transaction is not in an open state.

    General

    • 📇 Issue #5068: Sanitize gRPC call metadata before logging.
    • Issue #5147: Added check for service in init_kubernetes.sh script.

    🚀 commits.txt

  • v0.7.2-rc0 Changes

    September 15, 2020

    🚀 This release contains several bug fix and performance improvements over 0.7.1:

    Segment Store

    • Issue #5102: TableSegment Rollover Size not properly set.
    • Issue #5039: RollingStorage can trim Header Chunk on FileSystemStorage.
    • Issue #3637: Enable compaction on all Table Segments.
    • 🚀 Issue #4784: Release throttle in the event an unhandled exception comes from netty

    Client

    • Issue #5062: Handle SegmentSealedException during Append Setup.
    • ✅ Issue #4817: Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

    Controller

    • Issue #4476: Truncate internal streams to reclaim storage space.
    • 🔊 Issue #4985: Additional Logs for tracking controller periodic retention.
    • Issue #4944: Change granularity of Time based Retention Config in REST API.
    • 👻 Issue #4920: From noteTime api, throw exception rather than set the response code.
    • Issue #4913: Filter partially created streams from list of streams.
    • Issue #4815: Ensure TxnFailedException is thrown only if Transaction is not in an open state.

    General

    • 📇 Issue #5068: Sanitize gRPC call metadata before logging.
    • Issue #5147: Added check for service in init_kubernetes.sh script.

    🚀 commits.txt

  • v0.7.1 Changes

    May 29, 2020

    🚀 This release contains several bug fix and performance improvements over 0.7.0:

    Segment Store

    • Issue #3992: (Data Recovery) Reconcile BookKeeperLog with BookKeeper Ledgers.
    • Issue #4273: (Bug Fix) Revised Throtting Metrics.
    • Issue #4642: (Bug Fix) Load TLS certificates from /etc/secret-volume/ca-bundle.
    • 🛠 Issue #4652: (Bug Fix) Fixed Reconciliation of partially-flushed Storage append.
    • Issue #4654: (Bug Fix) DataLogWriterNotPrimaryException thrown indefinitely.
    • 🛠 Issue #4702: (Bug Fix) Fixed a deadlock condition in the Read Index.
    • Issue #4706: (Bug Fix) Fix SegmentStoreMetrics.
    • 📇 Issue #4721: (Bug Fix/Data Recovery) Auto-reconciliation of BookKeeperLog Metadata.
    • Issue #4743: (Bug Fix) Infinite loop in StorageWriter if deleting a segment while flushing.

    Client

    • 🔊 Issue #4633: (Bug Fix) Improve controller client logs.
    • 👻 Issue #4638: (Bug Fix) Propagate retryable exception to the client instead of returning Status.Failure on all exceptions.
    • Issue #4669: (Bug Fix) Fix serialization in PositionImpl.
    • 🐎 Issue #4720: (Performance) Reduce cpu use on client.
    • Issue #4766: (Bug Fix) Ensure BatchClient handles partial response from SSS.

    Controller

    • Issue #4631: (Bug Fix) Failed transactions metric incorrectly reported.
    • ⚡️ Issue #4639: (Bug Fix) Table store helper to retry only on unknownhost and auth failure for update requests.

    General

    • 🔧 Issue #4347: (Bug Fix) Enable auto-reload of modified logging configuration at runtime.
    • Issue #4628: Add license copy to binary artifacts.
    • Issue #4673: Bump up versions of some of the dependencies.

    🚀 commits.txt

  • v0.7.1-rc1 Changes

    May 27, 2020

    🚀 This release contains several bug fix and performance improvements over 0.7.0:

    Segment Store

    • Issue #3992: (Data Recovery) Reconcile BookKeeperLog with BookKeeper Ledgers.
    • Issue #4273: (Bug Fix) Revised Throtting Metrics.
    • Issue #4642: (Bug Fix) Load TLS certificates from /etc/secret-volume/ca-bundle.
    • 🛠 Issue #4652: (Bug Fix) Fixed Reconciliation of partially-flushed Storage append.
    • Issue #4654: (Bug Fix) DataLogWriterNotPrimaryException thrown indefinitely.
    • 🛠 Issue #4702: (Bug Fix) Fixed a deadlock condition in the Read Index.
    • Issue #4706: (Bug Fix) Fix SegmentStoreMetrics.
    • 📇 Issue #4721: (Bug Fix/Data Recovery) Auto-reconciliation of BookKeeperLog Metadata.
    • Issue #4743: (Bug Fix) Infinite loop in StorageWriter if deleting a segment while flushing.

    Client

    • 🔊 Issue #4633: (Bug Fix) Improve controller client logs.
    • 👻 Issue #4638: (Bug Fix) Propagate retryable exception to the client instead of returning Status.Failure on all exceptions.
    • Issue #4669: (Bug Fix) Fix serialization in PositionImpl.
    • 🐎 Issue #4720: (Performance) Reduce cpu use on client.
    • Issue #4766: (Bug Fix) Ensure BatchClient handles partial response from SSS.

    Controller

    • Issue #4631: (Bug Fix) Failed transactions metric incorrectly reported.
    • ⚡️ Issue #4639: (Bug Fix) Table store helper to retry only on unknownhost and auth failure for update requests.

    General

    • 🔧 Issue #4347: (Bug Fix) Enable auto-reload of modified logging configuration at runtime.
    • Issue #4628: Add license copy to binary artifacts.
    • Issue #4673: Bump up versions of some of the dependencies.

    🚀 commits.txt