All Versions
57
Latest Version
Avg Release Cycle
26 days
Latest Release
-

Changelog History
Page 3

  • v4.3.3 Changes

    June 17, 2020
    • ⚡️ Typescript types updated to support ssl options required in latest pg module. PR from @asafh

  • v4.3.2 Changes

    June 07, 2020
    • ⚡️ Typescript types updated to removed old done() function on subscribe callback. PR from @GriffinSchneider

  • v4.3.1 Changes

    May 25, 2020
    • ➕ Added missing Typescript type for pg intervals used by includeMetadata

  • v4.3.0 Changes

    May 21, 2020
    • ➕ Added includeMetadata option to fetch(), subscribe(), and their derivatives to allow including all job attributes during fetch. PR from @kevboh
    • 🛠 Fixed state monitoring Typescript type defs. PR from @brianmcd
    • 🛠 Fixed issue with maintenance job creation if the instance was previously shut down during execution

  • v4.2.0 Changes

    May 08, 2020
    • ⚡️ publishOnce() updated to fall back to the queue if the key argument is missing.
    • ⬆️ Upgraded uuid dependency to version 8.0.0.

  • v4.1.0 Changes

    March 31, 2020
    • 🚧 Retention policies added for internal maintenance queues to reduce the number of records in the job table.
    • 🛠 Fixed issue in some multi-master use cases where too many maintenance jobs were being created.
    • 🔄 Changed deleteQueue(name) and deleteAllQueues() behavior to only impact pending queue items and not delete completed or active jobs.
    • ➕ Added getQueueSize(name) to retrieve the current size of a queue.
    • ➕ Added clearStorage() as a utility function if and when needed to empty all job storage, archive included.
    • ⬆️ Restored older schema migrations to allow upgrading directly to version 4 from version 1.1 and higher.
    • ⬆️ Upgraded pg dependency to version 8.0.0.

  • v4.0.1 Changes

    March 20, 2020
    • ⏪ Restored BYODB support for Knex.js

  • v4.0.0 Changes

    March 16, 2020

    🔄 Changes

    • 🚧 start() is now fully multi-master ready and supported for installation, schema migrations and maintenance operations.
    • ➕ Added default configurations. The following options can now be set in the constructor and will apply to all usages of publish() or subscribe() on the instance unless overridden on the functions themselves.
      • Subscribe
      • polling interval
      • Publish
      • Expiration
      • Retries
      • Retention (new)
    • ⬆️ MAJOR: Replaced expiration pg interval string configuration in publish() with specific integer options for better validation and api consistency. If the expireIn option is detected after upgrading, you will see a warning such as the following, which will only be emitted once per instance. As mentioned above, all of these options can become defaults if used in the constructor configuration.
      (node:1) [pg-boss-w01] Warning: 'expireIn' option detected.  This option has been removed.  Use expireInSeconds, expireInMinutes or expireInHours
    • Removed:
      • expireIn
    • Added:
      • expireInSeconds
      • expireInMinutes
      • expireInHours
      • 🚧 MAJOR: Added retention policies for created jobs. In v3, maintenance operations archived completed jobs, but this policy ignored jobs which were created and never fetched.
    • Added the following configuration options to publish() and new PgBoss()
      • retentionSeconds
      • retentionMinutes
      • retentionHours
      • retentionDays
      • 🚧 MAJOR: Replaced maintenance pg interval string configurations with specific integer options for better validation and api consistency
    • Removed:
      • deleteArchivedJobsEvery
      • archiveCompletedJobsEvery
    • Added:
      • archiveIntervalSeconds
      • archiveIntervalMinutes
      • archiveIntervalHours
      • archiveIntervalDays
      • deleteIntervalSeconds
      • deleteIntervalMinutes
      • deleteIntervalHours
      • deleteIntervalDays
      • 🚧 MAJOR: Consolidated the maintenance constructor options and removed any options for intervals less than 1 second.
    • Removed:
      • expireCheckInterval
      • expireCheckIntervalSeconds
      • expireCheckIntervalMinutes
      • archiveCheckInterval
      • archiveCheckIntervalSeconds
      • archiveCheckIntervalMinutes
      • deleteCheckInterval
    • Added:
      • maintenanceIntervalSeconds
      • maintenanceIntervalMinutes
      • MAJOR: Split static getMigrationPlans() function into 2 functions for clarity.
    • Removed:
      • uninstall argument from getMigrationPlans(schema, version)
    • Added:
      • getRollbackPlans(schema, version)
      • 🚚 MAJOR: Removed pgcrypto from installation script.

    Summary

    🚀 The breaking changes introduced in this release should not cause any run-time failures, as they are focused on maintenance and operations. However, if you use the deferred publishing options, read the section below regarding retention policy changes, as this version will now archive jobs which have been created but never fetched.

    Multi-master

    🚀 This release was originally started to support rolling deployments where a new instance was being started before another instance was turned off in a container orchestration system. When this happened, sometimes a race condition occurred between maintenance operations causing unpredictable deadlock errors (see issue #133). This was primarily because of the use of unordered data sets in CTEs from a DELETE ... RETURNING statement. However, instead of focusing on the SQL itself, the concurrency problem proved a far superior use case to resolve holistically, and this became a perfect example of pg-boss eating its own dog food via a dedicated maintenance queue (mentioned below).

    🚧 The result of using a queue for maintenance instead of timers such as setTimeout() is the same distributed concurrency benefit of using queues for other workloads. This is sometimes referred to as a multi-master configuration, where more than one instance is using start() simultaneously. If and when this occurs in your environment, only one of them will be able to fetch a job (maintenance or state monitoring) and issue the related SQL commands.

    Additionally, all schema operations, both first-time provisioning and migrations, are nested within advisory locks to prevent race conditions during start(). Internally, these locks are created using pg_advisory_xact_lock() which auto-unlock at the end of the transaction and don't require a persistent session or the need to issue an unlock. This should make it compatible with most connection poolers, such as pgBouncer in transactional pooling mode.

    🔧 One example of how this is useful would be including start() inside the bootstrapping of a pod in a ReplicaSet in Kubernetes. Being able to scale up your job processing using a container orchestration tool like k8s is becoming more popular, and pg-boss can be dropped into this system with no additional code or special configuration.

    Retention Policies

    🚧 As mentioned above, previously only completed jobs were included in the archive maintenance, but with one exception: completion jobs were also moved to the archive even though they were in created state. This would sometimes result in missed jobs if an onComplete subscription were to reach a backlogged state that couldn't keep up with the configured archive interval.

    0️⃣ A new set of retention options (listed above) have been added which control how long any job may exist in created state, original or completion. Currently, the default retention is 30 days, but even if it's customized it automatically carries over to the associated completion job as well.

    👷 Furthermore, this retention policy is aware of any deferred jobs, such as those created with publishAfter(). If you have future-dated or interval-deferred jobs, the retention policy start date is internally based on the deferred date, not the created timestamp.

    ⬆️ If you're upgrading from v3, a migration script will run and set the retention date on all jobs found in 'created' state. For example, if you use the option retentionDays: 7 in the constructor, then run start(), the migration will assign a retention date of 7 days after the created or deferred date, whichever is later.

    🚧 Maintenance and Monitoring queues

    To keep maintenance overhead as light as possible, the concurrency of each task (expiration, archiving, deletion) has been adjusted to one operation at a time and placed into dedicated queues prefixed with '__pgboss__'. The same was also done for the optional state count monitoring.

    🚚 pgcrypto extension install removed from provisioning script

    ⬆️ If you were running pg-boss as a superuser account in production to have it auto-provision the pgcrypto extension in a new database, this change might be viewed as a disadvantage. The primary principle at play in this decision is "It should be simple to uninstall anything which was installed". Adding an extension to a database cannot be scoped to a schema, and it requires superuser privilege. If pg-boss were to install pgcrypto, it would be unsafe to assume it could be later removed, as it may be in use elsewhere. Also, having a script embedded in the installation which requires superuser privilege sends the wrong message of the intent of how applications should be configured in production, where a least privilege model should always be used. As a reminder, below is a simple 1-liner to run in your database if it's not already installed. If you are upgrading pg-boss from a previous version, this is obviously not an issue.

    CREATE EXTENSION pgcrypto;

  • v3.2.2 Changes

    January 22, 2020
    • Deferring housekeeping operations on start to reduce deadlocks during concurrent start() instances

  • v3.2.1 Changes

    January 09, 2020
    • 🛠 Fixed rare deadlocks by stacking housekeeping operations one at a time during start().
    • ➕ Added archive(), purge() and expire() to exports for manual housekeeping if desired along with connect(). Use this only if you need it for special cases, as it's not a good idea to run these in parallel (see deadlock comment above).
    • ➕ Added index to archive table by date to improve housekeeping perf.
    • ✅ Node 8 is now officially the minimum supported version. Not only have I stopped testing anything lower than 8 in Travis, but I finally migrated to async await in this round of changes.
    • Typescript type defs.