noms

The versioned, forkable, syncable database

Description

Noms is different from other databases. It is:

* Content-addressed. If you have some data you want to put into Noms, you don't have to worry about whether it already exists. Duplicate data is automatically ignored. There is no update, only insert.

* Append-only. When you commit data to Noms, you aren't overwriting anything. Instead you're adding to a historical record. By default, data is never removed from Noms. You can see the entire history of the database, diff any two commits, or rewind to any previous point in time.

* Typed. Every value, dataset, and version of a database has a type, which is generated automatically as you add data. You can write code against the type of a Noms database, confident that you've handled all the cases you need to.

* Decentralized. If I give you a copy of my database, you and I can modify our copies disconnected from each other, and come back together and merge our changes efficiently and correctly days, weeks, or years later.

Programming language: Go

License: Apache License 2.0

Tags: NoSQL     database     Git     Decentralized    

Latest version: v7.4

README

Warning - This project is not active

Noms is not being maintained. You shouldn't use it, except maybe for fun or research.

If you are interested in something like Noms, you probably want Dolt (https://github.com/dolthub/dolt) which is a fork of this project and actively maintained.

Send me (aaron at aaronboodman.com) a message if you have questions.

Use Cases  |  Setup  |  Status  |  [Documentation](./doc/intro.md)  |  Contact

Welcome

Noms is a decentralized database philosophically descendant from the Git version control system.

Like Git, Noms is:

• Versioned: By default, all previous versions of the database are retained. You can trivially track how the database evolved to its current state, easily and efficiently compare any two versions, or even rewind and branch from any previous version.
• Synchronizable: Instances of a single Noms database can be disconnected from each other for any amount of time, then later reconcile their changes efficiently and correctly.

Unlike Git, Noms is a database, so it also:

• Primarily stores structured data, not files and directories (see: the Noms type system)
• Scales well to large amounts of data and concurrent clients
• Supports atomic transactions (a single instance of Noms is CP, but Noms is typically run in production backed by S3, in which case it is "effectively CA")
• Supports efficient indexes (see: Noms prolly-trees)
• Features a flexible query model (see: [GraphQL](./go/ngql/README.md))

A Noms database can reside within a file system or in the cloud:

• The (built-in) [NBS](./go/nbs) ChunkStore implementation provides two back-ends which provide persistence for Noms databases: one for storage in a file system and one for storage in an S3 bucket.

Finally, because Noms is content-addressed, it yields a very pleasant programming model.

Working with Noms is declarative. You don't INSERT new data, UPDATE existing data, or DELETE old data. You simply declare what the data ought to be right now. If you commit the same data twice, it will be deduplicated because of content-addressing. If you commit almost the same data, only the part that is different will be written.

Use Cases

[Decentralization](./doc/decent/about.md)

Because Noms is very good at sync, it makes a decent basis for rich, collaborative, fully-decentralized applications.

Mobile Offline-First Database

Embed Noms into mobile applications, making it easier to build offline-first, fully synchronizing mobile applications.

Install

1. Download the latest release:
   • Linux
   • Mac OS
2. Unzip the directory somewhere and add it to your $PATH
3. Verify Noms is installed correctly:

$ noms version
format version: 7.18
built from <developer build>

Run

Import some data:

go install github.com/attic-labs/noms/samples/go/csv/csv-import
curl 'https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv?accessType=DOWNLOAD' > /tmp/data.csv
csv-import /tmp/data.csv /tmp/noms::nycdemo

Explore:

noms show /tmp/noms::nycdemo

Should show:

struct Commit {
  meta: struct Meta {
    date: "2017-09-19T19:33:01Z",
    inputFile: "/tmp/data.csv",
  },
  parents: set {},
  value: [  // 236 items
    struct Row {
      countAmericanIndian: "0",
      countAsianNonHispanic: "3",
      countBlackNonHispanic: "21",
      countCitizenStatusTotal: "44",
      countCitizenStatusUnknown: "0",
      countEthnicityTotal: "44",
...

Status

Nobody is working on this right now. You shouldn't rely on it unless you're willing to take over development yourself.

Major Open Issues

These are the major things you'd probably want to fix before relying on this for most systems.

• Sync performance with long commit chains (https://github.com/attic-labs/noms/issues/2233)
• Migration (https://github.com/attic-labs/noms/issues/3363)
• Garbage Collection (https://github.com/attic-labs/noms/issues/3374)
• Query language
  • We started trying to hack in GraphQL but it's incomplete and maybe not the right thing. See: [ngql](./go/ngql/README.md)
• Various other smaller bugs and improvements

Learn More About Noms

For the decentralized web: [The Decentralized Database](doc/decent/about.md)

Learn the basics: [Technical Overview](doc/intro.md)

Tour the CLI: [Command-Line Interface Tour](doc/cli-tour.md)

Tour the Go API: [Go SDK Tour](doc/go-tour.md)

Contact Us

Interested in using Noms? Awesome! We would be happy to work with you to help understand whether Noms is a fit for your problem. Reach out at:

• Mailing List
• Twitter

Licensing

Noms is open source software, licensed by Attic Labs, Inc. under the Apache License, Version 2.0.