yildiz

Graph Database on top of Bigtable

Usage no npm install needed!

<script type="module">
  import yildiz from 'https://cdn.skypack.dev/yildiz';
</script>

README

Yildiz Graph Database

yildiz

Thin graph database layer on top of Google Bigtable.

Build Status Version MIT License node Swagger

Intro

YildizDB acts as a highly scaleable HTTP layer in front of Google Bigtable. It helps you store billions of relations between nodes in edges and resolve them in milliseconds. A small access layer helps you manage multiple namespaces with ease. YildizDB scales to hundrets of Terabytes. YildizDB excells in $N:N$ non nested relationships.

Features

  • simple namespacing (table separation) with prefixes
  • simple access management through tokens
  • multi-tenancy through table prefixes (with access management)
  • high read and write performance
  • fast read access under heavy write load
  • able to handle billions of edges and nodes
  • scales beyond Terabytes
  • lightweight deployments (small Node.js footprint)
  • ttl feature for all resources
  • HTTP Open API
  • Kubernetes HELM charts

Build for high throughput $N:N$

  • Highly async API based on fastify
  • Thin layer on top of Google's GRPC Bigtable API
  • Hashing and translating all string identifiers into integer representations via murmurhash3 automatically

Fast random access to a node's edge data $1:N$

  • Multiple complex caching layers
  • Custom Bigtable cache table speed up by Google Memorystore (Redis)
  • All Memorystore hits also speed up by In-Memory store
  • Fetch job that keeps active nodes refreshed in cache

Available clients

  • Node.js Client
  • Any Http Client can be used to access the HTTP-Interface

Usage

  • You will need a Google Cloud Project with a running Bigtable cluster
  • Additionally YildizDB requires a Memorystore (or Redis) instance
  • Configure ./config/bigtable.json accordingly
  • Install and start: npm install -g yildiz and yildizdb -p 3058 -l ./config/bigtable.json
  • A word on configuration can be found here
  • Use right alongside your code
  • Spawn server via http interface

Deployment

  • YildizDB is designed to be deployed as simple platform service
  • it requires Node.js > 9.x.x and Redis > 3.x
  • It simply requires its npm module as well as a config file that describes the connections to Bigtable and Memorystore (Redis)
  • It scales best with an HTTP load balancer in front of it e.g. NGINX
  • We run and scale it very successfully in Google's Kubernetes Engine
  • We also offer HELM charts
  • Read some more about the config file here

Metrics & Monitoring

  • Yildiz exposes Prometheus Metrics @ /admin/metrics
  • Read more about it here

Developing YildizDB

Developing YildizDB with Google Bigtable backend

  • Configure ./config/bigtable.json accordingly
  • Run tests via yarn test
  • Start via yarn http

Documentation

Disclaimer

  • This project is not affiliated with Google
  • License is MIT see