README

dioscuri

Do you:

🤨 think that HTTP and HTML are bloated?
😔 feel markdown has superfluous features?
🤔 find gopher too light?
🥰 like BRUTALISM?

Then Gemini might be for you (see this post or this one on why it’s cool).

Dioscuri (named for the gemini twins Castor and Pollux) is a tokenizer/lexer/parser/etc for gemtext (the text/gemini markup format). It gives you several things:

buffering and streaming interfaces that compile to HTML
interfaces to create unist compliant abstract syntax trees and serialize those back to gemtext
interfaces to transform to and from mdast (markdown ast)
parts that could be used to generate CSTs

These tools can be used if you now have markdown but want to transform it to gemtext. Or if you want to combine your posts into an RSS feed or on your “homepage”. And many other things!

Install

This package is ESM only: Node 12+ is needed to use it and it must be imported instead of required.

npm:

npm install dioscuri

Use
API
gast
- Root
- Break
- Heading
- Link
- List
- ListItem
- Pre
- Quote
- Text
Security
Related
License

Use

See each interface below for examples.

API

This package exports the following identifiers: buffer, stream, fromGemtext, toGemtext, fromMdast, toMdast. The raw compiler and parser are also exported. There is no default export.

`buffer(doc, encoding?, options?)`

Compile gemtext to HTML.

`doc`

Gemtext to parse (string or Buffer).

`encoding`

Character encoding to understand doc as when it’s a Buffer (string, default: 'utf8').

`options.defaultLineEnding`

Value to use for line endings not in doc (string, default: first line ending or '\n').

Generally, discuri copies line endings ('\n' or '\r\n') in the document over to the compiled HTML. In some cases, such as > a, extra line endings are added: <blockquote>\n<p>a</p>\n</blockquote>.

`options.allowDangerousProtocol`

Whether to allow potentially dangerous protocols in URLs (boolean, default: false). URLs relative to the current protocol are always allowed (such as, image.jpg). Otherwise, the allowed protocols are gemini, http, https, irc, ircs, mailto, and xmpp.

Returns

string — Compiled HTML.

Example

Say we have a gemtext document, example.gmi:

# Hello, world!

Some text

=> https://example.com An example

> A quote

* List

An our script, example.js, looks as follows:

import fs from 'fs'
import {buffer} from 'dioscuri'

var doc = fs.readFileSync('example.gmi')

console.log(buffer(doc))

Now, running node example.js yields:

<h1>Hello, world!</h1>
<br />
<p>Some text</p>
<br />
<div><a href="https://example.com">An example</a></div>
<br />
<blockquote>
<p>A quote</p>
</blockquote>
<br />
<ul>
<li>List</li>
</ul>

`stream(options?)`

Streaming interface to compile gemtext to HTML. options is the same as the buffering interface above.

Example

Assuming the same example.gmi as before and an example.js like this:

import fs from 'fs'
import {stream} from 'dioscuri'

fs.createReadStream('example.gmi')
  .on('error', handleError)
  .pipe(stream())
  .pipe(process.stdout)

function handleError(error) {
  throw error // Handle your error here!
}

Then running node example.js yields the same as before.

`fromGemtext(doc, encoding?)`

Parse gemtext to into an AST (gast). doc and encoding are the same as the buffering interface above.

Returns

Root.

Example

Assuming the same example.gmi as before and an example.js like this:

import fs from 'fs'
import {fromGemtext} from 'dioscuri'

var doc = fs.readFileSync('example.gmi')

console.dir(fromGemtext(doc), {depth: null})

Now running node example.js yields (positional info removed for brevity):

{
  type: 'root',
  children: [
    {type: 'heading', rank: 1, value: 'Hello, world!'},
    {type: 'break'},
    {type: 'text', value: 'Some text'},
    {type: 'break'},
    {type: 'link', url: 'https://example.com', value: 'An example'},
    {type: 'break'},
    {type: 'quote', value: 'A quote'},
    {type: 'break'},
    {type: 'list', children: [{type: 'listItem', value: 'List'}]}
  ]
}

`toGemtext(tree)`

Serialize gast.

Example

Say our script example.js looks as follows:

import {toGemtext} from 'dioscuri'

var tree = {
  type: 'root',
  children: [
    {type: 'heading', rank: 1, value: 'Hello, world!'},
    {type: 'break'},
    {type: 'text', value: 'Some text'}
  ]
}

console.log(toGemtext(tree))

Then running node example.js yields:

# Hello, world!

Some text

`fromMdast(tree, options?)`

Transform mdast to gast.

`options.endlinks`

Place links at the end of the document (boolean, default: false). The default is to place links before the next heading.

`options.tight`

Do not put blank lines between blocks (boolean, default: false). The default is to place breaks between each block (paragraph, heading, etc).

Returns

gast, probably. Some mdast nodes have no gast representation so they are dropped. If you pass one of those in as tree, you’ll get undefined out.

Example

Say we have a markdown document, example.md:

# Hello, world!

Some text, *emphasis*, **strong**\
`code()`, and ~~scratch that~~strikethrough.

Here’s a [link](https://example.com 'Just an example'), [link reference][*],
and images: [image reference][*], [](example.png 'Another example').

***

> Some
> quotes

*   a list
*   with another item

1.  “Ordered”
2.  List

```
A
Poem
```

```js
console.log(1)
```

| Name | Value |
| ---- | ----- |
| Beep | 1.2   |
| Boop | 3.14  |

*   [x] Checked
*   [ ] Unchecked

Footnotes[^†], ^[even inline].

[*]: https://example.org "URL definition"

[^†]: Footnote definition

An our script, example.js, looks as follows:

import fs from 'fs'
import gfm from 'micromark-extension-gfm'
import footnote from 'micromark-extension-footnote'
import fromMarkdown from 'mdast-util-from-markdown'
import mdastGfm from 'mdast-util-gfm'
import mdastFootnote from 'mdast-util-footnote'
import {fromMdast, toGemtext} from 'dioscuri'

var mdast = fromMarkdown(fs.readFileSync('example.md'), {
  extensions: [gfm(), footnote({inlineNotes: true})],
  mdastExtensions: [mdastGfm.fromMarkdown, mdastFootnote.fromMarkdown]
})

console.log(toGemtext(fromMdast(mdast)))

Now, running node example.js yields:

# Hello, world!

Some text, emphasis, strong code(), and strikethrough.

Here’s a link[1], link reference[2], and images: image reference[2], [3].

> Some quotes

* a list
* with another item

* “Ordered”
* List

```
A
Poem
```

```js
console.log(1)
```

```csv
Name,Value
Beep,1.2
Boop,3.14
```

* ✓ Checked
* ✗ Unchecked

Footnotes[a], [b].

=> https://example.com [1] Just an example

=> https://example.org [2] URL definition

=> example.png [3] Another example

[a] Footnote definition

[b] even inline

`toMdast(tree)`

Transform gast to mdast.

Returns

mdast, probably. Some gast nodes have no mdast representation so they are dropped. If you pass one of those in as tree, you’ll get undefined out.

Example

Say we have a gemtext document, example.gmi:

# Hello, world!

Some text

=> https://example.com An example

> A quote

* List

An our script, example.js, looks as follows:

import fs from 'fs'
import {fromGemtext, toMdast} from 'dioscuri'

var doc = fs.readFileSync('example.gmi')

console.dir(toMdast(fromGemtext(doc)), {depth: null})

Now, running node example.js yields (position info removed for brevity):

{
  type: 'root',
  children: [
    {
      type: 'heading',
      depth: 1,
      children: [{type: 'text', value: 'Hello, world!'}]
    },
    {
      type: 'paragraph',
      children: [{type: 'text', value: 'Some text'}]
    },
    {
      type: 'paragraph',
      children: [
        {
          type: 'link',
          url: 'https://example.com',
          title: null,
          children: [{type: 'text', value: 'An example'}]
        }
      ]
    },
    {
      type: 'blockquote',
      children: [
        {type: 'paragraph', children: [{type: 'text', value: 'A quote'}]}
      ]
    },
    {
      type: 'list',
      ordered: false,
      spread: false,
      children: [
        {
          type: 'listItem',
          spread: false,
          children: [
            {type: 'paragraph', children: [{type: 'text', value: 'List'}]}
          ]
        }
      ]
    }
  ]
}

gast

gast extends unist, a format for syntax trees, to benefit from its ecosystem of utilities.

`Root`

interface Root <: Parent {
  type: "root"
  children: [Break | Heading | Link | List | Pre | Quote | Text]
}

Root (Parent) represents a document.

`Break`

interface Break <: Node {
  type: "break"
}

Break (Node) represents a hard break.

`Heading`

interface Heading <: Literal {
  type: "heading"
  rank: 1 <= number <= 3
  value: string?
}

Heading (Literal) represents a heading of a section.

`Link`

interface Link <: Literal {
  type: "link"
  url: string
  value: string?
}

Link (Literal) represents a resource.

A url field must be present. It represents a URL to the resource.

`List`

interface List <: Parent {
  type: "list"
  children: [ListItem]
}

List (Parent) represents an enumeration.

`ListItem`

interface ListItem <: Literal {
  type: "listItem"
  value: string?
}

ListItem (Literal) represents an item in a list.

`Pre`

interface Pre <: Literal {
  type: "per"
  alt: string?
  value: string?
}

Pre (Literal) represents preformatted text.

An alt field may be present. When present, the node represents computer code, and the field gives the language of computer code being marked up.

`Quote`

interface Quote <: Literal {
  type: "quote"
  value: string?
}

Quote (Literal) represents a quote.

`Text`

interface Text <: Literal {
  type: "text"
  value: string
}

Text (Literal) represents a paragraph.

Security

Gemtext is safe. As for the generated HTML: that’s safe by default. Pass allowDangerousProtocol: true if you want to live dangerously.

@derhuerst/gemini – Gemini protocol server & client
gemini-fetch – load Gemini protocol data the way you would fetch from HTTP in JavaScript

Usage no npm install needed!

README

dioscuri

Install

Contents

Use

API

buffer(doc, encoding?, options?)

doc

encoding

options.defaultLineEnding

options.allowDangerousProtocol

Returns

Example

stream(options?)

Example

fromGemtext(doc, encoding?)

Returns

Example

toGemtext(tree)

Example

fromMdast(tree, options?)

options.endlinks

options.tight

Returns

Example

toMdast(tree)

Returns

Example

gast

Root

Break

Heading

Link

List

ListItem

Pre

Quote

Text

Security

Related

License

`buffer(doc, encoding?, options?)`

`doc`

`encoding`

`options.defaultLineEnding`

`options.allowDangerousProtocol`

`stream(options?)`

`fromGemtext(doc, encoding?)`

`toGemtext(tree)`

`fromMdast(tree, options?)`

`options.endlinks`

`options.tight`

`toMdast(tree)`

`Root`

`Break`

`Heading`

`Link`

`List`

`ListItem`

`Pre`

`Quote`

`Text`