Ma_Sys.ma D5Man 2

WARNING: EXPERIMENTAL CODE

THIS IS HIGHLY EXPERIMENTAL AND UNDER DEVELOPMENT

Overview

Ma_Sys.ma D5Man 2 is a set of programs and auxiliary resources intended to build a locally run knowledge base. It consists of the following individual components:

d5manapi
Application Programming Interface providing access to page metadata by means of a REST (Representional State Transfer) interface. See d5manapi for details.
d5mantui
Terminal User Interface as an interactive means to query the D5Man API. Currently, it provides a keyword-based search (only). See d5mantui for details.
d5manexport
Multiple programs to export a special instance of pandoc’s Markdown by invoking the pandoc utility (not part of D5Man). Two export scripts are supplied: d5manexportpdf exports a single page to a printable PDF document; d5manexporthtml exports a whole directory structure of D5Man pages to a specified directory.
IAL (Information and Links)
The D5Man API allows for integration of pages which are not in D5Man’s Pandoc Markdown format but e.g. HTML by specifying separate files with D5Man-style metadta for such existing files. See Information and Links (IAL) for details.
Auxiliary Resources
The package provides a syntax file markdown.md as a replacement for VIM’s default Markdown style. This style makes sections more visible and is loosely inspired by the style used in D5Man Legacy.

This version “D5Man 2” replaces D5Man Legacy which was an attempt to achieve a similar outcome but in a much more sophisticated manner leading to excessive complexity.

D5Man Structure and Concepts

D5Man is centered around the concept of D5Man pages: A D5Man page is a text file written in a subset of pandoc’s Markdown together with a minimum amount of metadata in YAML format and with optional attachment files. The details of the format are described under D5Man Format 2.

Filesystem Structure

Two ways of organizing D5Man pages on file systems are recommended:

  1. Document-Root: A directory structure which consists only of D5Man pages (or other data processable by the D5Man API Server). On the top-level there is one directory per section and the respective sections contain one ore multiple D5Man documents each.
  2. Program-Root: A directory structure which consists of programs’ repositories. Here, pages are represented by the files named README.md in the subdirectories of the root.

A third option is detached, that is a directory unknown to the D5Man programs which contains one or more pages. Such files are not found by the D5Man API Server but can still be converted to PDF.

TODO ADD SOME EXAMPLES

Concept

D5Man is designed as a sort of personal Wiki that can have a published part. To achieve the separation between published and private parts, different sections are used. Each page is assigned a section, which by convention is a number of two digits (i.e. ranges from 10 to 99).

A typical workflow for creating a page is as follows:

  1. Create an empty page by copying from a template file to a file in a section directory or repository’s README.md TODO PROVIDE SUCH TEMPLATE FILE
  2. Populate the file with information.
  3. Optionally: Export the file to target format of choice and print it or upload it to the Internet.

To edit or recall a page, enter a prefix of the page’s name in d5mantui and once it appears in the search results, open it by pressing ENTER.

Benefits

Compared to other personal Wiki approaches, D5Man provides the following set of advantages:

Alternatives

TODO PROVIDE A LIST OF ALTERNATIVE SOFTWARE WHICH SERVES SIMILAR PURPOSES

D5Man Format 2

If you are interested in the old D5Man Legacy format, see d5man/legacy(32). Here, a selected subset of the elements from the rich syntax described in the Pandoc documentation is presented in order to obtain a sensible subset. Of course, there is no technical restriction for sticking to this subset.

D5Man’s text format is expected to always be read and written in UTF-8 encoding.

Metadata

A D5Man 2 Document begins with a header of following form:

---
section: 32
x-masysma-name: d5man2
title: Ma_Sys.ma D5Man 2
lang: en-US
author: ["Linux-Fan, Ma_Sys.ma (Ma_Sys.ma@web.de)"]
keywords: ["d5man", "d5man2", "d5manapi", "d5mantui", "ial"]
---

This header which follows YAML syntax is called the metadata in D5Man. It is a set of key-value assignments of form key: value. All fields which do not have any special meaning for Pandoc are prefixed x-masysma- as to indicate that they are additional fields used for D5Man. The use of the fields is as follows:

section (required)
Defines the section this page is part of. It is not really used in its Pandoc meaning (which would be the section for an actual manpage exported from the document), but the concept of D5Man manpages and actual manpages is similar to some extent (both provide textual information).
title (required)
Defines a document title (in legacy D5Man called description) which is a single large heading to go above the document.
lang (optional)
Gives the language in which the document (and/or its metadata) are written.
author (optional)
Gives a list of authors. Syntax ["Linux-Fan, Ma_Sys.ma..."] creates a YAML list with just one element in the example above.
keywords (required)
A list of keywords (also in YAML syntax) to assign to the page. Note that D5Man API search querys consider only x-masysma-name, section and keywords and matches case-sensitively against prefixes. It is thus often useful to provide sensible subsets of the page’s name in the keywords section. In legacy D5Man, this was called tags.
date (optional)
Specifies the date of document creation in YYYY/MM/DD HH:ii:ss format.
toc (optional)
Controls the generation of a table of contents for PDF exports (processed by pandoc only).
x-masysma-name (required)
Determines the page’s name. For newly created pages, it is recommended to chose names satisfying the regex [a-z0-9_/]+. Other names are supported, but may not contain any whitespace or other characters that are uncommon in file names processed by scripts (except for /). For Document-Root organizazion, the file name should be the page’s name with / replaced by _ and an additional .md suffix.
x-masysma-version, x-masysma-copyright (optional)
Specifies a version and copyright for the document (and the program it is describing). Fromat and use of these fields are entirely up to the user.
x-masysma-repository (optional)
Provides a link to the source code repository associated with the document and/or the software it describes.
x-masysma-website (optional)
Provides a link to the respective page on the (Ma_Sys.ma) Website. This allows e.g. Github users to find the Website which provides a correctly exported (i.e. readable) version of the distorted view that Github creates out of D5Man’s Markdown files.
x-masysma-owned (optional)
If present, this enables the inclusion of Ma_Sys.ma Logo and Icons in exported PDF files. Of course, the logos can also be replaced by different ones for local usage. Or one can leave out this key to avoid the use of logos in the export results altogether.
x-masysma-redirect (optional)
This field either gives an absolute URL (https://...) or a file name. In case a file name is given, the given file (relative to the attachment directory) is opened instead of opening the page when running from D5Man TUI. All pages available through IAL need to supply this field.
x-masysma-download (optional)
Specify an URL for downloading a file (used for Website generation).
x-masysma-web-priority, x-masysma-web-changefreq (optional)
Defines a priority (0.0–1.0) and a change frequency (monthly, weekly etc.) to be used in sitemaps generated during the XHTML export. Default is priority=0.4, changefreq=monthly.
x-masysma-expires (optional)
Expiry date. Same format as date. The meaning of this field is up to the user’s interpretation interpretation.

Section Structure

The section structure used by the Ma_Sys.ma is as given in the following table.

Sec Short Description
11 Documentation in the style of a classical man-page.
21 IAL as generated from documentation
22 IAL hand-added files
23 IAL internal
31 Website pages providing general website content (navigation, license, …).
32 Documentation for current Ma_Sys.ma developments (programs, scripts, etc.)
33 Legacy (TODO will this section actually be used?)
34 Creative section with Mods and Stories
35 not public: UNI notes
37 Blog, Knowledge Base, self-contained pages, other public notes
42 not public: user notes

Attachments

By convention, images included in the document are stored in a directory called the same as the page’s name with (/ replaced by _) and a suffix _att (instead of .md).

For instance, this README.md has name d5man2 thus the attachments would be stored in a directory d5man2_att next to the file. For page d5man/legacy, attachments go to d5man_legacy_att etc.

Additionally, images which are supplied in vector formats (SVG or PDF) are included by their file name without extension. This allows the LaTeX export to use a PDF file and the XHTML export to use a SVG file without changing the source file. Finally, D5Man’s XHTML export also instantiates a simple automatic conversion from PDF to SVG in order to avoid that redundant vector images need to be stored in the attachments directory.

Unlike legacy D5Man, an explicit list of all files attached is no longer needed to be declared in the documents themselves.

Top-Level Structure

Documents consist of the leading metadata block (see Metadata) followed by a D5Man document which consists of headings, lists, tables, code and paragraphs.

Headings

D5Man proposes three levels of headings. The top-level headings are underlined by equals signs. The second-level headings are prefixed by ## (hash-hash-space). The third-level headings are prefixed by ### (hash-hash-hash-space). The following code shows all the heading styles.

Top-Level Heading
=================

Top-Level (e.g. introductory) content.

## Second-Level Heading

Second-Level Content

### Third-Level Heading aka. List Title

Inner Content / End of example.

Lists

D5Man has numbered, unnumbered and definition lists. Legacy D5Man also proposed pro and contra-style lists which are as of now not retained in D5Man 2. Unnumberd list items are prefixed by an asterisk (*), numbered lists are prefixed by the item’s number followed by a dot (1., 2., etc.) and description lists’ contents are prefixed by a : at the beginning of the first line of the description list’s content. Note that for description lists, the offset from the left has to be exactly four characters wide (: / colon-space-space-space on the first line; ` ` space-space-space-space on the second line onwards). Here are examples for the respective list types.

Description List
:   This is the term being described.
    This is the second line of the term being described.
Second Description List Item
:   Another item to be described.

 1. First Item of a numbered list:
    This item has an additional line in source code.
 2. Second
 3. Third
 4. Fourth

 * Item 1 of the unnumberd list has
   two lines in source code.
 * Item 2 has a single line.
    * Nested Item a
    * Nested Item b
 * Item 3

Tables

TODO ASTAT

Code

Paragraphs and Inline Formatting

Paragraphs are just regular text separated by two newlines. Throughout the document’s text, it is possible to use inline formatting to place emphasis, links etc. It is described in the following.

Code
By using backtick-quotation, inline code can be expressed (`code` displays code). Escaping backticks inline requires them to be sourrounded by more backticks and space. See stackoverflow.com/82718 for details.
Emphasis
Like the legacy D5Man format, Markdown supports emphasis by surrounding the text to be emphasized with underscores e.g. _emphasized_ yields emphasized.
Superset and subset
Putting something in an index works by adding tilde symbols (~) around the part to be lowered, e.g. H~2~O for H2O. Elevating parts of a word is possible by surrounding it with hat symbols (^) e.g. 10^2^ for 102
Links
Links to URLs or other pages are of format [shortcut name](URL) e.g. [Example Page](http://www.example.com/) gives Example Page. If a link is given by URL only, it is given in angled-brackets like this: <http://www.example.com> gives http://www.example.com. To link to another D5Man page, its XHTML name needs to be given: [d5man/legacy(32)](d5man_legacy.xhtml) gives d5man/legacy(32). By convention, the link to another page is labelled by that page’s name followed by its section in parentheses. To link to pages in other sections, one needs to prefix ../SECTION to the link’s target due to the D5Man directory structure being organized in sections (even if it was originally a Program-Root structure, D5Man export always generates files as if they were part of a Document-Root structure). Note that unlike in legacy D5Man, links are expected to only work for the XHTML export. Navigating the hypertext directly inside the editor is no longer a supported use case.
Math
Inline Math is only supported for the PDF exports and can be expressed by LaTeX’ single-dollar notation, e.g. $\binom{1}{1}$ becomes (11)\binom{1}{1}.

For qotation and symbols, legacy D5Man used some automatic replacement logic. With the new version, this feature is no longer available, thus the use of UTF-8 symbols is suggested. On some Linux systems, quotation is easily available by [ALTGR]-[V] (), [ALTGR]-[B] () and [ALTGR]-[N] ().

Forced spaces (aka. non-breaking spaces) can be inserted by using the respective unicode symbols. As described by Thomas Keklak, a single non-breaking space can be entered in VIM by pressing [CTRL]-[K] [SPACE] [SPACE]. Similarly, a forced half-space can be entered by using the sequence [CTRL]-[V] [U] [2] [0] [2] [F].

TODO ARROWS

Images

The general syntax for images is ![CAPTION](FILE). By convention, FILE is given relative to the page’s file and if it is associated directly to the page, then it is placed in a directory with the page’s name concatenated with a trailing _att.

For instance, an attachment test.png for this very page would be loaded by specifying ![Test](d5man2_att/test.png).

Note that for .svg and .pdf files the extension of the image file name is normally not given in order to allow an automatic detection by pandoc/LaTeX to take place.

Compiling and Installing D5Man 2

d5manapi

d5mantui

d5manexportpdf

d5manexporthtml

USAGE d5manexport -o DESTDIR -i ROOT[,ROOT...] -s SECTION[,SECTION...] -u URLPREFIX [-m PDF2SVG] [-- PANDOCOPTIONS...]

TODO Needs to copy resource directories


Ma_Sys.ma Website 5 (1.0.0) – no Flash, no JavaScript, no Webfont, no Copy Protection, no Mobile First. No bullshit. No GUI needed. Works with any browser.

Created: 2019/12/06 13:16:32 | Revised: 2020/02/17 23:58:50 | Tags: d5man, d5man2, d5manapi, d5mantui, ial | Version: 2.0.0 | SRC (Pandoc MD) | GPL

Copyright (c) 2019, 2020 Ma_Sys.ma. For further info send an e-mail to Ma_Sys.ma@web.de.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.