Fosdem 2017 Nightly slides and video

elias wrote on March 1, 2017 at 6:22 am:

Free or Hyper-Free? That is the question.
(an alternative style which centered in writing clearer more correct code instead of improving by “bug hunting”)

Being Mozilla one of the most successful free software projects.

Please allow me to criticize what I see as one of the main obstacles in collaborative software development in general.

Mozilla has succeed, but that is not the case with many free software projects, which have been abandoned or had not attracted enough volunteers to maintain those programs.
Mozilla may also benefit of this ideas, which are not new and nor mine, about what programming is.

A program is a kind of mathematical proof. It is a sequence of instructions which transforms data to the intelligible state which is what we expect to accomplish the statement of the problem.
Each instruction may be seen as an axiom and their composition as the program.

E.W.Dijstra and C.A.R Hoare, developed axiomatic systems to derive programs from specifications.
That stuff is not very popular, some is too formal and hard to grasp for the majority of autodidact programmers, those who have learned as hackers (studying a lot of code written by others.)

There is, however, a LESS RIGID AND MORE INTUITIVE way to develop programs from specifications stating WHAT a program should do and WHY and HOW a program works. It is called LITERATE PROGRAMMING, it is a great idea that we owe to D.E.Knuth.

It is a common place to say that “the first thing to do to solve a problem is to understand the problem”

How much can one understand about the problem and the way it was attacked to solve it from the code?

On the average, it is very poorly documented and a great effort is needed to decipher what the program does. More over and worse, the programmer should a previous knowledge about the domain of the problem that the program is about, like communication protocols, graphics formats, etc. Also, the documentation of the program is commonly written after the program is written and almost always it is never synchronized.

The idea of LITERATE PROGRAMMING is to explain all about WHAT the program does, the domain of the problem, and explain HOW and WHY a piece of code solves the stated problem.
In a collaborative project, the programmer may explain what he leaved for a later version or how the program could be improved later after developing other modules.

A literate program, looks like a good physics book, where a phenomenon is explained in natural language, then the forces involved are translated into mathematical formulas which may be transformed with algebraic equivalences to isolate the unknown variable.

In a literate program, the domain knowledge is analogous to the description of the physical phenomenon, a first sketch of the program is a translation of the problem and the last equation to an optimized code,

In a literate program, every interested programmer can learn about the subject.

Instead of a succinct phrase telling something like this program computes X, it explains WHAT is X then why X is represented with which data structures and WHY also what precautions like dealing with endianness, garbage collection or what ever problems may emerge when porting.
Then the explanation of the computational method employed to transform the data, and how it was refined to improve performance, instead of an obscure programming trick.

Clarity of the program and a synchronous documentation are the practical advantages of this style. But CORRECTNESS is also achieved due to the introspection gained when explaining the statement of WHAT the program does.
It may or may not be as rigid as using formal axiomatic systems., but good results may be obtained in an intuitive relaxed way.

Why I called it Hyper-Free?

Because such clarity in programs invites volunteers to participate, they can enjoy learning about new subjects at the same time that they help to improve and extend the programs.

Also the literate programs may be written in LaTeX, with references to functions in other modules which are automatically translated to hyper-links and the source code of a whole program can be read as a book in a pdf file with hyper-links and cross reference indexes (for the printed version) all automatically generated. (any other markup text may work, but LaTeX is ideal)

Although programming errors will arise for many causes, the introspection needed to explain the domain of the problem, makes the maintaining of the system more °implementation refinement oriented”, than a “bug hunting” task.

The reason reason to invite you to adopt literate programming here, is due to your indubitable passion for free software which could to bring more joy to your practice, and the positive impact in other free software projects if you adopt this style. It could be invaluable for other collaborative projects in an Hyper-Free software and world!

Reply

elias wrote on March 1, 2017 at 10:20 am:

A minimalist and silly example of a literate program:
———-
/*

\documentclass{article}

\title{A silly literate program example}
\begin{document}
Given a positive integer number $n$ this function compute:
\[s(n)=\sum_{i=1}^{i=n} i\]
This program can be implemented as follows

\begin{verbatim}

int s(int n)
{ int i, acum=0;

for(i=0;i<=1;i++)
acum = acum + 1;

return acum;
}

\end{verbatim}

the previous version take $n$ steps to compute the series, but C.F.Gauss observed that taking the sequence:
\[1,2,\ldots,n-1,n\]
and the sequence
\[n,n-1,\ldots,2,1\]
which is the same with it's members in reverse order,
we can generate a new sequence adding member by member
\[1+n,2+(n-1),\ldots,(n-1)+2,n+1\]
which is an $n$ length sequence with all members equal to $n+1$
\[\sum_{i=1}^{i=n}i+[(n+1)-i]=\sum_{i=1}^{i=n}n+1=(n+1)\sum_{i=1}^{i=n}i=(n+1)n=2\times s(n)\]
then
\[s(n)=\frac{(n+1)n}{2}\]

which computes the series in one step and can be implemented as follows:
\begin{verbatim}
*/
int s(int n){ return ((n+1)*n)/2; }
/*
\end{verbatim}

this program is limited by the size of an int in the implememtation, it is desirable for a new implementation to define a data type Integer to represent any integer of arbitrary precision portable to any patform

\end{document}

% */
——–
this minimalist simple example can be compiled in LaTeX adding a comment % to the first line, or included in a C program, like
————–
#include
#include literateS.c

int main()
{
printf(“the s(5)=%d\n”,s(5));
return 0;
}
————–

There are several programs like web, cweb, and some LaTeX packages which process a single soource to produce a LaTeX source, and a program souce for different programming languages.
Those programs allow us to write a more embellished typesetting and modular structure.
I can’t a more elaborated example using cweb or similar programs special for literate programming, because I wrote some simple LaTeX macros, practical for the simple programs that I write.
But you can find, several sites dedicated to literate programming in the web.

Another example may describe for example, how images can be represented with RGB pixels and which data structure can be used, taking into account platform dependent issues like endianness and describe methods to transform to other representation like YUV420, and so on.

This is maybe enough to invite to use a Literate style.
Search “literate programming” in the web for more examples and tutorials.

Reply

1. elias wrote on March 20, 2017 at 8:30 am:

   Errata:
   the row:
   \[\sum_{i=1}^{i=n}i+[(n+1)-i]=\sum_{i=1}^{i=n}n+1=(n+1)\sum_{i=1}^{i=n}i=(n+1)n=2\times s(n)\]
   is wrong should say:

   \[2\times s(n)=\sum_{i=1}^{n}(n+1)=(n+1)\sum_{i=1}^{n}1=(n+1)n\]

   which simply says the sum of n times (n+1) is (n+1)n some expressions between angle brackets were taken as html markup and hidden for that reason.

   I forgot to mention that MINIX the inspiration of Linus Torvalds, was developed in the famous OS book by A.S.Tanembaum, explaining each piece of code.
   That OS book presents MINIX as a literate program.

   Reply

   1. elias wrote on March 22, 2017 at 10:12 am:

      Pascal, writing more documented programs imply more analysis and reflection producing code with less errors. In this style the program is documented at the time of coding.
      This style could be as successful as Rust, because both promote a more disciplined programming.
      Documenting what is incomplete at the time of programming may help to fix errors quicker.
      Other example is the Operating Systems book by A.S.Tanembaum, developing Minix.
      Feel free if the program does not display correctly. Or you judge that this comment has nothing to do with your post.

      Reply