BoF @ ECE 2016: Eclipse and AsciiDoc

At NumberFour, we use AsciiDoc for our specifications and documentation. We are even working on producing AsciiDoc API documentation from JSDoc-like comments (similar to JavaDoc) for our JavaScript language N4JS. Since we ran into a couple of problems with AsciiDoctor, I thought that it might be interesting to learn what other people in the Eclipse Community think about AsciiDoc(tor) and if we might be able to "join forces" to overcome certain obstacles. So I organized a "Birds of Feather" about "Documentation with AsciiDoc and Eclipse" at EclipseCon 2016 Europe. We were about 10 participants and this blog post serves more or less as a kind of meeting minutes of this BoF combined with our findings at NumberFour.

AsciiDoc is a lightweight markup language, similar to Markdown. The original AsciiDoc was written in Python, but Asciidoctor, probably the most popular tool nowadays, is written in Ruby with a variant AsciidoctorJ which provides Java support (via JRuby) and Asciidoctor.js for JavaScript (via Opal). From an Asciidoc document (.adoc) you can either create HTML, PDF or even EPub. Unfortunately, the PDF support is lacking some major features (e.g., footnotes are not supported yet), but there is a common workaround to produce DocBook XML files from AsciiDoctor and then use FOP to eventually create a beautiful PDF. Lars Vogel wrote a nice tutorial about AsciiDoc and AsciiDoctor. One of the great things about AsciiDoc is that GitHub provides basic support for AsciiDoc as well!

Writing AsciiDoc

The first question is how to write AsciiDoc. One solution is to use stand-alone editors such as Sublime (there is an AsciiDoc plug-in for syntax highlighting and coce completion which, together with the preview plug-in, is a really great solution).

But usually, you do not want to switch editors when you are already using Eclipse. Fortunately, there's an existing AsciiDoc editor which is part of Mylyn Wiki Text. This editor has some syntax highlighting, an outline view and even a preview.

This editor seems to be a good starting point for further support of AsciiDoc on the Eclipse platform, although it might have some shortcomings with regard to features and design. Regardless, the BoF participants (including Torkild, a Mylyn Docs committer) decided to use the mylyn-docs developers mailing list for further communication about AsciiDoc and Eclipse.

Apparently, having a good editor, possibly even with WYSIWYG or at least WYSIWYM support is one of the most important things required.

Building AsciiDoc

There are several ways of building AsciiDoc. The most simple one is about using the AsciiDoctor tools directly from the command line. There is also a Maven plugin provided by the AsciiDoctor project.

We haven't discussed that topic any further though. It seems as if the existing tools are sufficient even though there are some shortcomings when using custom macros as described below. You may have a look at the N4JS documentation pom.xml to get an idea of how this could look like.

Converting to AsciiDoc

During the BoF, the question came up how to convert existing documentation written in other formats to Asciidoc. One solution for that is Pandoc, which we (at NumberFour) are using to convert our LaTeX specification to Asciidoc. Jeremie Bresson has already blogged about the BoF and even provided a solution for using Mylyn Wikitext to convert Eclipse MediaWiki to Asciidoc. Thank you very much, Jeremie!

Using AsciiDoc as a Single Source

I already mentioned above that you can generate HTML and PDF from AsciiDoc. However, HTML is not HTML. We use AsciiDoc to generate

• the public web site (i.e., gh-pages)
• the Eclipse help
• the PDF specification

from a single asciidoc file. To give you an idea, have a look at an adoc source file and the generated web page. If you download the N4JS IDE, you can also have a look at the generated Eclipse help containing the same content. The PDF is not publicly available yet, we are currently working on migrating our language specification from LaTeX to AsciiDoc.

In order to generate Eclipse help, a table of contents file is required. At the moment, we generate them via the geneclipsetoc-maven-plugin Maven plugin, see our pom.xml for our configuration.

You may also use the Mylyn Docs to generate code, which is described in the Mylyn Docs Eclipse help. This also provides support for generating EPub, the first steps towards this support has already been described 2011 in Torkild's blog! It’s now a stable part of Mylyn Docs and has been for a few years. Currently it is in maintenance mode as there are few bugs and there is not much to add – apart for EPUB 3.0 support. You can find an EPUB examples (and presentations about Mylyn Docs in general) on Torkild's GitHub page.

Customizing AsciiDoc

Although AsciiDoc provides a lot of features ("macros" in Asciidoc terminology) for technical documentation, such as source blocks, warning blocks etc., we (at NumberFour) missed some functionality. We required additional support for

1. definitions -- a typical feature required in specifications
2. BibTex cites and bibliographies -- we already have a large BibTex database which we want to reuse
3. todos -- well, also a typical feature required in specifications and documentations
4. links to source code -- e.g., for adding links to either GitHub, or, in case of Eclipse help, to the files in the workspace
5. larger documents, e.g., includes similar to LaTeX's chapter folders

One problem we found is that Ruby-based custom macros calling native code cannot be used with AscidoctorJ. E.g., there already exists a custom Ruby macro which provides support for BibTex, but this does not work with AscidoctorJ. We want to write the macros in Java, since we plan to provide some editor support in Eclipse as well -- and then Java would be the better solution. Thus we already wrote our own Java-based, BibTex macro (fixing some smaller AsciidoctorJ problems on the way) and we are also working on solutions for the other topics. If you are interested in that, or if you have written Java-based macros for AsciiDoc as well -- let me know! We haven't open sourced our custom macros yet since they are still under (heavy) development. But we will probably do that once they are in a state to make them public.

Edit: Updated description of EPUB support -- thank you Torkild for the information!

Note: This blog post was initially posted at jevopi's developer blog (which is listed on Planet Eclipse). Since it is highly related to the N4JS project, we re-posted it here again.

Test-Driven Development with N4JS

The N4JS IDE validates and checks code as you type, making it a powerful tool for writing type safe applications that that are predictable, reliable and easily maintainable over time. This means that problem-solving takes place before code is deployed and your applications are safely exported.

As test-driven development is a key technique for the development of large-scale projects, the N4JS IDE is bundled with a robust test framework called Mangelhaft. Mangelhaft is an xUnit-like framework with an engine for executing tests and will report results to the IDE while running test suites.

Testing with Mangelhaft in the N4JS IDE 

While Node.js projects written with N4JS can easily be checked and tested with Mangelhaft within the IDE, the release of Mangelhaft on npm means that your Node.js projects will stay stable when deployed on a Continuous Integration server.

Using Mangelhaft with Travis CI

Using the example project bundled with the N4JS IDE, we can demonstrate how to use Mangelhaft with a Travis build. In the Travis .yml file, we define the language of the build as node_js. Currently, Java8 must also be installed as a requirement for the build:

    sudo: required
    language: node_js
    before_install:
      - sudo apt-get -qq update
      - sudo apt-get install -y oracle-java8-set-default
    node_js:
      - "5"
    services:
      - mongodb
  

Mangelhaft can either be installed globally ( npm install -g n4js-node-mangelhaft ) or integrated with a build by including Mangelhaft in the devDependencies of a project and exporting as an npm package. Mangelhaft will be downloaded and installed automatically when the Travis build begins:

Installing Mangelhaft on Travis CI Server

Default test scripts described in the package.json can then be run with the npm test command which calls n4js-mangelhaft.

Test results in Travis CI

A full list of available options can be listed with the command n4js-mangelhaft -h

Using Mangelhaft with Jenkins

Mangelhaft can also be integrated with a Jenkins job and fully supports detailed xUnit test results. View the status of your tests over time with the Jenkins UI and benefit from individual test reports:

n4js-mangelhaft \
      …
      --xunitReportFile ./build/tasks-test-report.xml \
      --xunitReportName tasks-test-report \
      --xunitReportPackage n4js.example.tasks.tests \
      …
  

Mangelhaft Test Results in Jenkins UI 

Mangelhaft Test Results in Jenkins UI

Generate comprehensive xUnit test data as frequently as required and drill down to the root of any issues that arise:

Individual Test Results in a Jenkins Build

Type Safe Workflow with Mangelhaft

Running npm tests with Mangelhaft ensures Node.js projects created with N4JS stay stable with errors detected and located quickly and easily. Download the N4JS IDE and begin creating type safe Node.js projects with the peace of mind of CI tests made possible with Mangelhaft.

Resources referred to in this post

• GitHub - N4JS Example Tasks
• Travis CI Build - N4JS Example Tasks
• n4js-node-mangelhaft on npm

N4JS Goes Open Source

Today NumberFour AG announced "N4JS,  An Open-Source Language and IDE for high

quality JavaScript development".

N4JS is built for JavaScript developers who seek to build large-scale and maintainable server-side JavaScript as well as Java developers who want to use JavaScript with familiar tool support, early validation and other Java benefits. It bridges the strengths of JavaScript and Java; the result is a typed JavaScript superset that is dynamic, flexible and type-safe.

N4JS Feature Highlights

The N4JS language and its IDE allow for high-quality JavaScript development of large Node.js projects. Here is a list of some of its features:

• super set of ECMAScript 5; most of ECMAScript 2015
• transpiles to V8-compatible ECMAScript (Node.js version 5)
• Java-like nominal types
• TypeScript-like structural types
• generics similar to Java 8
• async/await (TC39/ECMAScript proposal) with support for legacy code
• dependency injection using annotations similar to JSR330/Google Guice
• built-in support for testing, including xUnit-like test framework "mangelhaft"
• Node.js support: automatically download npms, export to npm

Visit the N4JS webpage to learn more about N4JS and download the latest N4JS IDE.

N4JS for Eclipse-Developers

You are a Eclipse plugin developer looking for a high-quality Java-based ECMAScript parser.
You develop DSLs wich are to be combined with JavaScript environments?

N4JS provides an Xtext-based ECMAScript 2015 parser. The created AST and type model are based on EMF, allowing for integration with other Eclipse modeling tools. Set-up is made easy due to provided Oomph set up script.The N4JS transpiler and the IDE are Eclipse-based.

Visit the N4JS GitHub page to access the source code of the N4JS IDE and the transpiler.

The N4JS Developer Blog

In this blog, the N4JS developer team at NumberFour will write about N4JS. Topics will include

• new features
• explanation of existing features
• discussion of language concepts
• description of design and implementation of the N4JS IDE and the transpiler

We are looking forward to your feedback! You can either leave comments, post to our mailing list, create issues, or, last but not least, create pull requests!