Metrics, new Macbook charger, testcontainers

Things I’m learning about, on this day of verbs:

  • What metrics matter: A guide for open source projects” has a lot of good information. It’s interesting.
  • I get to buy a new Macbook charger today; my “old” one is giving out, I think, which is frustrating.
  • Testcontainers looks like an interesting project. I may have to try it out. It’s enabling.
  • There is an Atlas of Endangered Alphabets. It’s fascinating.
  • Apparently January 18th is “Winnie the Pooh Day.” I had no idea. Even though we describe our cats in terms of Tiggerisms – and once had a cat named Tigger – it’s… vaguely confusing.
  • My exercise routine remains, but parts of it are missing this past week – namely, some of the short walks.

Art, JDBC, coffee, JSoup, Gradle, Hallelujah, Resetting

Things I may have learned recently-ish:

  • Painting pottery is fun. I’m not very good at it. I apparently also have a thing for beer steins… out of which I drink coffee, because beer isn’t very good in my opinion. But steins are also a LOT of coffee, so it’s impractical to drink coffee from steins.
  • Speaking of coffee, I’m sticking to black coffee straight up because of my diet/exercise regimen. I’m normally a sweet coffee kind of guy… this drinking of only bitter coffee is “interesting,” but not in the “most interesting man in the world” kind of way. Still trying to stick to it.
  • Virtuoso‘s JDBC driver is not especially reliable. And I really wish I’d kept a “total time elapsed” for this entire process. I didn’t even keep a “real time elapsed” – but I’m predicting it takes days.
  • I may have found a bug in JSoup. There’s a method that gets the representative text of a parent HTML node, and it’s removing one space character where I don’t think it’s supposed to. Will investigate further.
  • Had a discussion yesterday – well, sort of a discussion – with a programmer who was arguing against Kotlin, saying that it didn’t really do anything Java didn’t do. That’s a particularly reductionist argument; after all, Java doesn’t do anything C doesn’t do (Java is written in C when you go deep enough) and C doesn’t do anything machine code doesn’t do. Why doesn’t he just use machine code? (Or he could use Kotlin, which is remarkably expressive and saves a ton of time.) Of course, he also was arguing that non-nullable types were a waste of time – and how great Python was, so maybe his priorities were skewed way differently than mine.
  • Two friends may have finally seen an end to their job hunts, which is a good thing if true.
  • Gradle is nice, but I still prefer Maven for most things. Yesterday I considered getting module interdependencies working as a “milestone.” That’s a crappy milestone to have. This stuff should just work.
  • Leonard Cohen’s “Hallelujah” – particularly the original release – is a great song. As a Christmas song, I’m not … sold (what the heck?) but as a song about perfect expressions from an imperfect source… the song’s recording even echoes its own form, in an awesome fractal. He’s pitchy, off the beat, all kinds of things… and yet I dare anyone listening to it to not be thinking “… Hallelujah!” in a sympathetic echo in their souls.
  • I first heard “Hallelujah,” as far as I know, on American Idol.
  • I altered a core setting (the audio in/output mix) on my A/D device (which I use to record audio) a week or so ago – and yesterday, when I went to use it, couldn’t get any sound out of it. Memo to self: when you change things, set them back! I’m usually a lot better about this, and don’t know why I got it wrong – maybe someone interrupted me so I didn’t fix the hardware before getting up? I don’t know.

JNDI

Things I have learned recently:

  • People still don’t really get JNDI, and the Java frameworks around today make it easy to ignore, even though it’s still a core technology. It’s not difficult to see how it can be confusing: context in JNDI is everything, and context makes it a challenge to create examples that make sense in the general case.
  • At some point I’d like to learn Go.
  • Not something I’ve learned, but something I’ve been reflecting on this morning because … uh… I have no idea why: I wonder if Adidas shoes are any good, or what they’re good for. I tend to wear Vans Ultrarange shoes these days because they’re light, comfortable, and last forever – I have two working pairs, one for working in the yard and one for wearing – but… Adidas.
  • I really wish officials and announcers wouldn’t show bias during football games. As an FSU guy, I’m really, really, really tired of this – but I’ve been watching other teams’ bowl games (because FSU didn’t go bowling this year, first time in 40+ years) and it happens for them, too, often egregiously. The announcers I don’t care as much about, but the referees… those guys need to be fair, for real. The fact that there’s no urgency in making sure they’re fair is incredibly frustrating and erodes the game. n one game, a team had two defenders ejected for targeting… and the other team had an obvious false start missed, and a few targeting possibilities ignored by the guys in stripes. Let’s just say nope to all that. There needs to be a way for the league to tell these refs what they’re missing, and to either call it fairly or get out. It’s gotten really bad over the last few years, with FSU losing multiple games due to bad or missed calls.

Blogs are so eh

Things I think I’ve learned today:

  • Blogs are so yesterday, man. It’s probably the medium and platform I’ve chosen – I don’t use Medium, for example, although I have an account there – and I don’t publish often enough, or with enough direct focus, to really attract users, because I’m really not trying to build a popular platform. But at the same time, my pride is hurt just a little that traffic is so low.
  • Crawler4j makes it really easy to write a working web spider in Java – or, in my case, Kotlin. Looked into scrapy for a bit, but couldn’t figure out why what should have been a trivial recipe was so hard to find, switched to Kotlin because I really just needed to make progress. Progress was made. Scrapy’s probably fine – maybe the information I was looking for was out there, right in front of my face, and I couldn’t recognize it, but that didn’t help me make the progress I needed.
  • Had a good example of pragmatism in action show up yesterday, too. We issued a pull request for a migration, and one of our developers pointed out a number of situations in the new codebase that could have been problematic. I asked him to demonstrate the errors (or potential errors) with tests so we could validate them, but he wasn’t sure how much effort would be involved in writing those tests… so we progressed with the merge. I think we made the right choice; he’s not wrong in his observations (we might have introduced errors with the new changes) but without validation, we can’t know, and we’d be chasing ghosts. We made notes that the code might be problematic, and we’re going to watch for problems that come from it.
  • Aaron’s Thinking Putty is cooler than it should be.

A New Maven Archetype for Starter Project

I’ve recently put together a new Maven archetype, based on something I saw in Freenode’s ##java channel a few weeks ago. Basically, someone had built their own archetype for “standard projects,” with a few sensible dependencies and defaults, and while I thought it was a worthwhile effort, it didn’t fit what I found myself typically doing.

So I built my own, at https://github.com/jottinger/starter-archetype.

Primary features are:

  • Java 8 as a default Java version
  • Typical dependencies
  • Maven Shade

Java 8

Look, Java 7 and older has been end-of-lifed; you can pay Oracle for support and fixes, but you shouldn’t unless you have a real reason to do so. Java 8 is the current Java version. You should be using it, and so should I. Therefore, I do.

It’s an unfortunate aspect of Maven that it defaults to an older version of the Java specification. My starter archetype presumes you want to live in the current year.

Typical Dependencies

My starter archetype has seven dependencies; they are the ones I either include without thinking about it (because I know I’m going to want or need them), or they’re the dependencies that are so common that few projects would blink at their inclusion.

The dependencies are organized into three groups: runtime dependencies, one compile-time dependency, and testing dependencies.

The compile-time dependency is Lombok; it helps remove boilerplate from Java code, so I can build a Java object with mutators, accessors, toString(), hashCode(), and equals() very simply:

@Data
public class Thing {
    // public String getName() and setString(String name) are built for me through Lombok
    String name;
}

The runtime dependencies are Guava, Logback, and Apache’s commons-lang3. Guava and commons-lang3 have a lot of overlap, but both are very common; logback is a logging library that leverages slf4j, so it’s a workable default logging library that doesn’t force you to stick with it if you don’t like it.

All together, they use up roughly 3.5MB of disk space for a starting classpath, with Guava using 2.3MB of it. Given how useful Guava, et al, are, I think this is entirely worthwhile; most projects will have these libraries (or something nearly like them), so it’s acceptable.

The testing libraries are TestNG, assertj, and H2.

It’s arguable that JUnit 5 might have caught up to TestNG in a lot of ways, but there are still some features I really like from TestNG that JUnit doesn’t have built-in support for (namely, data providers – although note that there is a project that provides data provider support for JUnit, unsurprisingly called junit-dataprovider).

AssertJ is a set of fluent assertions for Java. It’s not necessary – for example, I’ve used TestNG’s innate assertions for years without a problem – but the fluent assertion style is rather nice. The actual dependency is assertj-guava – which includes the base library for assertj – but I chose assertj-guava because of the inclusion of Guava as a default runtime dependency.

H2 is an embedded database. I use embedded databases so much for first-level integration tests that it seemed silly not to include it; I have a lot of sandbox projects that don’t use H2, but as soon as I do anything with a database, this gets included, so it makes sense as a trivial “default testing library.”

Maven Shade

I wanted to be able to generate an executable jar by default, not because I do that very much, but because it seemed to be a sane default. (Usually, my starter projects exist to support a test that demonstrates a feature, as opposed to being an independently useful project.)

Because I wanted others to be able to see more use out of my starter project, I added Maven Shade to create a viable entry point and an executable jar.

Using the archetype

Right now, you’d need to run the following sequence at least once to use the archetype:

git clone https://github.com/jottinger/starter-archetype.git
cd starter-archetype
mvn install

Then, to build a project with the archetype, you’d run:

mvn archetype:generate \
    -DarchetypeGroupId=com.autumncode \
    -DarchetypeArtifactId=starter-archetype \
    -DarchetypeVersion=1.0-SNAPSHOT

Future Enhancements

It’s still very much a work in progress. Things I’d like to do:

  • Migrate into the main Maven repositories (publish, in other words)
  • Add publishing support to the archetype itself
  • Include better throwaway demonstrations of the dependencies (a difficult task, as the default classes are meant to be thrown away en masse)
  • Figure out better default libraries, if possible

You’re welcome to fork the project, create issues, comment, use with wild abandon, as you like. It’s licensed under the Apache Source License, 2.0.

A Simple Grappa Tutorial

Grappa is a parser library for Java. It’s a fork of Parboiled, which focuses more on Scala as a development environment; Grappa tries to feel more Java-like than Parboiled does.

Grappa’s similar in focus to other libraries like ANTLR and JavaCC; the main advantage to using something like Grappa instead of ANTLR is in the lack of a processing phase. With ANTLR and JavaCC, you have a grammar file, which then generates a lexer and a parser in Java source code. Then you compile that generated source to get your parser.

Grappa (and Parboiled) represent the grammar in actual source code, so there is no external phase; this makes programming with them feel faster. It’s certainly easier to integrate with tooling, since there is no separate tool to invoke apart from the compiler itself.

I’d like to walk through a simple experience of using Grappa, to perhaps help expose how Grappa works.

The Goal

What I want to do is mirror a tutorial I found for ANTLR, “ANTLR 4: using the lexer, parser and listener with example grammar.” It’s an okay tutorial, but the main thing I thought after reading was: “Hmm, ANTLR’s great, everyone uses it, but let’s see if there are alternatives.”

That led me to Parboiled, but some Parboiled users recommended Grappa for Java, so here we are.

That tutorial basically writes a parser for drink orders. We’ll do more.

Our Bartender

Imagine an automated bartender: “What’re ya havin?”

Well… let’s automate that bartender, such that he can parse responses like “A pint of beer.” We can imagine more variations on this, but we’re going to center on one, until we get near the end of the tutorial: we’d also like to allow our bartender to parse orders from people who’re a bit too inebriated to use the introductory article: “glass of wine” (no a) should also be acceptable.

If you’re interested, the code is on GitHub, in my grappaexample repository.

Let’s take a look at our bartender‘s source code, just to set the stage for our grammar. (Actually, we’ll be writing multiple grammars, because we want to take it in small pieces.)

package com.autumncode.bartender;

import com.github.fge.grappa.Grappa;
import com.github.fge.grappa.run.ListeningParseRunner;
import com.github.fge.grappa.run.ParsingResult;

import java.util.Scanner;

public class Bartender {
    public static void main(String[] args) {
        new Bartender().run();
    }

    public void run() {
        final Scanner scanner = new Scanner(System.in);
        boolean done = false;
        do {
            writePrompt();
            String order = scanner.nextLine();
            done = order == null || handleOrder(order);
        } while (!done);
    }

    private void writePrompt() {
        System.out.print("What're ya havin'? ");
        System.out.flush();
    }

    private boolean handleOrder(String order) {
        DrinkOrderParser parser
                = Grappa.createParser(DrinkOrderParser.class);
        ListeningParseRunner<DrinkOrder> runner
                = new ListeningParseRunner<>(parser.DRINKORDER());
        ParsingResult<DrinkOrder> result = runner.run(order);
        DrinkOrder drinkOrder;
        boolean done = false;
        if (result.isSuccess()) {
            drinkOrder = result.getTopStackValue();
            done = drinkOrder.isTerminal();
            if (!done) {
                System.out.printf("Here's your %s of %s. Please drink responsibly!%n",
                        drinkOrder.getVessel().toString().toLowerCase(),
                        drinkOrder.getDescription());
            }
        } else {
            System.out.println("I'm sorry, I don't understand. Try again?");
        }
        return done;
    }
}

This isn’t the world’s greatest command line application, but it serves to get the job done. We don’t have to worry about handleOrder yet – we’ll explain it as we go through generating a grammar.

What it does

Grappa describes a grammar as a set of Rules. A rule can describe a match or an action; both matches and actions return boolean values to indicate success. A rule has failed when processing sees false in its stream.

Let’s generate a very small parser for the sake of example. Our first parser (ArticleParser) is going to do nothing other than detect an article – a word like “a”, “an”, or “the.”

Actually, those are all of the articles in English – there are other forms of articles, but English has those three and no others as examples of articles.

The way you interact with a parser is pretty simple. The grammar itself can extend BaseParser<T>, where T represents the output from the parser; you can use Void to indicate that the parser doesn’t have any output internally.

Therefore, our ArticleParser‘s declaration will be:

public class ArticleParser extends BaseParser<Void> {

We need to add a Rule to our parser, so that we can define an entry point from which the parser should begin. As a first stab, we’ll create a Rule called article(), that tries to match one of our words. With a trie. It’s cute that way.

A trie is a type of radix tree. They tend to be super-fast at certain kinds of classifications. Note that this method name may change in later versions of Grappa, because honestly, the actual search mechanism – the trie – isn’t important for the purpose of invoking the method.

public class ArticleParser extends BaseParser<Void> {
    public Rule article() {
        return trieIgnoreCase("a", "an", "the");
    }
}

This rule should match any variant of “a”, “A”, “an”, “tHe”, or anything like that – while not matching any text that doesn’t somehow fit in as an article. Let’s write a test that demonstrates this, using TestNG so we can use data providers:

package com.autumncode.bartender;

import com.github.fge.grappa.Grappa;
import com.github.fge.grappa.run.ListeningParseRunner;
import com.github.fge.grappa.run.ParsingResult;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import static org.testng.Assert.assertEquals;

public class ArticleTest {
    @DataProvider
    Object[][] articleData() {
        return new Object[][]{
                {"a", true},
                {"an", true},
                {"the", true},
                {"me", false},
                {"THE", true},
                {" a", false},
                {"a ", true},
                {"afoo", true},
        };
    }

    @Test(dataProvider = "articleData")
    public void testOnlyArticle(String article, boolean status) {
        ArticleParser parser = Grappa.createParser(ArticleParser.class);
        testArticleGrammar(article, status, parser.article());
    }
    
    private void testArticleGrammar(String article, boolean status, Rule rule) {
        ListeningParseRunner<Void> runner
                = new ListeningParseRunner<>(rule);
        ParsingResult<Void> articleResult = runner.run(article);
        assertEquals(articleResult.isSuccess(), status,
                "failed check on " + article + ", parse result was "
                        + articleResult + " and expected " + status);
    }
}

So what is happening here?

First, we create a global (for the test) ArticleParser instance through Grappa. Then we create a ListeningParseRunner, with the entry point to the grammar as a parameter; this builds the internal model for the parser (stuff we don’t really care about, but it is memoized, so we can use that code over and over again without incurring the time it takes for processing the grammar at runtime.)

I used a utility method because the form of the tests themselves doesn’t change – only the inputs and the rules being applied. As we add more to our grammar, this will allow us to run similar tests with different inputs, results, and rules.

After we’ve constructed our Parser’s Runner, we do something completely surprising: we run it, with ParsingResult<Void> articleResult = runner.run(article);. Adding in the TestNG data provider, this means we’re calling our parser with every one of those articles as a test, and checking to see if the parser’s validity – shown by articleResult.isSuccess() – matches what we expect.

In most cases, it’s pretty straightforward, since we are indeed passing in valid articles. Where we’re not, the parser says that it’s not a successful parse, such as when we pass it me.

There are three cases where the result might be surprising: " a", "a ", and "afoo". The whitespace is significant, for the parser; for our test, the article with a trailing space passes validation, as does “afoo“, while the article with the leading space does not.

The leading space is easy: our parser doesn’t consume any whitespace, and Grappa assumes whitespace is significant unless told otherwise (by Rules, of course.) So that space doesn’t match our article; it fails to parse, because of that.

However, the trailing space (and "afoo") is a little more odd. What’s happening there is that Grappa is parsing as much of the input as is necessary to fulfill the grammar; once it’s finished doing that, it doesn’t care about anything that follows the grammar. So once it matches the initial text – the “a” – it doesn’t care what the rest of the content is. It’s not significant that "foo" follows the “a“; it matches the “a” and it’s done.

We can fix that, of course, by specifying a better rule – one that includes a terminal condition. That introduces a core concept for Grappa, the “sequence().” (This will factor very heavily into our grammar when we add the ability to say “please” at the end of the tutorial.)

Author’s note: I use “terminal” to mean “ending.” So a terminal anything is meant to indicate finality. However, a “terminal” is also used to describe something that doesn’t delegate to anything else, in Grappa’s terms. So for Grappa, the use of “terminal” might not be the same as my use of the word “terminal”.

Let’s expand our ArticleParser a little more. Now it looks like:

package com.autumncode.bartender;

import com.github.fge.grappa.parsers.BaseParser;
import com.github.fge.grappa.rules.Rule;

public class ArticleParser extends BaseParser<Void> {
    public Rule article() {
        return trieIgnoreCase("a", "an", "the");
    }

    public Rule articleTerminal() {
        return sequence(
                article(),
                EOI
        );
    }
}

What we’ve done is added a new Rule – articleTerminal() – which contains a sequence. That sequence is “an article” — which consumes “a,” “an”, or “the” – and then the special EOI rule, which stands for “end of input.” That means that our simple article grammar won’t consume leading or trailing spaces – the grammar will fail if any content exists besides our article.

We can show that with a new test:

@DataProvider
Object[][] articleTerminalData() {
    return new Object[][]{
         {"a", true},
         {"an", true},
         {"the", true},
         {"me", false},
         {"THE", true},
         {" a", false},
         {"a ", false},
         {"afoo", false},
    };
}

@Test(dataProvider = "articleTerminalData")
public void testArticleTerminal(String article, boolean status) {
    ArticleParser parser = Grappa.createParser(ArticleParser.class);
    testArticleGrammar(article, status, parser.articleTerminal());
}

Now our test performs as we’d expect: it matches the article, and only the article – as soon as it has a single article, it expects the end of input. If it doesn’t find that sequence exactly, it fails to match and isSuccess() returns false.

It’s not really very kind for us to not accept whitespace, though: we probably want to parse " a " as a valid article, but not " a the" or anything like that.

It shouldn’t be very surprising that we can use sequence() for that, too, along with a few new rules from Grappa itself. Here’s our Rule for articles with surrounding whitespace:

public Rule articleWithWhitespace() {
    return sequence(
            zeroOrMore(wsp()),
            article(),
            zeroOrMore(wsp()),
            EOI
    );
}

What we’ve done is added two extra parsing rules, around our article() rule: zeroOrMore(wsp()). The wsp() rule matches whitespace – spaces and tabs, for example. The zeroOrMore() rule seems faintly self-explanatory, but just in case: it says “this rule will match if zero or more of the contained rules match.”

Therefore, our new rule will match however much whitespace we have before an article, then the article, and then any whitespace after the article – but nothing else. That’s fun to say, I guess, but it’s a lot more fun to show:

 @DataProvider
 Object[][] articleWithWhitespaceData() {
     return new Object[][]{
             {"a", true},
             {"a      ", true},
             {"     the", true},
             {"me", false},
             {" THE ", true},
             {" a an the ", false},
             {"afoo", false},
     };
 }

 @Test(dataProvider = "articleWithWhitespaceData")
 public void testArticleWithWhitespace(String article, boolean status) {
     ArticleParser parser = Grappa.createParser(ArticleParser.class);
     testArticleGrammar(article, status, parser.articleWithWhitespace());
 }

Believe it or not, we’re actually most of the way to being able to build our full drink order parser – we need to figure out how to get data from our parser (hint: it’s related to that <Void> in the parser’s declaration), but that’s actually the greatest burden we have remaining.

One other thing that’s worth noting as we go: our code so far actually runs twenty-three tests. On my development platform, it takes 64 milliseconds to run all twenty-three – the first one takes 49, where it’s building the parser for the first time. The rest take somewhere between 0 and 4 milliseconds – and I’m pretty sure that 4ms reading is an outlier. Our grammar isn’t complex, and I imagine we could write something without a grammar that would be faster – maybe HashSet<String>.contains(input.trim()) – but we’re about to step into territory that would end up being a lot less maintainable as our grammar grows.

I ran the tests one hundred times each and the same pattern showed up: every now and then you’d see a test that ran slower. My initial guess is that this is related to garbage collection or some other housekeeping chore on the JVM’s part, but I haven’t verified it.)

Getting Data out of our Parser

Grappa uses an internal stack of values to track and expose information. We can tell it the type of the stack values – and in fact, we already did so in our ArticleParser. It’s the <Void> we used – that says that we have a stack of Void values, which is a cute way of saying “no value at all.” (If you remember carefully, we pointed that out when we first started describing the ArticleParser – this is where that information is useful!)

Therefore, all we need to do is expose a type, and then manipulate that stack of values. We do so with a special type of Rule, a function that returns a boolean that indicates whether the Rule was successful.

Our goal with the article is to parse drink orders, of the general form of “a VESSEL of DRINK.” We already worked on a parser that demonstrates parsing the “a” there – it’s time to think about parsing the next term, which we’ll call a “vessel.” Or, since we’re using Java, a Vessel – which we’ll encapsulate in an Enum so we can easily add Vessels.

The Vessel itself is pretty simple:

package com.autumncode.bartender;

public enum Vessel {
    PINT,
    BOWL,
    GLASS,
    CUP,
    PITCHER,
    MAGNUM,
    BOTTLE,
    SPOON
}

What we want to do is create a parser such that we can hand it “a glass” and get Vessel.GLASS out of it.

Given that we’ve said that a parser can be constructed with the “return type”, that tells us that our VesselParser wants to extend BaseParser<Vessel>, and so it does. In fact, our VesselParser isn’t even very surprising, given what we’ve learned from our ArticleParser:

public class VesselParser extends BaseParser<Vessel> {
    static final Collection<String> vessels = Stream
            .of(Vessel.values())
            .map(Enum::name)
            .collect(Collectors.toList());

    public Rule vessel() {
        return trieIgnoreCase(vessels);
    }
}

What does this do? Well, most of it is building a List of the Vessel values, by extracting the values from Vessel. It’s marked static final so it will only initialize that List once; the Rule (vessel()) simply uses the exact same technique we used in parsing articles. It doesn’t actually do anything with the match, though. It would simply fail if it was handed text that did not match a Vessel type.

Incidentally, the Java Language Specification suggests the order of static final, in section 8.3.1, Field Modifiers.

Let’s try it out, using the same sort of generalized pattern we saw in our ArticleParser tests. (We’re going to add a new generalized test method, when we add in the type that should be returned, but this will do for now.)

public class VesselTest {
    private void testGrammar(String corpus, boolean status, Rule rule) {
        ListeningParseRunner<Vessel> runner
                = new ListeningParseRunner<>(rule);
        ParsingResult<Vessel> result = runner.run(corpus);
        assertEquals(result.isSuccess(), status,
                "failed check on " + corpus + ", parse result was "
                        + result + " and expected " + status);
    }
    
    @DataProvider
    Object[][] simpleVesselParseData() {
        return new Object[][]{
                {Vessel.PINT.name(), true,},
                {Vessel.BOWL.name(), true,},
                {Vessel.GLASS.name(), true,},
                {Vessel.CUP.name(), true,},
                {Vessel.PITCHER.name(), true,},
                {Vessel.MAGNUM.name(), true,},
                {Vessel.BOTTLE.name(), true,},
                {Vessel.SPOON.name(), true,},
                {"hatful", false,},
        };
    }

    @Test(dataProvider = "simpleVesselParseData")
    public void testSimpleVesselParse(String corpus, boolean valid) {
        VesselParser parser = Grappa.createParser(VesselParser.class);
        testGrammar(corpus, valid, parser.vessel());
    }
}

The idiom that Grappa uses – and that I will use, in any event – involves the use of the push() and match() methods.

Basically, when we match a Vessel – using that handy vessel() rule – what we will do is push() a value corresponding to the Vessel whose name corresponds to the Rule we just wrote. We can get the text of the Rule we just matched, with the rather-handily-named match() method.

It’s actually simpler to program than it is to describe:

 // in VesselParser.java
 public Rule VESSEL() {
     return sequence(
             vessel(), 
             push(Vessel.valueOf(match().toUpperCase()))
     );
 }

This is a rule that encapsulates the matching of the vessel name – thus, vessel() – and then, assuming the match is found, calls push() with the Vessel whose text is held in match().

That’s fine to say, but much better to show. Here’s a test of our VESSEL() rule, following the same sort of generalized pattern we saw for parsing articles, along with a new generalized test runner that examines the returned value if the input data is valid according to the grammar:

private void testGrammarResult(String corpus, boolean status, Vessel value, Rule rule) {
    ListeningParseRunner<Vessel> runner
            = new ListeningParseRunner<>(rule);
    ParsingResult<Vessel> result = runner.run(corpus);
    assertEquals(result.isSuccess(), status,
            "failed check on " + corpus + ", parse result was "
                    + result + " and expected " + status);
    if(result.isSuccess()) {
        assertEquals(result.getTopStackValue(), value);
    }
}

@DataProvider
Object[][] simpleVesselReturnData() {
    return new Object[][]{
            {Vessel.PINT.name(), true, Vessel.PINT},
            {Vessel.BOWL.name(), true, Vessel.BOWL},
            {Vessel.GLASS.name(), true, Vessel.GLASS},
            {Vessel.CUP.name(), true, Vessel.CUP},
            {Vessel.PITCHER.name(), true, Vessel.PITCHER},
            {Vessel.MAGNUM.name(), true, Vessel.MAGNUM},
            {Vessel.BOTTLE.name(), true, Vessel.BOTTLE},
            {Vessel.SPOON.name(), true, Vessel.SPOON},
            {"hatful", false, null},
    };
}

@Test(dataProvider = "simpleVesselReturnData")
public void testSimpleVesselResult(String corpus, boolean valid, Vessel value) {
    VesselParser parser = Grappa.createParser(VesselParser.class);
    testGrammarResult(corpus, valid, value, parser.VESSEL());
}

Note that we’re testing with a Rule of parser.VESSEL() – the one that simply matches a vessel name is named parser.vessel(), and the one that updates the parser’s value stack is parser.VESSEL().

This is a personal idiom. I reserve the right to change my mind if sanity demands it. In fact, I predict that I will have done just this by the end of this article.

So what this does is very similar to our prior test – except it also tests the value on the parser’s stack (accessed via
result.getTopStackValue()) against the value that our DataProvider says should be returned, as long as the parse was expected to be valid.

All this is well and good – we can hand it "glass" and get Vessel.GLASS — but we haven’t fulfilled everything we want out of a VesselParser. We want to be able to ask for " a pint " — note the whitespace! — and get Vessel.PINT. We need to add in our article parsing.

First, let’s write our tests, so we know when we’re done:

@DataProvider
Object[][] articleVesselReturnData() {
    return new Object[][]{
            {"a pint", true, Vessel.PINT},
            {"the bowl", true, Vessel.BOWL},
            {"  an GLASS", true, Vessel.GLASS},
            {"a     cup", true, Vessel.CUP},
            {"the pitcher    ", true, Vessel.PITCHER},
            {" a an magnum", false, null},
            {"bottle", true, Vessel.BOTTLE},
            {"spoon   ", true, Vessel.SPOON},
            {"spoon  bottle ", false, null},
            {"hatful", false, null},
            {"the stein", false, null},
    };
}

@Test(dataProvider = "articleVesselReturnData")
public void testArticleVesselResult(String corpus, boolean valid, Vessel value) {
    VesselParser parser = Grappa.createParser(VesselParser.class);
    testGrammarResult(corpus, valid, value, parser.ARTICLEVESSEL());
}

Our tests should be able to ignore the leading article and any whitespace. Any wrongful formation (as you see in " a an magnum") should fail, and any vessel type that isn’t valid ("hatful" and "the stein") should fail.

Our Rule is going to look like a monster, because it has to handle a set of possibilities, but it’s actually pretty simple. Let’s take a look, then walk through the grammar:

public Rule article() {
    return trieIgnoreCase("a", "an", "the");
}

public Rule ARTICLEVESSEL() {
    return sequence(
            zeroOrMore(wsp()),
            optional(
                    sequence(
                            article(),
                            oneOrMore(wsp())
                    )),
            VESSEL(),
            zeroOrMore(wsp()),
            EOI);
}

First, we added our article() Rule, from our ArticleParser. It might be tempting to copy all the whitespace handling from that parser as well, but we shouldn’t – all we care about is the articles themselves (“lexemes,” if we’re trying to look all nerdy.)

It’s the ARTICLEVESSEL() Rule that’s fascinating. What that is describing is a sequence, consisting of:

  • Perhaps some whitespace, expressed as zeroOrMore(wsp()).
  • An optional sequence, consisting of:
    • An article.
    • At least one whitespace character.
  • A vessel (which, since we’re using VESSEL(), means the parser’s stack is updated.)
  • Perhaps some whitespace.
  • The end of input.

Any input that doesn’t follow that exact sequence ("spoon bottle", for example) fails.

Believe it or not, we’re now very much on the downhill slide for our bar-tending program.

We need to add a preposition (“of”) and then generalized text handling for the type of drink, and we need to add the container type – but of this, only the type of drink will add any actual complexity to our parser.

Rounding out the Bartender

Our VesselParser is actually a pretty good model for the DrinkOrderParser that our Bartender will use. What we need to add is matching for two extra tokens: “of,” as mentioned, and then a generalized description of a drink.

We’re not going to be picky about the description; we could validate it (just like we’ve done for Vessel) but there are actual better lessons to be gained by leaving it free-form.

Let’s take a look at the operative part of Bartender again, which will set the stage for the full parser.

DrinkOrderParser parser
        = Grappa.createParser(DrinkOrderParser.class);
ListeningParseRunner<DrinkOrder> runner
        = new ListeningParseRunner<>(parser.DRINKORDER());
ParsingResult<DrinkOrder> result = runner.run(order);
DrinkOrder drinkOrder;
boolean done = false;
if (result.isSuccess()) {
    drinkOrder = result.getTopStackValue();
    done = drinkOrder.isTerminal();
    if (!done) {
        System.out.printf("Here's your %s of %s. Please drink responsibly!%n",
                drinkOrder.getVessel().toString().toLowerCase(),
                drinkOrder.getDescription());
    }
} else {
    System.out.println("I'm sorry, I don't understand. Try again?");
}
return done;

The very first thing we’re going to do is create a DrinkOrder class, that contains the information about our drink order.

public class DrinkOrder {
    Vessel vessel;
    String description;
    boolean terminal;
}

I’m actually using Lombok in the project (and the @Data annotation) but for the sake of example, imagine that we have the standard boilerplate accessors and mutators for each of those attributes. Thus, we can call setDescription(), et al, even though we’re not showing that code. We’re also going to have equals() and hashCode() created (via Lombok), as well as a no-argument constructor and another constructor for all properties.

In other words, it’s a fairly standard Javabean, but we’re not showing all of the boilerplate code – and thanks to Lombok, we don’t even need the boilerplate code. Lombok makes it for us.

If you do need the code for equals(), hashCode(), toString(), or the mutators, accessors, and constructors shown, you may be reading the wrong tutorial. How did you make it this far?

Before we dig into the parser – which has only one really interesting addition to the things we’ve seen so far – let’s take a look at our test. This is the full test, so it’s longer than some of our code has been. The DrinkOrderParser will be much longer.

public class DrinkOrderParserTest {
    private void testGrammarResult(String corpus, boolean status, DrinkOrder value, Rule rule) {
        ListeningParseRunner<DrinkOrder> runner
                = new ListeningParseRunner<>(rule);
        ParsingResult<DrinkOrder> result = runner.run(corpus);
        assertEquals(result.isSuccess(), status,
                "failed check on " + corpus + ", parse result was "
                        + result + " and expected " + status);
        if (result.isSuccess()) {
            assertEquals(result.getTopStackValue(), value);
        }
    }
    
    @DataProvider
    public Object[][] drinkOrderProvider() {
        return new Object[][]{
                {"a glass of water", true, new DrinkOrder(Vessel.GLASS, "water", false)},
                {"a pitcher of old 66", true, new DrinkOrder(Vessel.PITCHER, "old 66", false)},
                {"a    pint  of duck   vomit   ", true, new DrinkOrder(Vessel.PINT, "duck vomit", false)},
                {"a shoeful of motor oil", false, null},
                {"nothing", true, new DrinkOrder(null, null, true)},
        };
    }
    
    @Test(dataProvider = "drinkOrderProvider")
    public void testDrinkOrderParser(String corpus, boolean valid, DrinkOrder result) {
        DrinkOrderParser parser = Grappa.createParser(DrinkOrderParser.class);
        testGrammarResult(corpus, valid, result, parser.DRINKORDER());
    }
}

Most of this should be fairly simple; it’s the same pattern we’ve seen used in our other tests.

I don’t actually drink, myself, so… I keep imagining some biker bar in the American southwest selling a beer called “Old 66,” and in my imagination “duck vomit” is the kind of wine that comes in a resealable plastic bag.

A lot of the DrinkOrderParser will be very familiar. Let’s dive in and take a look at all of it and then break it down:

public class DrinkOrderParser extends BaseParser<DrinkOrder> {
    Collection<String> vessels = Stream
            .of(Vessel.values())
            .map(Enum::name)
            .collect(Collectors.toList());

    public boolean assignDrink() {
        peek().setDescription(match().toLowerCase().replaceAll("\\s+", " "));
        return true;
    }

    public boolean assignVessel() {
        peek().setVessel(Vessel.valueOf(match().toUpperCase()));
        return true;
    }

    public boolean setTerminal() {
        peek().setTerminal(true);
        return true;
    }

    public Rule ARTICLE() {
        return trieIgnoreCase("a", "an", "the");
    }

    public Rule OF() {
        return ignoreCase("of");
    }

    public Rule NOTHING() {
        return sequence(
                trieIgnoreCase("nothing", "nada", "zilch", "done"),
                EOI,
                setTerminal()
        );
    }

    public Rule VESSEL() {
        return sequence(
                trieIgnoreCase(vessels),
                assignVessel()
        );
    }

    public Rule DRINK() {
        return sequence(
                join(oneOrMore(firstOf(alpha(), digit())))
                        .using(oneOrMore(wsp()))
                        .min(1),
                assignDrink()
        );
    }

    public Rule DRINKORDER() {
        return sequence(
                push(new DrinkOrder()),
                zeroOrMore(wsp()),
                firstOf(
                        NOTHING(),
                        sequence(
                                optional
                                        ARTICLE(),
                                        oneOrMore(wsp())
                                ),
                                VESSEL(),
                                oneOrMore(wsp()),
                                OF(),
                                oneOrMore(wsp()),
                                DRINK()
                        )
                ),
                zeroOrMore(wsp()),
                EOI
        );
    }
}

We’re reusing the mechanism for creating a collection of Vessel references. We’re also repeating the Rule used to detect an article.

We’re adding a Rule for the detection of the preposition “of”, which is a mandatory element in our grammar. We use ignoreCase(), because we respect the rights of drunkards to shout at their barkeeps:

public Rule OF() {
    return ignoreCase("of");
}

Note how I’m skirting my own rule about naming. I said I was reserving the right to change my mind, and apparently I’ve done so even while writing this article. According to the naming convention I described earlier, it should be of() and not OF() because it doesn’t alter the parser’s stack. The same rule applies to ARTICLE(). It’s my content, I’ll write it how I want to unless I decide to fix it later.

I’m also creating methods to mutate the parser state:

protected boolean assignDrink() {
    peek().setDescription(match().toLowerCase().replaceAll("\\s+", " "));
    return true;
}

protected boolean assignVessel() {
    peek().setVessel(Vessel.valueOf(match().toUpperCase()));
    return true;
}

protected boolean setTerminal() {
    peek().setTerminal(true);
    return true;
}

These are a little interesting, in that they use peek(). The actual base rule in our grammar is DRINKORDER(), which immediately pushes a DrinkOrder reference onto the parser stack. That means that there is a DrinkOrder that other rules can modify at will; peek() gives us that reference. Since it’s typed via Java’s generics, we can call any method that DrinkOrder exposes.

These utilities all return true. None of them can fail, because they won’t be called unless a prior rule has matched; these methods are for convenience only. Actually, let’s show the NOTHING() and VESSEL() rules, so we can see how these methods are invoked:

public Rule NOTHING() {
    return sequence(
            trieIgnoreCase("nothing", "nada", "zilch", "done"),
            EOI,
            setTerminal(),
    );
}

public Rule VESSEL() {
    return sequence(
            trieIgnoreCase(vessels),
            assignVessel()
    );
}

This leaves two new rules to explain: DRINK() and DRINKORDER(). Here’s DRINK():

public Rule DRINK() {
    return sequence(
            join(oneOrMore(firstOf(alpha(), digit())))
                    .using(oneOrMore(wsp()))
                    .min(1),
            assignDrink()
    );
}

This rule basically builds a list of words. It’s a sequence of operations; the first builds the match of the words, and the second operation assigns the matched content to the DrinkOrder‘s description.

The match of the words is really just a sequence of alphanumeric characters. It requires at least one such sequence to exist, but will consume as many as there are in the input.

Now for the Rule that does most of the work: DRINKORDER().

public Rule DRINKORDER() {
    return sequence(
            push(new DrinkOrder()),
            zeroOrMore(wsp()),
            firstOf(
                    NOTHING(),
                    sequence(
                            optional(sequence(
                                    ARTICLE(),
                                    oneOrMore(wsp())
                            )),
                            VESSEL(),
                            oneOrMore(wsp()),
                            OF(),
                            oneOrMore(wsp()),
                            DRINK()
                    )
            ),
            zeroOrMore(wsp()),
            EOI
    );
}

Again, we have a sequence. It works something like this:

  • First, push a new DrinkOrder onto the stack, to keep track of our order’s state.
  • Consume any leading whitespace.
  • Either:
    • Check for the terminal condition (“nothing”, for example), or
    • Check for a new sequence, of the following form:
      • An optional sequence:
        • An article
        • Any trailing whitespace after the article
      • A vessel
      • One or more whitespace characters
      • The lexeme matching “of”
      • One or more whitespace characters
      • The drink description
  • Any trailing whitespace
  • The end of input

We’ve basically built most of this through our parsers, bit by bit; armed with the ability to peek() and push(), we can build some incredibly flexible parsers with fairly simple code.

Adding Politesse

All this has been great, so far. We can actually “order” from a Bartender, giving us this scintillating conversation:

$ java -jar bartender-1.0-SNAPSHOT.jar
What're ya havin'? a glass of water
Here's your glass of water. Please drink responsibly!
What're ya havin'? a toeful of shoe polish
I'm sorry, I don't understand. Try again?
What're ya havin'? a pint of indigo ink
Here's your pint of indigo ink. Please drink responsibly!
What're ya havin'? nothing
$

The only problem is that it’s not very humane or polite. We can’t say “please,” we can’t be very flexible. What we need is to add politesse to our grammar.

What we really want to do is modify our DrinkOrderParser so that we can ask for “a cup of pinot noir, 1986 vintage, please?” It should be able to tell us that we’ve ordered “pinot noir, 1986” and not “pinot noir, 1986, please?” — that’d be silly.

However, we need to alter our grammar in some core ways – particularly in how we match the drink names — and use a new Rule, testNot. First, though, let’s take a look at our test code, because that’s going to give us a workable indicator of whether we’ve succeeded or not.

public class PoliteDrinkOrderParserTest {
    private void testGrammarResult(String corpus, boolean status, DrinkOrder value, Rule rule) {
        ListeningParseRunner<DrinkOrder> runner
                = new ListeningParseRunner<>(rule);
        ParsingResult<DrinkOrder> result = runner.run(corpus);
        assertEquals(result.isSuccess(), status,
                "failed check on " + corpus + ", parse result was "
                        + result + " and expected " + status);
        if (result.isSuccess()) {
            assertEquals(result.getTopStackValue(), value);
        }
    }

    @DataProvider
    public Object[][] drinkOrderProvider() {
        return new Object[][]{
                {"a glass of water please", true, new DrinkOrder(Vessel.GLASS, "water", false)},
                {"a pitcher of old 66, please", true, new DrinkOrder(Vessel.PITCHER, "old 66", false)},
                {"a pitcher of old 66", true, new DrinkOrder(Vessel.PITCHER, "old 66", false)},
                {"a glass of pinot noir, 1986", true, new DrinkOrder(Vessel.GLASS, "pinot noir, 1986", false)},
                {"a glass of pinot noir, 1986, ok?", true, new DrinkOrder(Vessel.GLASS, "pinot noir, 1986", false)},
                {"glass of pinot noir, 1986, ok?", true, new DrinkOrder(Vessel.GLASS, "pinot noir, 1986", false)},
                {"cup , pinot noir, 1986 vintage, ok?", true, new DrinkOrder(Vessel.CUP, "pinot noir, 1986 vintage", false)},
                {"cup,pinot noir, 1986,ok!", true, new DrinkOrder(Vessel.CUP, "pinot noir, 1986", false)},
                {"a    pint  of duck   vomit   ", true, new DrinkOrder(Vessel.PINT, "duck vomit", false)},
                {"a    pint  of duck   vomit  , please ", true, new DrinkOrder(Vessel.PINT, "duck vomit", false)},
                {" pint , duck   vomit please  ", true, new DrinkOrder(Vessel.PINT, "duck vomit", false)},
                {"a shoeful of motor oil", false, null},
                {"nothing", true, new DrinkOrder(null, null, true)},
        };
    }

    @Test(dataProvider = "drinkOrderProvider")
    public void testDrinkOrderParser(String corpus, boolean valid, DrinkOrder result) {
        PoliteDrinkOrderParser parser = Grappa.createParser(PoliteDrinkOrderParser.class);
        testGrammarResult(corpus, valid, result, parser.ORDER());
    }
}

You’ll notice that we’ve changed some other things, too. Our original grammar was pretty simple in formal-ish terms:

DRINKORDER ::= nothing | article? vessel `of` drink
article ::= a | an | the
vessel ::= pint | bowl | glass | cup | pitcher | magnum | bottle | spoon
drink ::= [a-zA-Z0-9]*
nothing ::= nothing | nada | zilch | done

Note that this isn’t an actual formal grammar – I’m cheating. It just looks as if it might be something near formal, with a particular failure in the “drink” term.

Our new one seems to be more flexible:

DRINKORDER ::= (nothing | article? vessel of drink) interjection? eos?
article ::= a | an | the
vessel ::= pint | bowl | glass | cup | pitcher | magnum | bottle | spoon
of ::= , | of
drink ::= !interjection
interjection ::= ','? please | ok | okay | pls | yo 
eos ::= '.' | '!' | '?'
nothing ::= nothing | nada | zilch | done

Here, we’re no more actually formal than we were – the “!interjection” is trying to say that a drink is everything where a drink would be appropriate, up to the interjection.

I don’t care for Backus-Naur form, and I’m using something that looks like it because I thought it might help. Your mileage may vary as to whether I was correct or not.

At any rate, our new grammar should allow us to say “please” and eliminate the unnecessary “of” – although I’m not willing to concede that a bartender should respond well to “pint beer.” “pint, beer.” I can accept – but that comma is significant, by golly.

I’ll leave it as an exercise for the reader to make the comma not necessary – and to write the test that proves it.

However, one thing remains: we haven’t seen our grammar. Most of it’s the same: the article, the vessel, and the action rules (the things that construct our returned drink order) haven’t changed, but we have a slew of new rules (for the end of the sentence and the interjection) and we’ve modified some old ones (drink, and of). Let’s take a look at the changes held in PoliteDrinkOrderParser:

public Rule OF() {
    return firstOf(
            sequence(
                    zeroOrMore(wsp()),
                    COMMA(),
                    zeroOrMore(wsp())
            ),
            sequence(
                    oneOrMore(wsp()),
                    ignoreCase("of"),
                    oneOrMore(wsp())
            )
    );
}

public Rule DRINK() {
    return sequence(
            oneOrMore(
                    testNot(INTERJECTION()),
                    ANY
            ),
            assignDrink());
}

public Rule DRINKORDER() {
    return sequence(
            optional(sequence(
                    ARTICLE(),
                    oneOrMore(wsp())
            )),
            VESSEL(),
            OF(),
            DRINK()
    );
}

public Rule COMMA() {
    return ch(',');
}

public Rule INTERJECTION() {
    return sequence(
            zeroOrMore(wsp()),
            optional(COMMA()),
            zeroOrMore(wsp()),
            trieIgnoreCase("please", "pls", "okay", "yo", "ok"),
            TERMINAL()
    );
}

public Rule EOS() {
    return anyOf(".!?");
}

public Rule TERMINAL() {
    return sequence(zeroOrMore(wsp()),
            optional(EOS()),
            zeroOrMore(wsp()),
            EOI
    );
}

public Rule ORDER() {
    return sequence(
            push(new DrinkOrder()),
            zeroOrMore(wsp()),
            firstOf(DRINKORDER(), NOTHING()),
            optional(INTERJECTION()),
            TERMINAL()
    );
}

We’ve had to move whitespace handling around a little, too, because of the use of OF() to serve as a connector rather than the simple word “of.”

OF() now has to serve as a syntax rule for a single comma – with no whitespace – as you’d see in the string “pint,beer“. It also has to handle whitespace – as you’d find in pint , beer.

However, it needs to mandate whitespace for the actual word of – because pintofbeer doesn’t work.

Another exercise for the reader: fix OF() to handle “pint, of beer“.

DRINK() has a new sequenceoneOrMore(testNot(INTERJECTION()), ANY).

Pay attention to this.

This means to match everything (as per the ANY) that does not match the INTERJECTION() rule. The sequence order is important – it tries to match the rules in order, so it checks the tokens (by looking ahead) against INTERJECTION() first, and failing that check (and therefore succeeding in the match – remember, we’re looking for something that is not an INTERJECTION()) it checks to see if the text matches ANY.

Given that ANY matches anything, it succeeds – as long as the tokens are not tokens that match the INTERJECTION() rule.

And what does INTERJECTION() look like? Well, it’s a normal rule – this is where Grappa really shines. Our INTERJECTION() has optional whitespace and punctuation, and it’s own case insensitive matching:

public Rule INTERJECTION() {
    return sequence(
            zeroOrMore(wsp()),
            optional(COMMA()),
            zeroOrMore(wsp()),
            trieIgnoreCase("please", "pls", "okay", "yo", "ok"),
            TERMINAL()
    );
}

It also has the terminal condition for the order, because something might look like an interjection but wouldn’t be. Consider this input: “glass,water,please fine.” The ,please matches an INTERJECTION(), but because the INTERJECTION() includes the TERMINAL() rule – which means “optional whitespace, an optional end-of-sentence, optional whitespace, and then a definite end-of-input” – “,please fine” fails the INTERJECTION() match, and falls back to ANY.

EOI can match legally several times. That’s why we can match it in our INTERJECTION() rule while still having it match the end of our ORDER() rule. The nature of TERMINAL() – being a series of optional elements – means that if it’s matched as part of INTERJECTION() it won’t match at the end of ORDER(). Such is life.

We can also order something like this: “glass,water, please ok?” — and our drink would be a glass of “water, please” because “ok” would match the INTERJECTION() rule.

Our bartender’s a great guy, but he likes his grammar.

Our PoliteBartender class is different from our Bartender only in the Parser it uses and the originating Rule – and, of course, in the flexibility of the orders it accepts.

$ java -cp bartender-1.0-SNAPSHOT.jar com.autumncode.bartender.PoliteBartender
What're ya havin'? a glass of water
Here's your glass of water. Please drink responsibly!
What're ya havin'? a toeful of shoe polish
I'm sorry, I don't understand. Try again?
What're ya havin'? a pint of indigo ink, please
Here's your pint of indigo ink. Please drink responsibly!
What're ya havin'? A SPOON OF DOM PERIGNON, 1986, OK?
Here's your spoon of dom perignon, 1986. Please drink responsibly!
What're ya havin'? magnum,water,pls, please
Here's your magnum of water,pls. Please drink responsibly!
What're ya havin'? nothing
$

Colophon

By the way, much appreciation goes to the following individuals, who helped me write this in various important ways, and in no particular order:

  • Francis Galiegue, who helped by reviewing the text, by pointing out various errors in my grammars, and by writing Grappa in the first place
  • Chris Brenton, who reviewed (a lot!) and helped me tune the messaging
  • Andreas Kirschbaum, who also reviewed quite a bit for the article, especially in early form

I still prefer Java.

I still prefer Java over other languages.

The Background: Javabot

I’m a fairly regular contributor to javabot, an IRC bot written for the Freenode ##java channel. I don’t know that I’d be considered a major contributor (I’m not listed in the credits, for example, so maybe I do know – and I’m not a major contributor) but I have a few solved issues to my credit…

But I think my contributions to the Javabot project are nearing an end, and the reason is really rather sad: Kotlin. Javabot recently underwent a conversion from Java to Kotlin.

Kotlin, as a language, looks really neat. It has features that I think Java could use; it’s probably a viable alternative to Java and, to some degree, Scala. I know a number of programmers who are using Kotlin, and what they describe sounds good.

But I don’t know Kotlin. I know a lot of coders, and of those, very few programmers who use Kotlin – and even though Kotlin looks neat, I can’t use it apart from cottage projects like javabot. The ecosystem around Kotlin is just too small.

That means that any contribution I had for Javabot would be severely limited by my own skill level in Kotlin – which is not experienced enough to even describe as “kotlin newbie” but is still stuck at “laughable.” Any code I wrote for javabot would be more of a burden than a benefit, since the code would have to be severely vetted.

I wouldn’t want a contribution to be a burden – any contribution of mine would be intended to make the world better, not harder. So I think that my participation in the project is limited by my own good intentions.

Prefer Java. Really.

The result, then, is that I’d suggest that coders use Java, even though it’s not as cool or as full-featured as some other languages might be. I’d be willing to consider projects in Scala, which has a viable ecosystem at this point (and has no difficulty leveraging Java’s ecosystem), and even Groovy (which leverages the Java ecosystem more than its own, as far as I can tell).

That’s not to say that Kotlin doesn’t have viability in its future, nor is it to say that I have no interest in learning Kotlin – it’s just a recognition that a new language is a burden to contributors, and because I don’t want to shoulder nor cause that burden, I’d suggest sticking with a language that’s common for the programming environment in which you live.

Addendum

I don’t blame the author for moving to Kotlin – javabot is a cottage project, really, and he actually did ask contributors their opinions before migrating. I voted for the migration. I just didn’t anticipate how I feel about it today as a result.

Test-driven development can be great.

Test-driven development” is one of those things that causes hives among some programmers, who immediately stand up and plant the claim that it’s worthless, peurile, deceptive, and generates awful code… never mind that others manage to use it effectively.

I have been playing with a 2D cellular automaton, largely inspired by Stephen Wolfram’s “A New Kind of Science.” I have a version in Java, and a simpler implementation in Python. TDD made the Java version work properly, and a lack of TDD left a hole in the Python version.

“A New Kind of Science” wasn’t the only inspiration, of course. A friend of mine published a Javascript version of Conway’s Game of Life – a 3D automaton that’s pretty well-known – and a different friend of mine was looking for simple projects that he could use to help teach kids how to program, and a 2D automaton came to mind – so he asked for an implementation, which is why I wrote the Python version of the automaton.

This post is not about the automaton itself. I’ll write that up later. (If you’re interested, you can see the source.)

What is TDD?

Test-driven development is, loosely defined, a practice in which tests are written before anything else, without regard to correctness.

For example, if I want to write a program to generate “Hello, world,” I would write a test that validated that “Hello, world” was generated – before I had anything that might create the output. My tests would fail; they wouldn’t even compile until I had some sort of implementation written.

However, by writing a test before anything else:

  • I more or less force myself into having some sort of specification that says what “correctness” means for my program (it is “correct” when it generates “Hello, world”)
  • I also force myself into writing something that has a reasonable interface (because I’m writing how I think it would be called, before writing the guts of the implementation)

By writing the tests first, I’ve effectively given myself a criterion for completeness. When my tests pass, I know I’ve “finished,” because my tests define a specification.

There’s nothing wrong with the specification being incomplete, of course; it may so happen that later, I want to greet someone specific. By having tests in place, though, not only do I have a record of the specification, but I also have a way that I can add to the specification in such a way that I know I’m not breaking code – I would simply add more tests that corresponded with the changing specification, and I will know if my changes break other code.

How did TDD work out for my Automaton?

Here’s the thing: I wrote the Java implementation using test-driven development practices (TDD), and the automaton is kinda neat; it generates some fascinating patterns even without entropy or a variable starting cell structure. An example, of pattern 171, using a color rendering mechanism:

color-171

The Python version was written because a friend of mine wanted to consider using it for a class he’s teaching. The Python version is very much simpler than the Java version, because it doesn’t do as much (it can’t output to multiple formats, for example).

It was not written with testing in mind. Why would it be? I had written the Java version from tests first; I was only writing a simple port to Python.

It was also wrong. A 2D automaton can “grow” to the right and to the left, depending on the pattern it’s given; the Python version could only grow on the right, because I had an off-by-one error in a core routine.

TDD would have caught that early (and it did catch problems like that, in the Java version).

TDD also provided me the opportunity to fix the names of structures (renaming Dataset to Generation, for example) because the tests made it obvious that the names were inaccurate.

Could I have done it without TDD? Of course. TDD isn’t the only way to write programs well. It’s not the only tool used to work out good names, or good processes, or even to validate that programs work properly – the Python version of the automaton was fixed without TDD, for example.

If you’re wondering why I didn’t use TDD for the Python version, it’s because I’m too much of a newbie with Python to know how, yet – and as I’m not really a Python programmer, there’s not a lot of need. However, seeing the differences in the development process between my Python implementation and the Java implementation, I might look into TDD with Python anyway.

An Incomplete Introduction to OWLAPI

OWL is a language used to describe ontologies – systems of knowledge. It’s found primarily in the Semantic Web, which mostly means “hardly anywhere,” and part of the problem is that using OWL is really pretty difficult. You can find plenty of technical language, but very little that describes how and why you’d actually use this stuff in the real world.

As a result, finding real-world uses is difficult, and where it is used in the real world, the usage is inconsistent.

That’s a shame. OWL can be very useful; you can use it to figure out things, you can use it for actual classification and data storage.

However, the purpose of this tutorial is, sadly, not to address the “why” but the “how” of OWL, specifically focusing on the underlying library used for one of the best-known OWL projects, Protégé. That library is known as OWLAPI.

There are tutorials – sort of – that focus on OWLAPI, and the best of them (meaning: “most popular,” the one that gets thrown at you if you ask for resources) is a set of slides in PDF form, from a presentation at OWLED 2011.

The slides are useful, but also need the presentation content, which is not included. The slides are also based on an old revision of the project, so more recent users not only have to figure out why some of the slides are included, but how to use the code snippets in a more recent version of the library.

I’d like to change that, if I can.

Expectations

After this tutorial is finished, I expect readers to be able to create, read, and write ontologies, with a basic understanding of what the various common elements in ontologies are and how they’re used.

The code (and the original Markdown source of this page) are in a Github project: owlapi-tutorial. Feel free to comment or fork as desired.

I do not expect readers to necessarily be left with a complete understanding of OWL, or how it should be used in their organization or projects.

I do not expect to be able to accurately or completely describe every nuance of every bit of code I describe. Much of OWLAPI programming involves cargo cult programming: “I did this, it worked, therefore I do it every time.” I’m afraid that I don’t know how to get around this, but trying to limit the cargo cult mentality is partially why I’m trying to write this tutorial in the first place.

Requirements

I am using:

  • Java 8. This is the current live, supported version of Java; it’s also the oldest supported version of Java as of this writing (Java 7 has been end-of-lifed) and I don’t see the point of using an outmoded version of Java. I actually personally prefer Scala, and OWLAPI is verbose enough that Scala actually makes the resulting code far cleaner than it would be in Java, but I think it’s more important to be precise (as opposed to concise) when writing tutorials.
  • Maven. This is my preferred building environment. There’s nothing wrong with Gradle, et al; I just prefer Maven as I think it’s pretty much the lowest common denominator of functional build systems. Everyone has it, everyone can cater to it.
  • TestNG. I will be writing most of the code snippets as tests first. JUnit is arguably more “lowest common denominator” than TestNG is, but I find that TestNG works better. It’s my tutorial, I’ll use the test framework I want to use.

My project file for Maven is pretty simple, and isn’t likely to change much. This is not a Maven tutorial, so here it is in all its raw glory:

< ?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelversion>4.0.0</modelversion>

    <groupid>com.autumncode</groupid>
    <artifactid>owlapi-tutorial</artifactid>
    <version>1.0-SNAPSHOT</version>

    <dependencymanagement>
        <dependencies>
            <dependency>
                <groupid>net.sourceforge.owlapi</groupid>
                <artifactid>owlapi-distribution</artifactid>
                <version>[4.0.2,)</version>
            </dependency>
            <dependency>
                <groupid>net.sourceforge.owlapi</groupid>
                <artifactid>owlapi-rio</artifactid>
                <version>[4.0.2,)</version>
            </dependency>
            <dependency>
                <groupid>net.sourceforge.owlapi</groupid>
                <artifactid>jfact</artifactid>
                <version>4.0.2</version>
            </dependency>
            <dependency>
                <groupid>org.slf4j</groupid>
                <artifactid>slf4j-simple</artifactid>
                <version>1.7.7</version>
            </dependency>
            <dependency>
                <groupid>org.testng</groupid>
                <artifactid>testng</artifactid>
                <version>[6.9.8,)</version>
                <scope>test</scope>
            </dependency>
        </dependencies>
    </dependencymanagement>

    <dependencies>
        <dependency>
            <groupid>net.sourceforge.owlapi</groupid>
            <artifactid>owlapi-distribution</artifactid>
        </dependency>
        <dependency>
            <groupid>net.sourceforge.owlapi</groupid>
            <artifactid>owlapi-rio</artifactid>
        </dependency>
        <dependency>
            <groupid>org.slf4j</groupid>
            <artifactid>slf4j-simple</artifactid>
        </dependency>
        <dependency>
            <groupid>org.testng</groupid>
            <artifactid>testng</artifactid>
        </dependency>
        <dependency>
            <groupid>net.sourceforge.owlapi</groupid>
            <artifactid>jfact</artifactid>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupid>org.apache.maven.plugins</groupid>
                <artifactid>maven-compiler-plugin</artifactid>
                <version>3.3</version>
                <configuration>
                    <source />1.8
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

There are a few dependencies that relate to OWLAPI: owlapi-distribution, which is OWLAPI itself; owlapi-rio, which has some classes we need in order to parse and write OWL; lastly, there’s jfact, which is a reasoner implementation for OWL. (A reasoner is used to pull various types of information from OWL. I don’t know a ton about reasoners yet, and apparently JFact is pretty lightweight, but it’s enough for this tutorial.)

Terms

An “Ontology” is a system of knowledge. It is composed of descriptions of what things are as well as a set of things.

A “Class” is a description of what a thing is. A “Person,” for example, is a term you might use to describe a living being of a specific species (homo sapiens, generally), with gender and a name. Therefore, we might describe a “Person class”.

A class can have attributes (a concept familiar to Java coders); these are “properties,” which can be made of data or references (again, just like Java). Properties can have ranges or defined subsets of values.

A class can also have the concept of a superclass (again, just like Java): a Fireman class might be a valid subclass of Person. However, there is no limit to the number of superclasses; a Fireman might be a subclass of the Person, PublicServant, WearsRed, and DrivesTruck classes. Java can model some of this with interfaces, but it’s not an exact analogy.

An “Individual” is a concrete instance of a class – much like an “instance,” in Java parlance. “John” might be a fireman, which implies that he is a person; Sam might be a person (but not a fireman). Sam might have a dog, Fritz, who is not a Person (instead, is a Dog). All three would be individuals, with varying properties.

Working with OWLAPI

OWL uses the concept of an “IRI,” an “Internationalized Resource Identifier.” It’s a lot like a URI, but has a wider set of characters available to it; for the sake of clarity we’ll stay seven-bit-clean.

Basically, an IRI is a unique identifier for a concept, whether it’s the ontology itself, a class in the ontology, a data property, an individual, or any other unique concept represented in the ontology.

OWLAPI creates references to elements in an Ontology by asking a data factory for a reference of a specific type with a specific IRI. Relationships between elements are expressed as “axioms.” Therefore, the typical interaction with OWLAPI will look something like this:

  1. Create an initial reference with a unique IRI.
  2. Create another reference, with a different (and unique) IRI.
  3. Express an axiom using the two references.
  4. Apply the axiom as a change to the ontology.

Creating an Ontology

Before we can do anything with an ontology, we need to be able to create one. We’re going to create a stateful helper class in Java to serve as a simple delegate to OWLAPI; it will retain some simple references that we’ll end up using over and over again (the OWLDataFactory and OWLOntologyManager) and also provide a class into which we can place simple utility methods.

Many of the methods in our utility class (which I’ve decided will be called OntologyHelper) are one-liners, or close to it, especially early on.

The baseline for our OntologyHelper looks like this, with packages and all imports stripped out:

public class OntologyHelper {
    OWLOntologyManager m= OWLManager.createOWLOntologyManager();
    OWLDataFactory df=OWLManager.getOWLDataFactory();
}

We know we’ll be converting between java.lang.String and org.semanticweb.owlapi.model.IRI quite a bit, so we’ll add a conversion method:

public class OntologyHelper {
    OWLOntologyManager m= OWLManager.createOWLOntologyManager();
    OWLDataFactory df=OWLManager.getOWLDataFactory();

    public IRI convertStringToIRI(String ns) {
        return IRI.create(ns);
    }
}

Now let’s look at actually creating an OWLOntology. First, let’s write a test that creates the ontology and then checks to make sure the ontology conforms to what we think it should be:

@Test
public void createOntology() throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    IRI iri = oh.convertStringToIRI("http://autumncode.com/ontologies/2015/example.owl");
    OWLOntology ontology = oh.createOntology(iri);

    assertNotNull(ontology);
    assertEquals(iri,
            ontology.getOntologyID().getOntologyIRI().or(oh.convertStringToIRI("false")));
}

That last assertEquals is a bit scary. The first argument is the IRI we expect the ontology to have; the second argument is a work of art. Basically, we’re getting the ontology ID, part of which is the ontology’s IRI; however, an ontology might not have an IRI, so it’s actually returned in a Java Optional<iri>. We use .or() to specify a constant value (an IRI) to use just in case our ontology does not have an IRI.

Creating an ontology, given our m and df declarations from OntologyHelper, is a matter of calling m.createOntology with an IRI. Since working with String is a lot more easily conceived of than working with an IRI, we’ll create a convenience method to take a String argument, convert it to an IRI, and then delegate to another method to create the OWLOntology itself.

With that, then, let’s add some content to our OntologyHelper, consisting of the following two methods (only one of which is used by the test, since we pass in an IRI directly).

public OWLOntology createOntology(String iri) throws OWLOntologyCreationException {
    return createOntology(convertStringToIRI(iri));
}

public OWLOntology createOntology(IRI iri) throws OWLOntologyCreationException {
    return m.createOntology(iri);
}

Writing an Ontology

Writing an ontology is fairly simple. To write an ontology, we simply create a OWLOntologyDocumentTarget of some kind – examples include a StringDocumentTarget and a FileDocumentTarget, among others – and then call m.saveOntology(), supplying the ontology to save, the target, and (optionally) a format to use. (OWL has multiple formats, including OWL itself, RDF, Turtle, and Manchester format among others; the formats are out of scope for this tutorial. We’re sticking to the default OWL format.)

First, let’s write our test:

@Test
public void writeOntology() throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology ontology = oh.createOntology("http://autumncode.com/ontologies/2015/example.owl");
    StringDocumentTarget sdt = new StringDocumentTarget();
    oh.writeOntology(ontology, sdt);

    StringDocumentSource sds = new StringDocumentSource(sdt.toString());
    OWLOntology o = oh.readOntology(sds);
    assertEquals(oh.convertStringToIRI("http://autumncode.com/ontologies/2015/example.owl"),
            o.getOntologyID().getOntologyIRI().or(oh.convertStringToIRI("false")));
}

Here, we’re creating our ontology (just as we did earlier), but we’re writing it into a StringDocumentTarget. We then use that StringDocumentTarget to demonstrate reading the ontology, using the read content to validate the ontology ID.

Reading an Ontology

Now let’s show a little more about reading an ontology – and when we say “a little more,” we mean literally only a little more. We’ll also use the @DataProvider concept in TestNG to give us some flexibility for when we create more test data.

The weakness of the writeOntology test is that it is internal; it doesn’t have any external data. We don’t have any artifacts of the test to examine. We should be able to read an external artifact (a file, for example) and use it for reference and for examination.

With that, let’s take a look an an ontology very similar to the ones we’ve been creating, stored in src/test/resources/example1.owl:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/2015/example1.owl#"
         xml:base="http://autumncode.com/ontologies/2015/example1.owl"
         xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:owl="http://www.w3.org/2002/07/owl#"
         xmlns:xml="http://www.w3.org/XML/1998/namespace"
         xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
         xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/2015/example1.owl"></owl:ontology>
</rdf:rdf>

This ontology is empty, consisting only of the ontology reference itself. (It’s expressing the idea of a system of knowledge that refers to … no knowledge at all, including the assertion of the absence of knowledge.)

Our test code will use a data provider to construct a reference to an OWLOntologyDocumentSource (which will point to this file) and the expected IRI from it (in this case, "http://autumncode.com/ontologies/2015/example1.owl").

@DataProvider
Object[][] readDataProvider() {
    return new Object[][]{{
            new StreamDocumentSource(this.getClass().getResourceAsStream("/example1.owl")),
            "http://autumncode.com/ontologies/2015/example1.owl"
    }};
}

@Test(dataProvider = "readDataProvider")
public void readOntology(OWLOntologyDocumentSource source, String baseIRI) throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology ontology = oh.readOntology(source);
    assertNotNull(ontology);
    assertEquals(oh.convertStringToIRI(baseIRI),
            ontology.getOntologyID().getOntologyIRI().or(oh.convertStringToIRI("false")));
}

As usual, our OntologyHelper code is far more simple than the test code is:

public OWLOntology readOntology(OWLOntologyDocumentSource source)
        throws OWLOntologyCreationException {
    return m.loadOntologyFromOntologyDocument(source);
}

Adding a Class to an Ontology

Adding a class to an ontology is similar to adding a class to a Java package, with the main exception being that a class in an ontology doesn’t exist without a relationship to something else. Therefore, in Java we can say “a person is something,” whereas in OWL, we have to say “a person is something that…”

An OWL class can’t exist in a vacuum.

First, let’s create a simple relationship: we’ll define an ontology which can allow individuals to be people (i.e., “a person”) or firemen. Here’s our test code, which will ironically have to live in a vacuum itself until we write all of our utility methods:

@Test
public void testAddClassesSimple() throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/person.owl");
    OWLClass person = oh.createClass("http://autumncode.com/ontologies/person.owl#Person");
    OWLClass fireman = oh.createClass("http://autumncode.com/ontologies/person.owl#Fireman");
    OWLAxiomChange axiom = oh.createSubclass(o, fireman, person);
    oh.applyChange(axiom);

    Set<string> superclasses = o.getSubClassAxiomsForSubClass(fireman)
            .stream()
            .map(ax -> ax.getSuperClass().asOWLClass().toStringID())
            .collect(Collectors.toSet());
    assertEquals(1, superclasses.size());
    assertTrue(superclasses.contains(person.toStringID()));
}

We need to create OntologyHelper.createClass first, which looks like this:

public OWLClass createClass(String iri) {
    return createClass(convertStringToIRI(iri));
}
public OWLClass createClass(IRI iri) {
    return df.getOWLClass(iri);
}

We also need to define OntologyHelper.createSubclass. This will create a change for our ontology – a patch – that says that in the context of our ontology, a fireman is a type, a subclass, of person. It does not apply that patch; that’s the role of our OntologyHelper.applyChange method.

Here are those methods from OntologyHelper:

public OWLAxiomChange createSubclass(OWLOntology o, OWLClass subclass, OWLClass superclass) {
    return new AddAxiom(o,df.getOWLSubClassOfAxiom(subclass, superclass));
}

public void applyChange(OWLAxiomChange ... axiom) {
    applyChanges(axiom);
}

private void applyChanges(OWLAxiomChange ... axioms) {
    m.applyChanges(Arrays.asList(axioms));
}

After we apply the changes, we get the axioms related to subclass for our definition of what a fireman is, translate the axioms into the superclass name, and then check to make sure the resulting set is the right size (one) and contains the right value (a reference to our person definition).

The resulting OWL, if you’re interested, looks like this:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/person.owl#"
     xml:base="http://autumncode.com/ontologies/person.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/person.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Fireman -->

    <owl:class rdf:about="http://autumncode.com/ontologies/person.owl#Fireman">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/person.owl#Person"></rdfs:subclassof>
    </owl:class>

    <!-- Person -->

    <owl:class rdf:about="http://autumncode.com/ontologies/person.owl#Person"></owl:class>
</rdf:rdf>

In Java, we’ve basically expressed something like this:

public class Person { }
public class Fireman extends Person { }

Let’s take it a little farther. Imagine, if you will, SkyNet: a Terminator is both a Person and a Robot, but that doesn’t preclude the existence of a person who is not a robot, nor does it mean that all robots are people. Our test method is similar, but has two superclasses for the definition of a Terminator.

@Test
public void testAddClassesMoreComplex() throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/terminator.owl");
    OWLClass person = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Person");
    OWLClass robot = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Robot");
    OWLClass terminator = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Terminator");

    oh.applyChange(oh.createSubclass(o, terminator, person),
            oh.createSubclass(o, terminator, robot));

    Set</string><string> superclasses = o.getSubClassAxiomsForSubClass(terminator)
            .stream()
            .map(ax -> ax.getSuperClass().asOWLClass().toStringID())
            .collect(Collectors.toSet());
    assertEquals(2, superclasses.size());
    assertTrue(superclasses.contains(person.toStringID()));
    assertTrue(superclasses.contains(robot.toStringID()));
}

The OWL this structure generates looks like this:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/terminator.owl#"
     xml:base="http://autumncode.com/ontologies/terminator.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/terminator.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Person -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Person"></owl:class>

    <!-- Robot -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Robot"></owl:class>

    <!-- Terminator -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Terminator">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/terminator.owl#Person"></rdfs:subclassof>
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/terminator.owl#Robot"></rdfs:subclassof>
    </owl:class>
</rdf:rdf>

However, Java cannot express these relationships, as it doesn’t support multiple concrete superclasses.

It’s worth pointing out that the code that validates the relationships is horribly ugly, and not generally useful outside of the context of these tests. One normally wouldn’t bother with that sort of code.

Note that we’ve only expressed the idea that we can have an ontology that describes people, robots, and things that are both robots and people (i.e., Terminators). We could create Captain Kirk, the T-800, and R2-D2, all while ignoring the shrieks of rage from Trekkies, Star Wars fans, and whoever cares about the Terminator.

Time to fix that. We still won’t be able to describe any of these individuals besides their names and what class they are, but that’ll be enough for the next step.

Adding an Individual to an Ontology

Let’s start with the simplest structure possible: let’s create an ontology that has a class of Person, and add Captain Kirk to the ontology as an instance of a Person.

Our test code is actually pretty simple, but verifying that we’ve done what we expected is a bit of a pain.

Most of the testing code builds a map of classes to individuals, using an OWLReasoner, which is a class that extracts data from the ontology: we use it here to extract the direct instances of a type from a given class (after iterating through “all classes”). The ListMultimap is part of Guava, which is a transitive dependency of OWLAPI.

Note also the “AutoClosable trick,” which forces Java to dispose of the OWLReasoner when the block has completed execution.

@Test
public void addSimpleIndividual() throws Exception {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/person.owl");
    OWLClass person = oh.createClass("http://autumncode.com/ontologies/person.owl#Person");
    OWLIndividual captainKirk = oh.createIndividual("http://autumncode.com/ontologies/person.owl#Kirk");
    oh.applyChange(oh.associateIndividualWithClass(o, person, captainKirk));

    // test that the individual is what we expect
    final OWLReasoner reasoner = new JFactFactory().createReasoner(o);
    try (final AutoCloseable ignored = reasoner::dispose) {
        ListMultimap</string><string , String> classInstanceMap = ArrayListMultimap.create();
        o.getClassesInSignature()
                .stream()
                .forEach(clazz ->
                        reasoner.getInstances(clazz, true)
                                .getFlattened()
                                .stream()
                                .forEach(i ->
                                        classInstanceMap.put(clazz.toStringID(), i.toStringID())));
        // should have one class
        assertEquals(classInstanceMap.keySet().size(), 1);
        Set</string><string> people=new HashSet<>();
        people.add(captainKirk.toStringID());
        assertEquals(classInstanceMap.asMap().get(person.toStringID()), people);
    } 
}

We need three additional methods in our OntologyHelper class, all of which are fairly simple (two create the OWLIndividual references, and one returns an axiom that associates an individual with a class):

public OWLIndividual createIndividual(String iri) {
    return createIndividual(convertStringToIRI(iri));
}
private OWLIndividual createIndividual(IRI iri) {
    return df.getOWLNamedIndividual(iri);
}

public OWLAxiomChange associateIndividualWithClass(OWLOntology o,
                                                   OWLClass clazz,
                                                   OWLIndividual individual) {
    return new AddAxiom(o, df.getOWLClassAssertionAxiom(clazz, individual));
}

In Java, we’ve basically modeled something like this:

class Person {}
void buildData() {
    Person captainKirk=new Person();
}

Let’s go a little further, and add individuals to our Terminator knowledge base, for a more complex example.

Here’s the code that actually creates the ontology and the three classes (Person, Robot, Terminator) and the relationships between them (a Terminator is a Person and a Robot):

OntologyHelper oh = new OntologyHelper();
OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/terminator.owl");
OWLClass person = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Person");
OWLClass robot = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Robot");
OWLClass terminator = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Terminator");

oh.applyChange(oh.createSubclass(o, terminator, person),
        oh.createSubclass(o, terminator, robot));

Now we’d like to create the three individuals (“Sarah,” a human; “Tank,” a nonhumanoid robot, and “T800,” a Terminator):

OWLIndividual sarah = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#Sarah");
OWLIndividual tank = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#Tank");
OWLIndividual t800 = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#T800");
oh.applyChange(oh.associateIndividualWithClass(o, person, sarah),
        oh.associateIndividualWithClass(o, robot, tank),
        oh.associateIndividualWithClass(o, terminator, t800));

The OWL that represents this ontology looks like this:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/terminator.owl#"
     xml:base="http://autumncode.com/ontologies/terminator.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/terminator.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Person -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Person"></owl:class>

    <!-- Robot -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Robot"></owl:class>

    <!-- Terminator -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Terminator">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/terminator.owl#Person"></rdfs:subclassof>
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/terminator.owl#Robot"></rdfs:subclassof>
    </owl:class>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Individuals
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Sarah -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#Sarah">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Person"></rdf:type>
    </owl:namedindividual>

    <!-- T800 -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#T800">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Terminator"></rdf:type>
    </owl:namedindividual>

    <!-- Tank -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#Tank">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Robot"></rdf:type>
    </owl:namedindividual>
</rdf:rdf>

Here’s where things get interesting. We said that the T800 is a Terminator, not that it’s a Robot or a Person; our test needs to validate its participation in all three classes (Terminator, Robot, Person), even though our OWL doesn’t say that it’s a member of all three classes. This is where the OWLReasoner comes in; it’s able to infer the complete class hierarchy for an individual (or, as we’re using it here, it’s able to find all individuals who are members of a given class.)

This is the beginning of seeing some real power in OWL, even though we’re expressing something very simple: we can describe what things are (a Terminator is a Robot and a Person), express characteristics about individuals (a T800 is a Terminator) and extract meaningful data from the ontology (i.e., that the T800 is both a Robot and a Person, while the Tank is only a Robot.)

Here’s the complete test method, including the extraction of individuals into classes and the conversion of data into comparable sets:

@Test
public void addTerminatorIndividuals() throws Exception {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/terminator.owl");
    OWLClass person = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Person");
    OWLClass robot = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Robot");
    OWLClass terminator = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Terminator");

    oh.applyChange(oh.createSubclass(o, terminator, person),
            oh.createSubclass(o, terminator, robot));

    OWLIndividual sarah = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#Sarah");
    OWLIndividual tank = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#Tank");
    OWLIndividual t800 = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#T800");
    oh.applyChange(oh.associateIndividualWithClass(o, person, sarah),
            oh.associateIndividualWithClass(o, robot, tank),
            oh.associateIndividualWithClass(o, terminator, t800));

    final OWLReasoner reasoner = new JFactFactory().createReasoner(o);
    try (final AutoCloseable ignored = reasoner::dispose) {
        ListMultimap</string><string , String> classInstanceMap = ArrayListMultimap.create();
        o.getClassesInSignature()
                .stream()
                .forEach(clazz ->
                        reasoner.getInstances(clazz, false)
                                .getFlattened()
                                .stream()
                                .forEach(i ->
                                        classInstanceMap.put(clazz.toStringID(), i.toStringID())));

        // should have three classes
        assertEquals(classInstanceMap.keySet().size(), 3);

        Set</string><string> people = new HashSet<>(Arrays.asList(new String[]{sarah.toStringID(), t800.toStringID()}));
        Set</string><string> robots = new HashSet<>(Arrays.asList(new String[]{t800.toStringID(), tank.toStringID()}));
        Set</string><string> terminators = new HashSet<>(Arrays.asList(new String[]{t800.toStringID()}));

        assertEquals(new HashSet<>(classInstanceMap.asMap().get(person.toStringID())), people);
        assertEquals(new HashSet<>(classInstanceMap.asMap().get(robot.toStringID())), robots);
        assertEquals(new HashSet<>(classInstanceMap.asMap().get(terminator.toStringID())), terminators);
    }
}

Adding a Property to a Class

It’s potentially useful to be able to determine class relationships and individual participation in classes, but we’re not actually describing much yet. In our Terminator example, for example, we have one Tank, one T800, one Sarah Connor. Since we know that there are many T800s and many Tanks (and, depending on how much you liked Terminator:Genisys, multiple Sarahs as well) we need to have a way of differentiating individuals beyond their names.

Let’s start by looking at a simpler description of people, and build a genealogy structure. We’ll have three classes (Human, Female, and Male), and add two properties to Person (references to mother and father). Since Mother and Father are derived from Person, we can declare individuals by gender and inherit the properties referring to parentage from Human.

OntologyHelper oh = new OntologyHelper();
OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/genealogy.owl");
OWLClass human = oh.createClass("http://autumncode.com/ontologies/geneaology.owl#Human");
OWLClass male = oh.createClass("http://autumncode.com/ontologies/genealogy.owl#Male");
OWLClass female = oh.createClass("http://autumncode.com/ontologies/genealogy.owl#Female");
OWLObjectProperty hasFather =
        oh.createObjectProperty("http://autumncode.com/ontologies/genealogy.owl#hasFather");
OWLObjectProperty hasMother =
        oh.createObjectProperty("http://autumncode.com/ontologies/genealogy.owl#hasMother");
oh.applyChange(
        oh.createSubclass(o, male, human),
        oh.createSubclass(o, female, human),
        oh.associateObjectPropertyWithClass(o, hasFather, human, male),
        oh.associateObjectPropertyWithClass(o, hasMother, human, female)
);

We need to add associateObjectPropertyWithClass to OntologyHelper:

/**
 * With ontology o, property in refHolder points to a refTo.
 *
 * @param o The ontology reference
 * @param property the data property reference
 * @param refHolder the container of the property
 * @param refTo the class the property points to
 * @return a patch to the ontology
 */
public OWLAxiomChange associateObjectPropertyWithClass(OWLOntology o, 
                                                       OWLObjectProperty property, 
                                                       OWLClass refHolder, 
                                                       OWLClass refTo) {
    OWLClassExpression hasSomeRefTo=df.getOWLObjectSomeValuesFrom(property, refTo);
    OWLSubClassOfAxiom ax=df.getOWLSubClassOfAxiom(refHolder, hasSomeRefTo);
    return new AddAxiom(o, ax);
}

We also want to add another method to the OntologyHelper, to manage a disjunction of classes. Our test doesn’t use it (but our next one will): what it says is that the first class cannot be the second class. Note that a disjunction like this is not reflexive – saying that ‘a’ cannot be a ‘b’ does not imply that ‘b’ cannot be an ‘a’; you have to specify the disjunction both ways if you’re trying to express that relationship. We’ll see how that works soon.

/**
 * With ontology o, an object of class a cannot be simultaneously an object of class b.
 * This is not implied to be an inverse relationship; saying that a cannot be a b does not
 * mean that b cannot be an a.
 * @param o the ontology reference
 * @param a the source of the disjunction
 * @param b the object of the disjunction
 * @return a patch to the ontology
 */
public OWLAxiomChange addDisjointClass(OWLOntology o, OWLClass a, OWLClass b) {
    OWLDisjointClassesAxiom expression=df.getOWLDisjointClassesAxiom(a, b);
    return new AddAxiom(o, expression);
}

The test code generates the following OWL:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/genealogy.owl#"
     xml:base="http://autumncode.com/ontologies/genealogy.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/genealogy.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Object Properties
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- hasFather -->

    <owl:objectproperty rdf:about="http://autumncode.com/ontologies/genealogy.owl#hasFather"></owl:objectproperty>

    <!-- hasMother -->

    <owl:objectproperty rdf:about="http://autumncode.com/ontologies/genealogy.owl#hasMother"></owl:objectproperty>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Female -->

    <owl:class rdf:about="http://autumncode.com/ontologies/genealogy.owl#Female">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/geneaology.owl#Human"></rdfs:subclassof>
    </owl:class>

    <!-- Male -->

    <owl:class rdf:about="http://autumncode.com/ontologies/genealogy.owl#Male">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/geneaology.owl#Human"></rdfs:subclassof>
    </owl:class>

    <!-- Human -->

    <owl:class rdf:about="http://autumncode.com/ontologies/geneaology.owl#Human">
        <rdfs:subclassof>
            <owl:restriction>
                <owl:onproperty rdf:resource="http://autumncode.com/ontologies/genealogy.owl#hasFather"></owl:onproperty>
                <owl:somevaluesfrom rdf:resource="http://autumncode.com/ontologies/genealogy.owl#Male"></owl:somevaluesfrom>
            </owl:restriction>
        </rdfs:subclassof>
        <rdfs:subclassof>
            <owl:restriction>
                <owl:onproperty rdf:resource="http://autumncode.com/ontologies/genealogy.owl#hasMother"></owl:onproperty>
                <owl:somevaluesfrom rdf:resource="http://autumncode.com/ontologies/genealogy.owl#Female"></owl:somevaluesfrom>
            </owl:restriction>
        </rdfs:subclassof>
    </owl:class>
</rdf:rdf>

Now let’s start describing a genealogy. We’re describing a few generations of a family:

  • Thomas, w. Shirley: Michael and Vicki
  • Barry, w. Shirley: Joseph
  • Samuel, w. Mary: Andrew
  • Andrew, w. Vicki: Jonathan

Here’s the code to build the geneaology structure, including the exclusion of Male and Female, along with the actual family tree:

@Test
public void simpleParentage() throws Exception {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/genealogy.owl");
    OWLClass human = oh.createClass("http://autumncode.com/ontologies/geneaology.owl#Human");
    OWLClass male = oh.createClass("http://autumncode.com/ontologies/genealogy.owl#Male");
    OWLClass female = oh.createClass("http://autumncode.com/ontologies/genealogy.owl#Female");
    OWLObjectProperty hasFather =
            oh.createObjectProperty("http://autumncode.com/ontologies/genealogy.owl#hasFather");
    OWLObjectProperty hasMother =
            oh.createObjectProperty("http://autumncode.com/ontologies/genealogy.owl#hasMother");
    oh.applyChange(
            oh.createSubclass(o, male, human),
            oh.createSubclass(o, female, human),
            oh.addDisjointClass(o, female, male),
            oh.addDisjointClass(o, male, female),
            oh.associateObjectPropertyWithClass(o, hasFather, human, male),
            oh.associateObjectPropertyWithClass(o, hasMother, human, female)
    );

    OWLIndividual barry = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#barry");
    OWLIndividual shirley = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#shirley");
    OWLIndividual thomas = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#thomas");
    OWLIndividual michael = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#michael");
    OWLIndividual vicki = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#vicki");
    OWLIndividual joseph = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#joseph");
    OWLIndividual mary = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#mary");
    OWLIndividual samuel = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#samuel");
    OWLIndividual andrew = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#andrew");
    OWLIndividual jonathan = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#jonathan");

    oh.applyChange(
            oh.associateIndividualWithClass(o, male, barry),
            oh.associateIndividualWithClass(o, male, thomas),
            oh.associateIndividualWithClass(o, male, michael),
            oh.associateIndividualWithClass(o, male, joseph),
            oh.associateIndividualWithClass(o, male, samuel),
            oh.associateIndividualWithClass(o, male, andrew),
            oh.associateIndividualWithClass(o, male, jonathan),
            oh.associateIndividualWithClass(o, female, shirley),
            oh.associateIndividualWithClass(o, female, vicki),
            oh.associateIndividualWithClass(o, female, mary),
            oh.addObjectproperty(o, michael, hasMother, shirley),
            oh.addObjectproperty(o, michael, hasFather, thomas),
            oh.addObjectproperty(o, vicki, hasMother, shirley),
            oh.addObjectproperty(o, vicki, hasFather, thomas),
            oh.addObjectproperty(o, joseph, hasMother, shirley),
            oh.addObjectproperty(o, joseph, hasFather, barry),
            oh.addObjectproperty(o, andrew, hasMother, mary),
            oh.addObjectproperty(o, andrew, hasFather, samuel),
            oh.addObjectproperty(o, jonathan, hasMother, vicki),
            oh.addObjectproperty(o, jonathan, hasFather, andrew)
    );

// we'll be adding code here!
}

The OWL this generates is pretty long, unfortunately. But here are some relevant parts from it, the Male and Female class declarations, along with Jonathan’s individual reference:

<!-- Female -->

<owl:class rdf:about="http://autumncode.com/ontologies/genealogy.owl#Female">
    <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/geneaology.owl#Human"></rdfs:subclassof>
    <owl:disjointwith rdf:resource="http://autumncode.com/ontologies/genealogy.owl#Male"></owl:disjointwith>
</owl:class>

<!-- Male -->

<owl:class rdf:about="http://autumncode.com/ontologies/genealogy.owl#Male">
    <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/geneaology.owl#Human"></rdfs:subclassof>
</owl:class>

<!-- jonathan -->

<owl:namedindividual rdf:about="http://autumncode.com/ontologies/genealogy.owl#jonathan">
    <rdf:type rdf:resource="http://autumncode.com/ontologies/genealogy.owl#Male"></rdf:type>
    <hasfather rdf:resource="http://autumncode.com/ontologies/genealogy.owl#andrew"></hasfather>
    <hasmother rdf:resource="http://autumncode.com/ontologies/genealogy.owl#vicki"></hasmother>
</owl:namedindividual>

If we didn’t have that disjunction of Male and Female, there would be nothing in the ontology suggesting that a “has mother of” reference couldn’t point to, say, Samuel. (Let’s keep gender identity simple for the clarity’s sake.)

What happens if we actually add invalid data, though? Let’s try it.

OWLReasoner reasoner = new JFactFactory().createReasoner(o);
try (final AutoCloseable ignored = reasoner::dispose) {
    assertTrue(reasoner.isConsistent());
}
oh.applyChange(oh.associateIndividualWithClass(o, female, jonathan));
reasoner = new JFactFactory().createReasoner(o);
try (final AutoCloseable ignored = reasoner::dispose) {
    assertFalse(reasoner.isConsistent());
}

Here, we’re checking to make sure our ontology is consistent (without error) – then we’re intentionally adding a declaration that Jonathan is female (leaving the reference to Male intact). Therefore, Jonathan is both male and female, which our disjunction says is invalid… and when we check our reasoner again, it shows that the ontology is inconsistent.

Confession

Unfortunately, this is also where the OWLAPI starts to lose me.

The Semantic Web technologies not only provide a way to store information like our genealogy data, but query it; we should be able to examine Joseph and Vicki, for example, and find that they are siblings because one or both of their parents are the same.

Even better, I think we can even declare rules such that a reasoner could assert that they were siblings because of common parentage. Someone online even showed me how such a rule would be written in one of the OWL formats.

However, it’s not clear to me (yet) how to express such a relationship using OWLAPI. I can see a potential path using various axioms chained together; many of the OWLDataFactory methods even accept arrays of arguments, which would seem to enable this.

But I haven’t been able to properly or simply conceptualize how all of this would work in implementation rather than in theory, which speaks of two failures:

  1. The available material has failed to make it simple enough such that it’s accessible to newbies like me, and thus…
  2. I have failed to completely grasp the subject matter enough to write authoritatively about it.

I have no problem acknowledging the second failure.

I’m writing this tutorial not to say “here’s a complete reference” but instead to preserve what I have learned in using OWLAPI incompletely, as a data transferral protocol and as a static data store rather than as a reasoning engine.

If someone more experienced than I would like to collaborate on this to correct both my ignorance and the tutorial’s incompleteness, you’re welcome and invited to participate.

Back to Regular Programming

The reasoning process that shows the invalidity of our data based on the hierarchy shows something important about OWL: it doesn’t throw out our invalid data, but provides a way to tell us that our data is incorrect based on the assertions we’ve made.

In other words, we said that a Human cannot be both Male and Female, and then we told it that a human was, in fact, both. (I’m avoiding cis gender issues, thank you; this is a simple example and isn’t meant to be a treatise on human biology or gender issues.) Because of the assertion of singular gender (and then the use of data that violated the assertion), our ontology is no longer consistent with itself – and the reasoner was able to tell us that.

If we had greater reasoning skills, we could derive actual family models – determine cousins, uncles, aunts, siblings, and more – and have the model tell us of gaps and unexpected relationships, all expressed and generated programmatically.

However, at present most of that is beyond your author’s knowledge and skill, and is beyond this tutorial’s scope.

Review

  • We have created, saved, and loaded ontologies, which represents a system of knowledge
  • We have created classes in an ontology, which represent ways of thinking about items.
    • We have seen that classes can be present in a hierarchy, such that elements of class B are also elements of type A.
    • We have seen that we can also declare classes as optionally being exclusive of one another. Therefore, we can say that if something is of class A, it cannot legitimately be of class B at the same time.
  • We have seen how we can create individuals in an ontology.
    • We have seen how we can add attributes to the individuals such that we define their classes, and we’ve seen that an individual is not limited to one class.
    • We have added references to other individuals to a given individual, and also declare that the references must be drawn from a specific domain. (We can say that a reference, A, must be to members of class B.)
  • We have seen how to use a reasoner to validate that our ontology, the class structure, and the individuals in it are consistent, with no contradictory information.

One thing we’ve not done, though, is add simple data to an individual.

Adding a Property Value to an Individual

When we add a reference to an individual as a property of another individual (“John’s mother is Andrea”), we are adding what OWLAPI calls an “Object Property.” Another type of property is a Data Property, which can be used in almost the exact same way.

Let’s go back to the Terminator. We’ll declare a simple ontology that tracks the T800, and tracks whether it is an enemy or not.

Here’s our ontology declaration as a test. After we see this, we’ll take a look at the OntologyHelper methods that make it work:

@Test
public void addDataToIndividual() throws Exception {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/terminator.owl");
    OWLClass terminator = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Terminator");
    OWLDataProperty killsHumans =
            oh.createDataProperty("http://autumncode.com/ontologies/terminator.owl#killsHumans");
    OWLIndividual t800 = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#t800");
    OWLIndividual pops = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#pops");
    oh.applyChange(
            oh.associateIndividualWithClass(o, terminator, pops),
            oh.associateIndividualWithClass(o, terminator, t800),
            oh.addDataToIndividual(o, t800, killsHumans, true),
            oh.addDataToIndividual(o, pops, killsHumans, false)
    );
}

Now the OntologyHelper methods for creating the data property and adding data to a given individual:

public OWLDataProperty createDataProperty(String iri) {
    return createDataProperty(convertStringToIRI(iri));
}

public OWLDataProperty createDataProperty(IRI iri) {
    return df.getOWLDataProperty(iri);
}

public OWLAxiomChange addDataToIndividual(OWLOntology o, OWLIndividual individual, 
        OWLDataProperty property, String value) {
    OWLLiteral literal = df.getOWLLiteral(value, OWL2Datatype.XSD_STRING);
    return new AddAxiom(o, df.getOWLDataPropertyAssertionAxiom(property, individual, literal));
}

public OWLAxiomChange addDataToIndividual(OWLOntology o, OWLIndividual individual, 
        OWLDataProperty property, boolean value) {
    OWLLiteral literal = df.getOWLLiteral(value);
    return new AddAxiom(o, df.getOWLDataPropertyAssertionAxiom(property, individual, literal));
}

public OWLAxiomChange addDataToIndividual(OWLOntology o, OWLIndividual individual, 
        OWLDataProperty property, int value) {
    OWLLiteral literal = df.getOWLLiteral(value);
    return new AddAxiom(o, df.getOWLDataPropertyAssertionAxiom(property, individual, literal));
}

These are, as you might expect, almost exact analogies for the methods that add object properties to individuals. The main difference here is that we actually deal with data, so we have overloaded argument types for the addDataToIndividual method. (The getOWLDataPropertyAssertionAxiom method can take the actual primitive values and render the proper types.)

So what does the generated data look like? That’s simple enough:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/terminator.owl#"
     xml:base="http://autumncode.com/ontologies/terminator.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:terminator="http://autumncode.com/ontologies/terminator.owl#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/terminator.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Data properties
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- killsHumans -->

    <owl:datatypeproperty rdf:about="http://autumncode.com/ontologies/terminator.owl#killsHumans"></owl:datatypeproperty>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Terminator -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Terminator"></owl:class>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Individuals
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- pops -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#pops">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Terminator"></rdf:type>
        <killshumans rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">false</killshumans>
    </owl:namedindividual>

    <!-- t800 -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#t800">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Terminator"></rdf:type>
        <killshumans rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</killshumans>
    </owl:namedindividual>
</rdf:rdf>

Final Thoughts

What we’ve tried to cover is actually pretty light: “how to add data into an ontology using OWLAPI,” really. There are supposed to be a lot of features provided by OWL that we’ve not even begun to touch; we should, for example, be able to infer which T800s are to be avoided in our last example (based on whether the individual kills humans or not.)

At the very least, we should be able to write a query to determine whether a T800 is a bad guy or not. However, that’s out of the scope of this tutorial at this point; I mostly wanted to preserve what I knew of OWLAPI for others to use.

Java Authors, Learn Java

I was reading a book on Neo4J and encountered this sentence:

What is the secret of Neo4J’s speed?

No, Neo4j developers haven’t invented a superfast algorithm for the military. Nor is Neo4j’s speed a product of the fantastic speed of the technologies it relies on (it’s implemented in Java after all!) (Emphasis mine. Quote is from Manning Publications’ “Neo4j in Action,” by Vokotic, Watt, and others.)

Come on, people! It’s 2015 now! Isn’t it time to put away the myth that Java’s intrinsically slow?