Solutions are not One-Size Fits All

One of the most enjoyable things about being on social media – especially a “new social media” like Mastodon – is seeing all of the energy people represent for solving the problems they see.

It’s also one of the worst things about new environments, because people have a natural myopia in how they see problems that have solutions, and they tend to see their solution – something that works – as the solution.

One of the most enjoyable things about being on social media – especially a “new social media” like Mastodon – is seeing all of the energy people represent for solving the problems they see.

It’s also one of the worst things about new environments, because people have a natural myopia in how they see problems that have solutions, and they tend to see their solution – something that works – as the solution.

When I first started writing, it was for Alan Williamson at Java Developer Journal. Alan – who is a great guy – and I had some arguments about Java’s direction and future, if I recall – Alan was seeing cracks in the edifice, and I thought his writing about those cracks was a little … overblown, shall we say, and he challenged me to put five hundred words in print for it.

I’m a sucker for challenges like that, so naturally I did it – and it turned into a series of op-ed closers for JDJ, and those turned into my joining the editorial staff as an editor for J2EE, and that eventually turned into a position as Editor-in-Chief for JDJ for a short while, until the owner and I had a crisis of intent and I resigned, because he was doing things with my name on the masthead with which I could not have approved, ever.

It was his product, and he had every right to use it how he saw fit – but I couldn’t attach my name to it. It didn’t matter if I agreed with him or not in his feelings about the matter; I just couldn’t be part of it.

But one of my favorite op-ed pieces for JDJ was an editorial called “There Is No Magic Bullet,” if memory serves. (I cannot find a copy of it online at the moment; there’s a site that says they have archives but it’s down as I write this. Joy!)

The summary should be pretty obvious: I was writing that there’s no one-size fits all solution. You get to solve each problem as it comes to you, examining axiomatically. That’s why we write new programs, day after day after week after year, because every problem is different. Even when problems have similar solutions, their starting points are not the same.

People on social media probably remember this – but social media doesn’t give you the room to observe it, so even if they remember that magic bullets aren’t real, they rarely say it out loud.

And since they can’t say it where others can see it (or hear it, I guess), they end up training themselves to stop thinking it, because it’s wasted thought.

So: want to solve carbon crisis? Electric vehicles everywhere! Today!

… Electric vehicles are a magic bullet. What works in a specific capitol city isn’t going to work in the rural areas. Fuel supply for hydrogen vehicles, even electric grid support for EVs… the problems there are going to be the same as they were for mixed-fuel vehicles. Those rural areas would be crippled by the mandates being demanded and dictated.

Here’s the thing: those rural areas, despite representing so few votes and surely being populated by GOP-voting morons, are where the cities get their food from.

It’s important to have data centers with highly paid devops staff, lawyers, accountants, high end coffee shops, clothing stores where the smallest purchase is $400 USD. But if the people who run those places die from starvation, none of those things matter. The lights go off, the cockroaches and the rats take over, the stone we’re on keeps revolving around the sun.

People in the cities who burden the people who feed them are playing stupid games and can only win stupid prizes if they lose.

The magic bullets the literati keeps suggesting, over and over again, without reprieve or reason or limit, would cripple the literati, and kill them in many cases.

That’s why the trucker protest in Canada was such a big deal, after all: Canada messed around and found out, in a very small way, what they were doing, and as a result went martial law, because why bother learning when you have political power and a system that has subjects instead of citizens, right?

Subjects can be dictated to. That’s what happened to the truckers, who had a legitimate protest.

In the United States, so far we’re still citizens. We’re a little harder to dictate to, no matter what our politicians keep trying. We comply, because we’ve had sixty years of our education system demanding compliance, but our core is still steel and what we’re seeing today is a little more spine than we’ve had for a while; compliance isn’t working, and can’t work.

It’s funny, too, because the grievance culture is planting the seeds of its own opposition: if it’s okay to protest in the name of what’s right, and what’s “right” depends on your context, well, that means a farmer actually has the moral impetus to push back when someone says something that endangers their livelihood.

The point is not that our solutions are wrong. Electric vehicles – the example I latched on early in this – are not bad things, at all. I’d prefer a hybrid, myself, but I can see a future in which an EV is right for me… but where I am, right now? An EV would cripple me. I live too far out in the boonies for the grid, and the distances EVs can travel just aren’t good enough yet.

If I lived in a city, my context would be different, and maybe the grid would be powerful enough to charge my EV (and the EVs of all of my neighbors, at the same time), and the distances wouldn’t matter enough; heck, I might just use public transport instead, really.

But … where I live, right now? There is no public transport. I can probably get an Uber or Lyft out here… but realistically, if I can’t drive my own vehicle, I’m begging one of my neighbors, with their gas-fueled cars, for a ride.

Because the solutions I see that mandate EVs are “magic bullets,” and they don’t work for where I live.

And the other thing about magic bullets is that they are, well, magic.

They’re solutions for the general case, often extrapolated from scant data (or no data, in some situations, just hopes and dreams), without testing against the actual real-world situations for which they’re offered.

I’m pretty liberal, when it comes down to it – classically liberal, really, as opposed to what people think of as liberal now. But I’m also fairly conservative in application, because I want change, but I want it to be the right change, and I want it to be advanced through observation.

We have a problem? Okay, what are our options? What do those options mean? What are the long term costs? How long do they take? How long do we have? Is there an emotional investment in a solution? If so, is that wise?

And what I see my fellow humans doing, more than anything else, is screaming “We have a problem! This is the solution!” with no thought, just… react, react, react, with no emotional restraint.

For example, I saw a post about gun control, where the poster was saying “This bad thing with guns happened to me and mine, how can people still have guns when this happened to me?”

I fully empathize. The emotion of being there is a real thing, and I have no intention of invalidate that person’s lived experience.

But rationally… guns were used to attack this person. And their response is to disarm everyone? Is that really the right response?

My thought is that while guns are dangerous (they’re weapons, and represent power, and power is dangerous), the long term response should rationally be that this person’s family should be advocating for a balance of power, not advocating against a balance of power.

You saw this when Trump took office: guns were the worst! But then… the power went to a person we did not and do not like. All of a sudden, guns were part of the resistance that prevented Trump from trying to become a dictator (because of course the military wouldn’t push back, even though it definitely would.)

And during the last six years, the party that was the most anti-gun has been purchasing new guns more than any other sector in the United States.

That’s amusing… and smart.

And it’s a message: that magic bullet (“no guns!”) was a bad solution. That’s how most such advocacy goes, and we should remember it. If it takes longer to type, or longer to say, or even diminishes the impact of your message… choose rationality over extremism.

Always.

Programming is also Teaching

Programming can be thought of as something that takes place as part of interacting with a culture – a culture with two very different audiences. One “audience” is the CPU, and the other audience is made of other programmers – and those other programmers are typically the ones who get ignored, or at least mistreated.

(Note: This article was originally written in 2009 or so, and published on TheServerSide.com, where it’s no longer easily accessible. So I’m posting it here.)

Programming has two goals.

One goal is to do something, of course: calculate an amortization table, present a list of updated feeds, snipe someone on Ebay, or perhaps smash a human player’s army. This goal is focused at a computing environment.

The other goal is – or should be – to transfer knowledge between programmers. This has a lot of benefits: it increases the number of people who understand a given piece of code, it frees a developer to do new things (since he’s no longer the only person who can maintain a given program or process), and it often provides better performance – since showing Deanna your code gives her a chance to point out where your code can improve. Of course, this can be a two-edged sword, because Deanna may have biases that affect the code she writes (and therefore, what you might learn.)

The CPU as an Audience

The CPU is an unforgiving target, but its very nature as a fixed entity (in most cases!) means it has well-known characteristics that can be compensated for and, in some cases, exploited.

The language in use has its own inflexible rules; an example can be seen in C, where the normal “starting point” for a program is a function with the signature “int main(int, char **)“. Of course, depending on your environment, you can circumvent that by writing “int _main(int, char **),” and for some other environments, you’re not expected to write main() at all; you’re expected to write an event handler that the library-supplied main() calls when appropriate.

The point is simple, though: there are rules, and while exceptions exist, one can easily construct a valid decision tree determining exactly what will happen given a program’s object code. Any errors can be resolved by modifying the decision tree to fit what is actually happening (i.e., by correcting errors.)

This is crucially important; flight code, for example, can be and has to be validated by proving out the results of every statement. If the CPU was not strictly deterministic, this would be impossible and we’d all be hoping that our pilot really was paying close attention every moment of every flight.

High-level languages like C (and virtually every other language not called “assembler”) were designed to abstract the actual CPU from the programmer while preserving deterministic properties, with the abstractions growing in scale over time.

Virtual machines bend the rules somewhat, by offering just-in-time compilation; Sun’s VM, for example, examines the most common execution path in a given class and optimizes the resulting machine code to follow that path. If the common path changes later in the run, then it can (and will) recompile the bytecode to run in the most efficient way possible.

Adaptive just-in-time compilation means that what you write isn’t necessarily what executes. While it’s possible to predict exactly what happens in a VM during execution, the number of state transitions is enormous and not normally something your average bear would be willing to undertake.

Adaptive JIT also affects what kind of code you can write to yield efficient runtimes. More on this later; it’s pretty important.

The Programmer as an Audience

The other members of your coding team are the other readers of your code. Rather than reading object code like a CPU does, they read source, and it’s crucially important how you write that source – because you have to not only write it in such a way that the compiler can generate good object code, you have to write it in such a way that humans (including you!) can read it.

To understand how people understand code, we need to understand how people understand.

How People Learn

People tend to learn slowly. This doesn’t mean that they’re stupid; it only means that they’re human.

A paper written in the 1950’s called “The Magical Number Seven, Plus or Minus Two” described how people learn: this is a poor summary, and I recommend that you read the original paper to learn more if you’re interested.

Basically, people learn by integrating chunks of information. A chunk is a unit of information, which can be thought of as mirroring how a neuron works in the human brain.

People can generally integrate seven chunks at a time, plus or minus two depending on various circumstances.

Learning takes place when one takes the chunks one already understands and adds a new chunk of information such that the resulting set of information is cohesive. Thus, the “CPU as an Audience” heading above starts with simple, commonly-understood pieces of information (“CPUs are predictable,” “C programs start at this function”) and refines it to add exceptions and alternatives. For some, the “chunk count” of the paragraphs on C’s starting points make up roughly four chunks – easily integrated by most programmers due to a low chunk count.

If the reader doesn’t know what C is, or doesn’t know what C function declarations mean or look like, those become new chunks to integrate, which may prove a barrier to learning.

Adoption of C++

Another example of chunking in action can be seen in the adoption of C++. Because of its similarity to C – in use by most programmers at the time – it was easily adopted. As it grows in features, adding namespaces, templates, and other changes, adoption is slower now because not only is there more to understand to C++ than there was, but it’s different enough from the “normal” language C that it requires a good bit more integration of new chunks than it did.

The result is that idiomatic C++ – where idioms are “the normal and correct way to express things” – is no longer familiar to C programmers. That’s not a bad thing – unless your goal is having your friendly neighborhood C programmer look at your C++ code.

It’s just harder, because there’s more to it and because it’s more different than it used to be.

Here’s the thing: people don’t really want to learn

This is where things get hard: we have to realize that, on average, people really don’t want to learn all that much. We, as programmers, tend to enjoy learning some things, but in general people don’t want to learn that stop signs are no longer red, but are now flashing white; we want the way things were because they’re familiar. We want control over what we learn.

Our experiences become a chunk to integrate, and since learning is integration of chunks into a cohesive unit, new information can clash with our old information – which is often uncomfortable. Of course, experience can help integrate new information – so the fifth time you see a flashing white stop sign (instead of the octogonal red sign so many are familiar with), you will be more used to it and start seeing it as a stop sign and not something that’s just plain weird.

That said, it’s important to recognize that the larger the difference between what people need to know and what they already know, the harder it will be for them to integrate the new knowledge. If you use closures in front of someone who’s not familiar with anonymous blocks of executable code, you have to be ready for them to mutter that they prefer anonymous implementations of interfaces; named methods are good. It’s what they know. They’re familiar with the syntax. It’s safe.

This is why “Hello, World” is so important for programmers. It allows coders to focus on fairly limited things; most programmers quickly understand the edit/compile/run cycle (which often has a “debug” phase or, lately, a “test” phase, thank the Maven) and “Hello, World” lets them focus on only how a language implements common tasks.

Think about it: you know what “Hello, World” does. It outputs “Hello, World.” Simple, straightforward, to the point. Therefore, you look for the text in the program, and everything else is programming language structure; it gives you an entry point to look for, a mechanism to output text, and some measure of encapsulated routines (assuming the language has such things, and let’s be real: any language you’re working with now has something like them.)

This also gives programmers a means by which to judge how much work they actually have to do to do something really simple. The Windows/C version of “Hello, World,” as recommended by early programming manuals, was gigantic – in simple console-oriented C, it’s four lines or so, and with the Windows API, it turns into nearly seventy. This gives programmers an idea (for better or for worse) what kind of effort that simple tasks will require – even if, as in the case of Windows, a simple message actually has a lot of work to do. (In all fairness, any GUI “Hello World” has this problem.)

So how do you use how people learn to your advantage?

There’s a fairly small set of things to keep in mind when you’re trying to help people learn things quickly.

First, start from a known position – meaning what’s known to the other person. This may involve learning what the other person knows and the terminology they use; they may be using a common pattern, but call it something completely different than you do. Therefore, you need to establish a common ground, using terminology they know or by introducing terminology that will be useful in conversation.

Second, introduce changes slowly. This may be as simple as introducing the use of interfaces instead of concrete classes before diving into full-blown dependency injection and aspect-oriented programming, for example. Small changes give your audience a chance to “catch up” – to integrate what you’re showing them in small, easy-to-digest chunks.

Third, demonstrate idioms. If your language or approach has idiomatic processes that don’t leap off the page (i.e., what most closure-aware people consider idiomatic isn’t idiom at all to people who aren’t familiar with closures), you need to make sure your audience has a chance to see what the “experts” do – because chances are they won’t reach the idioms on their own, no matter how simple they seem to you.

Related to the previous point, try to stay as close to the audience as you can. Idioms are great, but if the person to whom you’re talking has no way to relate to what you’re showing them, there’s no way you’re going to give them anything to retain. Consider the Schwartzian Transform: it creates a map from a set, where the key is the sort field and the value is the element being sorted, sorts based on the keys, then creates a set in the order of the (now-sorted) keys. It uses a function to generate a sortable key in place, which could be a closure.

If your audience doesn’t understand maps well, or the audience is conditioned to think in a certain way (ASM, FORTH, COBOL, maybe even Java?) the Schwartzian Transform can look like black magic; Java programmers have a better chance, but even there it can look very odd because of the mechanism used to generate the sort keys. In Java, it’s not idiomatic, so you’d approach the Schwartzian Transform in small steps to minimize the difficulty integrating for the audience.

Conclusion

Programming is not just telling your CPU what to do, but it’s also teaching your fellow coders how you work, and learning from them how they work. It’s a collaborative effort that can yield excellent programs and efficient programmers, but can also offer confusing user interfaces and discouraged coders.

The difference is in whether the coding team is willing to take the time to code not just for the CPU or for the team, but to both the CPU and the team.

The CPU is usually easier to write for because it’s less judgemental and more predictable, but a CPU will never become great. It’s got a set of capabilities, fixed in place at manufacturing. It will always act the same way.

A team, though… a team of mediocre, inexperienced coders who work together and write for the benefit of the team has the capability to become a great team, and they can take that learning approach to create other great teams.

It all comes down to whether the team sees its work as simply writing code… or writing with the goal of both code and learning.

Installing GraalVM on OSX with SDKMan

Want to install GraalVM on OSX? It’s easy.

First, get SDKMan. Trust me. You want it. Almost as much as brew, if you’re doing anything with the JVM. You’ll want to install bash – via brew – because SDKMan uses bash and the OSX bash shell is badly outdated.

Once you have SDKMan installed and available in your shell, execute the following command:

$ sdk install java 19.3.0.2.r11-grl

If you don’t install it as the default JVM, you can select it as the current JVM with this:

$ sdk default java 19.3.0.2.r11-grl

You can check it with java -version:

$ java -version
openjdk version "11.0.5" 2019-10-15
OpenJDK Runtime Environment GraalVM CE 19.3.0.2 (build 11.0.5+10-jvmci-19.3-b06)
OpenJDK 64-Bit Server VM GraalVM CE 19.3.0.2 (build 11.0.5+10-jvmci-19.3-b06, mixed mode, sharing)

This installs the latest GraalVM installation for Java 11, as of when this was written. Enjoy!

JNDI

Things I have learned recently:

  • People still don’t really get JNDI, and the Java frameworks around today make it easy to ignore, even though it’s still a core technology. It’s not difficult to see how it can be confusing: context in JNDI is everything, and context makes it a challenge to create examples that make sense in the general case.
  • At some point I’d like to learn Go.
  • Not something I’ve learned, but something I’ve been reflecting on this morning because … uh… I have no idea why: I wonder if Adidas shoes are any good, or what they’re good for. I tend to wear Vans Ultrarange shoes these days because they’re light, comfortable, and last forever – I have two working pairs, one for working in the yard and one for wearing – but… Adidas.
  • I really wish officials and announcers wouldn’t show bias during football games. As an FSU guy, I’m really, really, really tired of this – but I’ve been watching other teams’ bowl games (because FSU didn’t go bowling this year, first time in 40+ years) and it happens for them, too, often egregiously. The announcers I don’t care as much about, but the referees… those guys need to be fair, for real. The fact that there’s no urgency in making sure they’re fair is incredibly frustrating and erodes the game. n one game, a team had two defenders ejected for targeting… and the other team had an obvious false start missed, and a few targeting possibilities ignored by the guys in stripes. Let’s just say nope to all that. There needs to be a way for the league to tell these refs what they’re missing, and to either call it fairly or get out. It’s gotten really bad over the last few years, with FSU losing multiple games due to bad or missed calls.

Blogs are so eh

Things I think I’ve learned today:

  • Blogs are so yesterday, man. It’s probably the medium and platform I’ve chosen – I don’t use Medium, for example, although I have an account there – and I don’t publish often enough, or with enough direct focus, to really attract users, because I’m really not trying to build a popular platform. But at the same time, my pride is hurt just a little that traffic is so low.
  • Crawler4j makes it really easy to write a working web spider in Java – or, in my case, Kotlin. Looked into scrapy for a bit, but couldn’t figure out why what should have been a trivial recipe was so hard to find, switched to Kotlin because I really just needed to make progress. Progress was made. Scrapy’s probably fine – maybe the information I was looking for was out there, right in front of my face, and I couldn’t recognize it, but that didn’t help me make the progress I needed.
  • Had a good example of pragmatism in action show up yesterday, too. We issued a pull request for a migration, and one of our developers pointed out a number of situations in the new codebase that could have been problematic. I asked him to demonstrate the errors (or potential errors) with tests so we could validate them, but he wasn’t sure how much effort would be involved in writing those tests… so we progressed with the merge. I think we made the right choice; he’s not wrong in his observations (we might have introduced errors with the new changes) but without validation, we can’t know, and we’d be chasing ghosts. We made notes that the code might be problematic, and we’re going to watch for problems that come from it.
  • Aaron’s Thinking Putty is cooler than it should be.

A Simple Grappa Tutorial

Grappa is a parser library for Java. It’s a fork of Parboiled, which focuses more on Scala as a development environment; Grappa tries to feel more Java-like than Parboiled does.

Grappa’s similar in focus to other libraries like ANTLR and JavaCC; the main advantage to using something like Grappa instead of ANTLR is in the lack of a processing phase. With ANTLR and JavaCC, you have a grammar file, which then generates a lexer and a parser in Java source code. Then you compile that generated source to get your parser.

Grappa (and Parboiled) represent the grammar in actual source code, so there is no external phase; this makes programming with them feel faster. It’s certainly easier to integrate with tooling, since there is no separate tool to invoke apart from the compiler itself.

I’d like to walk through a simple experience of using Grappa, to perhaps help expose how Grappa works.

The Goal

What I want to do is mirror a tutorial I found for ANTLR, “ANTLR 4: using the lexer, parser and listener with example grammar.” It’s an okay tutorial, but the main thing I thought after reading was: “Hmm, ANTLR’s great, everyone uses it, but let’s see if there are alternatives.”

That led me to Parboiled, but some Parboiled users recommended Grappa for Java, so here we are.

That tutorial basically writes a parser for drink orders. We’ll do more.

Our Bartender

Imagine an automated bartender: “What’re ya havin?”

Well… let’s automate that bartender, such that he can parse responses like “A pint of beer.” We can imagine more variations on this, but we’re going to center on one, until we get near the end of the tutorial: we’d also like to allow our bartender to parse orders from people who’re a bit too inebriated to use the introductory article: “glass of wine” (no a) should also be acceptable.

If you’re interested, the code is on GitHub, in my grappaexample repository.

Let’s take a look at our bartender‘s source code, just to set the stage for our grammar. (Actually, we’ll be writing multiple grammars, because we want to take it in small pieces.)

package com.autumncode.bartender;

import com.github.fge.grappa.Grappa;
import com.github.fge.grappa.run.ListeningParseRunner;
import com.github.fge.grappa.run.ParsingResult;

import java.util.Scanner;

public class Bartender {
    public static void main(String[] args) {
        new Bartender().run();
    }

    public void run() {
        final Scanner scanner = new Scanner(System.in);
        boolean done = false;
        do {
            writePrompt();
            String order = scanner.nextLine();
            done = order == null || handleOrder(order);
        } while (!done);
    }

    private void writePrompt() {
        System.out.print("What're ya havin'? ");
        System.out.flush();
    }

    private boolean handleOrder(String order) {
        DrinkOrderParser parser
                = Grappa.createParser(DrinkOrderParser.class);
        ListeningParseRunner<DrinkOrder> runner
                = new ListeningParseRunner<>(parser.DRINKORDER());
        ParsingResult<DrinkOrder> result = runner.run(order);
        DrinkOrder drinkOrder;
        boolean done = false;
        if (result.isSuccess()) {
            drinkOrder = result.getTopStackValue();
            done = drinkOrder.isTerminal();
            if (!done) {
                System.out.printf("Here's your %s of %s. Please drink responsibly!%n",
                        drinkOrder.getVessel().toString().toLowerCase(),
                        drinkOrder.getDescription());
            }
        } else {
            System.out.println("I'm sorry, I don't understand. Try again?");
        }
        return done;
    }
}

This isn’t the world’s greatest command line application, but it serves to get the job done. We don’t have to worry about handleOrder yet – we’ll explain it as we go through generating a grammar.

What it does

Grappa describes a grammar as a set of Rules. A rule can describe a match or an action; both matches and actions return boolean values to indicate success. A rule has failed when processing sees false in its stream.

Let’s generate a very small parser for the sake of example. Our first parser (ArticleParser) is going to do nothing other than detect an article – a word like “a”, “an”, or “the.”

Actually, those are all of the articles in English – there are other forms of articles, but English has those three and no others as examples of articles.

The way you interact with a parser is pretty simple. The grammar itself can extend BaseParser<T>, where T represents the output from the parser; you can use Void to indicate that the parser doesn’t have any output internally.

Therefore, our ArticleParser‘s declaration will be:

public class ArticleParser extends BaseParser<Void> {

We need to add a Rule to our parser, so that we can define an entry point from which the parser should begin. As a first stab, we’ll create a Rule called article(), that tries to match one of our words. With a trie. It’s cute that way.

A trie is a type of radix tree. They tend to be super-fast at certain kinds of classifications. Note that this method name may change in later versions of Grappa, because honestly, the actual search mechanism – the trie – isn’t important for the purpose of invoking the method.

public class ArticleParser extends BaseParser<Void> {
    public Rule article() {
        return trieIgnoreCase("a", "an", "the");
    }
}

This rule should match any variant of “a”, “A”, “an”, “tHe”, or anything like that – while not matching any text that doesn’t somehow fit in as an article. Let’s write a test that demonstrates this, using TestNG so we can use data providers:

package com.autumncode.bartender;

import com.github.fge.grappa.Grappa;
import com.github.fge.grappa.run.ListeningParseRunner;
import com.github.fge.grappa.run.ParsingResult;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import static org.testng.Assert.assertEquals;

public class ArticleTest {
    @DataProvider
    Object[][] articleData() {
        return new Object[][]{
                {"a", true},
                {"an", true},
                {"the", true},
                {"me", false},
                {"THE", true},
                {" a", false},
                {"a ", true},
                {"afoo", true},
        };
    }

    @Test(dataProvider = "articleData")
    public void testOnlyArticle(String article, boolean status) {
        ArticleParser parser = Grappa.createParser(ArticleParser.class);
        testArticleGrammar(article, status, parser.article());
    }
    
    private void testArticleGrammar(String article, boolean status, Rule rule) {
        ListeningParseRunner<Void> runner
                = new ListeningParseRunner<>(rule);
        ParsingResult<Void> articleResult = runner.run(article);
        assertEquals(articleResult.isSuccess(), status,
                "failed check on " + article + ", parse result was "
                        + articleResult + " and expected " + status);
    }
}

So what is happening here?

First, we create a global (for the test) ArticleParser instance through Grappa. Then we create a ListeningParseRunner, with the entry point to the grammar as a parameter; this builds the internal model for the parser (stuff we don’t really care about, but it is memoized, so we can use that code over and over again without incurring the time it takes for processing the grammar at runtime.)

I used a utility method because the form of the tests themselves doesn’t change – only the inputs and the rules being applied. As we add more to our grammar, this will allow us to run similar tests with different inputs, results, and rules.

After we’ve constructed our Parser’s Runner, we do something completely surprising: we run it, with ParsingResult<Void> articleResult = runner.run(article);. Adding in the TestNG data provider, this means we’re calling our parser with every one of those articles as a test, and checking to see if the parser’s validity – shown by articleResult.isSuccess() – matches what we expect.

In most cases, it’s pretty straightforward, since we are indeed passing in valid articles. Where we’re not, the parser says that it’s not a successful parse, such as when we pass it me.

There are three cases where the result might be surprising: " a", "a ", and "afoo". The whitespace is significant, for the parser; for our test, the article with a trailing space passes validation, as does “afoo“, while the article with the leading space does not.

The leading space is easy: our parser doesn’t consume any whitespace, and Grappa assumes whitespace is significant unless told otherwise (by Rules, of course.) So that space doesn’t match our article; it fails to parse, because of that.

However, the trailing space (and "afoo") is a little more odd. What’s happening there is that Grappa is parsing as much of the input as is necessary to fulfill the grammar; once it’s finished doing that, it doesn’t care about anything that follows the grammar. So once it matches the initial text – the “a” – it doesn’t care what the rest of the content is. It’s not significant that "foo" follows the “a“; it matches the “a” and it’s done.

We can fix that, of course, by specifying a better rule – one that includes a terminal condition. That introduces a core concept for Grappa, the “sequence().” (This will factor very heavily into our grammar when we add the ability to say “please” at the end of the tutorial.)

Author’s note: I use “terminal” to mean “ending.” So a terminal anything is meant to indicate finality. However, a “terminal” is also used to describe something that doesn’t delegate to anything else, in Grappa’s terms. So for Grappa, the use of “terminal” might not be the same as my use of the word “terminal”.

Let’s expand our ArticleParser a little more. Now it looks like:

package com.autumncode.bartender;

import com.github.fge.grappa.parsers.BaseParser;
import com.github.fge.grappa.rules.Rule;

public class ArticleParser extends BaseParser<Void> {
    public Rule article() {
        return trieIgnoreCase("a", "an", "the");
    }

    public Rule articleTerminal() {
        return sequence(
                article(),
                EOI
        );
    }
}

What we’ve done is added a new Rule – articleTerminal() – which contains a sequence. That sequence is “an article” — which consumes “a,” “an”, or “the” – and then the special EOI rule, which stands for “end of input.” That means that our simple article grammar won’t consume leading or trailing spaces – the grammar will fail if any content exists besides our article.

We can show that with a new test:

@DataProvider
Object[][] articleTerminalData() {
    return new Object[][]{
         {"a", true},
         {"an", true},
         {"the", true},
         {"me", false},
         {"THE", true},
         {" a", false},
         {"a ", false},
         {"afoo", false},
    };
}

@Test(dataProvider = "articleTerminalData")
public void testArticleTerminal(String article, boolean status) {
    ArticleParser parser = Grappa.createParser(ArticleParser.class);
    testArticleGrammar(article, status, parser.articleTerminal());
}

Now our test performs as we’d expect: it matches the article, and only the article – as soon as it has a single article, it expects the end of input. If it doesn’t find that sequence exactly, it fails to match and isSuccess() returns false.

It’s not really very kind for us to not accept whitespace, though: we probably want to parse " a " as a valid article, but not " a the" or anything like that.

It shouldn’t be very surprising that we can use sequence() for that, too, along with a few new rules from Grappa itself. Here’s our Rule for articles with surrounding whitespace:

public Rule articleWithWhitespace() {
    return sequence(
            zeroOrMore(wsp()),
            article(),
            zeroOrMore(wsp()),
            EOI
    );
}

What we’ve done is added two extra parsing rules, around our article() rule: zeroOrMore(wsp()). The wsp() rule matches whitespace – spaces and tabs, for example. The zeroOrMore() rule seems faintly self-explanatory, but just in case: it says “this rule will match if zero or more of the contained rules match.”

Therefore, our new rule will match however much whitespace we have before an article, then the article, and then any whitespace after the article – but nothing else. That’s fun to say, I guess, but it’s a lot more fun to show:

 @DataProvider
 Object[][] articleWithWhitespaceData() {
     return new Object[][]{
             {"a", true},
             {"a      ", true},
             {"     the", true},
             {"me", false},
             {" THE ", true},
             {" a an the ", false},
             {"afoo", false},
     };
 }

 @Test(dataProvider = "articleWithWhitespaceData")
 public void testArticleWithWhitespace(String article, boolean status) {
     ArticleParser parser = Grappa.createParser(ArticleParser.class);
     testArticleGrammar(article, status, parser.articleWithWhitespace());
 }

Believe it or not, we’re actually most of the way to being able to build our full drink order parser – we need to figure out how to get data from our parser (hint: it’s related to that <Void> in the parser’s declaration), but that’s actually the greatest burden we have remaining.

One other thing that’s worth noting as we go: our code so far actually runs twenty-three tests. On my development platform, it takes 64 milliseconds to run all twenty-three – the first one takes 49, where it’s building the parser for the first time. The rest take somewhere between 0 and 4 milliseconds – and I’m pretty sure that 4ms reading is an outlier. Our grammar isn’t complex, and I imagine we could write something without a grammar that would be faster – maybe HashSet<String>.contains(input.trim()) – but we’re about to step into territory that would end up being a lot less maintainable as our grammar grows.

I ran the tests one hundred times each and the same pattern showed up: every now and then you’d see a test that ran slower. My initial guess is that this is related to garbage collection or some other housekeeping chore on the JVM’s part, but I haven’t verified it.)

Getting Data out of our Parser

Grappa uses an internal stack of values to track and expose information. We can tell it the type of the stack values – and in fact, we already did so in our ArticleParser. It’s the <Void> we used – that says that we have a stack of Void values, which is a cute way of saying “no value at all.” (If you remember carefully, we pointed that out when we first started describing the ArticleParser – this is where that information is useful!)

Therefore, all we need to do is expose a type, and then manipulate that stack of values. We do so with a special type of Rule, a function that returns a boolean that indicates whether the Rule was successful.

Our goal with the article is to parse drink orders, of the general form of “a VESSEL of DRINK.” We already worked on a parser that demonstrates parsing the “a” there – it’s time to think about parsing the next term, which we’ll call a “vessel.” Or, since we’re using Java, a Vessel – which we’ll encapsulate in an Enum so we can easily add Vessels.

The Vessel itself is pretty simple:

package com.autumncode.bartender;

public enum Vessel {
    PINT,
    BOWL,
    GLASS,
    CUP,
    PITCHER,
    MAGNUM,
    BOTTLE,
    SPOON
}

What we want to do is create a parser such that we can hand it “a glass” and get Vessel.GLASS out of it.

Given that we’ve said that a parser can be constructed with the “return type”, that tells us that our VesselParser wants to extend BaseParser<Vessel>, and so it does. In fact, our VesselParser isn’t even very surprising, given what we’ve learned from our ArticleParser:

public class VesselParser extends BaseParser<Vessel> {
    static final Collection<String> vessels = Stream
            .of(Vessel.values())
            .map(Enum::name)
            .collect(Collectors.toList());

    public Rule vessel() {
        return trieIgnoreCase(vessels);
    }
}

What does this do? Well, most of it is building a List of the Vessel values, by extracting the values from Vessel. It’s marked static final so it will only initialize that List once; the Rule (vessel()) simply uses the exact same technique we used in parsing articles. It doesn’t actually do anything with the match, though. It would simply fail if it was handed text that did not match a Vessel type.

Incidentally, the Java Language Specification suggests the order of static final, in section 8.3.1, Field Modifiers.

Let’s try it out, using the same sort of generalized pattern we saw in our ArticleParser tests. (We’re going to add a new generalized test method, when we add in the type that should be returned, but this will do for now.)

public class VesselTest {
    private void testGrammar(String corpus, boolean status, Rule rule) {
        ListeningParseRunner<Vessel> runner
                = new ListeningParseRunner<>(rule);
        ParsingResult<Vessel> result = runner.run(corpus);
        assertEquals(result.isSuccess(), status,
                "failed check on " + corpus + ", parse result was "
                        + result + " and expected " + status);
    }
    
    @DataProvider
    Object[][] simpleVesselParseData() {
        return new Object[][]{
                {Vessel.PINT.name(), true,},
                {Vessel.BOWL.name(), true,},
                {Vessel.GLASS.name(), true,},
                {Vessel.CUP.name(), true,},
                {Vessel.PITCHER.name(), true,},
                {Vessel.MAGNUM.name(), true,},
                {Vessel.BOTTLE.name(), true,},
                {Vessel.SPOON.name(), true,},
                {"hatful", false,},
        };
    }

    @Test(dataProvider = "simpleVesselParseData")
    public void testSimpleVesselParse(String corpus, boolean valid) {
        VesselParser parser = Grappa.createParser(VesselParser.class);
        testGrammar(corpus, valid, parser.vessel());
    }
}

The idiom that Grappa uses – and that I will use, in any event – involves the use of the push() and match() methods.

Basically, when we match a Vessel – using that handy vessel() rule – what we will do is push() a value corresponding to the Vessel whose name corresponds to the Rule we just wrote. We can get the text of the Rule we just matched, with the rather-handily-named match() method.

It’s actually simpler to program than it is to describe:

 // in VesselParser.java
 public Rule VESSEL() {
     return sequence(
             vessel(), 
             push(Vessel.valueOf(match().toUpperCase()))
     );
 }

This is a rule that encapsulates the matching of the vessel name – thus, vessel() – and then, assuming the match is found, calls push() with the Vessel whose text is held in match().

That’s fine to say, but much better to show. Here’s a test of our VESSEL() rule, following the same sort of generalized pattern we saw for parsing articles, along with a new generalized test runner that examines the returned value if the input data is valid according to the grammar:

private void testGrammarResult(String corpus, boolean status, Vessel value, Rule rule) {
    ListeningParseRunner<Vessel> runner
            = new ListeningParseRunner<>(rule);
    ParsingResult<Vessel> result = runner.run(corpus);
    assertEquals(result.isSuccess(), status,
            "failed check on " + corpus + ", parse result was "
                    + result + " and expected " + status);
    if(result.isSuccess()) {
        assertEquals(result.getTopStackValue(), value);
    }
}

@DataProvider
Object[][] simpleVesselReturnData() {
    return new Object[][]{
            {Vessel.PINT.name(), true, Vessel.PINT},
            {Vessel.BOWL.name(), true, Vessel.BOWL},
            {Vessel.GLASS.name(), true, Vessel.GLASS},
            {Vessel.CUP.name(), true, Vessel.CUP},
            {Vessel.PITCHER.name(), true, Vessel.PITCHER},
            {Vessel.MAGNUM.name(), true, Vessel.MAGNUM},
            {Vessel.BOTTLE.name(), true, Vessel.BOTTLE},
            {Vessel.SPOON.name(), true, Vessel.SPOON},
            {"hatful", false, null},
    };
}

@Test(dataProvider = "simpleVesselReturnData")
public void testSimpleVesselResult(String corpus, boolean valid, Vessel value) {
    VesselParser parser = Grappa.createParser(VesselParser.class);
    testGrammarResult(corpus, valid, value, parser.VESSEL());
}

Note that we’re testing with a Rule of parser.VESSEL() – the one that simply matches a vessel name is named parser.vessel(), and the one that updates the parser’s value stack is parser.VESSEL().

This is a personal idiom. I reserve the right to change my mind if sanity demands it. In fact, I predict that I will have done just this by the end of this article.

So what this does is very similar to our prior test – except it also tests the value on the parser’s stack (accessed via
result.getTopStackValue()) against the value that our DataProvider says should be returned, as long as the parse was expected to be valid.

All this is well and good – we can hand it "glass" and get Vessel.GLASS — but we haven’t fulfilled everything we want out of a VesselParser. We want to be able to ask for " a pint " — note the whitespace! — and get Vessel.PINT. We need to add in our article parsing.

First, let’s write our tests, so we know when we’re done:

@DataProvider
Object[][] articleVesselReturnData() {
    return new Object[][]{
            {"a pint", true, Vessel.PINT},
            {"the bowl", true, Vessel.BOWL},
            {"  an GLASS", true, Vessel.GLASS},
            {"a     cup", true, Vessel.CUP},
            {"the pitcher    ", true, Vessel.PITCHER},
            {" a an magnum", false, null},
            {"bottle", true, Vessel.BOTTLE},
            {"spoon   ", true, Vessel.SPOON},
            {"spoon  bottle ", false, null},
            {"hatful", false, null},
            {"the stein", false, null},
    };
}

@Test(dataProvider = "articleVesselReturnData")
public void testArticleVesselResult(String corpus, boolean valid, Vessel value) {
    VesselParser parser = Grappa.createParser(VesselParser.class);
    testGrammarResult(corpus, valid, value, parser.ARTICLEVESSEL());
}

Our tests should be able to ignore the leading article and any whitespace. Any wrongful formation (as you see in " a an magnum") should fail, and any vessel type that isn’t valid ("hatful" and "the stein") should fail.

Our Rule is going to look like a monster, because it has to handle a set of possibilities, but it’s actually pretty simple. Let’s take a look, then walk through the grammar:

public Rule article() {
    return trieIgnoreCase("a", "an", "the");
}

public Rule ARTICLEVESSEL() {
    return sequence(
            zeroOrMore(wsp()),
            optional(
                    sequence(
                            article(),
                            oneOrMore(wsp())
                    )),
            VESSEL(),
            zeroOrMore(wsp()),
            EOI);
}

First, we added our article() Rule, from our ArticleParser. It might be tempting to copy all the whitespace handling from that parser as well, but we shouldn’t – all we care about is the articles themselves (“lexemes,” if we’re trying to look all nerdy.)

It’s the ARTICLEVESSEL() Rule that’s fascinating. What that is describing is a sequence, consisting of:

  • Perhaps some whitespace, expressed as zeroOrMore(wsp()).
  • An optional sequence, consisting of:
    • An article.
    • At least one whitespace character.
  • A vessel (which, since we’re using VESSEL(), means the parser’s stack is updated.)
  • Perhaps some whitespace.
  • The end of input.

Any input that doesn’t follow that exact sequence ("spoon bottle", for example) fails.

Believe it or not, we’re now very much on the downhill slide for our bar-tending program.

We need to add a preposition (“of”) and then generalized text handling for the type of drink, and we need to add the container type – but of this, only the type of drink will add any actual complexity to our parser.

Rounding out the Bartender

Our VesselParser is actually a pretty good model for the DrinkOrderParser that our Bartender will use. What we need to add is matching for two extra tokens: “of,” as mentioned, and then a generalized description of a drink.

We’re not going to be picky about the description; we could validate it (just like we’ve done for Vessel) but there are actual better lessons to be gained by leaving it free-form.

Let’s take a look at the operative part of Bartender again, which will set the stage for the full parser.

DrinkOrderParser parser
        = Grappa.createParser(DrinkOrderParser.class);
ListeningParseRunner<DrinkOrder> runner
        = new ListeningParseRunner<>(parser.DRINKORDER());
ParsingResult<DrinkOrder> result = runner.run(order);
DrinkOrder drinkOrder;
boolean done = false;
if (result.isSuccess()) {
    drinkOrder = result.getTopStackValue();
    done = drinkOrder.isTerminal();
    if (!done) {
        System.out.printf("Here's your %s of %s. Please drink responsibly!%n",
                drinkOrder.getVessel().toString().toLowerCase(),
                drinkOrder.getDescription());
    }
} else {
    System.out.println("I'm sorry, I don't understand. Try again?");
}
return done;

The very first thing we’re going to do is create a DrinkOrder class, that contains the information about our drink order.

public class DrinkOrder {
    Vessel vessel;
    String description;
    boolean terminal;
}

I’m actually using Lombok in the project (and the @Data annotation) but for the sake of example, imagine that we have the standard boilerplate accessors and mutators for each of those attributes. Thus, we can call setDescription(), et al, even though we’re not showing that code. We’re also going to have equals() and hashCode() created (via Lombok), as well as a no-argument constructor and another constructor for all properties.

In other words, it’s a fairly standard Javabean, but we’re not showing all of the boilerplate code – and thanks to Lombok, we don’t even need the boilerplate code. Lombok makes it for us.

If you do need the code for equals(), hashCode(), toString(), or the mutators, accessors, and constructors shown, you may be reading the wrong tutorial. How did you make it this far?

Before we dig into the parser – which has only one really interesting addition to the things we’ve seen so far – let’s take a look at our test. This is the full test, so it’s longer than some of our code has been. The DrinkOrderParser will be much longer.

public class DrinkOrderParserTest {
    private void testGrammarResult(String corpus, boolean status, DrinkOrder value, Rule rule) {
        ListeningParseRunner<DrinkOrder> runner
                = new ListeningParseRunner<>(rule);
        ParsingResult<DrinkOrder> result = runner.run(corpus);
        assertEquals(result.isSuccess(), status,
                "failed check on " + corpus + ", parse result was "
                        + result + " and expected " + status);
        if (result.isSuccess()) {
            assertEquals(result.getTopStackValue(), value);
        }
    }
    
    @DataProvider
    public Object[][] drinkOrderProvider() {
        return new Object[][]{
                {"a glass of water", true, new DrinkOrder(Vessel.GLASS, "water", false)},
                {"a pitcher of old 66", true, new DrinkOrder(Vessel.PITCHER, "old 66", false)},
                {"a    pint  of duck   vomit   ", true, new DrinkOrder(Vessel.PINT, "duck vomit", false)},
                {"a shoeful of motor oil", false, null},
                {"nothing", true, new DrinkOrder(null, null, true)},
        };
    }
    
    @Test(dataProvider = "drinkOrderProvider")
    public void testDrinkOrderParser(String corpus, boolean valid, DrinkOrder result) {
        DrinkOrderParser parser = Grappa.createParser(DrinkOrderParser.class);
        testGrammarResult(corpus, valid, result, parser.DRINKORDER());
    }
}

Most of this should be fairly simple; it’s the same pattern we’ve seen used in our other tests.

I don’t actually drink, myself, so… I keep imagining some biker bar in the American southwest selling a beer called “Old 66,” and in my imagination “duck vomit” is the kind of wine that comes in a resealable plastic bag.

A lot of the DrinkOrderParser will be very familiar. Let’s dive in and take a look at all of it and then break it down:

public class DrinkOrderParser extends BaseParser<DrinkOrder> {
    Collection<String> vessels = Stream
            .of(Vessel.values())
            .map(Enum::name)
            .collect(Collectors.toList());

    public boolean assignDrink() {
        peek().setDescription(match().toLowerCase().replaceAll("\\s+", " "));
        return true;
    }

    public boolean assignVessel() {
        peek().setVessel(Vessel.valueOf(match().toUpperCase()));
        return true;
    }

    public boolean setTerminal() {
        peek().setTerminal(true);
        return true;
    }

    public Rule ARTICLE() {
        return trieIgnoreCase("a", "an", "the");
    }

    public Rule OF() {
        return ignoreCase("of");
    }

    public Rule NOTHING() {
        return sequence(
                trieIgnoreCase("nothing", "nada", "zilch", "done"),
                EOI,
                setTerminal()
        );
    }

    public Rule VESSEL() {
        return sequence(
                trieIgnoreCase(vessels),
                assignVessel()
        );
    }

    public Rule DRINK() {
        return sequence(
                join(oneOrMore(firstOf(alpha(), digit())))
                        .using(oneOrMore(wsp()))
                        .min(1),
                assignDrink()
        );
    }

    public Rule DRINKORDER() {
        return sequence(
                push(new DrinkOrder()),
                zeroOrMore(wsp()),
                firstOf(
                        NOTHING(),
                        sequence(
                                optional
                                        ARTICLE(),
                                        oneOrMore(wsp())
                                ),
                                VESSEL(),
                                oneOrMore(wsp()),
                                OF(),
                                oneOrMore(wsp()),
                                DRINK()
                        )
                ),
                zeroOrMore(wsp()),
                EOI
        );
    }
}

We’re reusing the mechanism for creating a collection of Vessel references. We’re also repeating the Rule used to detect an article.

We’re adding a Rule for the detection of the preposition “of”, which is a mandatory element in our grammar. We use ignoreCase(), because we respect the rights of drunkards to shout at their barkeeps:

public Rule OF() {
    return ignoreCase("of");
}

Note how I’m skirting my own rule about naming. I said I was reserving the right to change my mind, and apparently I’ve done so even while writing this article. According to the naming convention I described earlier, it should be of() and not OF() because it doesn’t alter the parser’s stack. The same rule applies to ARTICLE(). It’s my content, I’ll write it how I want to unless I decide to fix it later.

I’m also creating methods to mutate the parser state:

protected boolean assignDrink() {
    peek().setDescription(match().toLowerCase().replaceAll("\\s+", " "));
    return true;
}

protected boolean assignVessel() {
    peek().setVessel(Vessel.valueOf(match().toUpperCase()));
    return true;
}

protected boolean setTerminal() {
    peek().setTerminal(true);
    return true;
}

These are a little interesting, in that they use peek(). The actual base rule in our grammar is DRINKORDER(), which immediately pushes a DrinkOrder reference onto the parser stack. That means that there is a DrinkOrder that other rules can modify at will; peek() gives us that reference. Since it’s typed via Java’s generics, we can call any method that DrinkOrder exposes.

These utilities all return true. None of them can fail, because they won’t be called unless a prior rule has matched; these methods are for convenience only. Actually, let’s show the NOTHING() and VESSEL() rules, so we can see how these methods are invoked:

public Rule NOTHING() {
    return sequence(
            trieIgnoreCase("nothing", "nada", "zilch", "done"),
            EOI,
            setTerminal(),
    );
}

public Rule VESSEL() {
    return sequence(
            trieIgnoreCase(vessels),
            assignVessel()
    );
}

This leaves two new rules to explain: DRINK() and DRINKORDER(). Here’s DRINK():

public Rule DRINK() {
    return sequence(
            join(oneOrMore(firstOf(alpha(), digit())))
                    .using(oneOrMore(wsp()))
                    .min(1),
            assignDrink()
    );
}

This rule basically builds a list of words. It’s a sequence of operations; the first builds the match of the words, and the second operation assigns the matched content to the DrinkOrder‘s description.

The match of the words is really just a sequence of alphanumeric characters. It requires at least one such sequence to exist, but will consume as many as there are in the input.

Now for the Rule that does most of the work: DRINKORDER().

public Rule DRINKORDER() {
    return sequence(
            push(new DrinkOrder()),
            zeroOrMore(wsp()),
            firstOf(
                    NOTHING(),
                    sequence(
                            optional(sequence(
                                    ARTICLE(),
                                    oneOrMore(wsp())
                            )),
                            VESSEL(),
                            oneOrMore(wsp()),
                            OF(),
                            oneOrMore(wsp()),
                            DRINK()
                    )
            ),
            zeroOrMore(wsp()),
            EOI
    );
}

Again, we have a sequence. It works something like this:

  • First, push a new DrinkOrder onto the stack, to keep track of our order’s state.
  • Consume any leading whitespace.
  • Either:
    • Check for the terminal condition (“nothing”, for example), or
    • Check for a new sequence, of the following form:
      • An optional sequence:
        • An article
        • Any trailing whitespace after the article
      • A vessel
      • One or more whitespace characters
      • The lexeme matching “of”
      • One or more whitespace characters
      • The drink description
  • Any trailing whitespace
  • The end of input

We’ve basically built most of this through our parsers, bit by bit; armed with the ability to peek() and push(), we can build some incredibly flexible parsers with fairly simple code.

Adding Politesse

All this has been great, so far. We can actually “order” from a Bartender, giving us this scintillating conversation:

$ java -jar bartender-1.0-SNAPSHOT.jar
What're ya havin'? a glass of water
Here's your glass of water. Please drink responsibly!
What're ya havin'? a toeful of shoe polish
I'm sorry, I don't understand. Try again?
What're ya havin'? a pint of indigo ink
Here's your pint of indigo ink. Please drink responsibly!
What're ya havin'? nothing
$

The only problem is that it’s not very humane or polite. We can’t say “please,” we can’t be very flexible. What we need is to add politesse to our grammar.

What we really want to do is modify our DrinkOrderParser so that we can ask for “a cup of pinot noir, 1986 vintage, please?” It should be able to tell us that we’ve ordered “pinot noir, 1986” and not “pinot noir, 1986, please?” — that’d be silly.

However, we need to alter our grammar in some core ways – particularly in how we match the drink names — and use a new Rule, testNot. First, though, let’s take a look at our test code, because that’s going to give us a workable indicator of whether we’ve succeeded or not.

public class PoliteDrinkOrderParserTest {
    private void testGrammarResult(String corpus, boolean status, DrinkOrder value, Rule rule) {
        ListeningParseRunner<DrinkOrder> runner
                = new ListeningParseRunner<>(rule);
        ParsingResult<DrinkOrder> result = runner.run(corpus);
        assertEquals(result.isSuccess(), status,
                "failed check on " + corpus + ", parse result was "
                        + result + " and expected " + status);
        if (result.isSuccess()) {
            assertEquals(result.getTopStackValue(), value);
        }
    }

    @DataProvider
    public Object[][] drinkOrderProvider() {
        return new Object[][]{
                {"a glass of water please", true, new DrinkOrder(Vessel.GLASS, "water", false)},
                {"a pitcher of old 66, please", true, new DrinkOrder(Vessel.PITCHER, "old 66", false)},
                {"a pitcher of old 66", true, new DrinkOrder(Vessel.PITCHER, "old 66", false)},
                {"a glass of pinot noir, 1986", true, new DrinkOrder(Vessel.GLASS, "pinot noir, 1986", false)},
                {"a glass of pinot noir, 1986, ok?", true, new DrinkOrder(Vessel.GLASS, "pinot noir, 1986", false)},
                {"glass of pinot noir, 1986, ok?", true, new DrinkOrder(Vessel.GLASS, "pinot noir, 1986", false)},
                {"cup , pinot noir, 1986 vintage, ok?", true, new DrinkOrder(Vessel.CUP, "pinot noir, 1986 vintage", false)},
                {"cup,pinot noir, 1986,ok!", true, new DrinkOrder(Vessel.CUP, "pinot noir, 1986", false)},
                {"a    pint  of duck   vomit   ", true, new DrinkOrder(Vessel.PINT, "duck vomit", false)},
                {"a    pint  of duck   vomit  , please ", true, new DrinkOrder(Vessel.PINT, "duck vomit", false)},
                {" pint , duck   vomit please  ", true, new DrinkOrder(Vessel.PINT, "duck vomit", false)},
                {"a shoeful of motor oil", false, null},
                {"nothing", true, new DrinkOrder(null, null, true)},
        };
    }

    @Test(dataProvider = "drinkOrderProvider")
    public void testDrinkOrderParser(String corpus, boolean valid, DrinkOrder result) {
        PoliteDrinkOrderParser parser = Grappa.createParser(PoliteDrinkOrderParser.class);
        testGrammarResult(corpus, valid, result, parser.ORDER());
    }
}

You’ll notice that we’ve changed some other things, too. Our original grammar was pretty simple in formal-ish terms:

DRINKORDER ::= nothing | article? vessel `of` drink
article ::= a | an | the
vessel ::= pint | bowl | glass | cup | pitcher | magnum | bottle | spoon
drink ::= [a-zA-Z0-9]*
nothing ::= nothing | nada | zilch | done

Note that this isn’t an actual formal grammar – I’m cheating. It just looks as if it might be something near formal, with a particular failure in the “drink” term.

Our new one seems to be more flexible:

DRINKORDER ::= (nothing | article? vessel of drink) interjection? eos?
article ::= a | an | the
vessel ::= pint | bowl | glass | cup | pitcher | magnum | bottle | spoon
of ::= , | of
drink ::= !interjection
interjection ::= ','? please | ok | okay | pls | yo 
eos ::= '.' | '!' | '?'
nothing ::= nothing | nada | zilch | done

Here, we’re no more actually formal than we were – the “!interjection” is trying to say that a drink is everything where a drink would be appropriate, up to the interjection.

I don’t care for Backus-Naur form, and I’m using something that looks like it because I thought it might help. Your mileage may vary as to whether I was correct or not.

At any rate, our new grammar should allow us to say “please” and eliminate the unnecessary “of” – although I’m not willing to concede that a bartender should respond well to “pint beer.” “pint, beer.” I can accept – but that comma is significant, by golly.

I’ll leave it as an exercise for the reader to make the comma not necessary – and to write the test that proves it.

However, one thing remains: we haven’t seen our grammar. Most of it’s the same: the article, the vessel, and the action rules (the things that construct our returned drink order) haven’t changed, but we have a slew of new rules (for the end of the sentence and the interjection) and we’ve modified some old ones (drink, and of). Let’s take a look at the changes held in PoliteDrinkOrderParser:

public Rule OF() {
    return firstOf(
            sequence(
                    zeroOrMore(wsp()),
                    COMMA(),
                    zeroOrMore(wsp())
            ),
            sequence(
                    oneOrMore(wsp()),
                    ignoreCase("of"),
                    oneOrMore(wsp())
            )
    );
}

public Rule DRINK() {
    return sequence(
            oneOrMore(
                    testNot(INTERJECTION()),
                    ANY
            ),
            assignDrink());
}

public Rule DRINKORDER() {
    return sequence(
            optional(sequence(
                    ARTICLE(),
                    oneOrMore(wsp())
            )),
            VESSEL(),
            OF(),
            DRINK()
    );
}

public Rule COMMA() {
    return ch(',');
}

public Rule INTERJECTION() {
    return sequence(
            zeroOrMore(wsp()),
            optional(COMMA()),
            zeroOrMore(wsp()),
            trieIgnoreCase("please", "pls", "okay", "yo", "ok"),
            TERMINAL()
    );
}

public Rule EOS() {
    return anyOf(".!?");
}

public Rule TERMINAL() {
    return sequence(zeroOrMore(wsp()),
            optional(EOS()),
            zeroOrMore(wsp()),
            EOI
    );
}

public Rule ORDER() {
    return sequence(
            push(new DrinkOrder()),
            zeroOrMore(wsp()),
            firstOf(DRINKORDER(), NOTHING()),
            optional(INTERJECTION()),
            TERMINAL()
    );
}

We’ve had to move whitespace handling around a little, too, because of the use of OF() to serve as a connector rather than the simple word “of.”

OF() now has to serve as a syntax rule for a single comma – with no whitespace – as you’d see in the string “pint,beer“. It also has to handle whitespace – as you’d find in pint , beer.

However, it needs to mandate whitespace for the actual word of – because pintofbeer doesn’t work.

Another exercise for the reader: fix OF() to handle “pint, of beer“.

DRINK() has a new sequenceoneOrMore(testNot(INTERJECTION()), ANY).

Pay attention to this.

This means to match everything (as per the ANY) that does not match the INTERJECTION() rule. The sequence order is important – it tries to match the rules in order, so it checks the tokens (by looking ahead) against INTERJECTION() first, and failing that check (and therefore succeeding in the match – remember, we’re looking for something that is not an INTERJECTION()) it checks to see if the text matches ANY.

Given that ANY matches anything, it succeeds – as long as the tokens are not tokens that match the INTERJECTION() rule.

And what does INTERJECTION() look like? Well, it’s a normal rule – this is where Grappa really shines. Our INTERJECTION() has optional whitespace and punctuation, and it’s own case insensitive matching:

public Rule INTERJECTION() {
    return sequence(
            zeroOrMore(wsp()),
            optional(COMMA()),
            zeroOrMore(wsp()),
            trieIgnoreCase("please", "pls", "okay", "yo", "ok"),
            TERMINAL()
    );
}

It also has the terminal condition for the order, because something might look like an interjection but wouldn’t be. Consider this input: “glass,water,please fine.” The ,please matches an INTERJECTION(), but because the INTERJECTION() includes the TERMINAL() rule – which means “optional whitespace, an optional end-of-sentence, optional whitespace, and then a definite end-of-input” – “,please fine” fails the INTERJECTION() match, and falls back to ANY.

EOI can match legally several times. That’s why we can match it in our INTERJECTION() rule while still having it match the end of our ORDER() rule. The nature of TERMINAL() – being a series of optional elements – means that if it’s matched as part of INTERJECTION() it won’t match at the end of ORDER(). Such is life.

We can also order something like this: “glass,water, please ok?” — and our drink would be a glass of “water, please” because “ok” would match the INTERJECTION() rule.

Our bartender’s a great guy, but he likes his grammar.

Our PoliteBartender class is different from our Bartender only in the Parser it uses and the originating Rule – and, of course, in the flexibility of the orders it accepts.

$ java -cp bartender-1.0-SNAPSHOT.jar com.autumncode.bartender.PoliteBartender
What're ya havin'? a glass of water
Here's your glass of water. Please drink responsibly!
What're ya havin'? a toeful of shoe polish
I'm sorry, I don't understand. Try again?
What're ya havin'? a pint of indigo ink, please
Here's your pint of indigo ink. Please drink responsibly!
What're ya havin'? A SPOON OF DOM PERIGNON, 1986, OK?
Here's your spoon of dom perignon, 1986. Please drink responsibly!
What're ya havin'? magnum,water,pls, please
Here's your magnum of water,pls. Please drink responsibly!
What're ya havin'? nothing
$

Colophon

By the way, much appreciation goes to the following individuals, who helped me write this in various important ways, and in no particular order:

  • Francis Galiegue, who helped by reviewing the text, by pointing out various errors in my grammars, and by writing Grappa in the first place
  • Chris Brenton, who reviewed (a lot!) and helped me tune the messaging
  • Andreas Kirschbaum, who also reviewed quite a bit for the article, especially in early form

I still prefer Java.

I still prefer Java over other languages.

The Background: Javabot

I’m a fairly regular contributor to javabot, an IRC bot written for the Freenode ##java channel. I don’t know that I’d be considered a major contributor (I’m not listed in the credits, for example, so maybe I do know – and I’m not a major contributor) but I have a few solved issues to my credit…

But I think my contributions to the Javabot project are nearing an end, and the reason is really rather sad: Kotlin. Javabot recently underwent a conversion from Java to Kotlin.

Kotlin, as a language, looks really neat. It has features that I think Java could use; it’s probably a viable alternative to Java and, to some degree, Scala. I know a number of programmers who are using Kotlin, and what they describe sounds good.

But I don’t know Kotlin. I know a lot of coders, and of those, very few programmers who use Kotlin – and even though Kotlin looks neat, I can’t use it apart from cottage projects like javabot. The ecosystem around Kotlin is just too small.

That means that any contribution I had for Javabot would be severely limited by my own skill level in Kotlin – which is not experienced enough to even describe as “kotlin newbie” but is still stuck at “laughable.” Any code I wrote for javabot would be more of a burden than a benefit, since the code would have to be severely vetted.

I wouldn’t want a contribution to be a burden – any contribution of mine would be intended to make the world better, not harder. So I think that my participation in the project is limited by my own good intentions.

Prefer Java. Really.

The result, then, is that I’d suggest that coders use Java, even though it’s not as cool or as full-featured as some other languages might be. I’d be willing to consider projects in Scala, which has a viable ecosystem at this point (and has no difficulty leveraging Java’s ecosystem), and even Groovy (which leverages the Java ecosystem more than its own, as far as I can tell).

That’s not to say that Kotlin doesn’t have viability in its future, nor is it to say that I have no interest in learning Kotlin – it’s just a recognition that a new language is a burden to contributors, and because I don’t want to shoulder nor cause that burden, I’d suggest sticking with a language that’s common for the programming environment in which you live.

Addendum

I don’t blame the author for moving to Kotlin – javabot is a cottage project, really, and he actually did ask contributors their opinions before migrating. I voted for the migration. I just didn’t anticipate how I feel about it today as a result.

Test-driven development can be great.

Test-driven development is, loosely defined, a practice in which tests are written before anything else, without regard to correctness. For example, if I want to write a program to generate “Hello, world,” I would write a test that validated that “Hello, world” was generated – before I had anything that might create the output. When my tests pass, I know I’ve “finished,” because my tests define a specification. By having tests in place, though, not only do I have a record of the specification, but I also have a way that I can add to the specification in such a way that I know I’m not breaking code – I would simply add more tests that corresponded with the changing specification, and I will know if my changes break other code. Here’s the thing: I wrote the Java implementation using test-driven development practices (TDD), and the automaton is kinda neat; TDD also provided me the opportunity to fix the names of structures (renaming `Dataset` to `Generation`, for example) because the tests made it obvious that the names were inaccurate. However, seeing the differences in the development process between my Python implementation and the Java implementation, I might look into TDD with Python anyway.

Test-driven development” is one of those things that causes hives among some programmers, who immediately stand up and plant the claim that it’s worthless, peurile, deceptive, and generates awful code… never mind that others manage to use it effectively.

I have been playing with a 2D cellular automaton, largely inspired by Stephen Wolfram’s “A New Kind of Science.” I have a version in Java, and a simpler implementation in Python. TDD made the Java version work properly, and a lack of TDD left a hole in the Python version.

“A New Kind of Science” wasn’t the only inspiration, of course. A friend of mine published a Javascript version of Conway’s Game of Life – a 3D automaton that’s pretty well-known – and a different friend of mine was looking for simple projects that he could use to help teach kids how to program, and a 2D automaton came to mind – so he asked for an implementation, which is why I wrote the Python version of the automaton.

This post is not about the automaton itself. I’ll write that up later. (If you’re interested, you can see the source.)

What is TDD?

Test-driven development is, loosely defined, a practice in which tests are written before anything else, without regard to correctness.

For example, if I want to write a program to generate “Hello, world,” I would write a test that validated that “Hello, world” was generated – before I had anything that might create the output. My tests would fail; they wouldn’t even compile until I had some sort of implementation written.

However, by writing a test before anything else:

  • I more or less force myself into having some sort of specification that says what “correctness” means for my program (it is “correct” when it generates “Hello, world”)
  • I also force myself into writing something that has a reasonable interface (because I’m writing how I think it would be called, before writing the guts of the implementation)

By writing the tests first, I’ve effectively given myself a criterion for completeness. When my tests pass, I know I’ve “finished,” because my tests define a specification.

There’s nothing wrong with the specification being incomplete, of course; it may so happen that later, I want to greet someone specific. By having tests in place, though, not only do I have a record of the specification, but I also have a way that I can add to the specification in such a way that I know I’m not breaking code – I would simply add more tests that corresponded with the changing specification, and I will know if my changes break other code.

How did TDD work out for my Automaton?

Here’s the thing: I wrote the Java implementation using test-driven development practices (TDD), and the automaton is kinda neat; it generates some fascinating patterns even without entropy or a variable starting cell structure. An example, of pattern 171, using a color rendering mechanism:

color-171

The Python version was written because a friend of mine wanted to consider using it for a class he’s teaching. The Python version is very much simpler than the Java version, because it doesn’t do as much (it can’t output to multiple formats, for example).

It was not written with testing in mind. Why would it be? I had written the Java version from tests first; I was only writing a simple port to Python.

It was also wrong. A 2D automaton can “grow” to the right and to the left, depending on the pattern it’s given; the Python version could only grow on the right, because I had an off-by-one error in a core routine.

TDD would have caught that early (and it did catch problems like that, in the Java version).

TDD also provided me the opportunity to fix the names of structures (renaming Dataset to Generation, for example) because the tests made it obvious that the names were inaccurate.

Could I have done it without TDD? Of course. TDD isn’t the only way to write programs well. It’s not the only tool used to work out good names, or good processes, or even to validate that programs work properly – the Python version of the automaton was fixed without TDD, for example.

If you’re wondering why I didn’t use TDD for the Python version, it’s because I’m too much of a newbie with Python to know how, yet – and as I’m not really a Python programmer, there’s not a lot of need. However, seeing the differences in the development process between my Python implementation and the Java implementation, I might look into TDD with Python anyway.

An Incomplete Introduction to OWLAPI

OWL is a language used to describe ontologies – systems of knowledge. It’s found primarily in the Semantic Web, which mostly means “hardly anywhere,” and part of the problem is that using OWL is really pretty difficult. You can find plenty of technical language, but very little that describes how and why you’d actually use this stuff in the real world.

As a result, finding real-world uses is difficult, and where it is used in the real world, the usage is inconsistent.

That’s a shame. OWL can be very useful; you can use it to figure out things, you can use it for actual classification and data storage.

However, the purpose of this tutorial is, sadly, not to address the “why” but the “how” of OWL, specifically focusing on the underlying library used for one of the best-known OWL projects, Protégé. That library is known as OWLAPI.

There are tutorials – sort of – that focus on OWLAPI, and the best of them (meaning: “most popular,” the one that gets thrown at you if you ask for resources) is a set of slides in PDF form, from a presentation at OWLED 2011.

The slides are useful, but also need the presentation content, which is not included. The slides are also based on an old revision of the project, so more recent users not only have to figure out why some of the slides are included, but how to use the code snippets in a more recent version of the library.

I’d like to change that, if I can.

Expectations

After this tutorial is finished, I expect readers to be able to create, read, and write ontologies, with a basic understanding of what the various common elements in ontologies are and how they’re used.

The code (and the original Markdown source of this page) are in a Github project: owlapi-tutorial. Feel free to comment or fork as desired.

I do not expect readers to necessarily be left with a complete understanding of OWL, or how it should be used in their organization or projects.

I do not expect to be able to accurately or completely describe every nuance of every bit of code I describe. Much of OWLAPI programming involves cargo cult programming: “I did this, it worked, therefore I do it every time.” I’m afraid that I don’t know how to get around this, but trying to limit the cargo cult mentality is partially why I’m trying to write this tutorial in the first place.

Requirements

I am using:

  • Java 8. This is the current live, supported version of Java; it’s also the oldest supported version of Java as of this writing (Java 7 has been end-of-lifed) and I don’t see the point of using an outmoded version of Java. I actually personally prefer Scala, and OWLAPI is verbose enough that Scala actually makes the resulting code far cleaner than it would be in Java, but I think it’s more important to be precise (as opposed to concise) when writing tutorials.
  • Maven. This is my preferred building environment. There’s nothing wrong with Gradle, et al; I just prefer Maven as I think it’s pretty much the lowest common denominator of functional build systems. Everyone has it, everyone can cater to it.
  • TestNG. I will be writing most of the code snippets as tests first. JUnit is arguably more “lowest common denominator” than TestNG is, but I find that TestNG works better. It’s my tutorial, I’ll use the test framework I want to use.

My project file for Maven is pretty simple, and isn’t likely to change much. This is not a Maven tutorial, so here it is in all its raw glory:

< ?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelversion>4.0.0</modelversion>

    <groupid>com.autumncode</groupid>
    <artifactid>owlapi-tutorial</artifactid>
    <version>1.0-SNAPSHOT</version>

    <dependencymanagement>
        <dependencies>
            <dependency>
                <groupid>net.sourceforge.owlapi</groupid>
                <artifactid>owlapi-distribution</artifactid>
                <version>[4.0.2,)</version>
            </dependency>
            <dependency>
                <groupid>net.sourceforge.owlapi</groupid>
                <artifactid>owlapi-rio</artifactid>
                <version>[4.0.2,)</version>
            </dependency>
            <dependency>
                <groupid>net.sourceforge.owlapi</groupid>
                <artifactid>jfact</artifactid>
                <version>4.0.2</version>
            </dependency>
            <dependency>
                <groupid>org.slf4j</groupid>
                <artifactid>slf4j-simple</artifactid>
                <version>1.7.7</version>
            </dependency>
            <dependency>
                <groupid>org.testng</groupid>
                <artifactid>testng</artifactid>
                <version>[6.9.8,)</version>
                <scope>test</scope>
            </dependency>
        </dependencies>
    </dependencymanagement>

    <dependencies>
        <dependency>
            <groupid>net.sourceforge.owlapi</groupid>
            <artifactid>owlapi-distribution</artifactid>
        </dependency>
        <dependency>
            <groupid>net.sourceforge.owlapi</groupid>
            <artifactid>owlapi-rio</artifactid>
        </dependency>
        <dependency>
            <groupid>org.slf4j</groupid>
            <artifactid>slf4j-simple</artifactid>
        </dependency>
        <dependency>
            <groupid>org.testng</groupid>
            <artifactid>testng</artifactid>
        </dependency>
        <dependency>
            <groupid>net.sourceforge.owlapi</groupid>
            <artifactid>jfact</artifactid>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupid>org.apache.maven.plugins</groupid>
                <artifactid>maven-compiler-plugin</artifactid>
                <version>3.3</version>
                <configuration>
                    <source />1.8
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

There are a few dependencies that relate to OWLAPI: owlapi-distribution, which is OWLAPI itself; owlapi-rio, which has some classes we need in order to parse and write OWL; lastly, there’s jfact, which is a reasoner implementation for OWL. (A reasoner is used to pull various types of information from OWL. I don’t know a ton about reasoners yet, and apparently JFact is pretty lightweight, but it’s enough for this tutorial.)

Terms

An “Ontology” is a system of knowledge. It is composed of descriptions of what things are as well as a set of things.

A “Class” is a description of what a thing is. A “Person,” for example, is a term you might use to describe a living being of a specific species (homo sapiens, generally), with gender and a name. Therefore, we might describe a “Person class”.

A class can have attributes (a concept familiar to Java coders); these are “properties,” which can be made of data or references (again, just like Java). Properties can have ranges or defined subsets of values.

A class can also have the concept of a superclass (again, just like Java): a Fireman class might be a valid subclass of Person. However, there is no limit to the number of superclasses; a Fireman might be a subclass of the Person, PublicServant, WearsRed, and DrivesTruck classes. Java can model some of this with interfaces, but it’s not an exact analogy.

An “Individual” is a concrete instance of a class – much like an “instance,” in Java parlance. “John” might be a fireman, which implies that he is a person; Sam might be a person (but not a fireman). Sam might have a dog, Fritz, who is not a Person (instead, is a Dog). All three would be individuals, with varying properties.

Working with OWLAPI

OWL uses the concept of an “IRI,” an “Internationalized Resource Identifier.” It’s a lot like a URI, but has a wider set of characters available to it; for the sake of clarity we’ll stay seven-bit-clean.

Basically, an IRI is a unique identifier for a concept, whether it’s the ontology itself, a class in the ontology, a data property, an individual, or any other unique concept represented in the ontology.

OWLAPI creates references to elements in an Ontology by asking a data factory for a reference of a specific type with a specific IRI. Relationships between elements are expressed as “axioms.” Therefore, the typical interaction with OWLAPI will look something like this:

  1. Create an initial reference with a unique IRI.
  2. Create another reference, with a different (and unique) IRI.
  3. Express an axiom using the two references.
  4. Apply the axiom as a change to the ontology.

Creating an Ontology

Before we can do anything with an ontology, we need to be able to create one. We’re going to create a stateful helper class in Java to serve as a simple delegate to OWLAPI; it will retain some simple references that we’ll end up using over and over again (the OWLDataFactory and OWLOntologyManager) and also provide a class into which we can place simple utility methods.

Many of the methods in our utility class (which I’ve decided will be called OntologyHelper) are one-liners, or close to it, especially early on.

The baseline for our OntologyHelper looks like this, with packages and all imports stripped out:

public class OntologyHelper {
    OWLOntologyManager m= OWLManager.createOWLOntologyManager();
    OWLDataFactory df=OWLManager.getOWLDataFactory();
}

We know we’ll be converting between java.lang.String and org.semanticweb.owlapi.model.IRI quite a bit, so we’ll add a conversion method:

public class OntologyHelper {
    OWLOntologyManager m= OWLManager.createOWLOntologyManager();
    OWLDataFactory df=OWLManager.getOWLDataFactory();

    public IRI convertStringToIRI(String ns) {
        return IRI.create(ns);
    }
}

Now let’s look at actually creating an OWLOntology. First, let’s write a test that creates the ontology and then checks to make sure the ontology conforms to what we think it should be:

@Test
public void createOntology() throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    IRI iri = oh.convertStringToIRI("http://autumncode.com/ontologies/2015/example.owl");
    OWLOntology ontology = oh.createOntology(iri);

    assertNotNull(ontology);
    assertEquals(iri,
            ontology.getOntologyID().getOntologyIRI().or(oh.convertStringToIRI("false")));
}

That last assertEquals is a bit scary. The first argument is the IRI we expect the ontology to have; the second argument is a work of art. Basically, we’re getting the ontology ID, part of which is the ontology’s IRI; however, an ontology might not have an IRI, so it’s actually returned in a Java Optional<iri>. We use .or() to specify a constant value (an IRI) to use just in case our ontology does not have an IRI.

Creating an ontology, given our m and df declarations from OntologyHelper, is a matter of calling m.createOntology with an IRI. Since working with String is a lot more easily conceived of than working with an IRI, we’ll create a convenience method to take a String argument, convert it to an IRI, and then delegate to another method to create the OWLOntology itself.

With that, then, let’s add some content to our OntologyHelper, consisting of the following two methods (only one of which is used by the test, since we pass in an IRI directly).

public OWLOntology createOntology(String iri) throws OWLOntologyCreationException {
    return createOntology(convertStringToIRI(iri));
}

public OWLOntology createOntology(IRI iri) throws OWLOntologyCreationException {
    return m.createOntology(iri);
}

Writing an Ontology

Writing an ontology is fairly simple. To write an ontology, we simply create a OWLOntologyDocumentTarget of some kind – examples include a StringDocumentTarget and a FileDocumentTarget, among others – and then call m.saveOntology(), supplying the ontology to save, the target, and (optionally) a format to use. (OWL has multiple formats, including OWL itself, RDF, Turtle, and Manchester format among others; the formats are out of scope for this tutorial. We’re sticking to the default OWL format.)

First, let’s write our test:

@Test
public void writeOntology() throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology ontology = oh.createOntology("http://autumncode.com/ontologies/2015/example.owl");
    StringDocumentTarget sdt = new StringDocumentTarget();
    oh.writeOntology(ontology, sdt);

    StringDocumentSource sds = new StringDocumentSource(sdt.toString());
    OWLOntology o = oh.readOntology(sds);
    assertEquals(oh.convertStringToIRI("http://autumncode.com/ontologies/2015/example.owl"),
            o.getOntologyID().getOntologyIRI().or(oh.convertStringToIRI("false")));
}

Here, we’re creating our ontology (just as we did earlier), but we’re writing it into a StringDocumentTarget. We then use that StringDocumentTarget to demonstrate reading the ontology, using the read content to validate the ontology ID.

Reading an Ontology

Now let’s show a little more about reading an ontology – and when we say “a little more,” we mean literally only a little more. We’ll also use the @DataProvider concept in TestNG to give us some flexibility for when we create more test data.

The weakness of the writeOntology test is that it is internal; it doesn’t have any external data. We don’t have any artifacts of the test to examine. We should be able to read an external artifact (a file, for example) and use it for reference and for examination.

With that, let’s take a look an an ontology very similar to the ones we’ve been creating, stored in src/test/resources/example1.owl:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/2015/example1.owl#"
         xml:base="http://autumncode.com/ontologies/2015/example1.owl"
         xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:owl="http://www.w3.org/2002/07/owl#"
         xmlns:xml="http://www.w3.org/XML/1998/namespace"
         xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
         xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/2015/example1.owl"></owl:ontology>
</rdf:rdf>

This ontology is empty, consisting only of the ontology reference itself. (It’s expressing the idea of a system of knowledge that refers to … no knowledge at all, including the assertion of the absence of knowledge.)

Our test code will use a data provider to construct a reference to an OWLOntologyDocumentSource (which will point to this file) and the expected IRI from it (in this case, "http://autumncode.com/ontologies/2015/example1.owl").

@DataProvider
Object[][] readDataProvider() {
    return new Object[][]{{
            new StreamDocumentSource(this.getClass().getResourceAsStream("/example1.owl")),
            "http://autumncode.com/ontologies/2015/example1.owl"
    }};
}

@Test(dataProvider = "readDataProvider")
public void readOntology(OWLOntologyDocumentSource source, String baseIRI) throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology ontology = oh.readOntology(source);
    assertNotNull(ontology);
    assertEquals(oh.convertStringToIRI(baseIRI),
            ontology.getOntologyID().getOntologyIRI().or(oh.convertStringToIRI("false")));
}

As usual, our OntologyHelper code is far more simple than the test code is:

public OWLOntology readOntology(OWLOntologyDocumentSource source)
        throws OWLOntologyCreationException {
    return m.loadOntologyFromOntologyDocument(source);
}

Adding a Class to an Ontology

Adding a class to an ontology is similar to adding a class to a Java package, with the main exception being that a class in an ontology doesn’t exist without a relationship to something else. Therefore, in Java we can say “a person is something,” whereas in OWL, we have to say “a person is something that…”

An OWL class can’t exist in a vacuum.

First, let’s create a simple relationship: we’ll define an ontology which can allow individuals to be people (i.e., “a person”) or firemen. Here’s our test code, which will ironically have to live in a vacuum itself until we write all of our utility methods:

@Test
public void testAddClassesSimple() throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/person.owl");
    OWLClass person = oh.createClass("http://autumncode.com/ontologies/person.owl#Person");
    OWLClass fireman = oh.createClass("http://autumncode.com/ontologies/person.owl#Fireman");
    OWLAxiomChange axiom = oh.createSubclass(o, fireman, person);
    oh.applyChange(axiom);

    Set<string> superclasses = o.getSubClassAxiomsForSubClass(fireman)
            .stream()
            .map(ax -> ax.getSuperClass().asOWLClass().toStringID())
            .collect(Collectors.toSet());
    assertEquals(1, superclasses.size());
    assertTrue(superclasses.contains(person.toStringID()));
}

We need to create OntologyHelper.createClass first, which looks like this:

public OWLClass createClass(String iri) {
    return createClass(convertStringToIRI(iri));
}
public OWLClass createClass(IRI iri) {
    return df.getOWLClass(iri);
}

We also need to define OntologyHelper.createSubclass. This will create a change for our ontology – a patch – that says that in the context of our ontology, a fireman is a type, a subclass, of person. It does not apply that patch; that’s the role of our OntologyHelper.applyChange method.

Here are those methods from OntologyHelper:

public OWLAxiomChange createSubclass(OWLOntology o, OWLClass subclass, OWLClass superclass) {
    return new AddAxiom(o,df.getOWLSubClassOfAxiom(subclass, superclass));
}

public void applyChange(OWLAxiomChange ... axiom) {
    applyChanges(axiom);
}

private void applyChanges(OWLAxiomChange ... axioms) {
    m.applyChanges(Arrays.asList(axioms));
}

After we apply the changes, we get the axioms related to subclass for our definition of what a fireman is, translate the axioms into the superclass name, and then check to make sure the resulting set is the right size (one) and contains the right value (a reference to our person definition).

The resulting OWL, if you’re interested, looks like this:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/person.owl#"
     xml:base="http://autumncode.com/ontologies/person.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/person.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Fireman -->

    <owl:class rdf:about="http://autumncode.com/ontologies/person.owl#Fireman">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/person.owl#Person"></rdfs:subclassof>
    </owl:class>

    <!-- Person -->

    <owl:class rdf:about="http://autumncode.com/ontologies/person.owl#Person"></owl:class>
</rdf:rdf>

In Java, we’ve basically expressed something like this:

public class Person { }
public class Fireman extends Person { }

Let’s take it a little farther. Imagine, if you will, SkyNet: a Terminator is both a Person and a Robot, but that doesn’t preclude the existence of a person who is not a robot, nor does it mean that all robots are people. Our test method is similar, but has two superclasses for the definition of a Terminator.

@Test
public void testAddClassesMoreComplex() throws OWLException {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/terminator.owl");
    OWLClass person = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Person");
    OWLClass robot = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Robot");
    OWLClass terminator = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Terminator");

    oh.applyChange(oh.createSubclass(o, terminator, person),
            oh.createSubclass(o, terminator, robot));

    Set</string><string> superclasses = o.getSubClassAxiomsForSubClass(terminator)
            .stream()
            .map(ax -> ax.getSuperClass().asOWLClass().toStringID())
            .collect(Collectors.toSet());
    assertEquals(2, superclasses.size());
    assertTrue(superclasses.contains(person.toStringID()));
    assertTrue(superclasses.contains(robot.toStringID()));
}

The OWL this structure generates looks like this:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/terminator.owl#"
     xml:base="http://autumncode.com/ontologies/terminator.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/terminator.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Person -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Person"></owl:class>

    <!-- Robot -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Robot"></owl:class>

    <!-- Terminator -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Terminator">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/terminator.owl#Person"></rdfs:subclassof>
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/terminator.owl#Robot"></rdfs:subclassof>
    </owl:class>
</rdf:rdf>

However, Java cannot express these relationships, as it doesn’t support multiple concrete superclasses.

It’s worth pointing out that the code that validates the relationships is horribly ugly, and not generally useful outside of the context of these tests. One normally wouldn’t bother with that sort of code.

Note that we’ve only expressed the idea that we can have an ontology that describes people, robots, and things that are both robots and people (i.e., Terminators). We could create Captain Kirk, the T-800, and R2-D2, all while ignoring the shrieks of rage from Trekkies, Star Wars fans, and whoever cares about the Terminator.

Time to fix that. We still won’t be able to describe any of these individuals besides their names and what class they are, but that’ll be enough for the next step.

Adding an Individual to an Ontology

Let’s start with the simplest structure possible: let’s create an ontology that has a class of Person, and add Captain Kirk to the ontology as an instance of a Person.

Our test code is actually pretty simple, but verifying that we’ve done what we expected is a bit of a pain.

Most of the testing code builds a map of classes to individuals, using an OWLReasoner, which is a class that extracts data from the ontology: we use it here to extract the direct instances of a type from a given class (after iterating through “all classes”). The ListMultimap is part of Guava, which is a transitive dependency of OWLAPI.

Note also the “AutoClosable trick,” which forces Java to dispose of the OWLReasoner when the block has completed execution.

@Test
public void addSimpleIndividual() throws Exception {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/person.owl");
    OWLClass person = oh.createClass("http://autumncode.com/ontologies/person.owl#Person");
    OWLIndividual captainKirk = oh.createIndividual("http://autumncode.com/ontologies/person.owl#Kirk");
    oh.applyChange(oh.associateIndividualWithClass(o, person, captainKirk));

    // test that the individual is what we expect
    final OWLReasoner reasoner = new JFactFactory().createReasoner(o);
    try (final AutoCloseable ignored = reasoner::dispose) {
        ListMultimap</string><string , String> classInstanceMap = ArrayListMultimap.create();
        o.getClassesInSignature()
                .stream()
                .forEach(clazz ->
                        reasoner.getInstances(clazz, true)
                                .getFlattened()
                                .stream()
                                .forEach(i ->
                                        classInstanceMap.put(clazz.toStringID(), i.toStringID())));
        // should have one class
        assertEquals(classInstanceMap.keySet().size(), 1);
        Set</string><string> people=new HashSet<>();
        people.add(captainKirk.toStringID());
        assertEquals(classInstanceMap.asMap().get(person.toStringID()), people);
    } 
}

We need three additional methods in our OntologyHelper class, all of which are fairly simple (two create the OWLIndividual references, and one returns an axiom that associates an individual with a class):

public OWLIndividual createIndividual(String iri) {
    return createIndividual(convertStringToIRI(iri));
}
private OWLIndividual createIndividual(IRI iri) {
    return df.getOWLNamedIndividual(iri);
}

public OWLAxiomChange associateIndividualWithClass(OWLOntology o,
                                                   OWLClass clazz,
                                                   OWLIndividual individual) {
    return new AddAxiom(o, df.getOWLClassAssertionAxiom(clazz, individual));
}

In Java, we’ve basically modeled something like this:

class Person {}
void buildData() {
    Person captainKirk=new Person();
}

Let’s go a little further, and add individuals to our Terminator knowledge base, for a more complex example.

Here’s the code that actually creates the ontology and the three classes (Person, Robot, Terminator) and the relationships between them (a Terminator is a Person and a Robot):

OntologyHelper oh = new OntologyHelper();
OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/terminator.owl");
OWLClass person = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Person");
OWLClass robot = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Robot");
OWLClass terminator = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Terminator");

oh.applyChange(oh.createSubclass(o, terminator, person),
        oh.createSubclass(o, terminator, robot));

Now we’d like to create the three individuals (“Sarah,” a human; “Tank,” a nonhumanoid robot, and “T800,” a Terminator):

OWLIndividual sarah = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#Sarah");
OWLIndividual tank = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#Tank");
OWLIndividual t800 = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#T800");
oh.applyChange(oh.associateIndividualWithClass(o, person, sarah),
        oh.associateIndividualWithClass(o, robot, tank),
        oh.associateIndividualWithClass(o, terminator, t800));

The OWL that represents this ontology looks like this:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/terminator.owl#"
     xml:base="http://autumncode.com/ontologies/terminator.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/terminator.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Person -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Person"></owl:class>

    <!-- Robot -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Robot"></owl:class>

    <!-- Terminator -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Terminator">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/terminator.owl#Person"></rdfs:subclassof>
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/terminator.owl#Robot"></rdfs:subclassof>
    </owl:class>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Individuals
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Sarah -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#Sarah">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Person"></rdf:type>
    </owl:namedindividual>

    <!-- T800 -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#T800">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Terminator"></rdf:type>
    </owl:namedindividual>

    <!-- Tank -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#Tank">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Robot"></rdf:type>
    </owl:namedindividual>
</rdf:rdf>

Here’s where things get interesting. We said that the T800 is a Terminator, not that it’s a Robot or a Person; our test needs to validate its participation in all three classes (Terminator, Robot, Person), even though our OWL doesn’t say that it’s a member of all three classes. This is where the OWLReasoner comes in; it’s able to infer the complete class hierarchy for an individual (or, as we’re using it here, it’s able to find all individuals who are members of a given class.)

This is the beginning of seeing some real power in OWL, even though we’re expressing something very simple: we can describe what things are (a Terminator is a Robot and a Person), express characteristics about individuals (a T800 is a Terminator) and extract meaningful data from the ontology (i.e., that the T800 is both a Robot and a Person, while the Tank is only a Robot.)

Here’s the complete test method, including the extraction of individuals into classes and the conversion of data into comparable sets:

@Test
public void addTerminatorIndividuals() throws Exception {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/terminator.owl");
    OWLClass person = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Person");
    OWLClass robot = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Robot");
    OWLClass terminator = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Terminator");

    oh.applyChange(oh.createSubclass(o, terminator, person),
            oh.createSubclass(o, terminator, robot));

    OWLIndividual sarah = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#Sarah");
    OWLIndividual tank = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#Tank");
    OWLIndividual t800 = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#T800");
    oh.applyChange(oh.associateIndividualWithClass(o, person, sarah),
            oh.associateIndividualWithClass(o, robot, tank),
            oh.associateIndividualWithClass(o, terminator, t800));

    final OWLReasoner reasoner = new JFactFactory().createReasoner(o);
    try (final AutoCloseable ignored = reasoner::dispose) {
        ListMultimap</string><string , String> classInstanceMap = ArrayListMultimap.create();
        o.getClassesInSignature()
                .stream()
                .forEach(clazz ->
                        reasoner.getInstances(clazz, false)
                                .getFlattened()
                                .stream()
                                .forEach(i ->
                                        classInstanceMap.put(clazz.toStringID(), i.toStringID())));

        // should have three classes
        assertEquals(classInstanceMap.keySet().size(), 3);

        Set</string><string> people = new HashSet<>(Arrays.asList(new String[]{sarah.toStringID(), t800.toStringID()}));
        Set</string><string> robots = new HashSet<>(Arrays.asList(new String[]{t800.toStringID(), tank.toStringID()}));
        Set</string><string> terminators = new HashSet<>(Arrays.asList(new String[]{t800.toStringID()}));

        assertEquals(new HashSet<>(classInstanceMap.asMap().get(person.toStringID())), people);
        assertEquals(new HashSet<>(classInstanceMap.asMap().get(robot.toStringID())), robots);
        assertEquals(new HashSet<>(classInstanceMap.asMap().get(terminator.toStringID())), terminators);
    }
}

Adding a Property to a Class

It’s potentially useful to be able to determine class relationships and individual participation in classes, but we’re not actually describing much yet. In our Terminator example, for example, we have one Tank, one T800, one Sarah Connor. Since we know that there are many T800s and many Tanks (and, depending on how much you liked Terminator:Genisys, multiple Sarahs as well) we need to have a way of differentiating individuals beyond their names.

Let’s start by looking at a simpler description of people, and build a genealogy structure. We’ll have three classes (Human, Female, and Male), and add two properties to Person (references to mother and father). Since Mother and Father are derived from Person, we can declare individuals by gender and inherit the properties referring to parentage from Human.

OntologyHelper oh = new OntologyHelper();
OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/genealogy.owl");
OWLClass human = oh.createClass("http://autumncode.com/ontologies/geneaology.owl#Human");
OWLClass male = oh.createClass("http://autumncode.com/ontologies/genealogy.owl#Male");
OWLClass female = oh.createClass("http://autumncode.com/ontologies/genealogy.owl#Female");
OWLObjectProperty hasFather =
        oh.createObjectProperty("http://autumncode.com/ontologies/genealogy.owl#hasFather");
OWLObjectProperty hasMother =
        oh.createObjectProperty("http://autumncode.com/ontologies/genealogy.owl#hasMother");
oh.applyChange(
        oh.createSubclass(o, male, human),
        oh.createSubclass(o, female, human),
        oh.associateObjectPropertyWithClass(o, hasFather, human, male),
        oh.associateObjectPropertyWithClass(o, hasMother, human, female)
);

We need to add associateObjectPropertyWithClass to OntologyHelper:

/**
 * With ontology o, property in refHolder points to a refTo.
 *
 * @param o The ontology reference
 * @param property the data property reference
 * @param refHolder the container of the property
 * @param refTo the class the property points to
 * @return a patch to the ontology
 */
public OWLAxiomChange associateObjectPropertyWithClass(OWLOntology o, 
                                                       OWLObjectProperty property, 
                                                       OWLClass refHolder, 
                                                       OWLClass refTo) {
    OWLClassExpression hasSomeRefTo=df.getOWLObjectSomeValuesFrom(property, refTo);
    OWLSubClassOfAxiom ax=df.getOWLSubClassOfAxiom(refHolder, hasSomeRefTo);
    return new AddAxiom(o, ax);
}

We also want to add another method to the OntologyHelper, to manage a disjunction of classes. Our test doesn’t use it (but our next one will): what it says is that the first class cannot be the second class. Note that a disjunction like this is not reflexive – saying that ‘a’ cannot be a ‘b’ does not imply that ‘b’ cannot be an ‘a’; you have to specify the disjunction both ways if you’re trying to express that relationship. We’ll see how that works soon.

/**
 * With ontology o, an object of class a cannot be simultaneously an object of class b.
 * This is not implied to be an inverse relationship; saying that a cannot be a b does not
 * mean that b cannot be an a.
 * @param o the ontology reference
 * @param a the source of the disjunction
 * @param b the object of the disjunction
 * @return a patch to the ontology
 */
public OWLAxiomChange addDisjointClass(OWLOntology o, OWLClass a, OWLClass b) {
    OWLDisjointClassesAxiom expression=df.getOWLDisjointClassesAxiom(a, b);
    return new AddAxiom(o, expression);
}

The test code generates the following OWL:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/genealogy.owl#"
     xml:base="http://autumncode.com/ontologies/genealogy.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/genealogy.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Object Properties
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- hasFather -->

    <owl:objectproperty rdf:about="http://autumncode.com/ontologies/genealogy.owl#hasFather"></owl:objectproperty>

    <!-- hasMother -->

    <owl:objectproperty rdf:about="http://autumncode.com/ontologies/genealogy.owl#hasMother"></owl:objectproperty>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Female -->

    <owl:class rdf:about="http://autumncode.com/ontologies/genealogy.owl#Female">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/geneaology.owl#Human"></rdfs:subclassof>
    </owl:class>

    <!-- Male -->

    <owl:class rdf:about="http://autumncode.com/ontologies/genealogy.owl#Male">
        <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/geneaology.owl#Human"></rdfs:subclassof>
    </owl:class>

    <!-- Human -->

    <owl:class rdf:about="http://autumncode.com/ontologies/geneaology.owl#Human">
        <rdfs:subclassof>
            <owl:restriction>
                <owl:onproperty rdf:resource="http://autumncode.com/ontologies/genealogy.owl#hasFather"></owl:onproperty>
                <owl:somevaluesfrom rdf:resource="http://autumncode.com/ontologies/genealogy.owl#Male"></owl:somevaluesfrom>
            </owl:restriction>
        </rdfs:subclassof>
        <rdfs:subclassof>
            <owl:restriction>
                <owl:onproperty rdf:resource="http://autumncode.com/ontologies/genealogy.owl#hasMother"></owl:onproperty>
                <owl:somevaluesfrom rdf:resource="http://autumncode.com/ontologies/genealogy.owl#Female"></owl:somevaluesfrom>
            </owl:restriction>
        </rdfs:subclassof>
    </owl:class>
</rdf:rdf>

Now let’s start describing a genealogy. We’re describing a few generations of a family:

  • Thomas, w. Shirley: Michael and Vicki
  • Barry, w. Shirley: Joseph
  • Samuel, w. Mary: Andrew
  • Andrew, w. Vicki: Jonathan

Here’s the code to build the geneaology structure, including the exclusion of Male and Female, along with the actual family tree:

@Test
public void simpleParentage() throws Exception {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/genealogy.owl");
    OWLClass human = oh.createClass("http://autumncode.com/ontologies/geneaology.owl#Human");
    OWLClass male = oh.createClass("http://autumncode.com/ontologies/genealogy.owl#Male");
    OWLClass female = oh.createClass("http://autumncode.com/ontologies/genealogy.owl#Female");
    OWLObjectProperty hasFather =
            oh.createObjectProperty("http://autumncode.com/ontologies/genealogy.owl#hasFather");
    OWLObjectProperty hasMother =
            oh.createObjectProperty("http://autumncode.com/ontologies/genealogy.owl#hasMother");
    oh.applyChange(
            oh.createSubclass(o, male, human),
            oh.createSubclass(o, female, human),
            oh.addDisjointClass(o, female, male),
            oh.addDisjointClass(o, male, female),
            oh.associateObjectPropertyWithClass(o, hasFather, human, male),
            oh.associateObjectPropertyWithClass(o, hasMother, human, female)
    );

    OWLIndividual barry = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#barry");
    OWLIndividual shirley = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#shirley");
    OWLIndividual thomas = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#thomas");
    OWLIndividual michael = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#michael");
    OWLIndividual vicki = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#vicki");
    OWLIndividual joseph = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#joseph");
    OWLIndividual mary = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#mary");
    OWLIndividual samuel = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#samuel");
    OWLIndividual andrew = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#andrew");
    OWLIndividual jonathan = oh.createIndividual("http://autumncode.com/ontologies/genealogy.owl#jonathan");

    oh.applyChange(
            oh.associateIndividualWithClass(o, male, barry),
            oh.associateIndividualWithClass(o, male, thomas),
            oh.associateIndividualWithClass(o, male, michael),
            oh.associateIndividualWithClass(o, male, joseph),
            oh.associateIndividualWithClass(o, male, samuel),
            oh.associateIndividualWithClass(o, male, andrew),
            oh.associateIndividualWithClass(o, male, jonathan),
            oh.associateIndividualWithClass(o, female, shirley),
            oh.associateIndividualWithClass(o, female, vicki),
            oh.associateIndividualWithClass(o, female, mary),
            oh.addObjectproperty(o, michael, hasMother, shirley),
            oh.addObjectproperty(o, michael, hasFather, thomas),
            oh.addObjectproperty(o, vicki, hasMother, shirley),
            oh.addObjectproperty(o, vicki, hasFather, thomas),
            oh.addObjectproperty(o, joseph, hasMother, shirley),
            oh.addObjectproperty(o, joseph, hasFather, barry),
            oh.addObjectproperty(o, andrew, hasMother, mary),
            oh.addObjectproperty(o, andrew, hasFather, samuel),
            oh.addObjectproperty(o, jonathan, hasMother, vicki),
            oh.addObjectproperty(o, jonathan, hasFather, andrew)
    );

// we'll be adding code here!
}

The OWL this generates is pretty long, unfortunately. But here are some relevant parts from it, the Male and Female class declarations, along with Jonathan’s individual reference:

<!-- Female -->

<owl:class rdf:about="http://autumncode.com/ontologies/genealogy.owl#Female">
    <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/geneaology.owl#Human"></rdfs:subclassof>
    <owl:disjointwith rdf:resource="http://autumncode.com/ontologies/genealogy.owl#Male"></owl:disjointwith>
</owl:class>

<!-- Male -->

<owl:class rdf:about="http://autumncode.com/ontologies/genealogy.owl#Male">
    <rdfs:subclassof rdf:resource="http://autumncode.com/ontologies/geneaology.owl#Human"></rdfs:subclassof>
</owl:class>

<!-- jonathan -->

<owl:namedindividual rdf:about="http://autumncode.com/ontologies/genealogy.owl#jonathan">
    <rdf:type rdf:resource="http://autumncode.com/ontologies/genealogy.owl#Male"></rdf:type>
    <hasfather rdf:resource="http://autumncode.com/ontologies/genealogy.owl#andrew"></hasfather>
    <hasmother rdf:resource="http://autumncode.com/ontologies/genealogy.owl#vicki"></hasmother>
</owl:namedindividual>

If we didn’t have that disjunction of Male and Female, there would be nothing in the ontology suggesting that a “has mother of” reference couldn’t point to, say, Samuel. (Let’s keep gender identity simple for the clarity’s sake.)

What happens if we actually add invalid data, though? Let’s try it.

OWLReasoner reasoner = new JFactFactory().createReasoner(o);
try (final AutoCloseable ignored = reasoner::dispose) {
    assertTrue(reasoner.isConsistent());
}
oh.applyChange(oh.associateIndividualWithClass(o, female, jonathan));
reasoner = new JFactFactory().createReasoner(o);
try (final AutoCloseable ignored = reasoner::dispose) {
    assertFalse(reasoner.isConsistent());
}

Here, we’re checking to make sure our ontology is consistent (without error) – then we’re intentionally adding a declaration that Jonathan is female (leaving the reference to Male intact). Therefore, Jonathan is both male and female, which our disjunction says is invalid… and when we check our reasoner again, it shows that the ontology is inconsistent.

Confession

Unfortunately, this is also where the OWLAPI starts to lose me.

The Semantic Web technologies not only provide a way to store information like our genealogy data, but query it; we should be able to examine Joseph and Vicki, for example, and find that they are siblings because one or both of their parents are the same.

Even better, I think we can even declare rules such that a reasoner could assert that they were siblings because of common parentage. Someone online even showed me how such a rule would be written in one of the OWL formats.

However, it’s not clear to me (yet) how to express such a relationship using OWLAPI. I can see a potential path using various axioms chained together; many of the OWLDataFactory methods even accept arrays of arguments, which would seem to enable this.

But I haven’t been able to properly or simply conceptualize how all of this would work in implementation rather than in theory, which speaks of two failures:

  1. The available material has failed to make it simple enough such that it’s accessible to newbies like me, and thus…
  2. I have failed to completely grasp the subject matter enough to write authoritatively about it.

I have no problem acknowledging the second failure.

I’m writing this tutorial not to say “here’s a complete reference” but instead to preserve what I have learned in using OWLAPI incompletely, as a data transferral protocol and as a static data store rather than as a reasoning engine.

If someone more experienced than I would like to collaborate on this to correct both my ignorance and the tutorial’s incompleteness, you’re welcome and invited to participate.

Back to Regular Programming

The reasoning process that shows the invalidity of our data based on the hierarchy shows something important about OWL: it doesn’t throw out our invalid data, but provides a way to tell us that our data is incorrect based on the assertions we’ve made.

In other words, we said that a Human cannot be both Male and Female, and then we told it that a human was, in fact, both. (I’m avoiding cis gender issues, thank you; this is a simple example and isn’t meant to be a treatise on human biology or gender issues.) Because of the assertion of singular gender (and then the use of data that violated the assertion), our ontology is no longer consistent with itself – and the reasoner was able to tell us that.

If we had greater reasoning skills, we could derive actual family models – determine cousins, uncles, aunts, siblings, and more – and have the model tell us of gaps and unexpected relationships, all expressed and generated programmatically.

However, at present most of that is beyond your author’s knowledge and skill, and is beyond this tutorial’s scope.

Review

  • We have created, saved, and loaded ontologies, which represents a system of knowledge
  • We have created classes in an ontology, which represent ways of thinking about items.
    • We have seen that classes can be present in a hierarchy, such that elements of class B are also elements of type A.
    • We have seen that we can also declare classes as optionally being exclusive of one another. Therefore, we can say that if something is of class A, it cannot legitimately be of class B at the same time.
  • We have seen how we can create individuals in an ontology.
    • We have seen how we can add attributes to the individuals such that we define their classes, and we’ve seen that an individual is not limited to one class.
    • We have added references to other individuals to a given individual, and also declare that the references must be drawn from a specific domain. (We can say that a reference, A, must be to members of class B.)
  • We have seen how to use a reasoner to validate that our ontology, the class structure, and the individuals in it are consistent, with no contradictory information.

One thing we’ve not done, though, is add simple data to an individual.

Adding a Property Value to an Individual

When we add a reference to an individual as a property of another individual (“John’s mother is Andrea”), we are adding what OWLAPI calls an “Object Property.” Another type of property is a Data Property, which can be used in almost the exact same way.

Let’s go back to the Terminator. We’ll declare a simple ontology that tracks the T800, and tracks whether it is an enemy or not.

Here’s our ontology declaration as a test. After we see this, we’ll take a look at the OntologyHelper methods that make it work:

@Test
public void addDataToIndividual() throws Exception {
    OntologyHelper oh = new OntologyHelper();
    OWLOntology o = oh.createOntology("http://autumncode.com/ontologies/terminator.owl");
    OWLClass terminator = oh.createClass("http://autumncode.com/ontologies/terminator.owl#Terminator");
    OWLDataProperty killsHumans =
            oh.createDataProperty("http://autumncode.com/ontologies/terminator.owl#killsHumans");
    OWLIndividual t800 = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#t800");
    OWLIndividual pops = oh.createIndividual("http://autumncode.com/ontologies/terminator.owl#pops");
    oh.applyChange(
            oh.associateIndividualWithClass(o, terminator, pops),
            oh.associateIndividualWithClass(o, terminator, t800),
            oh.addDataToIndividual(o, t800, killsHumans, true),
            oh.addDataToIndividual(o, pops, killsHumans, false)
    );
}

Now the OntologyHelper methods for creating the data property and adding data to a given individual:

public OWLDataProperty createDataProperty(String iri) {
    return createDataProperty(convertStringToIRI(iri));
}

public OWLDataProperty createDataProperty(IRI iri) {
    return df.getOWLDataProperty(iri);
}

public OWLAxiomChange addDataToIndividual(OWLOntology o, OWLIndividual individual, 
        OWLDataProperty property, String value) {
    OWLLiteral literal = df.getOWLLiteral(value, OWL2Datatype.XSD_STRING);
    return new AddAxiom(o, df.getOWLDataPropertyAssertionAxiom(property, individual, literal));
}

public OWLAxiomChange addDataToIndividual(OWLOntology o, OWLIndividual individual, 
        OWLDataProperty property, boolean value) {
    OWLLiteral literal = df.getOWLLiteral(value);
    return new AddAxiom(o, df.getOWLDataPropertyAssertionAxiom(property, individual, literal));
}

public OWLAxiomChange addDataToIndividual(OWLOntology o, OWLIndividual individual, 
        OWLDataProperty property, int value) {
    OWLLiteral literal = df.getOWLLiteral(value);
    return new AddAxiom(o, df.getOWLDataPropertyAssertionAxiom(property, individual, literal));
}

These are, as you might expect, almost exact analogies for the methods that add object properties to individuals. The main difference here is that we actually deal with data, so we have overloaded argument types for the addDataToIndividual method. (The getOWLDataPropertyAssertionAxiom method can take the actual primitive values and render the proper types.)

So what does the generated data look like? That’s simple enough:

< ?xml version="1.0"?>
<rdf:rdf xmlns="http://autumncode.com/ontologies/terminator.owl#"
     xml:base="http://autumncode.com/ontologies/terminator.owl"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xml="http://www.w3.org/XML/1998/namespace"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:terminator="http://autumncode.com/ontologies/terminator.owl#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
    <owl:ontology rdf:about="http://autumncode.com/ontologies/terminator.owl"></owl:ontology>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Data properties
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- killsHumans -->

    <owl:datatypeproperty rdf:about="http://autumncode.com/ontologies/terminator.owl#killsHumans"></owl:datatypeproperty>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Classes
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- Terminator -->

    <owl:class rdf:about="http://autumncode.com/ontologies/terminator.owl#Terminator"></owl:class>

    <!-- 
    ///////////////////////////////////////////////////////////////////////////////////////
    //
    // Individuals
    //
    ///////////////////////////////////////////////////////////////////////////////////////
     -->

    <!-- pops -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#pops">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Terminator"></rdf:type>
        <killshumans rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">false</killshumans>
    </owl:namedindividual>

    <!-- t800 -->

    <owl:namedindividual rdf:about="http://autumncode.com/ontologies/terminator.owl#t800">
        <rdf:type rdf:resource="http://autumncode.com/ontologies/terminator.owl#Terminator"></rdf:type>
        <killshumans rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</killshumans>
    </owl:namedindividual>
</rdf:rdf>

Final Thoughts

What we’ve tried to cover is actually pretty light: “how to add data into an ontology using OWLAPI,” really. There are supposed to be a lot of features provided by OWL that we’ve not even begun to touch; we should, for example, be able to infer which T800s are to be avoided in our last example (based on whether the individual kills humans or not.)

At the very least, we should be able to write a query to determine whether a T800 is a bad guy or not. However, that’s out of the scope of this tutorial at this point; I mostly wanted to preserve what I knew of OWLAPI for others to use.

Java Authors, Learn Java

I was reading a book on Neo4J and encountered this sentence:

What is the secret of Neo4J’s speed?

No, Neo4j developers haven’t invented a superfast algorithm for the military. Nor is Neo4j’s speed a product of the fantastic speed of the technologies it relies on (it’s implemented in Java after all!) (Emphasis mine. Quote is from Manning Publications’ “Neo4j in Action,” by Vokotic, Watt, and others.)

Come on, people! It’s 2015 now! Isn’t it time to put away the myth that Java’s intrinsically slow?