Repost: Half-docs, non-docs, and… docs

I found a reference this morning on OSGi’s “buddy classloading” in reference to Spring, OSGi, and GigaSpaces. In addition to being, like, informative (thanks, Steve), it also had a reference to something Steve called “non-docs”:

To press button B, click the B button.

That’s perfect. It sums up OSGi documentation perfectly. But there’s another level of poor documentation, far more common when it comes to tech: half-docs.

Imagine this sort of thing, if you would:

Half-docs are what you get when something is documented, but not documented cleanly or illustrated well. You might have references. You might have hundreds of pages of references, in fact. But it’s all stuff that assumes you grok everything; just like I mentioned in the “what happened to JDO?” post, it’s a quality of reference that serves you well provided you didn’t use the reference to get started in the first place.

Look at Spring-DM. Let’s be clear here: I love this stuff. (Well, platonically.) Spring and OSGi is a disruptive technology, in the best possible way.

But getting it to work? This requires lots and lots of stab and attack for me, where I try something, get a failure, try to figure out why, try something, get a failure, try to figure out why, rinse and repeat ad nauseum. Eventually something sort of works, and I’m wondering why, still.

When I add a custom Spring schema to an OSGi module, this is what the docs tell me, from section 5.4 of the Spring-DM docs:

All bundles deployed in the OSGi space (whether they are Spring-powered or not) are scanned by Spring-DM for custom Spring namespace declaration (by checking the bundle space for META-INF/spring.handlers and META-INF/spring.schemas). If these are found, Spring-DM will make the schemas and the namespaces available through an OSGi service that will be automatically used by Spring-powered bundles. This mean that if you deploy a bundle that uses a custom schema, all you have to do is deploy the library that provides the namespace parser and the schema.

Well… okay. I don’t know what those files are supposed to contain; are they just supposed to exist? (I mean, hey: checking for META-INF/spring.schemas would yield a hit, right?)

Well… no. According to Appendix B (section 5) of the Spring documentation, it’s a little more involved; these are plain text files that resolve a URL to where the schema or namespace handlers are in the bundle itself.

Okay, fine; but adding this yielded a sequence of classloader issues that stretched for maybe seven packages, so that the NamespaceHandler (which worked beautifully and transparently, BTW, once the XSD was located) could find all of the Spring classes it needed.

I had declared a dependency on the Spring OSGi modules in my module, though, which makes me wonder why I had to specify all those imported bundles… or why the specification of all of those bundles wasn’t made explicit to me when I look at the docs.

Thus: half-docs. They make sure the information’s available to you, but only if you look hard. Bleah.

This actually sums up my approach to trying to write docs, BTW: “What other resources are needed in addition to this doc, in order for someone to be able to run what I’m showing them?” I’m sure I’m not perfect; in addition to getting things wrong sometimes (as the Terracotta people are fond of pointing out), I have my own assumptions to deal with, where I know something and don’t realise when others don’t.

I try, though.

Author’s Note: This is a repost from maybe 2009 or so.

Repost: Adding high availability in Terracotta DSO

Terracotta DSO is a package for distributing references in a heap across virtual machines. (Thus: Java. I thought it included C#, but Geert Bevin reminded me that I’m an idiot.) That means that if you have a Map, for example, you can set it to be shared, and your application can share it with other VMs without having to be coded for that purpose.

The only real requirement on the part of your code is that it be written correctly. As in, really correctly. (Want help with this? Check out Using Terracotta for Configuration Management, an article I helped write for TheServerSide.com.)

Luckily, DSO helps you do that by pointing out failures when you screw up.

Anyway, the way DSO works topologically is through a hub and spoke architecture. That means that your VMs are clients, tied to a central server. The hub, then, might seem a potential failure point; if your hub goes down, your clients stop working.

That would be bad. Luckily, DSO has availability options to change the nature of what the “hub” looks like.

The DSO client uses a configuration file named by default as “tc-config.xml.” It has a reference to a server instance that looks something like this:

<servers>
  <server host="localhost">
    <data>%(user.home)/terracotta/server-data</data>
    <logs>%(user.home)/terracotta/server-logs</logs>
  </server>
</servers>

Note the server hostname: it’s important, surprisingly enough. Adding high availability to DSO is only a matter of changing this block in your configuration file.

What we’re configuring here is active/passive failover. That means there’s a primary server instance and a passive server instance. The two server instances sync up, so if the primary goes down, the passive has all of its data; the clients are perfectly able to switch from using one server to the other as their active status changes, so from the client perspective, if the primary server dies, nothing happens.

When the former primary instance returns, well, it’s not the primary any more; it becomes the passive server instance. From the client perspective, all of this is under the covers.

This is Very Good.

So: here’s a configuration I used (tested with Vista running one DSO server with ip 192.168.1.106, running the other DSO server under a Windows XP VMWare image with ip 192.168.1.132):

<servers>
  <server host="192.168.1.106" name="m1">
    <data>%(user.home)/terracotta1/server-data</data>
    <logs>%(user.home)/terracotta1/server-logs</logs>
    <dso>
      <persistence>
        <mode>permanent-store</mode>
      </persistence>
    </dso>
  </server>
  <server host="192.168.1.132" name="m2">
    <data>%(user.home)/terracotta2/server-data</data>
    <logs>%(user.home)/terracotta2/server-logs</logs>
    <dso>
      <persistence>
        <mode>permanent-store</mode>
      </persistence>
    </dso>
  </server>
</servers>

When you start the server, you use a command line argument, like this, with the configuration file in $TERRACOTTA/bin:

./start-tc-server.sh -n m1

This tells the server instance to use the configuration “m1” – yes, I know, but I didn’t have hostnames setup and I wasn’t sure how NAT would affect what I was doing – and the two instances will work out between them which is active and which is passive.

The end result is that I was able to run my shared map program while changing the servers around – starting and stopping them in-process, for example – without the client program knowing or caring. (Note that the client program needs the same basic configuration data; it can get this by having its own copy of the configuration file, which is what I did, or it can load it from the servers, too, or any number of variants. It just needs the relevant data. Happy now, Geert?)

A side-benefit of the configuration is that the DSO cache is persistent – you don’t need high availability for persistency, but you do need persistency for high availability.

It’s good stuff.

Continue reading “Repost: Adding high availability in Terracotta DSO”

Repost: Whatever happened to JDO?

Someone on IRC asked about JDO (Java Data Objects) this morning: “Why isn’t JDO all that widespread?” Well… that’s a good question. JDO was supposed to be such a great technology that it really should have done much better than it did in the industry.

Let’s be real here: JDO wasn’t bad technology at all. Kodo, for example, was used by TheServerSide.com and it worked quite well. (There was a performance… “problem” in October 2005, that had nothing to do with Kodo and everything to do with a craptastic use case of Kodo. Since then, Kodo’s parent company, SolarMetric, was bought by BEA, and subsequently snapped up by Oracle. I don’t know Oracle’s plans for Kodo; the sort-of-Oracle link to Kodo may or may not work for you, like the links to other products Oracle bought from BEA.)

Kodo wasn’t the only implementation of JDO, either; there were other commercial vendors (I’d name them except I’d be going off of memory and I’m pretty sure I’d get them wrong) and there was even an open source implementation, JPox, if memory serves. It wasn’t availability that killed the JDO star.

It was attitude.

Here’s the thing: Hibernate already existed by the time JDO should have taken over. And Hibernate was, for better or for worse, working and simple and people sort of talked about it. JDO… had a book. A series of books, actually. I tried to read them.

They reminded me of the book I have on JavaSpaces. The JavaSpaces book – which I’ve been rereading lately – is a good reference for someone who’s familiar with the technology. Once you know JavaSpaces, the book’s a wonderful refresher course; knowing JavaSpaces like I do (I’m pretty good at it, if I say so myself), the book is very clear, makes perfect sense, tells me stuff I need to know.

But I know it already, you say! Well… yeah.

There’s the problem. When I started to use JavaSpaces, the book was… interesting. And nothing more. It didn’t even tell me where to find a good implementation! It didn’t tell me how to get started with JavaSpaces – it told me how to keep going once I got started.

Bzzt. There’s the flaw; JDO had the same attitude of “We’ll help you rock once you start rocking, but getting started? You’re on your own.”

Kodo – the JDO implementation with which I’m the most familiar – did a decent job of helping you get started, but it still didn’t give me the warm fuzzies Hibernate or JPA or even db4o did. I don’t know what the quantitative differences were. I’ve tried to figure them out, but I still can’t tell you what the secret sauce was, besides something in the attitude.

All I can tell you is that it’s the same thing for a lot of disruptive, fantastic technologies. JSF managed to fix it, but JDO didn’t; JPA managed to fix it (by being part of a specification, at least), but SDO didn’t; JCache managed to fix it (by being implemented in lots of cache specifications, even if the spec itself is incomplete and has gone nowhere) but JINI and JavaSpaces didn’t.

Ouch.

Author’s Note: This is one of the old blog posts I decided to rescue from the migration to my new site. I don’t remember when it was originally posted; maybe 2008? Anyway, it had some interesting things I thought were worth saving.