I have seeeeeeen the light, brothers and sisters!
Okay, that's a little hokey - but one test with OpenSpaces opened my eyes.
I work for GigaSpaces Technologies. I'm expected to be pro-GigaSpaces, especially since I joined the company because I believe in the product. That said, I'm still learning. Learning a lot.
So: my test was pretty simple. I've got a text classifier, trained per user. What I was doing initially was simply validating that I could load and train (and classify) data on a per-user basis; I was going to optimize by caching the classifier's data so I didn't have to keep loading it from the datastore (which is, in this case, GigaSpaces XAP.)
This used two JVMs: one running the XAP instance (the datastore), the other running the tests, so there was a communication layer between the datastore and the tests.
Four tests, each timed:
- Create a classifier. This involves creating via
new, and then loading all matching data for that user out of the datastore, then adding a listener to the classifier so training events are stored properly. - Create a classifier (repeating step #1!) and then training the classifier with two separate corpora, many times over.
- Repeat step one, so we can measure the time it takes to load a trained classifier.
- Repeat step one - again! - and using the classifier data to get the classifications of two corpora.
The results were stunning. There was no optimization here; step one, repeated over and over, basically did very simple stuff:
Classifier c=new ClassifierImpl();
loadClassifier(c, space);
c.addListener(new ClassifierListener() {...});Over and over and over again. No storage of the data in the classifier at all.
What surprised me was the timing results:
- Step one took ~300ms. (Whoa.)
- Step two: a whopping 1700ms, sometimes more.
- Step three... here's where the test jumped the rails.
It's important to note that this was a successful test. Every test passed, worked perfectly.
When I say "it jumped the rails," it's because the results surprised the heck out of me.
Step three took anywhere from 0 to 3ms. Including the whole
new/loadbit. - Step four: See step three. Same results. 0-3ms.
I'm okay with the slow step two - training is a rare event, in context, and the nature of the training exercises in that method is... really bad. Very wasteful; it stems the corpora over and over and over again, preserves no intermediate steps as part of the training. It's all good.
I still need to measure the impact on the heap, but holy smokes... This product rocks.