tag:blogger.com,1999:blog-6167429707177122136.post1168441826384590257..comments2008-01-05T03:11:39.172-08:00Comments on Quest for the Dawn: Food for thoughtEdson Tirellihttp://www.blogger.com/profile/06799293335230465902noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-6167429707177122136.post-50247004572370934342007-05-12T15:08:00.000-07:002007-05-12T15:08:00.000-07:00Here's some random thoughts, hopefully they make s...Here's some random thoughts, hopefully they make sense. I did think about having a registry like component which keeps the temporal facts in time order. The idea I had is that when an object is asserted, it is passed to the registry. Let's call it temporal registry for a lack of better name.<BR/><BR/>The simplest way I can think of is to have the temporal registry calculate the elapsed time between current time and the top fact in the registry. To keep performance acceptable I was thinking the granularity should be at the minute level. More granular than that, it could be rather expensive, especially in a real-time system where there's a constant stream of events. The registry could easily get overwhelmed.<BR/><BR/>I know firms like fidelity generate upwards of 5% of the trading volume on NYSE and their systems handle 2K transactions per second for 18hours a day. There's a four hour period where the transaction rate dips down, but there's always transactions happening. I believe the time corresponds to the gap between NYSE and the japanese stock market.<BR/><BR/>Anyway, back to the temporal registry. Using the real system clock probably isn't practical for a couple of reasons. The trading systems I know of use synchronized time, so it's not the system time.<BR/>The engine could use an internal timestamp, assuming the system time is already insync with GMT time.<BR/><BR/>The downside I see with a temporal registry is this. At best, it can only gaurantee the fact won't be retracted before the expiration time. It won't be able to gaurantee the fact will be removed exactly at expiration time. The rule engine can only promise that any matches after the expiration, will not fire a rule.<BR/><BR/>Johan Lindberg actually tried the polling approach in his python RETE engine and he saw the CPU usage go crazy :)<BR/><BR/>Chances are, the events will arrive out of order, especially trading systems. In the case of fidelity, they have offices in Boston, NY, Los Angeles, SF, Tokyo, London, and a half dozen other major cities. Those systems are literally going all the time, and the latency varies even in their own private network. For a trading system, I think the expiration time for the order should be set when the order is sent. That means the rule engine doesn't have control, and it also implies an absolute expiration time. As long as the rule engine can gaurantee the OMS ignores stale orders and remove them lazily, that's the best it can do.<BR/><BR/>A cheaper way of doing the temporal registry is to use a service rule as I mentioned. In an event driven system, the agenda should have at most 1 activation. This is because the activations are fired immediately and the rule have to be written properly. After the activation fires, the rule engine can run the service rule to remove expired facts. I would do it at the assert method. The pseudo logic might look like this.<BR/><BR/>assertObject(Object)<BR/> rootnode.assert(object)<BR/> // run the service rule(s)<BR/> executeServiceRule<BR/><BR/>When the executeServiceRule runs, it should retract all stale facts. This approach assumes activations are fired immediately, similar to JESS runUntilHalt. In jamocha, the rule would be declared (no-agenda true). I like explicitly declaring a rule by-passes the agenda, rather than runUntilHalt. Even in an event driven systems, there are times when I want a rule to not fire immediately. This gives users greater flexibility and mixed mode execution.<BR/><BR/>hope that makes some sense.woolfelhttps://www.blogger.com/profile/13814445471254728002noreply@blogger.com