How It Works

Through an exclusive partnership with HAWK Network Defense, Clear Technologies has solved the error-prone, time-consuming management of logs to prevent infiltration events. HAWK Network Defense has a patent-pending Naïve-Bayesian Histogram Analysis technology that transforms the tedious and time consuming tasks of event logging into a dynamic, powerful experience that proactively mitigates risk. Not only will the analyst be able to rely on experience of the tool to prevent threats, but also be able to utilize his own experience by writing, through regular expression, rules that will place a score on specific inter-organizational nuances which are not a threat.

How it Works:

Dynamic Log Analysis™ enables the average analyst to utilize a team of resources that can differentiate events that are not of threat, so that real threats can be identified and prevented.  Dynamic Log Analysis™ refers to an event driven solution that iteratively assesses the probability that certain types of events will produce a threat. Using a Naïve-Bayesian Histogram algorithm (refer to Bayesian Histogram Analysis Explained – by Phil Godwin) to assign ‘scores’ as well as utilizing Boolean rule sets, the system learns and places importance on certain types of correlated events. The system then assigns a ‘score’ to the threat. Dynamic Log Analysis’s scoring technology determines the priority of an ‘event’ for alerting and responding and its Multi-Decision Tree Matching Algorithm increases speed of matching of events to rules developed by the administrator.  By combining these two processes, the time to identify, respond and remediate an event is greatly reduced.   

Scoring

Just as a team of ‘super-detectives’ uses their shared experiences to identify and place emphasis on significant threats, the Bayesian Histogram algorithm and Boolean Ruleset assigns a score to define the magnitude of a threat. The ‘score’ is then placed in the database and the administrator is alerted on the most perilous threats. The unique total score is determined by utilizing the naïve Bayesian learning algorithm, the Boolean rule-set, as well as information acquired during the normalization and matching process.  All of the gathered information is taken into account before the total score is determined.

In its simplest form, the solution performs the following:

Once the event, which is any user action, log entry, security notification, and performance statistic, has been selected for processing, its contents are inserted into the database.  After database insertion, the event goes through the unique multifaceted scoring process that first includes a determination of the naïve Bayesian score by analyzing the standard deviation. The system is then able to match against those target events that have not been previously identified. In addition, this Naïve-Bayesian algorithm is specifically designed to match against known or trained information.  Together, the engine establishes an operating baseline, and to looks for deviations against this standard norm.

Next, Bayesian score is included along with the existing event properties to be processed by the Boolean rule-sets, which is list of rules associated with a positive or negative score.  Once a Boolean rule-set is matched against a provided event, the associated score is added to the existing score, which in most cases is zero. Once all the rules have been compared against the event, a total score is determined, allowing future actions to be taken based upon the pre-configured score threshold.

At this stage, the unique total score only applies to a single event.  By assigning each event a unique score, an analyst is able receive alerts on isolated, specific events that exceed a specified score threshold. In addition, isolating and assigning a unique score to each event enables the analyst to conduct a trend analysis and rapidly adjust to changes in overall activity.

Dynamic Log Analysis’s Multi-Decision Tree Matching Algorithm

In the same way a team of ‘super-detectives’ relies on their shared knowledge and experiences in order to quickly match threats to specific, predetermined high-risk behavior, the decision and matching technology then matches the provided event to its related ‘rule’ faster.  This technology is designed in three layers.

When an event is received by the Dynamic Log Analysis™ engine, it converts the received information into a normalized event, matches it against its pre-defined rule set and is then separated into two types; compiled modules, and a textual rule-set.  The textual rule-sets are separated into three basic classifications that provide the means for matching against our rule-set: triggers, rule-groups, and rules.  A trigger is a regular expression that must match a threat in order for the rules within the module to continue processing.  If it does, the event proceeds to one of the rules groups and within the rule group, a rule is applied. A rule contains all the given information Dynamic Log Analysis™ requires for improved matching, correlation, and scoring.  Each rule contains the alert name, category, knowledgebase id, host and network packet information, as well as audit procedure information for compliance monitoring and scoring. The final rule, upon successful match, allows the administrator to assign the specific information to the event’s normalized hash table.  The final rule allows for multiple matching rules as well as using the ‘not’ indicator.  Once these activities have been completed, the event is passed into the processing queue for archiving, scoring and additional correlation.

Dynamic Log Analysis’s Information Event Console

Lastly, in the same manner that a team of ‘super detectives’ combines all of their respective experiences and knowledge into one shared, cohesive view to visualize the extent of the threat, the Dynamic Log Analysis™ Information Event Console presents an overall view of the highest and lowest priority alerts, all arranged by severity of correlation.   Further, it acts as the management and data retrieval interface with the relational database, provides a historical retrieval of logged information, and, over secure encrypted sessions, provides role based access controls.