Apache Eagle v0.4.0发布，一个高效分布式的流式策略引擎

   <p style="text-align: center;"><img alt="" src="https://simg.open-open.com/show/e8c1df5024fa87531298f5f0c1e3571a.png" /></p>    <p>Apache Eagle 是 eBay公司开源的分布式实时安全监控，提供了一套高效分布式的流式策略引擎，具有高实时、可伸缩、易扩展、交互友好等特点，同时集成机器学习对用户行为建立Profile以实现实时智能实时地保护Hadoop生态系统中大数据的安全。</p>    <p><img alt="" src="https://simg.open-open.com/show/a7e7b47196a41f7fc695c7659c4f102c.png" /></p>    <p>Eagle的数据行为监控方案可用于如下几类典型场景：</p>    <ul>     <li>监控Hadoop中的数据访问流量</li>     <li>检测非法入侵和违反安全规则的行为</li>     <li>检测并防止敏感数据丢失和访问</li>     <li>实现基于策略的实时检测和预警</li>     <li>实现基于用户行为模式的异常数据行为检测</li>    </ul>    <p>Eagle具有如下特点：</p>    <ul>     <li>高实时： 我们充分理解安全监控中高度实时和快速反应的重要性，因此设计Eagle之初，我们竭尽可能地确保能在亚秒级别时间内产生告警，一旦综合多种因素确订为危险操作，立即采取措施阻止非法行为。</li>     <li>可伸缩：在eBay Eagle 被部署在多个大型Hadoop集群上，这些集群拥有数百PB的数据，每天有8亿以上的数据访问时间，因此Eagle必须具有处理海量实时数据的高度可伸缩能力。</li>     <li>简单易用：可用性也是Eagle产品的核心设计原则之一。通过Eagle的Sandbox，使用者仅需数分钟便可以设置好环境并开始尝试。为了使得用户体验尽可能简单，我们内置了许多很好的例子，只需简单地点击几步鼠标，便可以轻松地完成策略地创建和添加。</li>     <li>用户Profile：Eagle 内置提供基于机器学习算法对Hadoop中用户行为习惯建立用户Profile的功能。我们提供多种默认的机器学习算法供你选择用于针对不同HDFS特征集进行建模，通过历史行为模型，Eagle可以实时地检测异常用户行为并产生预警。</li>     <li>开源：Eagle一直根据开源的标准开发，并构建于诸多大数据领域的开源产品之上，因此我们决定以Apache许可证开源Eagle，以回馈社区，同时也期待获得社区的反馈、协作与支持。</li>    </ul>    <h2>更新日志</h2>    <p><strong>** Highlights **</strong><br />     * JBDC Metadata Storage Extension<br />     * Topology management in remote mode including start/stop/status operations<br />     * Auditlogparser for MapR's audit log<br />     * Oozie auditlog integration for Oozie security monitoring<br />     * Add applicaiton "maprFSAuditLog"<br />     * Refactor bin/eagle-sandbox-starter.sh to make it easier to use</p>    <p><strong>新特性</strong><br />     * [EAGLE-169] - Dynamic security event correlation in Eagle<br />     * [EAGLE-203] - Metrics feature support merge chart<br />     * [EAGLE-225] - Create eagle bootstrap scripts for examples <br />     * [EAGLE-226] - Refactor Eagle scripts to avoid heavily depending on Hortonworks Sandbox<br />     * [EAGLE-232] - Create local Kafka/Zookeeper/Storm runner tools for quickstart examples and add related scripts to start/top zk/kafka<br />     * [EAGLE-238] - Support scheduling topology in local mode including start/stop/status operations<br />     * [EAGLE-266] - Integrate MkDocs for eagle-docs: http://www.mkdocs.org/<br />     * [EAGLE-271] - Topology management in remote mode including start/stop/status operations<br />     * [EAGLE-272] - Support topology management in UI including creating topology and monitoring status<br />     * [EAGLE-282] - Auditlogparser for MapR's audit log <br />     * [EAGLE-284] - Connect to MapR's CLDB service<br />     * [EAGLE-298] - Oozie auditlog integration for Oozie security monitoring<br />     * [EAGLE-307] - Add applicaiton "maprFSAuditLog" </p>    <p><strong>改进</strong><br />     * [EAGLE-103] - add comments to readme to tell users: currently, eagle is tested under jdk1.7.x, may have compile error with jdk1.8.x<br />     * [EAGLE-182] - Replace Legacy "dataSource" field with "application" in UI request<br />     * [EAGLE-185] - UI create cache after building<br />     * [EAGLE-190] - JBDC Metadata Storage Extension<br />     * [EAGLE-193] - UI metric dashboard support sortable<br />     * [EAGLE-194] - UI show exception alert if service error<br />     * [EAGLE-195] - policy metric display with interval of 5 min or customized interval<br />     * [EAGLE-196] - eagle-topology.sh should have jar file path as parameter<br />     * [EAGLE-201] - Change maven group name to org.apache.eagle instead of eagle<br />     * [EAGLE-205] - Metric dashboard support multi metrics<br />     * [EAGLE-207] - Management page add tips<br />     * [EAGLE-208] - UI metric dashboard should support order & rename<br />     * [EAGLE-216] - Added RM Policy and GC Policies in Resource<br />     * [EAGLE-223] - Notification plugin to enable multiple instance of given alert plugin <br />     * [EAGLE-237] - Add development tools for quickly starting zookeeper, kafka and webservice without depending on sandbox<br />     * [EAGLE-248] - Rename directories according industrial common sense<br />     * [EAGLE-287] - Make EagleStore as the default notification method<br />     * [EAGLE-288] - Need to add "Alert De-Dup Interval" setting in "PolicyObjectBase" <br />     * [EAGLE-295] - Add configuration value to enable application Manager<br />     * [EAGLE-303] - Refactor message format in the email template.<br />     * [EAGLE-305] - Add a config tip to the document for "Application Manager Tutorial" - setting "appCommandLoaderEnabled=true"<br />     * [EAGLE-306] - add metadata for showing "Topology" tab in left-nav by default<br />     * [EAGLE-315] - Add tutorial for mapr audit log monitoring<br />     * [EAGLE-316] - Feature topology should not be added into an application<br />     * [EAGLE-339] - Create HBase tables if not exists <br />     * [EAGLE-340] - refactor bin/eagle-sandbox-starter.sh to make it easier to use </p>    <p><strong>Bug修复</strong><br />     * [EAGLE-8] - In eagle-check-env.sh shell , Itbad way to check kafka installation<br />     * [EAGLE-18] - Follow up with infra about website creation<br />     * [EAGLE-157] - policy metric should be refreshed every minute<br />     * [EAGLE-171] - Policy listing table is messed up by too long policy name.<br />     * [EAGLE-172] - Scripting string is allowed to create policy rules.<br />     * [EAGLE-173] - Mark/Un-mark a sensitivity type does not sync status mark in the table list.<br />     * [EAGLE-176] - Metric dashboard UI keep api refresh after page switch<br />     * [EAGLE-192] - Uncaught ReferenceError: damControllers is not defined (doc.js:7628)<br />     * [EAGLE-200] - GC Log Monitoring  Not Working<br />     * [EAGLE-210] - UI application group not display correctly<br />     * [EAGLE-211] - Fix sometime unit test failing at TestSiddhiStateSnapshotAndRestore<br />     * [EAGLE-212] - Fix AlertDataSourceEntity Bug in Hive web<br />     * [EAGLE-213] - Updates fail for MySql  <br />     * [EAGLE-214] - Policy edit page need auto switch application<br />     * [EAGLE-217] - Fix unstable unit tests about state snapshot management<br />     * [EAGLE-224] - Column not found to EAGLE_METRIC when using JDBC<br />     * [EAGLE-227] - java.lang.NoClassDefFoundError: org/apache/commons/pool/impl/CursorableLinkedList$ListIter<br />     * [EAGLE-228] - org.apache.eagle.notification.plugin.NotificationPluginManagerImpl - fail invoking plugin's onAlert, continue  java.lang.NullPointerException: null<br />     * [EAGLE-229] - java.lang.IncompatibleClassChangeError: class net.sf.extcos.internal.JavaResourceAccessor$AnnotatedClassVisitor has interface org.objectweb.asm.ClassVisitor as super class<br />     * [EAGLE-230] - Exception in persisting entitiesService side exception: org.codehaus.jackson.map.JsonMappingException: Conflicting setter definitions for property "alertContext"<br />     * [EAGLE-235] - org.codehaus.jackson.map.JsonMappingException: Conflicting setter definitions for property "alertContext"<br />     * [EAGLE-239] - Alert list and details are not correctly displayed<br />     * [EAGLE-240] - java.lang.ArrayIndexOutOfBoundsException thrown by MetricKeyCodeDecoder<br />     * [EAGLE-242] -  Import the notification plugin metadata when initializing<br />     * [EAGLE-254] - HdfsAuditLog topology keeps alerting for one piece of log<br />     * [EAGLE-258] - Automatically add apache-github and apache-git in pr tools<br />     * [EAGLE-269] - Comparisons between 'LONG VARCHAR (UCS_BASIC)' and 'LONG VARCHAR (UCS_BASIC)' are not supported<br />     * [EAGLE-270] - JDBC: Create table fail for some of the tables<br />     * [EAGLE-273] -  Issue with creating MySql tables , only 14 were created out of 24, reason being varchar(30000) for multiple columns lead to exceeding the maximum row size of 65,535 bytes.<br />     * [EAGLE-274] - 2016-04-15 15:50:20 b.s.d.worker [ERROR] Error on initialization of server mk-worker java.lang.RuntimeException: java.lang.ClassNotFoundException: org.slf4j.impl.Log4jLoggerAdapter<br />     * [EAGLE-275] - Eagle email alert bug: $elem["dataSource"] Alert Detected<br />     * [EAGLE-291] - JDBC: Update transactions fail in PostgreSQL<br />     * [EAGLE-292] - Updated hbase policy failed: Data too long for column 'policyDef' when using mysql storage<br />     * [EAGLE-294] - If a policy metadata field is not set, null attributes can not be able to add into input stream for SiddhiCEP <br />     * [EAGLE-297] - Email with authentication can not be validated and sent out.<br />     * [EAGLE-300] - Disable spring debug log by default in webservice<br />     * [EAGLE-301] - Tables omitted for using mysql<br />     * [EAGLE-304] - Enable Advanced dedup configuration in policy definition <br />     * [EAGLE-308] - Consistency issue: deleting a topology doesn't delete existing topology-execution bound to it.<br />     * [EAGLE-310] - already existing active topology status not displayed when a deleted topology+execution re-created with same name<br />     * [EAGLE-311] - operations of items listed on topology-management monitoring page require buffering loading approaches<br />     * [EAGLE-313] - normally stopped topology-execution shows error message in the description column<br />     * [EAGLE-319] - java.sql.SQLSyntaxErrorException caught when querying from table topologyExecutionEntity<br />     * [EAGLE-321] - java.lang.NoSuchMethodError: com.google.protobuf.LazyStringList.getUnmodifiableView<br />     * [EAGLE-326] - typo found in eagle documentation<br />     * [EAGLE-327] - java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer<br />     * [EAGLE-330] - Hive ql.Parser can't parser a hive query sql with keywords<br />     * [EAGLE-338] - fix topology-assembly build issue because of module name change<br />     * [EAGLE-346] - ClassNotFoundException thrown out when topology is executing<br />     * [EAGLE-355] - UI advanced policy expression can't parse<br />     * [EAGLE-356] - Fix Authentication problem to query resource manager web service</p>    <p><strong>Task</strong><br />     * [EAGLE-73] - Put docker steps to site tutorial<br />     * [EAGLE-221] - Support cusomized notification type in policy editor<br />     * [EAGLE-222] - Documentation for eagle alert plugin mechnism<br />     * [EAGLE-280] - Update logstash-kafka-conf.md<br />     * [EAGLE-309] - Add code formatter template</p>    <p><strong>Sub-task</strong><br />     * [EAGLE-219] - Use PUT method for updating request when possible in front-end.</p>    <h2>下载</h2>    <ul>     <li><a href="/misc/goto?guid=4958992387262374095">apache-eagle-0.4.0-incubating-src.tar.gz</a></li>    </ul>