Spring for Apache Hadoop 1.0 RC1 发布
jopen 12年前
Spring Hadoop为开发 Apache Hadoop 应用程序提供支持,充分利用 Spring, Spring Batch 和 Spring Integration 等项目提供的功能。
Changes in version 1.0 RC1 (2012-10-07) --------------------------------------- General * 引入 Hive, Pig 运行器用于执行声明脚本 * 重构 (Cascading, M/R, Hive, Pig) 运行器,用 Callables 代替 FactoryBeans * 重命名 'pig' 至 'pig-factory' 和 'pig-ref' 至 'pig-factory-ref' * 重命名 'hive-client' 至 'hive-client-factory' 和 'hive-client-ref' 至 'hive-client-factory-ref' * 引入所有运行器(Cascading, M/R, Hive, Pig) 的执行前和执行后的动作 * 引入嵌入式执行 Hadoop Jar * 改进 spring-hadoop.xsd 命名空间 * 改进,完善和扩大的参考文档 * 改进 artifacts pom * 升级至 Spring Batch 2.1.9 * 升级至 Hive 0.9.0 * 升级至 Pig 0.10.0 * 升级至 Gradle 1.2 Package o.s.data.hadoop.cascading * Introduced FlowFactoryBean Package o.s.data.hadoop.configuration * Fixed potential cycle with FileSystem url registration Package o.s.data.hadoop.fs * Added codecs support to hdfs resources * Refined DistributedCache fragment creation for CDH4/Hadoop 0.23 distros * Introduced options for closing the FileSystem * Fine-tuned the DistributedCache API for setting cache entries Package o.s.data.hadoop.hbase * Refined resource management of HBase tables Package o.s.data.hadoop.hive * Addressed swallowed exception occuring script execution * Improved HiveQL parsing for multi-line statements * Introduced variable binding and substitution per Hive script * Refined namespace to preserve parameter ordering * Introduced HiveClient factory (to deal with thread-safety issues) * Introduced HiveTemplate & callback * Introduced extended exception conversion to DataAccessException * Introduced HiveRunner Package o.s.data.hadoop.mapreduce * Introduced scope attribute for job definitions * Introduced verbose flag to job tasklet * Introduced more options for job and streaming namespace * Introduced jar executor * Refined Tool and Jar execution to prevent class loading leaks * Refactored JobRunner FactoryBean into a Callable * Introduced namespace for job-runner * Removed path validation from JobFactoryBean Package o.s.data.hadoop.pig * Refined namespace to preserve parameter ordering * Introduced PigServer factory (to deal with thread-safety issues) * Introduced PigTemplate & callback * Introduced extended exception conversion to DataAccessException * Refined execution of Pig scripts * Introduced PigRunner Package o.s.data.hadoop.scripting * Refactored HdfsScriptFactoryBean into HdfsScriptRunner * Script definitions no longer cause execution on container lookup