micmiu – 软件开发+生活点滴

基本环境参数：

• Mac OSX 10.9.1
• Maven 3.0.4
• Java 1.6.0_65
• Hadoop 2.2.0

源码编译的基本步骤：

1、checkout 源码

ps: hadoop2.2.0的源码有个bug ：修改 <源码跟目录>/hadoop-common-project/hadoop-auth/pom.xml中添加如下依赖：

2、安装 protoc

到 Protocol Buffers 的官网 https://code.google.com/p/protobuf/ 下载2.5.0的安装源文件进行安装：

安装完成后，执行 protoc –vresion 验证是否安装成功。

3. 安装cmake

• 官方（http://www.cmake.org/cmake/resources/software.html）下载对应系统的版本进行安装
• Mac OSX 已经安装了MacPorts 可以直接执行：sudo port install cmake  即可

4. 执行mvn 命令进行编译了

在Mac OSX系统下compile、package 都能成功，如果指定-Pdist,native 生成native lib 不成功，查阅有关官方介绍发现：Hadoop本地库只支持*nix平台，已经广泛使用在GNU/Linux平台上，但是不支持 Cygwin  和 Mac OS X 。当然有人已经给出了Mac OSX系统编译本地库的patch ，具体的方法可以参见文章：

Hadoop 2.x build native library on Mac os x （http://www.micmiu.com/bigdata/hadoop/hadoop-build-native-library-on-mac-os-x/ ）

5.常见错误及解决方法

ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.0.0-SNAPSHOT:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: ‘protoc –version’ did not return a version -> [Help 1]

上面的错误原因是：没有安装protoc

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program “cmake” (in directory “/Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/native”): error=2, No such file or directory
[ERROR] around Ant part …<exec dir=”/Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/native” executable=”cmake” failonerror=”true”>… @ 4:154 in /Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/antrun/build-main.xml

上面的错误原因是： 没有安装 cmake

ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part …<exec dir=”/Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/native” executable=”make” failonerror=”true”>… @ 7:153 in /Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2
around Ant part …<exec dir=”/Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/native” executable=”make” failonerror=”true”>… @ 7:153 in /Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 2
around Ant part …<exec dir=”/Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/native” executable=”make” failonerror=”true”>… @ 7:153 in /Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
… 19 more
Caused by: /Users/micmiu/no_sync/opensource_code/hadoop/trunk/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:7: exec returned: 2

在Mac OSX 系统下编译本地库会由提示上面的错误信息，具体解决办法看上面的 第 【4】小节的介绍。

6、参考：

• http://wiki.apache.org/hadoop/HowToContribute
  
• http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/NativeLibraries.html
• https://issues.apache.org/jira/browse/HADOOP-3659 

—————–  EOF @Michael Sun —————–