Monday, November 25, 2013

Building Hadoop cannot access org.mortbay.component.AbstractLifeCycle error or class file for org.mortbay.component.AbstractLifeCycle not found

Here is a snippet of the error message I got.


[ERROR] /home/dc/bigtop-0.6.0/build/hadoop/rpm/BUILD/hadoop-2.0.5-alpha-src/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[86,13] cannot access org.mortbay.component.AbstractLifeCycle
class file for org.mortbay.component.AbstractLifeCycle not found
    server = new Server(0);
[ERROR] /home/dc/bigtop-0.6.0/build/hadoop/rpm/BUILD/hadoop-2.0.5-alpha-src/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[96,29] cannot access org.mortbay.component.LifeCycle
class file for org.mortbay.component.LifeCycle not found
    server.getConnectors()[0].setHost(host);
[ERROR] /home/dc/bigtop-0.6.0/build/hadoop/rpm/BUILD/hadoop-2.0.5-alpha-src/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[98,10] cannot find symbol
symbol  : method start()
location: class org.mortbay.jetty.Server
[ERROR] /home/dc/bigtop-0.6.0/build/hadoop/rpm/BUILD/hadoop-2.0.5-alpha-src/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[104,12] cannot find symbol
symbol  : method stop()
location: class org.mortbay.jetty.Server


There are a bunch of web posts on this error. A large percentage of the web posts instruct the user to delete the m2 repo. This won't work. Here is an example of a post.


There is a patch which fixes this problem. The author of the patch adds a maven dependency called jetty-util. 
Let's figure out why this is correct. 

You can look at the contents of jetty jars. The pom for this class hadoop-2.0.5-alpha-src/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java is under the hadoop-auth directory. 

This error is because it can't find org.jetty.components where the Lifecycle and AbstractLifecycle classes are. 

There is one dependency for jetty in the pom.xml file which is: 

      >groupid<org.mortbay.jetty
      jetty
      test

Fortunately there is a website which shows the classes for maven jetty artifacts which correspond to jars. This is much easier than having to track down the jars, expanding them to look for classes or tracking down the source and looking for packages. 

http://grepcode.com/project/repo1.maven.org/maven2/org.mortbay.jetty/jetty/ shows the Jetty versions for the jetty artifact. Click on version 6.0.26 and you can see there is no org.mortbay.component package where org/mortbay/component/AbstractLifeCycle.java should be. 




Change the url to jetty-util to 
http://grepcode.com/project/repo1.maven.org/maven2/org.mortbay.jetty/jetty-util/ and click under 6.0.26 and you can see there is a org.mortbay.component package






Add this dependency to the pom.xml file under hadoop-2.0.6-alpha-src/hadoop-common-project/hadoop-auth/pom.xml

     
      org.mortbay.jetty
      jetty-util
      test
     

which you can also do by downloading the patch text file and running the patch command. If you are using bigtop to build, make the change to the source code, tar and gz the source code, place it under dl, delete the hadoop directory under build/ and run >make rpm again to see the error go away. 






12 comments:

  1. great article... ran into a problem today with the latest build and came across this one while trying to find a solution .

    To add However :

    1) There are two builds for Hadoop that can be downloaded . One is the src which is 19 MB and the other is something of 104 MB .
    2) When you download the patch , you would need to apply it to the 104 MB thing and then rebuild and i was not able to do that .
    2) The pom file entry applies to the 19MB src thing . However realize that each of the projects has a POM file .
    3) So i was trying to add this in the root folder pom.xml and nothing was happening .
    4) What finally helped me was that i navigated to the path :
    http://svn.apache.org/viewvc?view=revision&revision=1543190
    This is where the original changes have been added .
    5) and this tells that this entry needs to be made in the " hadoop-common-project/hadoop-auth/pom.xml "
    6) With this entry now made in the correct pom.xml , this error goes away .

    ReplyDelete
  2. Thank for sharing this great Hadoop tutorials Blog post. I will use your command when upgrade hadoop.
    Hadoop Training in hyderabad

    ReplyDelete
  3. This comment has been removed by a blog administrator.

    ReplyDelete
  4. Thanks for sharing this informative blog, it was a good presentation everything you have shared here was crystal clear... keep blogging. sap training center in Chennai

    ReplyDelete
  5. Thanks for sharing your knowledge with us... keep posting
    Informatica training in chennai

    ReplyDelete

  6. Nice article i was really impressed by seeing this article, it was very interesting and it is very useful for Learners.I get a lot of great information from this blog. Thank you for your sharing this informative blog.AWS Training in chennai | AWS Training chennai | AWS course in chennai


    ReplyDelete
  7. Your posts is really helpful for me.Thanks for your wonderful post. I am very happy to read your post.very nice !!! VMWare course chennai | VMWare certification in chennai | VMWare certification chennai

    ReplyDelete
  8. I really enjoyed while reading your article, the information you have mentioned in this post was damn good. Keep sharing your blog with updated and useful information.
    Regards,

    Python Training in Chennai|Python Training|Fita Chennai reviews|FITA Chennai complaints

    ReplyDelete
  9. Great content thanks for sharing this informative blog which provided me technical information keep posting.
    qlikview training in chennai

    ReplyDelete