Saturday, August 15, 2009

Hudson: svn authentication cancelled

The Hudson was building all projects without any issues since it was installed. All of a sudden yesterday my inbox was filled with failure reports, Hudson was not able to build any of the projects. Logs were having stacktrace like mentioned below-

ERROR: svn: authentication cancelled
org.tmatesoft.svn.core.SVNCancelException: svn: authentication cancelled
    at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.cancel(SVNErrorManager.java:37)
    at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.cancel(SVNErrorManager.java:32)
    at org.tmatesoft.svn.core.internal.wc.DefaultSVNAuthenticationManager.getNextAuthentication(DefaultSVNAuthenticationManager.java:200)
    at hudson.scm.FilterSVNAuthenticationManager.getNextAuthentication(FilterSVNAuthenticationManager.java:42)
    at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:537)
    at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:273)
    at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:261)
    at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:516)
    at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:98)
    at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1001)
    at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.testConnection(DAVRepository.java:97)
    at hudson.scm.SubversionSCM$DescriptorImpl.checkRepositoryPath(SubversionSCM.java:1686)
    at hudson.scm.SubversionSCM.repositoryLocationsExist(SubversionSCM.java:1826)
    at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:506)
    at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:469)
    at hudson.model.AbstractProject.checkout(AbstractProject.java:898)
    at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:391)
    at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:340)
    at hudson.model.Run.run(Run.java:1090)
    at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:301)
    at hudson.model.ResourceController.execute(ResourceController.java:93)
    at hudson.model.Executor.run(Executor.java:122)
Skipping sonar analysis due to bad build status FAILURE
Finished: FAILURE


The stack trace talks about authentication cancellation not failure, which is quite strange. I still thought that it might be an authentication failure and tried to authenticate against SVN server using same credentials as used by Hudson. Authentication was successful using various methods (browser, command line client, TortoiseSVN). Google search did not provided any relevant information and I was running out of clues :-(

After putting lots of pondering I remembered that day before I did a manual (from console) project checkout on the same machine and during checkout svn client (1.6.4) asked to save the password and I said 'no'. Will that make any difference? I think it should not.

Just for the sake of trial I deleted the '~/.subversion' directory and did a Hudson build after that. Now Hudson was able to do SVN checkout/update and build project. After several trials, I concluded that if password is not saved during manual checkout, Hudson will fail with above-mentioned error. Though it will wok fine if password was saved. Another observation was that Hudson SVN client (SVNKit 1.3.0) does not create '~/.subversion/auth', while 'svn' command does, irrespective of one chooses to save the password or not.

Though I do not know why Hudson (SVNKit) cancels the SVN authentication but removing the '~/.subversion' directory solves the issue.

16 Comments:

Kerry Wilson said...

thanks a lot, you just saved me some time.

Anonymous said...

Thanks!

Anonymous said...

Although, it makes me wonder why things were working fine before... all of my builds failed this morning without any changes to the workspace or job setting... just running them again fixed the issue... very strange.

Anonymous said...

Thanks a ton Vinod. This helped me save a lot of time. Was wondering what had gone wrong and was even thinking of re-installing hudson!!

Anonymous said...

I was trying to write a ant script to checkout code from svn.I did got a similar error something like
checkout:
[svn] started ...
[svn] svn: authentication cancelled
[svn] svn: authentication cancelled
[svn] failed !

dont know where can I find ~/.subversion directory to delete it.I am trying to checkout for the very first time and its failing.However when trying the same credentials via TortoiseSVN ,it goes without any error.I am kind of new to this and urgent help is needed.
Thanks

Vinod Singh said...

~/ means current user's home directory.

krishna said...

I get the same error. But I don't see a .subversion directory under ~/ . Please help

Anonymous said...

I'm having the same problem, but deleting the .subversion directory doesn't solve the problem. This seems like a fundamental flaw in hudson which makes it unusable for me, I'm going to have to look for a different CI build tool. Can't believe this HUGE bug has been outstanding for over 3 years and they haven't bothered fixing it.

Gawon said...

1. Make sure you update the credentials in the job configs / (you may want/need to override global settings)
2. check log on svn server to find our which user is being used
3. There is no need to delete the .subversion directory of the user hudson is running with. The clearing the subfolder "auth" of all contained files is sufficient.

Anonymous said...

Clearing auth is not a sane option as well as erasing whole ~/.subversion.

I've looked inside of the auth files after this bug was hit and i see that passwd information is missing, i.e. hudson erased it (in direct or indirect way).

after commands like:

chown root -R ~hudson/.subversion/auth/svn.simple;
chmod 755 ~hudson/.subversion/auth/svn.simple;
chmod 644 ~hudson/.subversion/auth/svn.simple/*

i haven't observed this issue again.

Anonymous said...

Same problem ... removing ~/.subversion' directory don't solves the issue

Anonymous said...

Thanks for the helpful post. Your particular solution did not solve the problem for me, but I did eventually find a solution in the Jenkins users mailing list here: http://jenkins.361315.n4.nabble.com/Cannot-create-SVN-authentication-credentials-td3433093.html. Essentially, the problem is that the auth folder is not created under the Subversion folder and so there are no credentials stored on disk for the Subversion plugin to use. If you download an svn client and execute a command that requires authentication, the svn plugin for Hudson/Jenkins will start working again. This looks to be quite the flaw in the svn plugin with several bugs reported to both the Hudson and Jenkins teams that has not yet been fixed.

Om Prakash Mohapatra said...
This comment has been removed by the author.
Om Prakash Mohapatra said...

"After putting lots of pondering I remembered that day before I did a manual (from console) project checkout on the same machine and during checkout svn client (1.6.4) asked to save the password and I said 'no'. Will that make any difference? I think it should not." - This helped me in resolving that. I did a SVN update manually and opted for remember password; then started the build and it ran successfully.

Anonymous said...

Thank you - this helped resolved my issue - our jenkins had master and 2 slaves - the error stack was as follows:

ERROR: Failed to check out https://svn.yourcompanydomaingoeshere.com/svn/yoursvndirectorypathgoeshere
org.tmatesoft.svn.core.SVNException: svn: OPTIONS /svn/yoursvndirectorypathgoeshere failed
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:298)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:283)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:271)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:533)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:98)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1011)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getLatestRevision(DAVRepository.java:180)
at org.tmatesoft.svn.core.wc.SVNBasicClient.getRevisionNumber(SVNBasicClient.java:482)
at org.tmatesoft.svn.core.wc.SVNBasicClient.getLocations(SVNBasicClient.java:876)
at org.tmatesoft.svn.core.wc.SVNBasicClient.createRepository(SVNBasicClient.java:534)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doCheckout(SVNUpdateClient.java:901)
at hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:84)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:136)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:788)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:769)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:753)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2309)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.tmatesoft.svn.core.SVNErrorMessage: svn: OPTIONS /svn/yoursvndirectorypathgoeshere failed
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:200)
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:146)
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:89)
... 26 more
...
FATAL: null
java.lang.NullPointerException
at java.util.ArrayList.addAll(ArrayList.java:530)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:743)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:685)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1325)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:676)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:581)
at hudson.model.Run.execute(Run.java:1516)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:236)

Eric Herrmann said...

Note that you may also get this exception when when checking out a URL that contains svn:externals.

Hudson needs to have separate credentials entered for each location that your main SCM location is externaling to. The workaround is to add each external as a separate SCM location, adding credentials for each, and then delete the locations. The credentials will persist in the job's subversion.credentials file.