Monday, August 24, 2009

Redmine with Subversion 1.6

Recently I updated SVN client to 1.6.x from 1.5.x on the machine where Redmine was installed. After this change Redmine stopped fetching further revisions from SVN repositories for existing projects and for new projects it was showing following error message on 'Repository' tab-

The entry or revision was not found in the repository.
In Redmine log file I could not find any worthwhile information. There was just one liner-
Completed in 0.34931 (2 reqs/sec) | Rendering: 0.01537 (4%) | DB: 0.01871 (5%) | 500 Internal Server Error [http://myhost/redmine/repositories/show/myproject]
Then I looked inside Apache (Redmine was fronted by Apache + Passenger) error log file, it had error messages like given below-

    -----------------------------------------------------------------------
    ATTENTION!  Your password for authentication realm:

      <http://serverIP:80=""> My SVN

    can only be stored to disk unencrypted!  You are advised to configure
    your system so that Subversion can store passwords encrypted, if
    possible.  See the documentation for details.

    You can avoid future appearances of this warning by setting the value
    of the 'store-plaintext-passwords' option to either 'yes' or 'no' in
    '/home/appsupport/.subversion/servers'.
    -----------------------------------------------------------------------
    Store password unencrypted (yes/no)? svn: Can't read stdin: End of file found


So svn client was asking whether to save the password or not and there was no one to answer that question, leading to abortion of the operation. Uncommenting following two lines in ~/.subversion/config file-
store-passwords = no
store-auth-creds = no
resolved this issue.

Saturday, August 15, 2009

Hudson: svn authentication cancelled

The Hudson was building all projects without any issues since it was installed. All of a sudden yesterday my inbox was filled with failure reports, Hudson was not able to build any of the projects. Logs were having stacktrace like mentioned below-

ERROR: svn: authentication cancelled
org.tmatesoft.svn.core.SVNCancelException: svn: authentication cancelled
    at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.cancel(SVNErrorManager.java:37)
    at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.cancel(SVNErrorManager.java:32)
    at org.tmatesoft.svn.core.internal.wc.DefaultSVNAuthenticationManager.getNextAuthentication(DefaultSVNAuthenticationManager.java:200)
    at hudson.scm.FilterSVNAuthenticationManager.getNextAuthentication(FilterSVNAuthenticationManager.java:42)
    at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:537)
    at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:273)
    at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:261)
    at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:516)
    at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:98)
    at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1001)
    at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.testConnection(DAVRepository.java:97)
    at hudson.scm.SubversionSCM$DescriptorImpl.checkRepositoryPath(SubversionSCM.java:1686)
    at hudson.scm.SubversionSCM.repositoryLocationsExist(SubversionSCM.java:1826)
    at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:506)
    at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:469)
    at hudson.model.AbstractProject.checkout(AbstractProject.java:898)
    at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:391)
    at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:340)
    at hudson.model.Run.run(Run.java:1090)
    at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:301)
    at hudson.model.ResourceController.execute(ResourceController.java:93)
    at hudson.model.Executor.run(Executor.java:122)
Skipping sonar analysis due to bad build status FAILURE
Finished: FAILURE


The stack trace talks about authentication cancellation not failure, which is quite strange. I still thought that it might be an authentication failure and tried to authenticate against SVN server using same credentials as used by Hudson. Authentication was successful using various methods (browser, command line client, TortoiseSVN). Google search did not provided any relevant information and I was running out of clues :-(

After putting lots of pondering I remembered that day before I did a manual (from console) project checkout on the same machine and during checkout svn client (1.6.4) asked to save the password and I said 'no'. Will that make any difference? I think it should not.

Just for the sake of trial I deleted the '~/.subversion' directory and did a Hudson build after that. Now Hudson was able to do SVN checkout/update and build project. After several trials, I concluded that if password is not saved during manual checkout, Hudson will fail with above-mentioned error. Though it will wok fine if password was saved. Another observation was that Hudson SVN client (SVNKit 1.3.0) does not create '~/.subversion/auth', while 'svn' command does, irrespective of one chooses to save the password or not.

Though I do not know why Hudson (SVNKit) cancels the SVN authentication but removing the '~/.subversion' directory solves the issue.

Friday, August 07, 2009

Yum hangs on Fedora 11 behind proxy

Yesterday I installed Fedora 11 on a new machine from a live CD. As usual I tried to update the system just after installation. This machine was behind a proxy server and http_proxy variable was configured to let yum get through proxy server. Yum hanged just after emitting two line on the console-

    [root@localhost ~]# yum update
    Loading "refresh-packagekit" plugin
    Setting up Package Sacks


Even after 1 hour it did not came out and I can't kill it using ctrl+c, killing the process from another terminal was the only option. Running yum in verbose mode also does not print any useful information-

    [root@localhost ~]# yum -v update
    Not loading "blacklist" plugin, as it is disabled
    Loading "refresh-packagekit" plugin
    Not loading "whiteout" plugin, as it is disabled
    Config time: 0.082
    Yum Version: 3.2.22
    Setting up Package Sacks


The wget http://www.yahoo.com/ was successfully downloading the web page that means internet connectivity and proxy was not an issue. Then I looked inside /etc/yum.repos.d/fedora.repo, to get some clues. All URLs mentioned in that file were using https, so I tried to do wget over https-

    [root@localhost ~]# wget https://www.yahoo.com/
    --2009-08-07 16:35:11--  https://www.yahoo.com/
    Resolving www.yahoo.com... 69.147.76.15
    Connecting to www.yahoo.com|69.147.76.15|:443...


and that too hanged after emitting above text. Now I see why yum is not able to update. Changing URLs from https to http in /etc/yum.repos.d/fedora.repo took yum slightly ahead from where it was hanging earlier-

    [root@localhost ~]# yum -v update
    Not loading "blacklist" plugin, as it is disabled
    Loading "refresh-packagekit" plugin
    Not loading "whiteout" plugin, as it is disabled
    Config time: 0.082
    Yum Version: 3.2.22
    Setting up Package Sacks

    fedora/metalink                                          |  10 kB     00:00
    fedora                                                   | 3.8 kB     00:00
    fedora/primary_db                                        | 8.4 MB     00:09


Now yum hanged after emitting above-mentioned text. Changing URLS from https to http in /etc/yum.repos.d/fedora-updates.repo made update successful.

This https issue seems to be specific to Fedora 11, as I never faced any difficulties on updating previous Fedora releases (6, 7, 8 and 9).

Tuesday, August 04, 2009

Updating a Linux box behind proxy server

Recently I had to update a Linux (Fedora 9 to be precise) box, which was behind a proxy server. To get through the proxy server I configured relevant environment variables as described here. After executing 'sudo yum update' command system seems to be hanged after emitting following line on the console-

Loaded plugins: refresh-packagekit

After a long wait yum failed with following error message on console-

[vinod@localhost ~]# sudo yum update
Loaded plugins: refresh-packagekit
Could not retrieve mirrorlist http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-9&arch=i386 error was
[Errno 4] IOError: <urlopen error (110, 'Connection timed out')>
Error: Cannot retrieve repository metadata (repomd.xml) for repository: fedora. Please verify its path and try again


OK, so yum was not able to get through proxy, but when I tried to access the same URL using wget-

[vinod@localhost ~]# wget http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-9&arch=i386

it was able to access it. Then what could be wrong :-/

After pondering for sometime, I realized that proxy configuration was done using a non-root user while yum was running as root (due to sudo). Environment variables configured for a user are not carried forwarded (to another user) by Linux when user is changed using su or sudo command. Configuring the proxy as root solved the problem and yum successfully updated system.

Monday, August 03, 2009

Getting started with Redmine

For sometime I was looking for a project management software, which can provide an integrated view of most of the project related artifacts/activities. Redmine was the most impressive tool, I came across. It has issue tracking, wiki, forum, documents, integration with third party tools (source repositories, LDAP etc.), plugin to integrate with Hudson and many more features. It has plugin based architecture, so possibilities are almost endless.

Redmine is written in Ruby (Ruby on Rail) and I know nothing about that other than spelling :-) I had to install several packages to get started with it. Redmine stores its data in a database and when it comes to open source DB, PostgreSQL is always my first choice. As a practice I prefer to serve all web applications through Apache and here I found an Apache module called Passenger, which can serve Ruby applications.

Here are my notes, which I took during installing Redmine and supporting applications.

  1. Install PostgreSQL for Linux
          $ ./postgresql-x.x.x-x.bin --mode text

    Follow the installer instructions to complete the installation. Make sure that PostgreSQL accepts connections from other than 'localhost' else you can't connect to DB from any other machine. This can be done by editing 'pg_hba' configuration file.

  2. Install Ruby, Redmine needs ruby 1.8.7-
          $ tar zxvf ruby-1.8.7-p174.zip
          $ cd ruby-1.8.7-p174
          $ ./configure --prefix=/opt/redmine/ruby-1.8.7
          $ make
          $ make install


    Add ruby to your classpath before executing further installation steps.

  3. Install ruby-gems, say rubygems-1.3.4
          $ unzip rubygems-1.3.4.zip
          $ cd rubygems-1.3.4
          $ ruby setup.rb

  4. Install rake
          $ unzip rake-0.8.7.zip
          $ cd rake-0.8.7
          $ ruby install.rb

  5. Install rails (Ruby On Rails) version 2.1.2 (required by Redmine)
          $ gem install rails -v 2.1.2

    The GEM needs internet connectivity to download and install the packages. Though there are ways to install packages locally, after downloading them manually. As I am new to Ruby so preferred the easiest installation method.

  6. Install PostgreSQL adapter for Ruby
    Add PostgreSQL's 'bin' directory in PATH before executing following command-
        $ gem install pg

  7. Install redmine

    • Unzip the downloaded archive
      $ tar zxvf redmine-0.8.4.tar.gz

    • Create 'redmine' database user
      CREATE ROLE redmine LOGIN PASSWORD 'redmine' NOSUPERUSER NOINHERIT CREATEDB NOCREATEROLE;

    • Create 'redmine' database
      CREATE DATABASE redmine WITH ENCODING='UTF8' OWNER=redmine;

    • Define PosgtreSQL DB in '/opt/redmine/redmine-0.8.4/config/database.yml'
             production:
               adapter: postgresql
               database: redmine
               host: localhost
               port: 5432
               username: redmine
               password: "redmine"

    • Create the database structure, by running the following command under the redmine installation directory. It will create tables and an administrator account.
          $ rake db:migrate RAILS_ENV="production"

    • Insert default configuration data in database, by running the following command:
          $ rake redmine:load_default_data RAILS_ENV="production"

  8. Install Passenger module to enable Apache to serve ruby applications
        $ tar zxvf passenger-2.2.4.tar.gz
        $ cd passenger-2.2.4/bin
        $ export APXS2=/opt/redmine/apache-2.2.11/bin/apxs
        $ ./passenger-install-apache2-module

  9. Configure Apache to server redmine or any other ruby application using passenger
    To publish a rubby application under a sub-uri, create a symbolic link to application's public directory under web root directory. Thats the way passenger works, pointing directly to application's public directory will not work here.

        $ ln -s /opt/redmine/redmine-0.8.4/public redmine

    Here is a sample configuration text from Apache's httpd-vhosts.conf file to serve Redmine-

        NameVirtualHost *:80

        LoadModule passenger_module /opt/redmine/passenger-2.2.4/ext/apache2/mod_passenger.so
        PassengerRoot /opt/redmine/passenger-2.2.4
        PassengerRuby /opt/redmine/ruby-1.8.7/bin/ruby

        <VirtualHost *:80>
            ServerAdmin webmaster@mydomain.com
            ServerName redmine

            DocumentRoot "/opt/apache-2.2.11/htdocs"
            RailsEnv production
            RailsBaseURI /redmine

            ErrorLog "logs/error_log"
            CustomLog "logs/access_log" common

            <Directory "/opt/redmine/redmine-0.8.4/public">
                Options FollowSymLinks
                AllowOverride None
                Order deny,allow
                Allow from all
            </Directory>
        </VirtualHost>
Now Redmine is available for action. Next steps could be configuring to integrate with LDAP and few more things or installing some plugins to enhance the functionality. Next I am looking for ways to migrate issue tracker from Bugzilla to Redmine, to have everything under one roof.