403 Forbidden with wget

We have few shell scripts, which access some HTTP URLs using wget as part of some task. wget failed to access some of the URLs with error message like below-

Resolving repo1.maven.org... 38.97.124.18
Connecting to repo1.maven.org|38.97.124.18|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2009-09-22 11:38:53 ERROR 403: Forbidden.

Though all failed URLs were accessible using browsers from same machine / user account. Reason of this failure is that wget does not send any information about itself (agent information) i.e. name and version of the browser etc. and some servers do not entertain requests without agent information. A simple workaround for this problem is to add a dummy agent information as shown below-

wget -U MyBrowser/1.0 URL_TO_DOWNLOAD

Now everything will work as expected.

Redirect HTTP to HTTPS on Apache

At times we need to run a website over HTTPS only and to retain the traffic coming to the website over HTTP, that needs to be redirected towards HTTPS. Though there are many ways to do so but this method looks easiest to me-

<VirtualHost *:80>
    ServerName mydomain.com

    RedirectPermanent / https://mydomain.com/
</VirtualHost>

<VirtualHost *:443>
    ServerName mydomain.com

    # other SSL configuration goes here
</VirtualHost>

If only parts of the website (say /secure) needs to be HTTPS enabled then redirection can be done something like-

        RedirectPermanent /secure https://mydomain.com/secure

Proxy setting on Linux

If your Linux box is behind a proxy server then to access the internet, proxy configuration will be required. The 'http_proxy' and 'ftp_proxy' environment variables hold the information about proxy server. Just execute the following commands-

$export http_proxy=http://<proxy-server-ip>:<port>
$export ftp_proxy=http://<proxy-server-ip>:<port>

Now wget, yum, apt-get etc. can use these variables to access http/s and ftp URLs out of office LAN. If proxy server needs authentication credentials then pass them as shown below-

$export http_proxy=http://<user>:<password>@<proxy-server-ip>:<port>
$export ftp_proxy=http://<user>:<password>@<proxy-server-ip>:<port>

To avoid executing these commands every time you login or reboot the machine then add them in ~/.bashrc file.

Using JAX-WS Handlers

The JAX-WS provides a good facility to do pre/post processing on SOAP messages using SOAPHandler. The handlers are useful for auditing, logging and potentialy some more functionality. In this entry I will try to explain a typical usage of handlers for logging the SOAP messages. Every handler class needs to implement javax.xml.ws.handler.soap.SOAPHandler interface as shown below-

public class LoggingHandler implements SOAPHandler<SOAPMessageContext> {

    @Override
    public Set<QName> getHeaders() {
        return null;
    }

    @Override
    public void close(MessageContext context) {
    }

    @Override
    public boolean handleFault(SOAPMessageContext context) {
        logToSystemOut(context);
        return true;
    }

    @Override
    public boolean handleMessage(SOAPMessageContext context) {
        logToSystemOut(context);
        return true;
    }

    private void logToSystemOut(SOAPMessageContext smc) {
        Boolean outboundProperty = (Boolean) smc.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY);

        if (outboundProperty.booleanValue()) {
            System.out.println("\nOutgoing message:");
        } else {
            System.out.println("\nIncoming message:");
        }

        SOAPMessage message = smc.getMessage();
        try {
            message.writeTo(System.out);
        } catch (Exception e) {
            System.out.println("Exception in handler: " + e);
        }
    }

}

Now create a handler confiuration file say 'handlers.xml' as shown below and put it in 'WEB-INF' directory of your application.

<handler-chains xmlns="http://java.sun.com/xml/ns/javaee">
   <handler-chain>
      <handler>
         <handler-class>my.server.pkg.LoggingHandler</handler-class>
      </handler>
   </handler-chain>
</handler-chains>

Using @HandlerChain annotation we can instruct the JAX-WS runtime to apply the handlers configured in the above-mentioned configuration on a web service endpoint-

@WebService(endpointInterface = "my.server.pkg.IMyService")
@HandlerChain(file = "handlers.xml")
public class MyServiceImpl implements IMyService {

Now all incoming and outgoing SOAP messages will be logged to standard output stream. Source code for this example is available here.
For developing and deploying web services see my earlier posts-
http://blog.vinodsingh.com/2008/09/building-jax-ws-web-service.html
http://blog.vinodsingh.com/2008/09/jax-ws-web-service-and-jboss.html

SVN with Apache + LDAP

We have been using CVS at work for long, though personally I am a big fan of SVN. Lately we thought to catch up with the SVN buzz and I was asked to have a Proof Of Concept before we officially move to SVN. Installation of SVN with Apache was pretty straightforward. Though LDAP integration was also simpler but at times it failed without any obvious reasons. Here are the steps for the entire process on Windows XP-

1. Install Apache 2.2.
2. Install SVN 1.5 by unzipping the distribution.
3. Add SVN\bin folder to PATH.
4. Copy mod_dav_svn.so to modules directory of Apache.
5. Add following line in Apache conf file-
LoadModule dav_module modules/mod_dav.so
LoadModule dav_svn_module modules/mod_dav_svn.so

# Following are required for LDAP authentication
LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
LoadModule ldap_module modules/mod_ldap.so

6. Download [svnindex.css and svnindex.xsl] from http://svn.collab.net/repos/svn/trunk/tools/xslt/
and put them in 'htdocs' folder of Apache. This will do nice listing of SVN contents.

7. Add the repository location in http conf file, this will have LDAP auth as well

#Location of repository
<Location>
DAV svn

SVNParentPath C:\svn
SVNListParentPath on
SVNIndexXSLT "/svnindex.xsl"

AuthType Basic
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthName "My SVN Setup"
AuthLDAPURL ldap://host:3268/dc=X,dc=Y?sAMAccountName?sub?(objectClass=user) NONE
AuthLDAPBindDN myUserID
AuthLDAPBindPassword myPassword
Require valid-user
</Location>

7. Run 'httpd' command from Apache

VOILA! YOU ARE DONE

I gave it a try on Fedora 4 as well, which comes with Apache 2.2 and SVN 1.4 pre-installed. Only thing that was missing was there 'mod_dav_svn'. Getting that to with YUM is pretty simple, just do-
yum install mod_dav_svn

Here we were using Active Directory as our LDAP server. Initially I used the port 389 in the AUthLDAPURL shown above and it worked fine for first 2 weeks but after that it failed without any obvious reasons with this error message in Apache log files-
[ldap_search_ext_s() for user failed][Operations Error]

It made it to work for brief periods by re-starting the Apache. Re-starting is not an option in production. After googling for some time I found suggestions to use port 3268 instead of 389. More than 2 weeks has been passed since I switched to port 3268 and it is working fine till date :-)

My next step will be to convert some CVS repositories to SVN. Will update soon about the results of this conversion.

Proxy authentication in Java

The usual corporate networks provide internet access via proxy servers and at times they require authentication as well. May applications do open the connections to servers which are external to the corporate intranet. So one has to do proxy authentication programmatically. Fortunately Java provides a transparent mechanism to do proxy authentications.

Create a simple class like below-

import java.net.Authenticator;

class ProxyAuthenticator extends Authenticator {

    private String user, password;

    public ProxyAuthenticator(String user, String password) {
        this.user = user;
        this.password = password;
    }

    protected PasswordAuthentication getPasswordAuthentication() {
        return new PasswordAuthentication(user, password.toCharArray());
    }
}

and put these lines of code before your code opens an URLConnection-

Authenticator.setDefault(new ProxyAuthenticator("user", "password"));
System.setProperty("http.proxyHost", "proxy host");
System.setProperty("http.proxyPort", "port");

Now all calls will successfully pass through the proxy authentication.

Monitor SOAP messages

While developing Web Services at times one has to look for what is going out and coming in during web service interaction. Eclipse has a built-in TCP/IP Monitor but somehow I did not have a good experience using it.

Apache SOAP project has small utility called TCP Tunnel/Monitor, which is available here. TCP Tunnel/Monitor can be started using follwing command-

set CLASSPATH=%CLASSPATH%;./soap.jar;
java org.apache.soap.util.net.TcpTunnelGui 80 localhost 8080

Here first option 80 is the port where TcpTunnel will listen on your machine (localhost). The second parameter is the target address (the local machine in this case). The final parameter 8080 is the port where it will forward the calls to desired server as specified in 2nd parameter.

The TCP Tunnel/Monitor application is really just a simple proxy application that includes a visual interface to view the request response XML. TCP Tunnel/Monitor will display a window with two text areas:




* The text area on the left shows the request.
* The text area on the right shows the response received from the target server.