Login to eBay with Java cURL

Lately, I played around with the eBay API using the Java SDK. It was quite easy to use but when it comes to use your application in production using the real eBay site (not the sandbox), you need to be certified. And It has a bunch of conditions that my application wasn't fulfilling.

A work around is simply not to use the eBay API and to programmatically simulate a real user connecting to eBay through the web interface and place bids as you would through your browser. To do it, I first tried to use the HttpClient developed by Apache. However I didn't manage to use it to simulate a login on eBay (it worked well on sandbox.ebay.com but for some reasons it didn't work on the ebay site and had a problem with cookies, even though HttpClient was supposed to handle them). So I decided to use cURL which is a very popular HTTP library especially in the C and PHP community. For Java, the author has also provided a port using JNI but was made towards Windows.

I describe below the different step to use it in your application under Linux. You need to have gcc, Java and curl installed on your computer.

Creating a javacurl.jar and javacurl.so

We want to create a jar file that we'll include in our Java application as well as a .so (shared library) that will be used by java at run-time.

You need to have curl installed on your computer as well as the development libraries (curl-dev). If you don't have it, download curl from http://curl.haxx.se/download.html and install it (./configure; make; sudo make install)

Then download java curl from http://curl.haxx.se/libcurl/java/

After unpacking the archive, edit the Makefile and make sure to have the following lines:
LIBCURL_PATH=/usr/local/
CPPFLAGS = -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/linux

comment all the lines starting by LDFLAGS and add the following line instead:
LDFLAGS = -lcurl -Wl -shared

To make things cleaner, we want the classes to belong to the package se.haxx.curl so edit CurlGlue.java and CurlWrite.java and add in the first line:
package se.haxx.curl;

Create a directory tree with se/haxx/curl:
mkdirhier se/haxx/curl

and copy the files CurlGlue.java and CurlWrite.java:
cp CurlGlue.java CurlWrite.java se/haxx/curl

There is a small mistake in the file CurlGlue.java which close the file description when the garbage collector is triggered (in the method finalize) so simply rename the method finalize() by close().

Compile them:
javac -classpath . se/haxx/curl/*.java

Create the jar file as follows:
jar cvf javacurl.jar se/haxx/curl/*.class

Generate the C header (.h) as follows:
javah -classpath . se.haxx.curl.CurlGlue

That will create the file: se_haxx_curl_CurlGlue.h

The file javacurl.c was written not to use a package, so let's edit it as follows:
replace:
#include "CurlGlue.h" /* the JNI-generated glue header file */
by
#include "se_haxx_curl_CurlGlue.h" /* the JNI-generated glue header file */

and replace all the occurences of: Java_ by Java_se_haxx_curl_
and CurlWrite by se_haxx_curl_CurlWrite.


Then create the .so as follows:
gcc -I $JAVA_HOME/include -I $JAVA_HOME/include/linux/ -I. -shared -o libjavacurl.so -lcurl javacurl.c

Voila, you've got the javacurl.java that you can include in your application and the file libjavacurl.so that you'll be using at run time.

Now if you like to compile the test program that connects to eBay (change the username and password in the code):
javac -classpath:javacurl.jar:. Test.java

to execute it:
java -classpath:javacurl.jar:. -Djava.library.path=. Test

The java.library.path should point to the directory where the libjavacurl.so is located. Alternatively, you can set your environment variable LD_LIBRARY_PATH to this directory so you don't need to specify it using -Djava.library.path.

Maven Artifact
If you use Maven (great tool, if you don't use it and are still using ANT, check this out you'll never regret it), you might want to create an artifact.

Let's create the artifact:
mvn install:install-file -Dfile=javacurl.jar -DgroupId=se.haxx -DartifactId=curl -Dversion=0.2.1 -Dpackaging=jar -DcreateChecksum=true

You can now use the dependency in your pom.xml
<dependency>
<groupid>se.haxx</groupid>
<artifactId>curl</artifactId>
<version>0.2.1</version>
</dependency>


Enjoy!

If someone has managed to connect to eBay using HTTPClient, please let me know.

Test Code:

import se.haxx.curl.*;
/*
* Test class to login to eBay
* change the username and password to yours.
*/
public class Test implements CurlWrite {
public final static String username="MY_EBAY_USERNAME";
public final static String password="MY_EBAY_PASSWORD";

public int handleString(byte s[])
{
/* output everything */
try {
System.out.write(s);
}
catch (java.io.IOException e) {
e.printStackTrace();
}
return 0;
}

public static void main(String[] args)
{
CurlGlue cg;

try {
Test cw = new Test();

// Register callback write function
cg = new CurlGlue();
cg.setopt(CurlGlue.CURLOPT_WRITEFUNCTION, cw);

// first, go to the login page to get the cookies.

cg.setopt(CurlGlue.CURLOPT_URL, "https://signin.ebay.com/aw-cgi/eBayISAPI.dll?SignIn");
cg.setopt(CurlGlue.CURLOPT_USERAGENT, "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.7) Gecko/2009030422 Ubuntu/8.10 (intrepid) Firefox/3.0.7");
cg.setopt(CurlGlue.CURLOPT_FOLLOWLOCATION, 1);
cg.setopt(CurlGlue.CURLOPT_COOKIEJAR, "cookie.txt");
cg.setopt(CurlGlue.CURLOPT_COOKIEFILE, "cookie.txt");
cg.perform();
cg.close();

// login using the username, password and the cookies we got from the login page.
cg.setopt(CurlGlue.CURLOPT_WRITEFUNCTION, cw);
cg.setopt(CurlGlue.CURLOPT_URL, "https://signin.ebay.com/aw-cgi/eBayISAPI.dll");
cg.setopt(CurlGlue.CURLOPT_USERAGENT, "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.7) Gecko/2009030422 Ubuntu/8.10 (intrepid) Firefox/3.0.7");
cg.setopt(CurlGlue.CURLOPT_POST, 1);
cg.setopt(CurlGlue.CURLOPT_POSTFIELDS, "MfcISAPICommand=SignInWelcome&siteid=0&co_partnerId=2&UsingSSL=0&ru=&pp=&pa1=&pa2=&pa3=&i1=-1&pageType=-1&userid=" + username + "&pass=" + password);
cg.setopt(CurlGlue.CURLOPT_FOLLOWLOCATION, 1);
cg.setopt(CurlGlue.CURLOPT_COOKIEJAR, "cookie.txt");
cg.setopt(CurlGlue.CURLOPT_COOKIEFILE, "cookie.txt");
cg.perform();

cg.close();

// now, if you like you can fetch the page: http://my.ebay.com/ws/eBayISAPI.dll?MyeBay and see that you're logged in
} catch (Exception e) {
e.printStackTrace();
}
}
}