How to Upload

There are two basic features to the ProteomeCommons.org Tranche network: adding data and getting data. This document describes how to add data to the network using the command line tool and the Java API.

If you are a researcher who wishes to use the GUI to upload data to the ProteomeCommons.org Tranche network, you may wish to visit the tutorial on using the Tranche GUI. However, to illustrate how the servers work, we will walk through the process of running the server using the GUI, creating a user, setting permissions, and uploading our first file on our localhost.

Brief network introduction

Anyone can host a Tranche server, and it is remarkably simple to do so.

The simplest way is to run the Tranche GUI tool, which uses Java Web Start so that installation and updates are automatic. Two other options include using the command line tool, or by running the server in a custom tool using our Java API. The latter two are discussed in this document.

Note that ProteomeCommons.org Tranche requires authentication. If you are interested in using the ProteomeCommons.org Tranche network, you will need to contact an administrator to generate a certificate and digital keys, or to sign an existing certificate. Until you register with ProteomeCommons.org Tranche, any attempt to upload will result in a security exception, preventing a file upload.

However, you can still use Tranche in networks where you are trusted. To make this scenario simple, you can run the tranche software on your localhost (127.0.0.1). The process of uploading requires:

To follow along with this exercise, launch the Tranche GUI tool. To begin, select the "Server Configuration" tab under the "Preferences" menu located at the top of the application.

The default port of 443 is listed in the port field. Decide if this is the port you would like to use or change it to another of your choosing, then click Start the server. (I will use 1500 instead of the default, 443, since it is less likely to be closed by various operating systems, though you can use whatever port you deem appropriate.)

An error message will be displayed if the port you have chosen is blocked by your OS or currently in use. The best way to resolve this issue is to determine whether or not your OS has closed the port. The easiest way to resolve this issue is to simply use another port, which is why I chose 1500 for this exercise.

If the server starts successfully you will be able to see the addition of the server under the "Server" tab. The new server will be displayed in the form tranche://ip_address:port.

Creating a user

To create a user, click on Create User Tool. This tool is located under the "Users" tab on the left hand menu. This will launch the Create User dialog box.

Fill in the required information as well as any relevant optional information, and click on Create User. The client will take a few seconds to create the user file, so wait for a confirmation message.

When the client is done creating the user file, it will alert you with a message saying "User file successfully made". You probably want to store this file somewhere readily accessible because it is required everytime you wish to upload to Tranche.

Now that you have created your user file, you can close the Create User dialog box.

Registering the user with the network

Since you are running the server on your own computer, you can grant permissions to users (via their user files) at your discretion.

By default any time a user starts a server they are logged on as that server's administrator; however, this also requires that before a server can be started, a "user file" must be created for the user and said user must also be logged in. If you wish to create additional administrators this can be accomplished fairly easily; however, it is not recommended that when creating new "user files" that they are given administrator priveledges.

To add an administrator, click on Add New Administrator, which is found on the left hand menu under the "Servers" tab. A choose user file dialog box should appear. Find and select the user file. When Tranche has successfully added the user, you will be alerted with a message that states that the user was added as an administrator for the server.

Next, we want to view the server configuration. Under the "Servers" tab find and select your server. On the left hand menu you can select "Configure Server". A Server Configuration dialog box should open.

If you again select Configure Server from the left hand menu of the "Servers" tab after selecting your server, and this time select the "Data Directories" tab, you can change the directory where uploaded data is stored as well as set size limits. This is especially useful for testing as well as dedicating separate drives for server storage.

When you are satisfied with the settings, click Save Changes. Assuming you set this particular user as an administrator, you should receive no objections. If you get an error message about priveledges, you need to add the user as an administrator.

After saving the changes, you can close this server configuration dialog box.

Testing the Upload Tool

This document doesn't aim to cover the GUI upload function, but to complete our exercise, click on Upload Tool. A Tranche Uploader dialog box will appear.

If we want the upload to work with our server we must modify the appropriate fields:

  1. Upload: select a test file to upload
  2. User: load the same user file we've been using
  3. Title: each uploaded file must have a title
  4. Description: if a directory of files is being uploaded a description must also be input
  5. In order to upload to only your server, you must specify/configure the uploader appropriately. To do this select "Options" from the top menu and then click on "Configuration". A dialog box will be displayed with all available servers, their status, and the user permissions for those servers. From this list, ensure that only your server is selected.

You can optionally uncheck Register this upload with ProteomeCommons.org's network. Since we supplied a server IP, the client won't attempt to upload to the Proteome Commons Tranche network.

Click "Save Changes" and you will be returned to the uploader dialog box.

Here's a screenshot of a sample dialog box before clicking Upload File(s):

Click Upload File(s). After a few seconds (assuming the file size is small), you should get a confirmation dialog box with the hash, which you should store. (Expect more time if file(s) is large.) The hash is used to download the file.

The key to exploring the software on the server running on your computer is to supply the URL for localhost (i.e., tranche://127.0.0.1:1500). This will prevent Tranche from attempting to upload to the ProteomeCommons.org Tranche network. Before you continue, you may want to explore the client software to get an idea of the scope of the Tranche project.

Uploading files using the command line tool

There is more information on the command-line quickstart guide. However, after downloading the ZIP archive, decompress it and type the following:

java -jar Tranche-Uploader.jar --help

Or

java -jar Tranche-Uploader.jar -h

The usage information will be displayed to standard output, including parameters and return codes.

It is important to note that uploads are not encrypted by default. To encrypt a project, use the flag -e followed by a passphrase that will be used to decrypt the project when downloaded. See the command-line quickstart guide for more information.

Java API

There are many reasons you may wish to create your own Java upload tools:

A complete simple sample upload client and a more verbose upload client exists that you may wish to look at now. However, we will walk through the steps of creating your own custom upload client.

If you are looking for sample code illustrating the Tranche Java API for downloading a file, or other common activities, visit the code snippets page. The upload code found above will also be cross-listed on the code snippets page.

Since user uploads are validated using X.509 certificates and encryption using your private key, writing upload code could involve handling the security issues using a security API. However, to abstract, the creation of "users" simplifies the validation process by encapsulating security.

An overview of the upload process involves codifying these steps:

AddFileTool

AddFileTool offers a simple API for uploading data to a Tranche server. Let's look at a code snippet to illustrate its use:

AddFileTool addTool = new AddFileTool(user.getCertificate(), user.getPrivateKey());
addTool.addServerURL("tranche://127.0.0.1:1500");
addTool.setTitle("The title.");
addTool.setDescription("The description.");
BigHash hash = addTool.addFile(new File("/path/to/file"));
System.out.println("The file's hash: " + hash);

We stated earlier that the creation of a "user" encapsulates our security concerns. In the case of this code snippet, our "user" offers simple access to the certificate and private key, which AddFileTool needs for authentication and (potentially) for security. (The "user" is actually an object of the class UserZipFile, which we will discuss shortly.)

This code snippet assumes we have a server running on our localhost, accessible via port 1500. The default port is 443, but I chose 1500 since unpriveledged users will more likely have access to that port. However, if you are hosting a server on the Tranche network, you will most likely use the default port to simplify usage.

The method addFile(File) uploads a file to the server we selected, and returns a hash, which you can print out. You should store the value of the hash, since it is needed to download the file from the server.

If you are developing the tool as a command line tool, you may find it convenient to simply redirect the output to a file for convenient storage, or, if you choose, you can use a file writer to create a new file or append to an existing file, which simplifies the record-keeping process of retaining hashes for future reference. In either case, if you use Tranche frequently, using a custom tool can greatly simplify the process of uploading data.

Users and UserZipFile

Say you have already created a user file with administrative priveledges. You can load that file in your custom application:

UserZipFile user = new UserZipFile(new File("/path/to/user_file.zip.encrypted"));
user.setPassphrase("super_secret_password");

Working with existing users is easy. It is easy to create a new user, but since we are uploading a file, we will have to set the permissions so the upload will be successful:

maker = new MakeUserZipFileTool();
maker.setName("John Smith");
maker.setPassphrase("super_secret_password");
maker.setUserFile(new File("John.zip.encrypted"));
UserZipFile user = (UserZipFile) maker.makeCertificate();
// Set user permissions as admin
user.setFlags((new SecurityUtil()).getProteomeCommonsAdmin().getFlags());

It is assumed that if you upload multiple times, you will only create the user once. To upload to the Tranche network, you will need to contact an administrator at ProteomeCommons.org, who will generate a certificate and key for you, or authenticate a certificate for you if you already have one that you wish to use. As long as you are uploading to a server you control however, you do not need to do this. See the introduction above for more information.

Servers

If you already have a server running, you do not need to run a server from your code. In fact, if you attempt to run a second server on the same host using the same port, the server will be unable to connect, citing a java.net.BindException. We can use this to our advantage — we can assume a server isn't running on a specific port, and even if it is, our application will still work, capitalizing on the advantage of exception handling.

FlatFileTrancheServer ffserver = new FlatFileTrancheServer(new File("/path/to/dir"));
ffserver.getConfiguration().getUsers().add(user);
try {
  Server server = new Server(ffserver, 1500);
  server.start();
}
catch (BindException e) {
  System.out.println("Port 1500 already bound, continuing assuming a Tranche server is already running...");
}

// ... code handling upload

try {server.setRun(false);} catch (Exception e){}
try {ffserver.close();} catch (Exception e){}

Notice that we are registering the user with the flat file server, and we are wrapping that server in a Server object. We will actually run the server as a thread, and notice that we catch a BindException. If you are confident that this exception will almost always be the result of an instance of Tranche already running, you may even leave out the message, meaning the user doesn't really need to know whether a server is running or not.

After we upload the file, we will want to close the server so that the operating system can free the socket.

The complete upload tool

There are two examples of the upload tools. The first assumes that you know the Tranche server that you would like to upload to and that you have an appropriate user file. The second example assumes that you want to use the Java API to set up a new server as well as create a new user file. Here is the first, simple code example. Here is the complete sample upload client Java source code, which is a modified version of our code snippets. Note that there are boolean variables you can change depending on whether you have an existing user file and whether you have a server running. (Even though the code will work regardless of whether or not a server is running, you may not want the application to attempt to create a server process, choosing for your application to fail if a process doesn't already exist.)

You may find it easier to adapt the above code rather than to start from scratch. If that is the case, there are many ways you can make a custom tool more useful. For example you could modify the code to:

Join group for updates

If you use either the command-line tools or API to access the Tranche network, you should join our Google Group, ProteomeCommons.org Tranche Command-Line Tools and API (link: http://groups.google.com/group/proteomecommonsorg-tranche-command-line-tools).

As a member, you will receive occasional information about new updates, bug fixes, etc. This should be a low-activity group with minimal information so that users..



Home   •   FAQ   •   Contact Us   •   Join email list   •   Collaborations   •   Credits