Comparison of web-based file transfer methodologies

Transferring large files over the internet has never been a simple task. Anyone that has ever tried to transfer a file larger than 100MB can vouch for the slow transfer speeds, multiple disconnects, data corruption, complexity of the task and security issues surrounding FTP.

The same problem exists for web developers trying to implement web-based file transfer functionality. Ideally, the end user should be able to upload or download large files without leaving the web browser to open an FTP client. Most web developers also require some kind of mechanism that will notify their front end (HTML/JavaScript) or backend (PHP, JSP, ASP) code when a transfer event (complete, cancelled, paused, error) happens, as well as which files were transferred.

Today there are a number of tools available, including HTML components, widgets, plugins, add-ons and Java applets that can transfer large files from a web browser.  We will review the most popular of these.  Please keep in mind that we are not reviewing specific products or applications but rather the technology and methodology behind them. For example, there are a handful of commercially available file transfer products that use Java applets, including our own file transfer applets. Every product differs in functionality, security and reliability but all use the same underlying Java applet technology.

There are several protocols that can be used to upload files. To the end user, the protocol is usually transparent. Such protocols can be HTTP POST, FTP(S), SFTP, WebDav or UDP. When using FTP(S), SFTP or UDP, the uploaded data doesn’t need to reside on the same server as your web server. Using HTTP POST will be more compatible with your web application, but it will not provide support for very large files or other advanced features. The HTTP protocol was not designed to transfer large files and there is usually a 2GB file size limit imposed by the server. Also with HTTP POST, support for resuming file transfers is defined but is not widely supported.

Web Based File Transfer Tools Examined

This list is fairly complete, containing every widely-used technology for transferring files on the web. If you have seen any other technologies/frameworks for uploading large files within a web browser, please leave a comment and I will be glad to add your suggestions. Please don’t advertise specific services; only valid technologies/frameworks or plugins will be considered (i.e. things I can deploy locally on my own server).

Standard HTML: input type=file

This method is built into all web browsers (old and new) and has been in existence as a standard for over 15 years. The backend code requires a server side script that can parse “multipart/form-data” HTTP POST requests and save them into a local file on the server. Some of libraries/files used to receive these POSTS on the server side include: apache-commons-file-upload for Java, PHP $_FILES array, FileUploader in ASP.

HTML5 file upload based on XHR2 and XMLHttpRequest objects

This is an extension of the standard HTML input type=file but provides hooks to expose a nice progress bar and provides the programmer the ability to specify timeouts. The back-end code must be able to parse “octet-stream” HTTP Post requests and save them to a local file on the server. XHR2 via the File object in JavaScript also provides the ability to check the file size and the ability to upload chunks, which makes re-trying and resuming possible for the programmer.
For a complete tutorial on XHR2 see the following article in our blog: XHR2 via JavaScript – HTTP Uploads.

Flash: Upload tools or widgets

Any Flash upload widget will still require the backend code similar to standard HTML. There are myriad commercial flash uploader tools on the market today, as well as a healthy choice of free open source uploaders such as SWFUpload. Flash technology only supports HTTP(S) connections; any flash based uploader will inherit the limitations of HTTP protocol.

Custom Web browser Plugins or Add-ons

There are many commercial software vendors with their own browser plugins/add-ons, which (if installed on the client’s machine) can initiate a file transfer within a web page. Most custom plugins will define their own protocol handlers, so instead of using ftp:// or https:// a custom handler (for example “custom://”) might be used. Since those plugins usually run external to the web browser, the integration with JavaScript or HTML is limited. Web-based session cookies might also not be shared, making capture of file transfer events and user authentication difficult in your web application.

ActiveX controls for uploading

A few years back when the Internet was dominated by Internet Explorer, there were many commercially available ActiveX components used for file transfer within a web browser. ActiveX controls are comparable to Java applets but lack the cross browser and cross platform capability. Most ActiveX uploads will support all the common data exchange protocols, including HTTP(s) and FTP(s).

Java Applets

These are small applications that run within the web browser using the Java plugin. Java applets can connect to a server with almost any protocol on the market including FTP(S), HTTP(S), SFTP, webdav and UDP. Oracle claims that Java is deployed on over 3 Billion devices (see about Java). The wide acceptance of Java across multiple browsers, desktops and platforms, makes it an ideal technology for web based file transfers. Due to such a wide deployment of Java, any security flaws are very quickly discovered and patched, at the same time these issues are broadly covered by the media. Java applet security model allows to access local files so they can be transferred over the network. Anything that is possible to program in the Java language is also possible to do in a Java applet. Many advanced features listed here require full programming capability of Java, including the ability to open UDP sockets for bandwidth optimization. UDP protocol will typically require a commercial file transfer server.

FTP Protocol Handler

Most recent web browsers can access FTP sites by simply typing “ftp://user:[email protected]” in the address bar of the web browser. Although this type of file transfer technically takes place in web browser, it doesn’t have any integration options with HTML or JavaScript and is not considered a true web-based approach. Creative use iFrames or pop-ups could yield to a web based transfer, but without hooks into the progress or post upload scripts.

Evaluation

When evaluating a file transfer tool or technology, most developers should be looking for the following features, grouped into 4 main sections: compatibility, basic features, integration and deployment, and advanced features. We will be using a scale 0-10 to grade the availability of each feature. 0 means the option is not available; 10 means the feature is fully available. 5 would mean partial support.

Compatibility

• Cross Browser: Cross browser and cross platform capability (no add-ons or plugins required).
• Back-end Compatibility: Compatibility with the back-end server. Not all technologies can be deployed on every server. If you have PHP server, you want to make sure that the technology you use is compatible with it.

 Winner: input type=file

The clear winner in this round is input type=file. If you are developing a web application where the compatibility is a “Must” and you are willing to trade off on advanced features, then input type=file is the best fit. HTML5 is a close second, if you don’t mind that older web browsers are not supported. ActiveX and a custom browser plugin are least desirable.

Basic Features

• Small Files: Ability to send small files (up to 100MB).
• Large Files: Ability to send large files (up to 2GB, for even larger files see the advanced features).
• Progress Bar: Show a progress bar to the end user.
• Cancel: Ability to cancel a transfer.
• Security: Must support a secure protocol (ex: https, ftps, sftp) and the overall framework to run/deploy the tool must also be secure.

Winner: Every technology shines in this round, with HTML5 being most compatible and input type=file being the least desirable.

For compatibility reasons, HTML5 is the unofficial winner here (when working with smaller files). For larger file sets, Java Applets, Flash and custom plugins are far better.

Integration and Deployment

• Interface with Web App: Interface with a web application via JavaScript or URL calling.
• Share Session Cookies: Ability to share web session cookies with the web application. Most web apps require the end user to login which creates a session cookie in the user’s web browser. It is beneficial for the upload tool to access protected URL’s of the web app.
• Central Upgrade: Central upgrade of all the client nodes, when one wants to be able to upgrade their entire deployment by simply updating the files on their web server. Some file transfer plugins are actually installed on the client’s machine and can not be remotely updated
• Custom L&F: Customizable look and feel, ideally via CSS but a configuration file can also be used.
• No Client Software: When no client software is required. Flash and Java Applets get partial points for using commonly deployed Plugins statistically present in over 80% of computers worldwide.

 Winner: HTML5, with Flash, Java Appplets and ActiveX as close seconds.

If your application doesn’t require to upload large files and you don’t need advanced features HTML5 is your best option. Java Applets and Flash are a close second, the least compatible would be a custom Plugin/Add-on or ftp:// handler.

Advanced Features

• Very Large Files: Ability to send very large files (2gb or more)
• Resume and Retry: If the connection drops, you want the file transfer tool to automatically retry and resume the file transfer.
• Select Multiple Folders: Select and send several files with a single click. Selecting one file at a time is tedious when many files must be sent
• Transfer Folders: Ability to send directory structures.
• Auto-Zip: Automatically zip files into a single archive before transfer to minimize the amount of data.
• Bandwidth Optimization: Optimize bandwidth using UDP or multiple TCP streams maximize the available bandwidth to accelerate the file transfer (requires a proprietary server on the backend).
• Checksum: Ensures the file transferred files are identical.

 Winner: Java Applets and Custom Plugin/Add-on

The winners are Java Applets and Custom Plugin/Add-ons.  Both technologies will provide all the advanced features however Java Applets will have better integration and cross-browser compatibility.

Consolidated Results

Below is a combined table with all the ratings and totals for each technology/tool:

… And the overall winners are:

For small to medium files, HTML5 wins; for large files, Java applets win.

The least desirable options are input type=file or ftp:// protocol handler, unless you are required to support ALL web browsers and platforms in the world or you are obsessed with cross browser and cross platform compatibility.

If the file sizes are small and your web application doesn’t require any advanced features, HTML5 is most likely the best option. If the files are larger you will most likely require some advanced features such as automatic retry and resume, making the technology of choice Java applets.

The bottom line is that in order to handle large file transfers correctly within a web browser, you will have to steer away from the HTML standards and move towards other technologies not necessarily embedded with all the web browsers.

What about mobile support?

If you are required to support web-based uploads from mobile devices, input type=file is your best option and even this option is flaky.

• On iOS the browse button doesn’t work at all (it’s probably related to the fact that you must use iTunes to transfer files to/from the iPhone/iPad)
• On Android, during our tests, the maximum file size we were able to transfer was less than 5 MB
• BlackBerry stock browser seemed to work the best, in our tests we got files up-to 10MB sent using input type=file.
• We haven’t tested Windows Mobile, leave us a comment if you have done some tests.

In conclusion, if you are serious about mobile support, none of the options outlined in this article will work well, you will be better in investing into a custom mobile upload app.

Other suggestions?

As mentioned earlier, if you have seen any other technologies/frameworks for uploading large files within a web browser, please leave a comment and I will be glad to add your suggestions. Please don’t advertise specific commercial products or services; only valid technologies/frameworks or plugins will be considered (i.e.: things I can deploy locally on my own server).