Sven "Hoaxter" Hoexter Radim "HSN" Kolar

hsn@sendmail.cz.nospam

2003-2004 Authors This document is a major rewrite of the old FSP FAQ. I started with this FAQ from scratch cause nearly all parts of the old FAQ where outdated. This new FAQ is maintained in DocBook SGML. Send patches to the fsp-devel Mailinglist (FIXME link to id gethelpml) or directly to me sven@du-gehoerst-mir.de-nospam. You should be able to find the latest SGML version of this FAQ in the fsp CVS at sf.net or here http://sven.stormbind.net/fsp/fsp-doc/fsp-faq.sgml (be aware the sf.net public CVS is often about a week behind the "real" CVS :( ) FSP stands for &fsp;. In general FSP is what anonymous ftp should be: a reliable and bandwidth friendly way to access publicly available data. FSP is a UDP based protocol for transferring files around. It has many benefits over FTP, mainly for running anonymous archives. FSP protocol is valuable in all kinds of environments because it is one of the only TCP/IP protocols that is not aggressive about bandwidth, while still being sufficiently fault tolerant. FSP is (and always has been) open source project with MIT/X11 source code license. See for more details. In the past various people maintained the FSP code base. At the moment Radim Kolar is maintaining FSP. FSP project is now hosted on Source Forge.NET. If you like to get involved take a look at the FSP Homepage and the FSP project page. The &fsp; has its strengths on slow lines and connections with a high packets lose like a Wireless LAN or radio packet networks. Because FSP is simple and lightweight, it is very well suited for use in embedded devices area. FSP protocol is not aggressive about bandwidth and was designed to resist against users which want to do some bandwidth-related attacks. It can support more users, because the same available bandwidth is divided into smaller parts. Because FSP uses less bandwidth, FSP transfers takes about 2-3 times longer than with TCP-based protocols. FSP protocol chooses reliability and simplicity over raw transfer speed. File transfer takes longer, but there are much more resistant against network failure. Entire connection management is moved from server to client side, which solves problems with aborted and timed out transfer. Because FSP is stateless protocol, it can survive even server reboot without interruption. We did some benchmarking so that you can see how fsp performance compared to various other command protocols used to transfer files. You should understand that FSP is slower than TCP based protocols by design. TCP protocol can have more than one packet flying in network (usually 2-3) while FSP was designed to make it impossible (See ). All TCP protocols have on normal networks (max 15% packet loss) higher transfer rate than FSP. FSP is clean winner when it comes to overloaded sites or lines. These benchmarks were performed on 10Mbit LAN with zero traffic. Transfer rate is quite slow even for 10Mb because one of testing machines was old AMD K5/90 and both machines used cheap clone ISA NE2000 Ethernet cards. Test file was 3656535 bytes long (FreeBSD kernel image). Test results shows, that FSP is about 2-2.5 times slower than TCP. tftp 15.2 secs 240.6 kB/s ftp 5.18 secs 688.7 kB/s http 5.32 secs 685.4 KB/s sftp 8.0 secs 446.4 KB/s fsp 12.7 secs 287.7 kB/s HTTP: widely supported by servers, clients and proxies. De-facto standard protocol for accessing information today. FSP: Can provide directory listings, which are not supported by standard HTTP/1.1 protocol. You need to have configured WebDAV server-side extension for that. FTP: widely supported by servers and clients, more difficult to proxy than HTTP, uses 2 TCP connections, can provide directory listings in site-specific format, more heavy service than HTTP. FSP: do not needs to make any connection, very lightweight service, directory listings are standardized by protocol. tftp is UDP (but TCP version also exists) used mainly for loading boot and flash ROM images from network. TFTP: more widely used, standardized, most network cards can boot image via TFTP. FSP: about 1.5-times faster because of bigger packet size, supports directory listings, can query file date and size, can provide random access to file and supports opening more than one file, supports access control, timeout management moved to client side, stateless. If you are providing anonymous archive (FTP,WWW), you should also offer FSP access. FSP daemon is a very lightweight process, never forks and you will never have more than one copy running around. On todays machines, it has zero performance impact. Running fspd allows people with overloaded or lousy lines to get files from your archive. It also helps people during spikes periods when archive is very busy. There are also some people which likes slower FSP downloads, because it saves their bandwidth for interactive tasks. It is not true that UDP based protocols are unsecure. &fsp; provides the same security level as anonymous FTP and is much more resistant against denial of service attack. It is nearly impossible to overload fsp server by flooding. On other side, TCP-based protocols are very simple to SYN flood. You can Download the FSP source Distribution from the sf.net mirror network or from ibblio. If you have the FSP tools already you can access the latest FSP release via fsp on hxt.homelinux.org or wrack.telelev.net both running fspd on port 2221. When you're interested in the latest, and of course greatest, development done in FSP please try a CVS checkout. Explanation and all needed information to do this can be found on the sf.net CVS page. First of all various people took the time to write nice documentation on FSP including the fspd Server and the client tools. Please read those fine documents before you start writing to the mailinglists. The &fsp; definition The latest version of the FAQ, The old FSP FAQ. For developing issue writte to the fsp-dev mainlinglist. For end-user questions and questions on how to configure your server/client write to the fsp-user Mailinglist. You can find more information on how to subscribe/unsubscribe to the mailinglist and the archive on the FSP mailinglist page on sf.net. Please before you write to one of the provided mailinglists read and understand ESRs HowTo ask smart questions and learn.to/quote FSP code base uses MIT/X11 license (also known as 2-point BSD license). This license is very friendly to users in commercional area. You can freely develop, use and sell your commercional implementation of &fsp;. Do not claim that you have wrote this software. Do not sue us if something breaks. Summary: As long as you will not remove our copyrights from source code and will not go with your complains to the court, you can do with this software anything you want. From the COPYRIGHT file in the source distribution (e-mail addresses removed): Authors: Radim Kolar (Current FSP maintainer) Andrew Doherty Michael Fischbein Cimarron D. Taylor Guido van Rossum Wen-King Su Philip G. Richards Michael Meskes Rich $alz gjc@mitech.com Joseph_Traub Sven (VMS port maintainer) Very Mad Students, University of Karlsruhe, FRG (VMS port) Copyright: All of the FSP code is free software. Most of fsp falls under two copyrights, the 2-point BSD copyright and one by Wen-King Su: Copyright (c) 1991 by Wen-King Su (wen-king@vlsi.cs.caltech.edu) You may copy or modify this file in any manner you wish, provided that this notice is always included, and that you hold the author harmless for any loss or damage resulting from the installation or use of this software. Other contributions to fsp fall under different copyrights: This file is Copyright 1992 by Philip G. Richards. All Rights Reserved. See the file README that came with this distribution for permissions on code usage, copying, and distribution. It comes with absolutely no warranty. Copyright (c) 1993 by Michael Meskes You may copy or modify this file in any manner you wish, provided that this notice is always included, and that you hold the author harmless for any loss or damage resulting from the installation or use of this software. This should give you a short overview over the development of the &fsp; in the past and today. FSP development started in a very old days. From source code and man page time stamps we can see that it was working and alive in 1988. FSP versions 1.0 and 2.0 was born in Dec 1991. After that two first releases FSP goes to the active life. FSP active development era ends in May 1993 when last official version 2.7.1 was released. It was quite popular at that time, mainly because it was superior way for transferring warez around when compared with FTP. Because of this, FSP protocol got a bad name and suffers from it even today. I see that FSP warez era as practical demonstration how can FSP perform nicely on overloaded sites and lines. After that then was some work on FSP in 1995 and which ends with now famous FSP version 2.8.1b3 in March 1996. This version is used on some servers today because Debian Woody has it, but 2.7.1 is still used on some forgotten machines in universities. It has also known security issues (fspd follows symlinks), but at this time there were viewed as feature, not a bug. Radim Kolar released 2.8.1b4 in 2001, which was just some bugfixed version of 2.8.1b3 untouched from 1997 when I ported it to OS/2. Active working on FSP again starts on 25 June 2003 when Version 2.8.1b5 goes out. Major parts of FSP server code was rewrited for higher performance and many old bugs in FSP code fixed. In the past various people worked on the &fsp;, see Radim Kolar started to work on the &fsp; again in June 2003. Since that time he's maintaining the source distribution. You can reach him on the mailinglists or via e-mail hsn@sendmail.cz-nospam. Since September 2003 Sven Hoexter is working on parts of the documentation. You can reach him on the mailinglists or via e-mail sven@du-gehoerst-mir.de-nospam. See There are several ways to obtain the fsp source distribution please take a look at At the moment we provide only RPM for RedHat 7.3. You can download the RPM file from the sf.net download page or from wrack.telelev.net. wrack.telelev.net provides the rpms in a apt-rpm usable way, for more information about apt-rpm and the packages on wrack.telelev.net take a look at http://sven.stormbind.net/aptrpm/ If you're running other rpm based distributions you can try to rebuild the source rpms. After download you should be able to unpack the source tarball, change into the source directory and use ./configure make su make install FIXME Information about the fspd Maybe I should start a second document describing all the possible configuration options in the fspd? Raw overview for this section: - install fspd -> see prior section - basic infos on fspd.conf - info about init scripts - info about the several .FSP_ files FIXME FIXME Use init scripts ;) FIXME FIXME Explain client tools and those f*cking shell script wrappers *hrhr* Check the X based fsp tools, maybe they'll compile ... FIXME Information about all the f* commands FSP Client is FTP-like interface for FSP. It look exactly like classic FTP does, but uses different transfer protocol. People usually find this program easier to use that multiple client programs in FSP protocol suite. If you are at least somewhat familiar with command line FTP and want to try FSP, this program is right for you. FSP client homepage is http://fspclient.sourceforge.net. If you want a GUI, the easiest way is to use FSP directly from your web browser. You need to download and install FSP Java LIB and Proxy server. Read included docs for install instructions. FSP protocol is well documented. You can find the latest protocol definition right here http://fsp.sourceforge.net/doc/PROTOCOL.txt or in the source distribution in the "doc" subdirectory. FSP is datagram-based protocol. FSP protocol itself do not require reliable underlying transport. FSP can operate even without any Layer 2 and Layer 3 transports only with some extra features disabled. FSP can be implemented in all kinds of environments. This makes FSP very suitable for embedded devices area. When used in TCP/IP based networks, UDP is used for transporting of FSP datagrams, this lowers protocol overhead by skipping TCP level, which is not needed, because FSP handles connection management by itself. This method is standardized in . FSP frames can be packed directly info Ethernet 802.2, USB or GSM frames. Biggest advantage of using FSP in that areas is simplicity. FSP protocol is very simple to implement. One of interesting parts of underlying network technology used by FSP is how protocol design restrict user from sending more than one packet into network. FSP uses very simple method: Every packet sent from server to client contains a 16-bit long, pseudo random key. Client needs this key for sending next request to server. Server will ignore requests with wrong key, unless there was no previous activity recorded for client IP address in last 60 seconds. Server remembers for each IP not only nextkey, but also previous key. When packet's key matches previously stored key -- it is resend from client. This method also allows ignore duplicate requests by server for action which should not be done more than once (for example mkdir). In each packet sent by client to server is sequence number. Server will echo this number back to the client, when packet is processed. There is no restriction on that number, client can use anything he wants. Primary purpose of this sequence number is possibility to put unique mark on every network packet. Client can use this information and fine-tune its retry algorithm by computing packet loss ratio of network, and duplicate packet rate.