  Frox FAQ
  James Hollingshead <frox at hollo.org>
  2001/12/09

  ______________________________________________________________________

  Table of Contents


  1. General
     1.1 What is frox?
     1.2 What can it do for me?
     1.3 What can't it do for me?
     1.4 Why doesn't it run on ... OS
     1.5 Where can I get it?

  2. Compiling and installing
     2.1 ./configure options
     2.2 Systems other than linux

  3. Configuration
     3.1 Configuring frox
     3.2 The Working Directory
     3.3 How should I configure my ftp clients?

  4. Caching
     4.1 HTTP caching vs local caching
     4.2 StrictCaching
     4.3 CacheOnFQDN
     4.4 I get an ``Unable to initialise cache'' error.
     4.5 I retrieved a file I had retrieved before and it doesn't seem to have been cached?

  5. Virus scanning
     5.1 Can I redirect requests to a virus scanning http proxy?

  6. Security
     6.1 User, Group and Chroot
     6.2 Access control lists
     6.3 Command Control Program

  7. Troubleshooting
     7.1 I can't connect to an ftp site through frox?
     7.2 Non transparent proxying won't work
     7.3 I still can't get it to work

  8. Network problems
     8.1 General
     8.2 Control connection problems
     8.3 Data connection problems


  ______________________________________________________________________

  11..  GGeenneerraall

  11..11..  WWhhaatt iiss ffrrooxx??

  Frox is a transparent ftp proxy for linux written under the GPL.

  11..22..  WWhhaatt ccaann iitt ddoo ffoorr mmee??

  It will transparently proxy your ftp clients (duh). This means that
  any clients you have that are behind the proxy will believe that they
  are connecting to an ftp server as normal, but will actually be
  connecting to frox. Frox will do the onward connection to the remote
  server.
  It can also be set up to do non-transparent proxying. In this case the
  ftp client can connect directly to frox, but instead of logging in
  with ``username'' should log in with ``username@ftp.wherever.org''.

  On either of these sorts of connections it can do caching of files you
  download, or converting of data connections from active-->passive
  which can make firewalling rules a lot easier/safer. There is also
  basic support for running a virus scanner on downloaded files.

  These extra options can all be enabled/disabled at compile time so if
  you don't want them you can still have a small binary - the default
  one is around 30K.

  11..33..  WWhhaatt ccaann''tt iitt ddoo ffoorr mmee??

  It does not implement ftp proxy over HTTP. This means if you configure
  a web browser (eg. Netscape/Mozilla/IE) to use frox as their ftp proxy
  it won't work. If you leave them with ftp-proxy unconfigured then they
  should be transparently proxied like anything else.

  IPv6 is completely unimplemented

  11..44..  WWhhyy ddooeessnn''tt iitt rruunn oonn ...... OOSS

  Transparent proxy system calls vary a lot from system to system. I
  only have success reports for linux and FreeBSD, but you may well have
  success on any UNIX like OS which uses ipfilter. See the section on
  ``systems other than linux'' for details and let me know if you have
  success or failure

  11..55..  WWhheerree ccaann II ggeett iitt??

  Latest versions should be downloadable from the homepage at:
  <http://frox.sourceforge.net/> or at;  <http://www.hollo.org/frox/>.

  22..  CCoommppiilliinngg aanndd iinnssttaalllliinngg

  Compile frox with:


               ./configure
               make
               make install (as root)



  22..11..  ..//ccoonnffiigguurree ooppttiioonnss


  +o  --enable-http-cache and --enable-local-cache compile in caching
     support.  You can give both these options if you wish.

  +o  --enable-virus-scan compiles in support for virus scanning
     downloaded files.

  +o  --enable-configfile=... allows you to set the default config file
     to something else eg.  /etc/frox.conf. Otherwise it defaults to
     /usr/local/etc/frox.conf

  +o  --enable-transparent-data compiles in support for transparently
     proxying the data connections.  Note you probably don't need this.
     Transparent proxying works without, and it entails security risks.
     If you wish to run on a 2.4.x kernel you will also need --enable-
     libiptc

  22..22..  SSyysstteemmss ootthheerr tthhaann lliinnuuxx

  Frox was originally written for linux, but thanks to some work by
  Sergey Matveychuk it should now run on BSD and possibly some other
  UNIX like OSes. The configure script should do things automatically
  for bsd systems, but their are still one or two things you should be
  aware of.

  The BSD transparent proxying code can use ipfilter or ipfw.  ipfw is
  the default, but you can use the ipfilter code by compiling with
  --enable-ipfilter. Other OSes which run ipfilter may also work in this
  way. If you try it could you drop me an email and let me know the
  result.

  You should also note that the TransparentData config file option and
  the --enable-transparent-data compile time option only works under
  linux. You probably don't need this functionality anyway, but if you
  really do then feel free to send me a patch. :)

  33..  CCoonnffiigguurraattiioonn

  33..11..  CCoonnffiigguurriinngg ffrrooxx

  By default frox uses /usr/local/etc/frox.conf as a configuration file,
  but this can be changed either by giving --enable-
  configfile=/etc/whatever/you/want to the ./configure script, or by the
  -f command line option.

  The sample config file is well commented and the best documentation
  for configuration. Note you will need to edit some of these options
  for it to work at all. You will also need to create a working
  directory for frox - see below.

  For the transparent proxying to work your kernel will need to be
  compiled with transparent proxy support and you will need to enable
  forwarding with ``echo 1 >> /proc/sys/net/ipv4/ip_forward''. You then
  need to redirect ftp requests passing through the box to frox. For
  kernel 2.2 this will be something like ``ipchains -A input -p tcp -s
  LOCALNET -d 0/0 21 -j REDIRECT 2121'', and for kernel 2.4, ``iptables
  -t nat -A PREROUTING -p tcp -s LOCALNET --dport 21 -j REDIRECT --to
  2121''.

  You can configure frox to run from inetd with the FromInetd option.
  This will not work well with either the local caching or with
  TransparentData, but otherwise should be ok.

  33..22..  TThhee WWoorrkkiinngg DDiirreeccttoorryy

  Frox needs a working directory, specified in the config file. This is
  used for temporary files and sockets, for creating cache files in, and
  is also the directory frox will chroot to. This directory should be
  owned by root, permissions 0755, and frox will create the directory
  structure it needs in here. If you are doing virus scanning or using
  the command control program then these binaries, with any libraries
  they need, must also be added to this directory.

  You may also have problems with resolving host names.  Frox needs to
  resolve host names if you are using non transparent proxying, or if
  you are doing caching with CacheOnFQDN set to yes. There are two ways
  to fix this:

  The quick and easy way is to define ResolvLoadHack in the config file
  to an address which is not defined in /etc/hosts. Frox will try and
  resolve this on startup and this will load the relevant resolver
  libraries. It doesn't matter if this address fails to resolve.

  The other solution is to copy the resolver libraries into the chroot
  jail. For me it works with host.conf and resolv.conf in
  _W_o_r_k_i_n_g_D_i_r/etc/, and libnss_dns.so.2 and libresolv.so.2 in
  _W_o_r_k_i_n_g_D_i_r/lib/

  33..33..  HHooww sshhoouulldd II ccoonnffiigguurree mmyy ffttpp cclliieennttss??

  Normally you don't have to. Certainly don't set the ftp proxy variable
  in your browser or set your ftp_proxy environment variable to
  http://anything.

  If you are using non-transparent ftp proxying (set ``DoNTP'' to
  ``yes'' in the config file) and you have a ftp client which supports
  this (such as ncftp, or debian's apt-get ftp method) then you need to
  set it up to login with ``username@host[:port]''. eg.
  ``anonymous@ftp.gnu.org'' or ``anonymous@ftp.gnu.org:21''. You can of
  course do this manually by typing this in when prompted for the
  username.

  You may have a mixture of clients being transparently proxied and
  clients being non transparently proxied. In this case if you set the
  NTPAddress to the address/port you have told your non transparently
  proxied clients to contact the proxy on then only they will see frox's
  login banner. Clients which are transparently redirected to frox will
  see the login banner of the remote host they are contacting and will
  not be offered the chance to login with username@host:port.

  44..  CCaacchhiinngg

  44..11..  HHTTTTPP ccaacchhiinngg vvss llooccaall ccaacchhiinngg

  With HTTP caching frox rewrites ftp retrieve requests in HTTP and
  sends them to a proxy server like squid which actually retrieves the
  file and does the caching. With local caching frox maintains its own
  cache of recently downloaded files on the hard disk.

  The main advantage of the HTTP method is that the cache can be shared
  with any web browsers which use the same proxy directly for their ftp
  requests. Local caching is much faster (especially for small files)
  and lighter on network resources.

  Caching needs to be compiled in by running ./configure with --enable-
  http-cache or --enable-local-cache. You can have both compiled in if
  you want - the cache method actually used (if any) is chosen in the
  config file.

  For HTTP caching your config file needs to say:


               CacheModule HTTP
               HTTPProxy 192.168.2.1:3128 # proxy address
               # and optionally...
               MinCacheSize 64000 # in bytes



  Files smaller than MinCacheSize will not be cached. Latency on small
  files can be really bad with this method since the HTTP proxy has to
  make a separate connection to the ftp server - setting MinCacheSize
  fairly high will help with this.


  For Local caching your config file should say:


               CacheModule Local
               # and optionally...
               CacheSize 100 # in MB



  The cached copies will be created within frox's working directory as
  specified in the config file. If CacheSize is unspecified the cache
  will grow indefinitely.

  44..22..  SSttrriiccttCCaacchhiinngg

  With strict caching off frox works out the URI for the file it is
  retrieving by sending a PWD to ask the server what the current working
  directory is. On a small minority of servers this may be broken, and a
  server may return the same answer to PWD for more than one directory.
  If additionally two such directories contain files with the same
  names, sizes, and times of modification then frox could cache the
  wrong file by mistake.

  StrictCaching tries to get around this by keeping track of all
  directory changes and making up its own path instead of asking the
  server for it. This has the disadvantage that ``cd pub; get readme''
  will not cache to the same file as ``get pub/readme'' or ``get
  /pub/readme'' or ``cd /pub; get readme''. As a result your cache hit
  rate will probably drop considerably, but you will have the extra
  security. Note that you can set this on a per server basis if there is
  a particular server about which you are worried.

  Thanks to Henrik Nordstrom for pointing this problem out and
  suggesting a solution.

  44..33..  CCaacchheeOOnnFFQQDDNN

  This option involves using FQDNs instead of IP addresses for the cache
  requests (either to squid or the local cache), and allowing squid to
  guess whether to use binary or ascii file transfer rather than using
  the value supplied by the ftp client. For transparently proxied
  connections this requires a reverse DNS lookup to get the FQDN, and
  may cause problems with un-synced mirrored ftp hosts all running off
  the same FQDN, although most of the time there will be no problems.
  The advantage is that you should increase the hit rate, in particular
  with squid based caching for files which have been previously
  downloaded through squid with a web browser.

  If all your clients are being non transparently proxied then this
  option should cause no problems. Otherwise use with some care,
  although it may be worth the extra hit rate.

  44..44..  II ggeett aann ````UUnnaabbllee ttoo iinniittiiaalliissee ccaacchhee'''' eerrrroorr..

  Check that frox was compiled with support for the cache module you are
  attempting to use, and that you have given the correct Cache options
  for that module. If doing http caching check that the http proxy name
  resolves.

  44..55..  II rreettrriieevveedd aa ffiillee II hhaadd rreettrriieevveedd bbeeffoorree aanndd iitt ddooeessnn''tt sseeeemm ttoo
  hhaavvee bbeeeenn ccaacchheedd??

  Files are not cached for non-anonymous ftp sessions, or if the ftp
  server doesn't support the MDTM and SIZE extensions.  Additionally if
  the host's DNS name resolves to more than one IP address then the file
  will be cached, but frox has no way of knowing that the different IP
  addresses refer to the same host - unless your ftp client happens to
  pick the same IP both times you will not get the cached file back.
  Using CacheOnFQDN may impove this behaviour.

  55..  VViirruuss ssccaannnniinngg

  You can configure frox to run a virus scanner on downloaded files
  between downloading them to the proxy and them being forwarded to the
  client. You need to ./configure with --enable-virus-scan, and set the
  config file variable VirusScanner to something like
  '"/usr/local/bin/yourviruscanner" "--options" "%s"'. You no longer
  need to be doing local caching for this to work - it will work with
  caching through squid or with no caching at all.

  Note that the whole variable is enclosed in single quotes ('), while
  the individual arguments within are enclosed with double quotes (").
  The "%s" will be replaced by the name of the file to scan. NB. unlike
  all the other paths in the config file, any pathnames or filenames in
  VirusScanner will _n_o_t be stripped in the case that you are running in
  a chroot jail. If your virus scanner exits with a value other than 0
  when a file is scanned and is virus free you will need to set VSOK to
  that value.

  There are currently some problems with this:

  +o  Only file downloads are scanned.

  +o  If the file contains a virus then the file will not be sent to the
     user, even if your virus scanner is able to clean the virus from
     the file.

  +o  The user will see a delay before the file starts to download as the
     whole thing must be downloaded to the proxy and be scanned before
     any of it can be passed to the client.

  If the VSProgressMsgs option is set to something other than 0 then
  every time that number of seconds passes during the delay a progress
  message will be sent to the client. On the plus side these may be
  displayed to users of command line ftp clients, and will probably
  prevent clients from giving up and timing out. On the down side the
  progress messages could confuse some ftp clients (I think they do with
  some Mozilla builds).

  55..11..  CCaann II rreeddiirreecctt rreeqquueessttss ttoo aa vviirruuss ssccaannnniinngg hhttttpp pprrooxxyy??

  Probably not. Most of these proxies send various http progress
  messages back to the client. Frox is unable to parse these without
  giving it a more extensive http implementation, and so it will
  probably fail. Also there will be a delay before the client starts to
  get the file, and it may well time out during this period. There is
  interest in being able to do this though, so if anyone wants to try
  coding it then they are welcome.

  66..  SSeeccuurriittyy

  66..11..  UUsseerr,, GGrroouupp aanndd CChhrroooott

  Unless compiled with --enable-run-as-root you must specify a non root
  user and group for frox to run as. It is strongly recommended that you
  do this rather than recompiling.

  You must specify WorkingDir for frox in the config file. By default it
  will also chroot to this directory. If you do not wish to run chrooted
  you must explicitly set DontChroot to "yes" in the config file.
  66..22..  AAcccceessss ccoonnttrrooll lliissttss

  Access to frox is controlled by the access control lists in the config
  file. These take the format

          ACL ACTION CLIENT - SERVER [PORTS]



  ACTION can be one of Deny or Allow.  CLIENT and SERVER are either a
  domain name, a *, an ip address, or an address/netmask in either
  x.x.x.x/y or x.x.x.x/y.y.y.y form.  PORTS is optional. If present it
  should contain one or more port numbers or ranges of port numbers,
  separated by ",". There should be no whitespace within the PORTS
  string. If no ports are specified then ports are not taken into
  account in matching that rule.

  Acls are checked after the client has connected to frox, but before
  frox connects to the server. If non-transparent proxying is enabled
  then the client will enter the username before the acl is checked. The
  acls are checked in the order they appear in the config file until one
  matches. If no acls match the connection is denied.

  66..33..  CCoommmmaanndd CCoonnttrrooll PPrrooggrraamm

  Frox has optional support for a ftp-proxy like command control program
  (ccp). This is a a bit of a rip off of the ftp-proxy system, but
  someone requested it so it's gone in. To use this you should compile
  with --enable-ccp, and set CCProgram to the program to use. This must
  be within the chroot jail with any libraries and/or shells it needs to
  run.

  The ccp will then be called for every command that the client sends.
  It will not be called for any commands that frox decides to send
  itself (eg. as part of determining whether a file is suitable for
  caching). It can make use of the following environment variables:

          FROX_CLIENT, FROX_SERVER: Ip addresses of client and server.
          FROX_SERVERNAME:          Server's domain name if known.
          FROX_SESSION:             Unique string for this session.
          FROX_COMMAND:             FTP command just given.
          FROX_PARAMATER:           Argument to that command.



  With a new connection the ccp will be called with the special
  FROX_COMMAND of +NEW. The server variables may subsequently change if
  a user name is given which contains a different server to contact.

  If the ccp exits with a value of 0 then the command is dealt with as
  normal. If it exits with 1 then the command is discarded. If it exits
  with 2 then the session is closed. If the ccp writes a line to stderr
  then it will be sent to the client over the control connection. If it
  writes a line to stdout it will be logged by frox. If the ccp returns
  a value other than 0 it is its responsibility to write a well formed
  ftp message to the client which tells it what is happening (eg. from a
  shell script by echo -ne "533 Permission denied\r\n"  >&2) . Failure
  to do this will result in the client and proxy getting out of sync.

  77..  TTrroouubblleesshhoooottiinngg

  77..11..  II ccaann''tt ccoonnnneecctt ttoo aann ffttpp ssiittee tthhrroouugghh ffrrooxx??

  Look at the log file that frox produces and see if it gives you any
  clues. It should say ``Connect from A to B''. If it doesn't say
  anything then you aren't getting through to frox at all - check the
  config file to see frox is listening on the right address/interface.
  Have you remembered to do the ipchains/iptables command that is in the
  INSTALL file?

  If the log file suggests frox is connecting back to itself then you
  are probably contacting it directly and non-transparent proxying is
  switched off. Either switch non-transparent proxying on in the config
  file, or make sure that your ftp client is not trying to connect
  directly to frox - either from you specifying the frox machine as the
  ftp host on the command line, or from the client trying to use it
  explicitly as a proxy.

  If you have a complicated network setup then check the``Network
  Problems'' section.

  Read the ``client configuration'' section again

  77..22..  NNoonn ttrraannssppaarreenntt pprrooxxyyiinngg wwoonn''tt wwoorrkk

  Check whether you have NTPAddress set. If this is set then it must be
  set to the address and port that the ftp clients are using to contact
  the proxy, otherwise clients will not be offered non transparent
  proxying. If you unset it then all clients will be offered non
  transparent proxying.

  77..33..  II ssttiillll ccaann''tt ggeett iitt ttoo wwoorrkk

  Feel free to drop me a line at frox at hollo.org.  Please let me know
  what version of frox you are running, what version of the linux
  kernel, whether you gave any options to ./configure at compile time,
  and what point exactly it fails at. The log file should give you some
  info, and if you are feeling brave you could recompile with the
  --enable-debug option to ./configure and the log file will give you
  lots of information.

  88..  NNeettwwoorrkk pprroobblleemmss

  88..11..  GGeenneerraall

  If you are running frox on a machine on your default route, and with
  little or no firewalling between it and either the outside world or
  your internal network you should not have network problems. Because of
  the nature of the ftp protocol though, if your setup is more
  complicated than this then you may run into difficulties.

  The easiest way of checking that your network setup isn't what is
  causing the problems is to (temporarily if you wish) run an ftp server
  on the machine which runs frox. If you can't ftp from your clients to
  this server and retrieve files O.K.  then your clients won't be able
  to ftp to frox either. Also try running an ftp client on the frox
  machine and connecting to a server in the outside world. If this
  doesn't work then frox obviously won't be able to do this either.

  88..22..  CCoonnttrrooll ccoonnnneeccttiioonn pprroobblleemmss

  With a control connection problem your clients will not be able to
  connect to ftp servers through frox at all. If transparently proxying
  then check that outgoing tcp traffic is being redirected to frox
  without changing the destination address (ie. you must route it there,
  not use DNAT), and that the machine running frox can make a tcp
  connection to the outside world successfully

  88..33..  DDaattaa ccoonnnneeccttiioonn pprroobblleemmss

  If the problem is with the data connection then your clients will be
  able to connect to the ftp server, login, and do things like print the
  current working directory. File retrievals and directory listings will
  fail, though you may find that passive mode ftp works and active
  doesn't, or visa versa.

  If you are having these problems I strongly reccomend that you set
  TransparentData to no, and ApConv to yes in the config file. The frox
  machine must be able to make tcp connections on any port to the
  outside world. It should also be possible to establish tcp connections
  in either direction between the frox machine and the client machines,
  and with no NAT being performed on these connections. With this setup
  you shouldn't have any problems.

  The above suggestions aren't strictly necessary, but things get more
  complicated without them.

  +o  ApConv = no: Either incoming tcp connections must be allowed from
     the outside world to the machine running frox, or they must be
     prevented only by a NATed (IP masquerading) firewall which is
     configured to allow ftp connections (eg. linux + ipt_nat_ftp.o
     module)

  +o  If frox cannot make tcp connections to the client then clients are
     limited to passive mode ftp.

  +o  If clients cannot make arbitrary tcp connections to frox then they
     are limited to active mode ftp.

  +o  NAT between client and frox: If DNAT occurs when the client
     connects to frox (ie. the client makes the connection to a
     different IP to that which the packets frox eventually receives are
     addressed to then you should set PASVAddress to the IP which the
     clients should use.  NB. This option doesn't actually exist yet,
     but let me know if you need it and I'll consider adding it.

  +o  TransparentData = yes: This can make things really unpleasant. Are
     you _s_u_r_e you need it? These are the problems which spring to
     mind...

     +o  Frox should be in your default route for outgoing tcp traffic.
        If you are using policy based routing to route only port 21 tcp
        traffic to frox then TransparentData will not work.

     +o  Beware that frox will make connections to your clients that
        purport to be from arbitrary hosts outside your network. If
        there is a firewall/router between your frox and your clients
        with IP spoofing protection it must let these packets through.

     +o  Have another read of SECURITY to remind yourself of the extra
        risks you are taking.



