Ftp-Proxy "how to" (forward and reverse proxy)

Home

Ftp-proxy was developed to help ftp clients transverse an OpenBSD PF firewall. The ftp protocol is notoriously difficult to use if a firewall is involved due to active and passive negotiation methods and its non-standard control and data connections. In 2005 a new proxy called "pftpx" was developed for the purpose of helping ftp clients contact ftp servers on either side of a firewall. The OpenBSD group eventually accepted pftpx as its standard ftp proxy and renamed it to "ftp-proxy". ftp-proxy is included by default on OpenBSD 4.0 and later.

ftp-proxy is a ftp proxy with many advanced features such as:

it handles all ftp modes: PORT, PASV, EPRT, EPSV
it handles ipv6
it scales: one process handles all sessions using libevent
it works with "strict" ftp clients (clients that want data connections to the same IP as the control connection)

What is the difference between a forward or reverse proxy?

Think of a single firewall protecting an internal LAN from the Internet. If your machines are on the inside (LAN) and you connect outward to the Internet then this is considered a "forward" direction. Connections made from the Internet to a server located inside the firewall are considered to be in the "reverse" direction.

Forward Proxy is a ftp proxy allowing clients on the inside of the firewall to connect to ftp servers on the Internet.
lan client --> firewall --> internet ftp server
Reverse Proxy is a ftp proxy used to allow public internet clients to access ftp servers on the inside of the firewall.
internet client --> firewall --> internal lan ftp server

How does a ftp proxy work?

Ftp-proxy is a proxy for the Internet File Transfer Protocol. FTP control connections are redirected into the proxy using the pf(4) rdr command, after which the proxy connects to the server on behalf of the client.

The proxy allows data connections to pass, rewriting and redirecting them so the correct addresses are used. All connections from the client to the server have their source address rewritten so they appear to come from the proxy. Consequently, all connections from the server to the proxy have their destination address rewritten, so they are redirected to the client. The proxy uses the pf(4) anchor facility for this. (man ftp-proxy)

In effect, the client is connecting to the proxy and the proxy is connecting to the ftp server. When a client initiates a connection the redirection rules (rdr) inside of PF intercept the packets. The client is routed to the ftp proxy and the the proxy translates any IP addresses from the internal network to the external interface (IP). The proxy then connects to the ftp server as a liaison for the client. When requests are returned from the server to the client the proxy forwards the responses.

Though the setup is different for a forward or reverse proxy the methodology is the same.

Is using a proxy secure?

The use of a proxy helps secure the client as well as the server. Since the ftp proxy is acting as the middle man all commands must pass though the proxy. If an illegal command is sent either from the client or the server the proxy will not accept it.

IMPORTANT NOTE: The following instructions separately cover FORWARD and REVERSE proxy setups. You can also configure both proxy types on the same machine without conflict.

Setting up a FORWARD ftp-proxy

A "forward" ftp proxy is used to allow internal LAN clients to connect to Internet ftp servers when connecting through a PF firewall. All control and data connection rules are made by the ftp-proxy daemon and inserted into PF using anchors.

Step 1: You have two choices in setting up a forward ftp proxy. The most common setup is using the default external ip address as the source of ftp connections from the firewall. This will be option 1. You can instead use option 2 to allow the ftp proxy to use an alternate aliased ip address on the external interface. This is useful if you have a DMZ and want to allow those machine to connect to external ftp servers using a dedicated ip address.
Option 1 - FTP proxy using the default ip on the external interface: Start the ftp-proxy daemon. The daemon is what will translate the clients requests and also make the pf rules to allow the data connections through.

The following command will start the daemon using the options:

queue named "bulk" (-q bulk)

tagging rules under the string "FTPPROXY" (-T FTPROXY)

debug level 7 (-D 7)

very verbose logging (-v)

You can start the daemon manually by using the following command:
/usr/sbin/ftp-proxy -q bulk -T FTPROXY -D 7 -v
To make the ftp proxy start on boot put the following line into your /etc/rc.conf.local :
ftpproxy_flags="-q bulk -T FTPROXY -D 7 -v"
Option 2 - FTP proxy using an alias on the external interface: Start the ftp-proxy daemon. This option uses the argument "-a" to tell the ftp proxy to source all of its connections from a specific ip address on the external interface. The daemon is what will translate the clients requests and also make the pf rules to allow the data connections through.

The following command will start the daemon using the options:

The proxy will use this as the source address for the control connection to a server. (-a 10.20.30.40)

queue named "bulk" (-q bulk)

tagging rules under the string "FTPPROXY" (-T FTPROXY)

debug level 7 (-D 7)

very verbose logging (-v)

You can start the daemon manually by using the following command:
/usr/sbin/ftp-proxy -a 10.20.30.40 -q bulk -T FTPROXY -D 7 -v
To make the ftp proxy start on boot put the following line into your /etc/rc.conf.local :
ftpproxy_flags="-a 10.20.30.40 -q bulk -T FTPROXY -D 7 -v"
Step 2: Edit the PF rules to add the anchors necessary for ftp-proxy. An anchor is used to insert and delete rules from PF dynamically as needed. Ftp-proxy will use these anchors to insert the rules necessary to let internal ftp clients connect to external ftp servers.

The ftp-proxy daemon listens on localhost (127.0.0.1) port 8021 by default. We need to have a redirection rule (rdr) to redirect all requests from the internal network ($IntIf:network) to any external ftp server (to any port ftp) and send those requests to the ftp-proxy daemon (lo0 port 8021).

If you use strict ordering in PF then here are the lines separated into sections. If you need an example of these anchors in a working PF rule set check out our Calomel.org PF config page.
################ Translation ###############################
# Ftp ( secure forward ftp proxy )
 nat-anchor "ftp-proxy/*"
 rdr-anchor "ftp-proxy/*"
 rdr on $IntIf inet proto tcp from $IntIf:network to any port ftp -> lo0 port 8021

################ Filtering #################################
# Ftp ( secure forward ftp proxy )
 anchor "ftp-proxy/*"
Step 3: Apply the new Pf rules using the following line:
pfctl -f /etc/pf.conf
Step 4: Test the connection using a client on the inside of the firewall. Do not use a client on the firewall itself for testing as the proxy can not be used transparently for local clients. Since the ftp daemon is running and the PF rules have been applied you should be able to use any client to connect to any external ftp site. For example, the following line will use ncftp and connect to the primary OpenBSD ftp server anonymously.
ncftp ftp.openbsd.org

NOTE: Calomel.org also offers a OpenBSD Pf Firewall "how to" ( pf.conf ) if you need it. We cover the "forward" ftp proxy example integrated into a working pf.conf rule set.

Setting up a REVERSE ftp-proxy

A "reverse" ftp proxy is used to allow internet clients to connect to internal lan ftp servers when connecting through a PF firewall. All control and data connection rules are made by the ftp-proxy daemon and inserted into PF using anchors.

In this example we will setup a reverse proxy to accept connections from any ip address to port 21 on the firewall and forward that to the proxy. The proxy in turn will connect to the ftp server on the internal lan. Our setup will look like so:

(any ip) --> (firewall ip port 21) --> (rdr to ftp-proxy port 8021) --> (internal ftp server port 21)

Step 1: Start the ftp-proxy daemon in the reverse proxy configuration. This means we need to tell the proxy what port to listen on as well as what ip and port to forward the connection to. Since the proxy normally listens on localhost port 8021 we will keep this setting. We must also tell the proxy to connect to our internal ftp server on ip 10.10.10.50 port 21. This is the line we will use:
/usr/sbin/ftp-proxy -q bulk -T FTPROXY -p 8021 -R 10.10.10.50 -P 21 -D7 -v
Step 2: Add the PF rules to allow external connections to be redirected to the proxy. This involves adding a redirection rule (rdr) and filter rule like so to pf:
################ Translation ###############################
# Ftp ( secure reverse ftp proxy )
 nat-anchor "ftp-proxy/*"
 rdr-anchor "ftp-proxy/*"
 rdr on $ExtIf proto tcp from any to ($ExtIf) port ftp tag FTPROXY -> lo0 port 8021

################ Filtering #################################
# Ftp ( secure reverse ftp proxy )
 anchor "ftp-proxy/*"
 pass in log on $ExtIf inet proto tcp from any to lo0 port 8021 flags S/SA tcpproxy state queue (bulk, ack) tagged FTPROXY
Step 3: Apply the new Pf rules using the following line:
pfctl -f /etc/pf.conf
Step 4: Test the connection using a client on the outside of the firewall. Since the ftp daemon is running and the PF rules have been applied you should be able to use any client to connect to the external interface of the firewall and be forwarded to the internal ftp server. For example here we are using ncftp to connect to the external name of the firewall on port 21.
ncftp external.interface.name.com

Want more speed? Make sure to also check out the Network Speed and Performance Guide. With a little time and understanding you could easily double your firewall's throughput.

Questions?

Can I use CARP with a reverse ftp proxy?

Yes, you can. Bind the reverse ftp proxy to the carp address not the external interface, for example carp0, and pass it to the internal ftp server. Remember carp0 is the virtual interface which is bound to the carp0 ip address.
################ Translation ###############################
# Ftp ( secure reverse ftp proxy )
rdr on $ExtIf proto tcp from any to carp0 port ftp tag FTPROXY -> lo0 port 8021

################ Filtering #################################
# Ftp ( secure reverse ftp proxy )
pass in log on $ExtIf inet proto tcp from any to lo0 port 8021 flags S/SA tcpproxy state queue (bulk, ack) tagged FTPROXY
A problem you might eventually see is when the CARP firewalls fail over. Current ftp connections to the ftp server will die when the backup firewall takes over because it does not have the ftp-proxy anchors from the first firewall. The anchors are not pfsync states and thus are not transfered to the backup firewall through pfsync. But, if the users issue a re-connect to your ftp server after the firewall fail over they will connect without issue.

Is there a limit to how many CARP bound ftp-proxy daemons I can run? The limit really depends on your hardware and how much bandwidth you have. We have setup CARP firewalls with as many as 12 separate CARP ip addresses bound to 12 different ftp-proxy daemons to 12 back end ftp servers. No problems at all. Just remember to run each ftp-proxy daemon on a different port (8021 for the first and 8022 for the second, and so on). Then just use Pf and the redirection rule (rdr) to direct the right client to the right ftp-proxy.

NOTE: Calomel.org also offers a PF CARP firewall fail over "how to" if you need it. We cover the setup and explain each step of setting up a secure CARP firewall fail over cluster.

How do I watch the pf logs in real time?

tcpdump -n -e -ttt -i pflog0

How do I cat the pf log file?

tcpdump -n -e -ttt -r pflog0

How do I tell pf to re-read the pf.conf file after I make a change?

pfctl -f /etc/pf.conf

Can multiple ftp-proxy daemons be run on the same machine?

Yes, as long as they are listening on different ports. The daemons will use the same ftp-proxy table to keep track of client to server sessions, but each proxy will use a sub-anchor so there is no contention between the daemons. Subanchors are named as follows: "ftp-proxy.$pid.$session", where $pid is the process id of ftp-proxy, and $session the ftp-proxy session sequence number. So multiple ftp-proxy's will never collide.

The ftp-proxy daemon dies unexpectedly sometimes. What can I do?

In some of the older versions of ftp-proxy (used to be called pftpx) the daemon used to die unexpectedly under load. A simple solution is to employ a ftp-proxy watcher though cron. The following cron job will look for the ftp-proxy daemon executed with "ftp-proxy -q bulk -T FTPROXY -D 7 -v". If the pid does not exist then the cron job will execute it.
#minute (0-59)
#|   hour (0-23)
#|   |    day of the month (1-31)
#|   |    |   month of the year (1-12 or Jan-Dec)
#|   |    |   |   day of the week (0-6 with 0=Sun or Sun-Sat)
#|   |    |   |   |   commands
#|   |    |   |   |   |
#### ftp-proxy watcher
*/15 *    *   *   *   /usr/bin/pgrep -f "ftp-proxy -q bulk -T FTPROXY -D 7 -v" 2>&1 >/dev/null || /usr/sbin/ftp-proxy -q bulk -T FTPROXY -D 7 -vv

Can you explain the different connection types of FTP and why it has problems with firewalls?

FTP runs exclusively over TCP. FTP servers by default listen on port 21 for incoming connections from FTP clients. A connection to this port from the FTP Client forms the control stream on which commands are passed to the FTP server from the FTP client and on occasion from the FTP server to the FTP client. FTP uses out-of-band control, which means it uses a separate connection for control and data. Thus, for the actual file transfer to take place, a different connection is required which is called the data stream. Depending on the transfer mode, the process of setting up the data stream is different.

In active mode, the FTP client opens a random port (> 1023), sends the FTP server the random port number on which it is listening over the control stream and waits for a connection from the FTP server. When the FTP server initiates the data connection to the FTP client it binds the source port to port 20 on the FTP server.

In order to use active mode, the client sends a PORT command, with the IP and port as argument. The format for the IP and port is "h1,h2,h3,h4,p1,p2". Each field is a decimal representation of 8 bits of the host IP, followed by the chosen data port. For example, a client with an IP of 192.168.0.1, listening on port 1025 for the data connection will send the command "PORT 192,168,0,1,4,1". The port fields should be interpreted as p1 x 256 + p2 = port, or, in this example, 4 x 256 + 1 = 1025.

In passive mode, the FTP server opens a random port (> 1023), sends the FTP client the server's IP address to connect to and the port on which it is listening (a 16 bit value broken into a high and low byte, like explained before) over the control stream and waits for a connection from the FTP client. In this case the FTP client binds the source port of the connection to a random port greater than 1023.

To use passive mode, the client sends the PASV command to which the server would reply with something similar to "227 Entering Passive Mode (127,0,0,1,78,52)". The syntax of the IP address and port are the same as for the argument to the PORT command.

In extended passive mode, the FTP server operates exactly the same as passive mode, however it only transmits the port number (not broken into high and low bytes) and the client is to assume that it connects to the same IP address that was originally connected to. Extended passive mode was added by RFC 2428 in September 1998.

While data is being transferred via the data stream, the control stream sits idle. This can cause problems with large data transfers through firewalls which time out sessions after lengthy periods of idleness. While the file may well be successfully transferred, the control session can be disconnected by the firewall, causing an error to be generated.

The FTP protocol supports resuming of interrupted downloads using the REST command. The client passes the number of bytes it has already received as argument to the REST command and restarts the transfer. In some command line clients for example, there is an often-ignored but valuable command, "reget" (meaning "get again") that will cause an interrupted "get" command to be continued, hopefully to completion, after a communications interruption.

Resuming uploads is not as easy. Although the FTP protocol supports the APPE command to append data to a file on the server, the client does not know the exact position at which a transfer got interrupted. It has to obtain the size of the file some other way, for example over a directory listing or using the SIZE command. Wikipedia, FTP

Questions, comments, or suggestions? Contact Calomel.org