"The Linux Gazette...making Linux just a little more fun!"

Proxyarp

From cly on Fri, 25 Dec 1998

My problem is, that I should route between to nets with linux, but it doesnt work.

I think, the problem is, that the routers sitting on the outer network dont know me as a router.

How can I solve this ?

Cly

Well, you're probably correct. If your host is a route to a block of machines, and other routers on your LAN don't know that you are the route, then they won't forward the packets to your --- and you won't get the opportunity to route them.

There are three solutions to the problem:

1. Add a static route to the other routers
2. Publish your routes via RIP, OSPF, or some other routing update protocol that's supported by your routers (running 'routed' or 'gated' on your host as appropriate). Note: this may require changes to the configurations on these other local routers --- since they may be configured to filter routing updates except from a well defined list of peers.
3. Use "proxyarp"

... The "proxyarp" trick will only work if the systems to which you are trying to route are using addresses that would normally be local to your other routers. I realize that that sentence didn't make any sense. So let's use an example:

(complete with ugly ASCII art)

              +--------+                 +----------+ 222.123.45.48/28
 (Internet) --| router |-----------------| proxyarp |------+
              +--------+                 +----------+      |
                          222.123.45.32/27                 |  +-----+
                                                           +--| cow |
                                                              +-----+

... in this example (which is similar to one I'm using at home --- though the IP addresses have been changed to discourage people from probing my network more than they already have) we have a router. In my case it's a DSL --- provided by my ISP and configured to give me a block of 30 addresses from 222.123.45.32 through 222.123.45.63, minus the network and broadcast address, of course). This is a "5 bit" subnet --- that is to say that it uses a netmask of 255.255.255.224 or 32-27=5.

I want to hide my systems behind a home brewed Linux firewall --- especially after I noticed three IMAP and one mountd scans within the first month of getting the line --- before I've even published any DNS entries pointing to these addresses!

So I put up a Linux system with two ethernet interfaces and a copy of the 2.0.36 kernel on it. The router is at 222.123.45.33 (the first address in the block). The "outer" interface on my "proxyarp" (firewall) is at 222.123.45.46 (the last usable address in the lower half of that range --- if I want to subnet the block into two grougs of fourteen addresses for a 4 bit subnet using a mask of 255.255.255.240). The inner interfaces (further away from my DSL router) is assigned 22.123.45.62 (the last usable address in the upper 4-bit subnet of my original 5-bit subnet. These are arbitrary choices that I make for my own convenience. They allow me to have simpler routing tables and packet filters on the inner LAN.

The problem here is that the DSL router doesn't know that it should forward packets to "cow" (at 222.123.45.49, for example) to my firewall system. So I issue the following command:


#!/bin/sh
/sbin/arp -s 222.123.45.48 -i eth0 -D eth1 netmask 255.255.255.240 pub

... on the firewall (proxyarp host).

Now, when the DSL router gets a packet for anything in the 222.123.45.32/27 range it does an ARP (address resolution protocol) transaction to determine which ethernet card the data frame should be addressed to.

This is a source of confusion in advanced routing. "Packets" are transport layer constructs. "Frames" are network layer constructs which encapsulate packets. Packets are addressed by IP addresses (the familiar sets of four "octets" or dotted decimal numbers). Frames are addressed differently according to the media over which they are sent. Ethernet frames addressed by their "MAC" (media access control) addresses --- a 48 bit number which is either from the card's "BIA" (burned in address) or set by ethernet driver (look at the "hw" option on your 'ifconfig' command's man page).

You can see your ethernet card's MAC address by reading the output of the 'ifcommand' They are usually represented something like:

               HWaddr 0A:60:B7:0F:F1:6E  Int:10 addr:0x300

Here's how ARP works. By addressing frames to the proper ethernet card --- all of the other ether cards on all of the other systems on a given LAN segment can filter out frames that aren't intended for them. This allows the host in which they are installed to work more efficiently --- since the driver software doesn't have to "look at" those frames (unless that card is in "promiscous mode" running 'sniffit' or 'tcpdump' for example). However, the routers and other local systems much find a way to map between the IP addresses and the corresponding MAC addresses. (We're only talking about ethernet here --- though similar mechanism exists for ARCnet and token ring; ATM is completely different)

ARP is that method. Any system on an ethernet segment that has a packet that appears to be destined for the local segment (according to its routing table) will send out an ethernet broadcast frame essentially yelling out: "Will the owner of IP address 222.123.45.61 please speak up!" Every host on that segment then must process an interrupt to filter that ARP request. The system that actually "owns" this IP address (has it assigned to their interface, or is acting as the proxyarp host for it) will send a response packet addressed just to the original requestor. The requestor will then cache this ARP entry (so the one ARP broadcast will normally occur at the beginning of the first TCP connection or UDP transaction and stay valid for a long time).

... The idea of "proxyarp" is that this 'arp' command, with the "pub" option tells the Linux kernel to respond to ARP requests for any of the addresses in the specified subnet. (It's also possible to publish an ARP for a single address, of course).

When the kernel gets a copy of the frames that are addressed to it's ethernet MAC it will process them. When it discovers that the packet encapsulated in this frame is not one of its addresses it will attempt to forward it. Actually in recent kernels it will refer to the value of the /proc/sys/net/ipv4/ip_forward flag and only forward the packet if that is set to one. So, you must use a command like:

echo 1 > /proc/sys/net/ipv4/ip_forward

... to do any packet routing --- proxyarp or otherwise. If the ip_forward flag is set to 0 then only those packets which originated from the kernel and from locally running processes will be forwarded.

Assuming that you have ip_forward set to 1, Linux will then look for a route to these addresses. So we must have valid routes on the firewall/proxyarp host for both of these networks. In our example the firewall machine has a routing table that looks something like:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        9 lo
222.123.45.32   0.0.0.0         255.255.255.240 U     0      0        9 eth0
222.123.45.48   0.0.0.0         255.255.255.240 U     0      0        9 eth1
0.0.0.0         222.123.45.33   0.0.0.0         UG    0      0      166 eth0

While the routing table on the DSL router probably looks something like:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        9 lo
222.123.45.32   0.0.0.0         255.255.255.224 U     0      0        9 eth0
0.0.0.0         RT.TO.MY.ISP    0.0.0.0         UG    0      0      166 ppp0

(where RT.TO.MY.ISP is address to my ISP's routers and thus to the Internet as a whole).

The routes on "cow" and system clients on that segment would look like:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        9 lo
222.123.45.48   0.0.0.0         255.255.255.240 U     0      0        9 eth0
0.0.0.0.0       222.123.45.63   255.255.255.240 UGH   0      0        9 eth0

... which shows that the addresses between 222.123.45.49 and 222.123.45.62 are all "Local" and that any other packets should be relayed to the firewall.

For any hosts that I put between the router and the firewall I'd have routes that looked like:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        9 lo
222.123.45.32   0.0.0.0         255.255.255.240 U     0      0        9 eth0
222.123.45.48   222.123.45.46   255.255.255.240 UGH   0      0        9 eth0
0.0.0.0.0       222.123.45.33   255.255.255.240 UGH   0      0        9 eth0

... which tells those systems that the 222.123.45.33 through 222.123.45.46 addresses on on the local segment, that DSL router is the gateway to the Internet (the default route), and that the "outer" interface on the firewall is the gateway to the 222.123.45.48 subnet.

It's also possible to act as a proxyarp for PPP, SLIP, and PLIP connections. In other words I can proxyarp/route between other types and combinations of interfaces besides the two ethernet cards I'm showing in this example. For PPP you can simply add the "proxyarp" directive to the appropriate 'options' file or command line.

There is a mini-HOWTO on ProxyARP by Bob Edwards of the Australian National University (home of Samba, and 'rsync' among other projects) at:

http://metalab.unc.edu/LDP/HOWTO/mini/Proxy-ARP-Subnet.html

I've copied the author on this message --- in case he'd like to make any corrections or it he finds my examples of use in updates to his HOWTO.

I hope that helps. When I dug up a link to that mini-HOWTO I found that Bob has added quite a bit to it. I seem to recall that it used to cover the PPP case pretty well and didn't describe the case between a couple of ethernet segments --- but I may be just hallucinating.

Now that I'm looking at it again I see that most of my message is duplicated effort. Oh well. Hopefully this message will offer an alternative to anyone who just doesn't "get it" from Bob's explanation.

I've probably flubbed some of the numbers in my example when I "sanitized" them. My address block is actually in the 0-31 range rather than the 32-63 range that I've described here.

Last month I also wrote a lengthy treatise on subnetting and routing. Heather tells me it will get published this month, along with many other messages. Please take a look at it since using proxyarp requires an understanding of general routing and subnetting issues. You'll understand proxyarp better if you compare it to "normal" subnetting.

As you can see from my message the arp command is easy. It's just understanding the rest of the routing that takes all the explanation.

Copyright © 1999, James T. Dennis
Published in The Linux Gazette Issue 36 January 1999

						a	b	c	1	2	3	4	5	6	7	9	10	11	12
						15	16	18	19	20	21	22	23	24	25	26	27	28
						29	31	32	33	34	35	36	37	38	39	40	41	42	44
45	46	47	48	49	50	51	52	53	54	55	56	57	60	61	62	63	64	65	66
67	69	72	76	77	78	79	80	81	82	84	85	86	87	91	94	95	96	97	98