The ideaFor many people the suggested solution will suffice. However, as more and more people do these days, I proxy all my traffic through my (EU-based) VPS. This is perfectly doable with the setup I described in my last blogpost. But it becomes tedious to set up if you have multiple devices. Therefore, I want to do all my routing on my VPS, as outlined in this diagram:
|----------| |----------| | | | | /------| VPN |------>| Netflix | |----------| |----------| / | | | | | | | |-------/ |----------| |----------| | Desktop |-----> | VPS | | | | |-------\ /------^-----\ |----------| |----------| \ / \ \-------| Internet | \ / \-----------/
The connection between my Desktop and VPS is an OpenVPN tunnel. The connection between the VPS and VPN is also an OpenVPN tunnel provided by Private Internet Access (PIA). We cannot simply duplicate the exact same logic from my last blogpost in this situation, because it's based off the UID of the Netflix application, and in the current setup the routing is done on a different machine than where the Netflix application runs. In this particular example the advantage of routing to the VPN service from PIA are limited, but once you get multiple devices (laptop, mobile phone...) on the left side of the VPS, and start using multiple VPN services or locations, it starts to pay off.
IPIPWe need to find a solution to tell the VPS what gateway to take. It would of course be possible to set up another VPN connection between my Desktop and VPS but that would be tedious, especially once the number of clients and gateways increases. In my current VPN setup OpenVPN has been configured in routed mode (using TUN interfaces rather than TAP) and that means that we cannot use any layer-2 specific features (like abusing the QoS bit to indicate the desired gateway). In my case I decided to with an IPIP (Ip over Ip) tunnel. When an IP Packet is sent through an IPIP-tunnel, the only thing that happens is that an extra Source and Destination header is added to the IP Packet, as is illustrated in the Wireshark capture below (click to enlarge):
Setting it upIn the steps outlined below I'll assume you've already set up a VPN connection between the Desktop and the VPS. If you have not, you could take a look at this blogpost. On both the client and server the interfaces are called tun0. Furthermore, I'll assume you've already configured a VPN on the VPS towards the US, called tunUS, as described in my last post.
Creating the IPIP-tunnelBecause the IPIP-tunnel can only be initialized once the OpenVPN connection between the Desktop (Client) and VPS (Server) has been established, I put the logic of creating this IPIP-tunnel in a script that would be called by OpenVPN once the connection is initialized (using the OpenVPN 'up' directive).
On the client side:
1 2 3 4 5 6 7 8 9 10 11
#!/bin/bash # make sure all routes have been initialized sleep 5 ip tunnel add tunUS-in-tun0 mode ipip remote 172.31.254.1 local 172.31.254.10 ip address add 172.31.252.2/30 dev tunUS-in-tun0 ip link set tunUS-in-tun0 up ip route add default via 172.31.252.1 src 172.31.252.2 table 2 ip route add 172.31.252.0/30 dev tunUS proto kernel scope link src 172.31.252.2 table 2
The 172.31.254.1 and .10 addresses are the OpenVPN gateway and client ip. I'd have preferred to use the variables that OpenVPN passes on to this script, but for some reason the gateway ip that's supplied is wrong (or technically right, but cannot be used for this purpose). On the server side we have a script that is almost the same:
1 2 3 4 5 6 7 8 9 10
#!/bin/bash # make sure all routes have been initialized sleep 5 ip tunnel add tunUS-in-tun0 mode ipip remote 172.31.254.10 local 172.31.254.1 ip address add 172.31.252.1/30 dev tunUS-in-tun0 ip link set tunUS-in-tun0 up ip route add 172.31.252.0/30 dev tunUS-in-tun0 proto kernel scope link src 172.31.252.1 table 2
Directing traffic on the clientNext up, we need to do direct the Netflix traffic from the Netflix application into the IPIP-tunnel. You could very well add this to the same script as the IPIP-tunnel on the client:
1 2 3
iptables -t mangle -A OUTPUT -m owner --gid-owner netflix -j MARK --set-mark 2 iptables -t nat -A POSTROUTING -o tunUS-in-tun0 -j SNAT --to 172.31.252.2 ip rule add fwmark 2 table 2
To verify that everything so far works, you can open a ping as user netflix on the client to a random (public) ip, and verify using tcpdump on the server that the traffic is coming in from the right ip on the tunUS-in-tun0 interface.
Forwarding traffic from Client to VPN on VPSTo get the traffic from the VPS to Netflix via the VPN service we still need to route all traffic from the tunUS-in-tun0 to the tunUS interface on the VPS:
1 2 3 4
echo 1 > /proc/sys/net/ipv4/ip_forward iptables -t nat -A POSTROUTING -s 172.31.252.0/24 -o tunUS -j SNAT --to-source 10.198.1.6 ip rule add from 172.31.252.0/24 lookup 2 ip route add default via 10.198.1.5 dev tunUS src 10.198.1.6
In this case, the 10.198.1.5 and .6 are my gateway and client ip from my VPN Service, I retrieved those using:
1 2 3 4
# ip a s tunUS 17: tunUS: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 100 link/none inet 10.198.1.6 peer 10.198.1.5/32 scope global tunUS
I'm thinking you could perfectly well retrieve those from OpenVPN in its export variables when invoking it using the up-script configuration directive. Beware, in the case of PIA these addresses are changed once a day or so, so hardcoding them may not be the way to go.
RP_FilterLinux has a feature called reverse-path filtering. This feature drops traffic from any ip it has no route for on that interface. Because it does not keep into account the use of multiple routing tables, we'll need to disable it: on both the client and the server.
1 2 3
for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 > $f done
Troubleshooting the setupWith all debugging, you begin initiating traffic on the client, as the Netflix application does:
client# sudo -u netflix ping 22.214.171.124
Then, you should see the traffic going out on the right interface:
1 2 3 4 5 6
client# sudo tcpdump -i tunUS-in-tun0 -ns0 icmp [sudo] password for dolf: tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tunUS, link-type RAW (Raw IP), capture size 65535 bytes 03:13:47.599418 IP 172.31.252.2 > 126.96.36.199: ICMP echo request, id 25294, seq 115, length 64 03:13:47.835465 IP 188.8.131.52 > 172.31.252.2: ICMP echo reply, id 25294, seq 115, length 64
Please take note of the IP addresses. If these don't match, things are bound to go wrong.
So once you've confirmed the traffic leaves your desktop from the right interface using the right ip addresses, it's time to hop on to the VPS, see if the traffic comes in there:
1 2 3 4 5 6 7
server# sudo tcpdump -i tunUS-in-tun0 -ns0 icmp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tunUS-in-tun0, link-type RAW (Raw IP), capture size 65535 bytes 03:12:38.890815 IP 172.31.252.2 > 184.108.40.206: ICMP echo request, id 25294, seq 64, length 64 03:12:39.104200 IP 220.127.116.11 > 172.31.252.2: ICMP echo reply, id 25294, seq 64, length 64 03:12:39.894865 IP 172.31.252.2 > 18.104.22.168: ICMP echo request, id 25294, seq 65, length 64 03:12:40.108261 IP 22.214.171.124 > 172.31.252.2: ICMP echo reply, id 25294, seq 65, length 64
The traffic you see here should match 1:1 with the output of the previous command. If it does not (or you have no traffic at all on this interface), there's something wrong with your IPIP tunnel.
The last step to see if all goes as it should, is to see if the traffic towards the VPN service leaves your VPS correctly:
1 2 3 4 5 6 7
server# sudo tcpdump -i tunUS -ns0 icmp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tunUS, link-type RAW (Raw IP), capture size 65535 bytes 03:17:38.141751 IP 10.198.1.6 > 126.96.36.199: ICMP echo request, id 25294, seq 363, length 64 03:17:38.355633 IP 188.8.131.52 > 10.198.1.6: ICMP echo reply, id 25294, seq 363, length 64 03:17:39.136641 IP 10.198.1.6 > 184.108.40.206: ICMP echo request, id 25294, seq 364, length 64 03:17:39.350106 IP 220.127.116.11 > 10.198.1.6: ICMP echo reply, id 25294, seq 364, length 64
The outgoing IP here should match that of the tunUS address. If you don't see any traffic at all, it's likely that one of your iptables, rule or routes was dropped. This can happen if your tunUS interface is removed (this happens if you e.g. restart OpenVPN - or PIA does).
RP_FilterIf you do see traffic coming in at one place, but it doesn't reach the application or next hop, it may not be related to any of your firewall rules, ip rules, or routes. It will probably mean the return path filtering is kicking in. You can debug this using:
echo 1 >/proc/sys/net/ipv4/conf/<interfacename>/log_martians
Edit September 24, 2013: I did some more digging. It turns out the rp_filtering will by defintion have to be disabled. The post has been modified to reflect this.
Some companies have not really yet grasped the concept of a global Internet, and like to restrict things to the public of just one country. In this case, the movie industry comes to mind, which allows Netflix to only air certain movies and tv shows in certain countries. Although I am located in The Netherlands, and Netflix recently opened their doors there as well, I'm still affected by geo-restrictions. In The Netherlands I'm served Dutch subs, and some tv shows are not available. Others, are available on DVD only in the US, whereas I can simply stream them in The Netherlands.
There's a simple solution of course, a VPN service. With most VPN services, you simply download a small application, and click & run. The VPN application will direct all your traffic to a host in a location of your choosing, and for the outside world it will look as if you were located at that location.
However, even though that works, I don't want to route all of my traffic through a US host (for you know, there's the NSA, MPAA, they got it all). So I needed a way to route all of the Netflix traffic through the VPN, while leaving the rest of my traffic unaffected.
My first attempt did not succeed. Most routing is based on destination IP, but Netflix uses Amazon and other generic CDNs for content distribution. This brings along two problems; The ip-adresses used are constantly changed, so it's near-impossible to track specific IP-addresses. Especially not, because Amazon hosts more than one website, which I'd like to access via my regular IP. The first attempt therefore used a hosts file that mapped the Amazon hostnames to alternative IP's, and based on those IP's I then did some destination based routing. The problem is that netflix uses many hostnames, therewith not really maintainable (I stopped collecting them after two-dozen). I didn't try configuring an HTTP proxy in Firefox, because I figured it wouldn't work in Silverlight. My first attempt can be found here.
Setting it upSo given that using destination IP's wasn't really an option to make any distinction on, I opted to do it based on UID. IPtables has the ability to have a firewall rule match on the user that initiated the traffic, so running netflix as its own user enables one to use a whole set of firewall rules dedicated to just Netflix.
We begin by installing netflix-desktop and setting up the Netflix-user:
1 2 3 4
apt-add-repository ppa:ehoover/compholio apt-get update apt-get install netflix-desktop useradd -m netflix
Allow a sudo'ed application to use XNetflix will not start though. It requires access to X, and we haven't done anything with the network yet either. To allow Netflix to use X, it needs access to your 'xauthority' file. You can find the location of this file by running the following command in Bash.
export | grep xauthority -i declare -x XAUTHORITY="/run/gdm/auth-for-dolf-7osFEz/database"
Then when you have this file, you can grant the Netflix user access:
chmod 640 /run/gdm/auth-for-dolf-7osFEz/database chown dolf:netflix /run/gdm/auth-for-dolf-7osFEz/database
Now you should be able to run Netflix with the following command. That is, you get a nice window, telling you that you do not have access from your country/ip.
DISPLAY=":0" XAUTHORITY="/run/gdm/auth-for-dolf-7osFEz/database" sudo -u netflix -H netflix-desktop
NetworkingNext up is the networking. First you'll need a VPN service. I went with Private Internet Access (PIA) because it works with Netflix, is cheap, and offers an OpenVPN service (some are stuck in the PPTP era). To ensure not all your traffic is automatically redirected to the VPN, and to not have to type your password everytime, you do have to modify the default config files PIA provides. E.g.:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
client dev tunUS # renamed the interface name for brevity. proto udp remote us-california.privateinternetaccess.com 1194 resolv-retry infinite nobind persist-key persist-tun ca ca.crt tls-client remote-cert-tls server auth-user-pass comp-lzo verb 1 reneg-sec 0 ca /etc/openvpn/privateinternetaccess/ca.crt # Add these lines: auth-user-pass /etc/openvpn/privateinternetaccess/passwd route-nopull keepalive 1 30
Linux supports multiple routing tables. When you show the contents of the routing table ('ip route show'), it shows the contents of the default routing table. Just compare the output of the following two commands:
1 2 3 4
> ip route show default via 192.168.1.1 dev eth0 proto static > ip route show table 254 default via 192.168.1.1 dev eth0 proto static
So after you've set up the VPN service, and you know there is more than one routing table you can use, all you need to do is configure a new routing table that sends all traffic through the VPN, and tell Linux to use that routing table for all traffic originating from the Netflix user:
1 2 3
ip route add default dev tunUS table 2 iptables -t mangle -A OUTPUT -m owner --gid-owner netflix -j MARK --set-mark 2 ip rule add fwmark 2 table 2
RP_FilterLinux has a feature called reverse-path filtering. This feature drops traffic from any ip it has no route for on that interface. Because it does not keep into account the use of multiple routing tables, we'll need to disable it:
1 2 3
for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 > $f done
Audio through PulseAudioIf you start Netflix as netflix user, you should get a nice GUI, and no more errors about being in the wrong country. However, once you start to play a movie or an episode of a tv serie, you'll have no sound yet. Assuming a default Ubuntu installation, Pulseaudio is used to manage all audio, and by default it doesn't support multiple users. Now you can try to get that right the right way, or you can go the quick-and-dirty way (my favorite). The only disadvantage is that if Netflix is playing, no other applications can use the sound system.
1 2 3 4
sudo usermod -a -G audio netflix sudo ln -s /home/dolf/.pulse-cookie /home/netflix/.pulse-cookie sudo chown dolf:audio .pulse-cookie sudo chmod 640 .pulse-cookie
Now you should be good to go, and all should be running.
I myself proxy all my traffic via my own VPS (located in the EU as well), and from there I route my traffic via the appropriate VPN service. But because the Netflix application runs locally, I cannot make a distinction based on GID on my VPS. In a next blog I'll show how you can still get that done. EDIT: New blogpost: Netflix - Application based routing over another VPN
Edit September 24, 2013: I did some more digging. It turns out the rp_filtering will by defintion have to be disabled. Therefore, the section 'RP_Filter' has been added.
Edit September 26, 2013: Added the keep-alive statement to the OpenVPN config so there are less timeouts and your routes etc should be more stable due to less changing IP's.
My current VPS provider Edis provides a block of IPv6 addresses, but unfortunately it's only 1 /112 block. I wanted to use this VPS as my VPN gateway, and provide all devices behind this VPN with IPv6 addresses.
There is also the problem of renumbering; were I ever to choose a different VPS provider (it's unlikely I'll stick to the same one for eternity), I'd have to assign a different IP address to all devices that reside behind my VPN. RFC 4192 describes how you can go about this renumbering.
To overcome this problem, one would usually assign two addresses to each device: One ULA address, and one publicly routable address. The ULA address would be used for all internal traffic and the publicly routable addresses for traffic that also goes beyond the private network.
The problem here is that the smallest block of addresses that can be used by OpenVPN is a /112 subnet. That limit appears to be an arbitrarily chosen limit (I think it could also be /126), but it's one we need to deal with whatsoever. This limit means that we cannot assign a IPv6 subnet to the public interface of the gateway, and a smaller part of that subnet to the side of the VPN interface.
ULAULA, introduced in RFC 4193, provide an equivalent to the IPv4 Private Address Spaces like 192.168.0.0/16.
The idea is that everybody can use their own ULA-space. To accomplish this, you can generate your own /48 prefix to use within your network. By making sure you have a unique prefix, it should be possible to connect any two private networks together (e.g. by means of a lan2lan vpn).
A prefix can be generated based on your MAC-address and the current time, the first of which should by design be unique as well. Sixxs provides a simple generator to generate your own ULA prefix.
Because there's a slim chance that your generated prefix may collide with that of someone else, Sixxs has also set up a registry of prefixes. This registry seems to be the defacto standard. You can register your prefix on the same page as the generator.
As means of an example I will use the fd6c:62d9:eb8c::/48 ULA prefix in this blog post. The Sixxs registry ascribes this prefix to 'Example'. If you're so smart to just copy/paste this prefix, it's your loss - eventually.
SolutionIn order to supply all devices behind my VPN with an IPv6 address, and provide them with connectivity to the internet (so not just a link-local address), and not having to renumber my internal addresses when I change ISP, I set up the following scheme:
- All devices behind my VPN use a ULA subnet only.
- The gateway performs 1:1 NAT, translating from my public /112, to a /112 subnet from my ULA /48 prefix.
- OpenVPN assigns addresses from a /112 subnet that are part of the ULA block.
"But NAT is bad!"
Well, that depends. When most people refer to NAT, they mean NAPT, Network Address Port Translation, as defined in RFC2663, section 4.1.2. With this type of NAT, one usually has 1 public ip address, and multiple private ip addresses. What packets are routed to what private ip is determined based on the destination port. However, all outgoing connections are required to be tracked by the NAT-gateway as well in order to know what private ip is meant to receive what incoming packet.
However, with the kind of NAT I describe here, the size of the subnet on the public side has the same size of that on the private side, both /112. All private addresses have a public counterpart, and therefore no tracking of connections is required (a firewall is!). Whenever a packet comes in, the public prefix is simply replaced by the ULA prefix, and the checksum is recalculated. The only disadvantage is that protocols that also carry addresses over layer 7 (e.g. SIP), may still be broken. But those protocols are rare, and most have already means to tackle problems like these.
The stepsSome say that IPv6 has been around for almost 15 years now, since it was published in RFC 2460 in 1998. However, means to actually implement the protocol are still being published and released up to today.
Until recently, the Linux kernel didn't support IPv6 NAT support natively, although some patches have been around for some time. This functionality was implemented in the recently released version 3.7.
I'm on Ubuntu 12.04 (Precise), and by default this doesn't have 3.7 yet. That may be coming in two weeks though with the release of 12.04.3. You can however also install a new version of the kernel with hardly any dependencies, and you'll need an accompanying version of Netfilter (iptables) as well:
1 2 3 4 5
apt-get install linux-generic-lts-raring wget http://ftp.us.debian.org/debian/pool/main/i/iptables/libxtables10_18.104.22.168-1_amd64.deb http://ftp.us.debian.org/debian/pool/main/i/iptables/iptables_22.214.171.124-1_amd64.deb dpkg -i libxtables10_126.96.36.199-1_amd64.deb dpkg -i iptables_188.8.131.52-1_amd64.deb
A reboot is required for the new kernel to come into effect.
OpenVPNThe next step is installing OpenVPN itself. For them IPv6 support is also relatively new, and you need version 2.3 to get all the bells and whistles:
1 2 3
echo deb http://repos.openvpn.net/repos/apt/precise-snapshots precise main > /etc/apt/sources.list.d/repos.openvpn.net-precise-snapshots.list apt-get update apt-get install openvpn=2.3.*
I set up the following configs, based off this tutorial which may provide some additional pointers and how to generate your certificates used for authentication.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
cat /etc/openvpn/openvpn.conf port 1194 proto udp dev tun ca ca.crt cert server.crt key server.key dh dh4097.pem ifconfig-pool-persist ipp.txt keepalive 10 120 comp-lzo user nobody group users persist-key persist-tun status openvpn-status.log verb 3 client-to-client server-ipv6 fd6c:62d9:eb8c::/112 server 172.31.253.0 255.255.255.128 push "route-ipv6 fd6c:62d9:eb8c::/48" push "route-ipv6 2000::/3"
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
remote server.example.com 1194 client port 1194 ca server.ca.crt cert client.crt key client.key comp-lzo dev tun tun-ipv6 tls-client proto udp persist-key persist-tun user dolf group dolf mtu-test status openvpn-status.log
NetworkingThen it's required to allow traffic to be forwarded from the private side of things to the public side, set the following values in /etc/sysctl.conf:
1 2 3 4
net.ipv6.conf.all.proxy_ndp = 1 net.ipv6.conf.all.forwarding = 1 net.ipv6.conf.all.autoconf = 0 net.ipv6.conf.all.accept_ra = 0
You can reload Sysctl by performing on the command line: sysctl -p
We also need to tell Iptables how to translate the ip addresses. Here, eth0 is the WAN interface, fd6c:62d9:eb8c::aa7 the address that OpenVPN assigns to my client, and 2606:2800:220:6d:26bf:1447:1097:aa7 the address that is mapped to.
1 2 3 4 5
ip6tables -F ip6tables -t nat -F ip6tables -X ip6tables -t nat -A POSTROUTING -o eth0 -s fd6c:62d9:eb8c::aa7 -j SNAT --to-source 2606:2800:220:6d:26bf:1447:1097:aa7
This is suboptimal, ideally Netfilter would allow to simply translate fd6c:62d9:eb8c::/112 to 2606:2800:220:6d:26bf:1447::/112. However, there are no plans to implement it; at least not anytime soon. Given that you now need 1 iptables rule per client, you'd ideally be able to script it using the client-connect directive from OpenVPN. However, currently OpenVPN doesn't yet pass the required ipv6 parameters to the script for the script to determine what client ip is was assigned, as described in this issue in the OpenVPN issue tracker. In my case this is not a real problem, since I only have a few devices on my personal VPN, but if you have many, you may want to look into fixing that issue yourself (and contribute back of course!).
In case you're interested, this is my networking config (/etc/network/interfaces):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet static address 257.235.256.92 netmask 255.255.255.0 network 257.235.256.0 broadcast 257.235.256.255 gateway 257.235.256.1 iface eth0 inet6 static autoconf 0 privext 0 address 2606:2800:220:6d:26bf:1447:1097:1 gateway 2606:2800:220::1 netmask 48 accept_ra 0 dns-nameservers 2001:4860:4860::8888 2001:4860:4860::8844
Public RoutingOnce you've got the VPN up and running, there's a connection between the VPS and datacenter gateway, and between the VPS and VPN clients. However, the datacenter gateway may not yet be aware where to route the addresses to. With IPv6, the Neighbor Discovery Protocol, NDP (RFC 3122) is used to locate devices within a network based on their ip address, just like ARP is used with IPv4.
By default, Linux only sends out NDP Advertisements for addresses that it has been assigned. I found ('tcpdump -nli any icmp6') that initially all traffic from VPN clients was going out successfully, but that the datacenter had no idea where to send the packets when they were returned from my destination.
The first thing I tried was configuring the ip address that should be routed to my VPS for a VPN client on my public interface:
ip a a 2606:2800:220:6d:26bf:1447:1097:aa7 dev eth0
It worked, but seems like a major hack. A more appropriate solution turned out to be configuring the neighboring ip settings manually.
ip -6 neigh add proxy 2606:2800:220:6d:26bf:1447:1097::aa7 dev eth0
This is more or less a correct way to do it, but still can be fairly tedious to maintain. There is also Neighbor Proxy Daemon IPv6, NPD6. This daemon simply sends out NDP Advertisements for all NDP Neighbor Solicitations for a given prefix.
1 2 3 4 5 6 7
cd /usr/src apt-get install subversion gcc svn checkout http://npd6.googlecode.com/svn/trunk/ npd6-read-only cd npd6-read-only/ apt-get install gcc make make install
Then configure the prefix directive in /etc/npd6.conf:
And finally restart NPD6:
Initially, NPD6 didn't want to start. If that's the case, check for typos in your config. In my case, it turned out that the last colon on the end of the prefix is pretty mandatory.
Once you've covered these steps, you should have your VPN setup up and running. Enjoy your new VPN, and welcome to the IPv6 era!